Custom Deepstack Models

[ghzgod]

Describe the solution you'd like
I have created a few custom deepstack models using this tutorial Deepstack Custom Models. I can then ask Deepstack to analyze an image via the custom model link EG: http://10.0.0.X:5000/v1/vision/custom/my_custom_model_name?image which will reply with the confidence of the detected object.

A good example of this would be not only detecting that a car is in the driveway, but who's car is in the driveway? Mine? My friend? etc.

I've been able to save triggered images from node-deepstack-ai by setting the purgeInterval to 0 (thanks for adding that) and then using those images I've collected overtime, to label and train.

So logically...I can see this as follows:

1. My driveway has motion, image stores to aiinput folder
2. Node-deepstack AI sends it to default deepstackUri for analysis.
3. Another trigger config in triggers.json sends that same image to my custom deepstack model link for analysis.
4. From there, I can take actions via MQTT.

Maybe allowing a deepstackUri override in triggers.json? And ignoring things like watchObjects as the only objects that will be detected are

1. A car I know or
2. Not a car I know

aka (A hot dog/not a hot dog 😄 ).

I am sure this is more complex but just an idea. I can't imagine if many people use custom models or not. Thank you in advance.

[neilenns]

Thank you for the suggestion, I have a build for you to try.

Change your docker-compose so instead of using latest it uses issue416. Then in your triggers.json add a customEndpoint property that points to the path for your custom model on each trigger that needs it:

"customEndpoint": "/v1/vision/custom/my_custom_model_name?image"

You still need to specify at least one watchObject but the code will ignore it.

I don't really have any way to test this so let me know if it works. If it does I'll figure out how to make the JSON schema not require watchObjects when customEndpoint is specified.

[ghzgod]

This is working beautifully and I am thankful because my node-red flow was more complex than it needed to be.

My mqtt alert shows the custom trained objects and their predicted values. Perfect! Thank you so much.

If you ever wanted to test this on your own, here is a custom license plate detector model if you have a camera facing a street with car traffic.

[neilenns]

Can you share the JSON you are getting in the mqtt messages? I'd love to see it. Thanks for the link to the license plate detector. Will see if I can integrate that into the dev stack for testing. Once I get the schema figured out I'll release the change and let you know when it's is available via latest.

…

________________________________ From: ghzgod <[email protected]> Sent: Monday, January 18, 2021 10:38:58 AM To: danecreekphotography/node-deepstackai-trigger <[email protected]> Cc: Neil Enns <[email protected]>; Assign <[email protected]> Subject: Re: [danecreekphotography/node-deepstackai-trigger] Custom Deepstack Models (#416) This is working beautifully and I am thankful because my node-red flow was more complex than it needed to be. My mqtt alert shows the custom trained objects and their predicted values. Perfect! Thank you so much. If you ever wanted to test this on your own, here is a custom license plate detector model<https://github.com/odd86/deepstack_licenceplate_model> if you have a camera facing a street with car traffic. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#416 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACIVHFTSCV6LMOICCZI4UDLS2R54FANCNFSM4WHJZRWQ>.

[ghzgod]

Sure, is this what you're looking for?

{
"analysisDurationMs": 4082,
"basename": "drivewayhd.20210118_195848070.jpg",
"fileName": "/aiinput/drivewayhd.20210118_195848070.jpg",
"formattedPredictions": "Toyota (60%), Nissan (67%)",
"name": "Driveway Custom",
"predictions": [
{
"confidence": 0.60056514,
"label": "Toyota",
"y_min": 26,
"x_min": 854,
"y_max": 1202,
"x_max": 1949
},
{
"confidence": 0.6700693,
"label": "Nissan",
"y_min": 78,
"x_min": 16,
"y_max": 1236,
"x_max": 630
}
],
"state": "on"
}

[neilenns]

Yep. That's super cool! I think I got the schema figured out, will have a pre-release build for you to try shortly.

[neilenns]

Ok this is now published under the dev image tag. Would you mind updating your config to that and verifying it still works? Also please also try various combinations of watchObjects and customEndpoint: it should complain if you specify both, complain if neither are specified, etc.

[ghzgod]

When you say "it should complain if you specify both" I thought earlier you mentioned that I had to have a value in there (watchObjects). Has that changed since issue416 image?

[neilenns]

Yes, I figured out how to enforce one or the other but not both via the JSON schema.

[ghzgod]

dev tag is working.

As far as triggers.json :

It does not complain if I have both watchObjects and customEndpoint set.

It does not complain if I have only customEndpoint and no watchObjects set.

It does not complain if I have only watchObjects and no customEndpoint set.

It does not complain if I have neither customEndpoint / watchObjects set.

The logs for all 4 of these conditions in triggers.json showed the image being analyzed. The only one that did not return results was when both customEndpoint watchObjects were unset which makes sense.

[neilenns]

Hmm. Ok, thanks. I'll have to do a bit more digging!

[neilenns]

So the reason it doesn't error is because... I never call the validation method 🤦‍♂️ I wonder how long that bug has been there. Sigh.

…

________________________________ From: ghzgod <[email protected]> Sent: Monday, January 18, 2021 12:12:28 PM To: danecreekphotography/node-deepstackai-trigger <[email protected]> Cc: Neil Enns <[email protected]>; State change <[email protected]> Subject: Re: [danecreekphotography/node-deepstackai-trigger] Custom Deepstack Models (#416) dev tag is working. As far as triggers.json : It does not complain if I have both watchObjects and 'customEndpoint' set. It does not complain if I have only 'customEndpoint' and no 'watchObjects' set. It does not complain if I have only 'watchObjects' and no 'customEndpoint' set. It does not complain if I have neither 'customEndpoint' / 'watchObjects' set. The logs for all 4 of these conditions in triggers.json showed the image being analyzed. The only one that did not return results was when both customEndpoint watchObjects were unset which makes sense. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub<#416 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACIVHFTWLN5J3MKBETMRAJLS2SI2ZANCNFSM4WHJZRWQ>.

[ghzgod]

Haha the wonderful world of coding. Trust me, I know how you feel.