Implementing custom model with object-detection example

[semroai]

Hi,

I'm trying to implement my own object detection model to my Axis P1375 camera. I've been using the object-detection example as a framework (MobileNetV2 w/ COCO), but I can't get it to work. My model has an input of 320x320 (EfficientDet1) which is reflected on the manifest cpu file. I've also changed the tensor sizes accordingly. However, when I try to run the app I get a few errors in the logs: "setupLarod: Unable to load model: Could not load model: Could not verify and build model from buffer."

Using journalctl -u larod via SSH I get an error saying: "The model is not a valid Flatbuffer buffer. Session 9: Could not verify and build model from buffer.

I'm a bit at a loss as to what else I have to modify to make my model work. Any help would be greatly appreciated!

[Corallo]

Hi Semorai
Could you try to run on your device
larod-client -g /path/to/model -c tflite-cpu
This way, we can make sure that the issue is with the model and not with the code to begin with.

If it fails, it means your model is for some reason non-compatible.
It is hard for us to say what is wrong with the model without looking at it.
You could try to load it on netron.app and compare it with the SSD mobilenet that we have in our example and see if you notice something odd. (Loops, quantize and de-quantize operations, custom operations etc).

[Corallo]

Yes, that should be possible, after you create your event like shown in the sent_event.c it will appear in the menu after you start the ACAP.

[semroai]

@Corallo, great. Is there something in particular that needs to be in the code for it to show up on as a possible condition? I've quickly loaded up the send_event application, but it doesn't show up as a possible condition after running it. Perhaps it needs something else to be initialized or triggered.

[semroai]

Hi @Corallo, I'm using this example to set up and declare a stateful event to the camera: https://axiscommunications.github.io/acap-documentation/docs/api/src/api/axevent/html/ax_event_property_state_declaration_example_8c-example.html

However, I can't seem to figure out how to toggle the state of the key-value set "enabled". The idea would be to toggle that state to true when an object has been detected and to false when nothing is being detected. I'd really appreciate it if you could share how to achieve this in the example. I was unable to get it to work using the documentation.

[Corallo]

Hello @semroai

We are currently working on the send_event example to make show how the event can be crated and be visible from the event system in th GUI.

We might take a bit longer to publish a fix, in the meantime you can try this:

Instead of setting up the event like currently showned in the example L129-146
Try the following:

    ax_event_key_value_set_add_key_values(
        key_value_set, &error, "topic0", "tnsaxis", "CameraApplicationPlatform",
        AX_VALUE_TYPE_STRING, "topic1", "tnsaxis", "Event_example", AX_VALUE_TYPE_STRING, NULL);

    ax_event_key_value_set_add_key_value(key_value_set, "topic2", "tnsaxis", "event",
                                         AX_VALUE_TYPE_STRING, NULL);
    ax_event_key_value_set_add_nice_names(key_value_set, "topic2", "tnsaxis", "event",
                                          "Event_example:Event", NULL);
    ax_event_key_value_set_add_key_value(key_value_set, "value", NULL, &start_value,
                                         AX_VALUE_TYPE_DOUBLE, NULL);
    ax_event_key_value_set_mark_as_data(key_value_set, "value", NULL, NULL);

then, to trigger the event:

L56
change the way it is done in the example with:

g_autoptr(GError) err = NULL;
   ax_event_key_value_set_add_key_values(key_value_set, &err, "value", NULL, &app_data->value,
                                          AX_VALUE_TYPE_DOUBLE, NULL);

Try first building the send_event application, with these changes.
You should see that after starting the applicaiton, a new condition will appear in the GUI:

You can for instance try to make the camera LED blink when this event is triggered.
Note: This event carries a double value, you can still use it just as a trigger in the GUI regardless of the value that it carries.

If this works, you can then try to port the send_event to your object detection example.

[semroai]

Hi @Corallo, thanks a lot for the information. I've tried to implement your changes, but the application won't start. It immediately stops running. However, I've managed to get the condition to appear using the example I mentioned in my previous comment. I just can't figure out how to toggle the "enabled" condition (key-value set) the right way. What is the correct procedure for changing a value in a set?

Example

My process would be something like this:

Set up and declare event with default value FALSE.

• Object is detected:
  - Change event value to TRUE
• Object is not detected:
  - Change event value to FALSE

Edit: I got the modifications to work on the example. Sorry about that.

[Corallo]

Hi @semroai
Also in the case of that other example, it should be enough to use

gboolean state = TRUE;
ax_event_key_value_set_add_key_value(set, "enabled", NULL, &state,
                                         AX_VALUE_TYPE_BOOL, NULL);

To send an event

[semroai]

Thanks a lot @Corallo, I got it to work.