Are tensorflow2 based trainings compatible with artpec8?

[egSat]

Hi,
I'm having troubles when loading a custom model, the error says:
ERROR in Inference: Failed to load model model.tflite (Could not load model: Could not build an interpreter of the model)
The model was trained with tensorflow2.

I converted the model to tflite and also tried converting it to google coral, but the issue remains.
So my questions are:
-Is there any issue with using tensorflow2?

• What kind of optimizations we need to make when converting to tensorflow lite (for example input and/or outputs to int8, etc)?
• Is it neccesary the step of converting to google-coral? If a model is compatible with google coral, does it mean that will run on artpec8?

Environment

• Axis device model: P3265
• Axis device firmware version: 10.11.73
• Version: sdk 1.2.1
• docker daemon: 1.2.3

Additional context

Thank you very much

[marbali8]

Hello!

Thank you for getting in touch with us. ARTPEC-8 cameras only support Tensorflow 1. There is a guide on how to train and convert your model from TensorFlow1 to Larod in the tensorflow-to-larod-artpec8 example. Specifically, the model conversion is done in convert_model.py.

[egSat]

Thank you very much. I'm trying to convert custom models trained from object-detection-api (tensorflow 1.15) , but I need to add more parameters to the conversor like --allow_custom_ops in order to work, but when loaded on the camera I get the same error (Could not load model: Could not build an interpreter of the model).

EDIT: actually I only need the flag --allow_custom_ops because of the operator TFLite_Detection_PostProcess. Otherwise the flag is not needed so I suppose shouldn't be an issue.
Others parameters used are: --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --input_shape=1,300,300,3 --input_array normalized_input_image_tensor --output_array TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 --inference_type=QUANTIZED_UINT8 --inference_input_type=QUANTIZED_UINT8 --mean_values=128 --std_dev_values=128 --default_ranges_min=0 --default_ranges_max=6

[Corallo]

Hi, unfortunately custom ops are not supported at the moment, you can't use that model directly on the camera

[egSat]

Hi, thank you very much for the answer.
But I tried the same model as used at the example of object-detector-python (which uses ssd_mobilenet_v2) but trained with custom objects using the object-detector-api of tensorflow 1.15, but I cannot get it to run.

The output of the example is a list of several TFLite_Detection_PostProcess values (line 43 of https://github.com/AxisCommunications/acap-computer-vision-sdk-examples/blob/main/object-detector-python/app/detector.py) so I suppose in the conversion to tflite the custom ops had to be done somehow on the model.
I also tried tf-lite model maker but only works with tensorflow 2 so it didn't work either.

best regards

[Corallo]

Alright, I see, so you are using a standard model (ssd_mobilenetv2) on a custom dataset? That should be working.

Did you implement the model by yourself?

If your model needs --allow_custom_ops in your conversion, it is in general a sign that it won't work on the hardware, because custom ops are not allowed at the moment.
The conversion should look like something like this

We also have this guide
that shows how to train a model from scratch.
In short, to retrain a model in the "simplest" way you should rely on the TensorFlow object detector api to take a pre-build model, and retrain it in your dataset.

Of course, you can also implement your model, but you need to make sure not to use custom ops.

[egSat]

Many thanks for the detailed answer.
For now I'm not able of tranforming the model ssd-mobilenet_v2_coco_2018_03_29 from object detection api without specifying --allow_custom_ops as TFLite_Detection_PostProcess is not integrated into tensorflow lite core.

To convert the model I use the script export_tflite_ssd_graph.py and next tflite_convert script, where fails if not set allow_custom_ops.
I'll search another versions of ssd-mobilenet-v2 models, maybe they have some diferences that make the conversion possible.

[Corallo]

Maybe give it a try with the ssd_mobilenet_v2_quantized_coco from the tensorflow model zoo This should be made to be easily converted to tflite without the custom ops.

[egSat]

Hi. thanks for the suggestion. I've tried this model but the custom operation parameter is still needed.
Anyway, I've managed to run the model, because I've found the issue I had. The container that holds the model, acap_dl-models (from object-detector-api) is not updating in the camera so it was loading an old model from the beginning. I've removed containers, images (by pruning both in my computer and on the camera) but the poblem persists. I notice it because I renamed the model to run so when running the docker-compose on the camera the container said that tried to load the old named model. The only way to get the new model loaded was by formatting the sd card attached in the camera.
Thank to this I can run the model ssd-mobilenet even when I convert the model to tflite with allow_custom_ops enabled.
The thing with new issue is that only I'm able to load the new model by formatting the sd card, I've tried several docker comands to remove everything but nothing works.

[Corallo]

Oh good to know!
So which one was the model that you used as base architecture in the end?
Yes, when you stop your application, you need to make sure to run the docker-compose down command with the -v flag to remove the volume with the old model.

[egSat]

I used ssd_mobilenet_v2_quantized_coco from model zoo of tensorflow 1. Now I'm trying other models, to see how fast is performed the inference while having the best model possible.
very good suggestion about using the docker-compose down commadn, it cleaned the containers with no issues :) i removed them separately before but didn't work