Using this model for object detection

[matiasV-TYN]

Hi,

I'm currently working on deploying a YOLO variant on the MAX 78002 for object detection and saw your issue on the ADI MAX GitHub which caught my attention.
So far I have tried downscaled variants of YOLO v1 but the results are not great. I've been trying to replicate the results from 2 papers based on the MAX 78000 which implemented YOLOv1 variants (BED project and TinyissimoYOLO), but I am nowhere near close to the reported results in said papers. Do you think the backbone from your model with a YOLO fully connected layer at the end for the output boxes mapping could work? I'm not sure what decisions and cuts where made with the application of classification in mind.

Thanks!

P.S.: I don't know if you found this out already but the ai8x-training repo has a yaml template generator. I haven't had the best luck with it but it's a start I guess.

[SanderGi]

This sounds like a cool project!

I absolutely think there’s a good chance the YOLOv8p backbone could work well for object detection. It is a mostly faithful port of YOLOv8n with the same layers and cross stage partials that make it really good at feature extraction compared to earlier YOLO iterations. Two architecture modifications were necessary to support MAX78000: simulate stride 2 with pooling and hardcode a resolution so adaptive pooling can be handled with a fixed average pooling instead. To reduce the size, I simply scaled down the channels in each layer and set the number of bottlenecks in each C2f block to 1 where YOLOv8n has 2 bottlenecks in some layers. For classification, I did not observe an advantage to having multiple bottlenecks and I was able to significantly downsize the classification head to keep the backbone quite large. For your object detection use case, you might find multiple bottlenecks helpful or find that reducing the backbone size in favor of a larger head is advantageous. You should be able to change the layerx_channel variables to experiment with this (which is what I did to find good sizes for classification). For optimal object detection, you’ll need to replace the classification head with an adaptation of the object detection head for the MAX. For classification, I found that what’s more important than the specific layer sizes and architecture is the training strategy and data quality/augmentation. The larger pretrained YOLOv8 checkpoints from ultralytics have been trained on more data with clever augmentation techniques (such as mosaic augmentation) that mean they generalize better. If you first finetune one of these larger models (or use the pretrained checkpoints directly if they work well for your task) and then distill to the YOLOv8p, you’ll see an improvement in performance. This will likely also apply to object detection.

Best of luck with your project and keep me posted on how it goes!

Thanks for the tip on the yaml template generator.

[matiasV-TYN]

This sounds like a cool project!

I absolutely think there’s a good chance the YOLOv8p backbone could work well for object detection. It is a mostly faithful port of YOLOv8n with the same layers and cross stage partials that make it really good at feature extraction compared to earlier YOLO iterations. Two architecture modifications were necessary to support MAX78000: simulate stride 2 with pooling and hardcode a resolution so adaptive pooling can be handled with a fixed average pooling instead. To reduce the size, I simply scaled down the channels in each layer and set the number of bottlenecks in each C2f block to 1 where YOLOv8n has 2 bottlenecks in some layers. For classification, I did not observe an advantage to having multiple bottlenecks and I was able to significantly downsize the classification head to keep the backbone quite large. For your object detection use case, you might find multiple bottlenecks helpful or find that reducing the backbone size in favor of a larger head is advantageous. You should be able to change the layerx_channel variables to experiment with this (which is what I did to find good sizes for classification). For optimal object detection, you’ll need to replace the classification head with an adaptation of the object detection head for the MAX. For classification, I found that what’s more important than the specific layer sizes and architecture is the training strategy and data quality/augmentation. The larger pretrained YOLOv8 checkpoints from ultralytics have been trained on more data with clever augmentation techniques (such as mosaic augmentation) that mean they generalize better. If you first finetune one of these larger models (or use the pretrained checkpoints directly if they work well for your task) and then distill to the YOLOv8p, you’ll see an improvement in performance. This will likely also apply to object detection.

Best of luck with your project and keep me posted on how it goes!

Thanks for the tip on the yaml template generator.

Thanks for the reply and all the info!
Will do!