Guidance on Replacing YOLOv8 Backbone with MobileNetV3 (Using Google Colab)

[eternalsummer01]

Hi everyone,

I'm planning to modify the YOLOv8 architecture to use MobileNetV3 as the backbone instead of the default one. I searched through previous issues and discussions but couldn't find much detailed information about this specific change.

So far, I understand that I’ll need to modify the .yaml configuration file in the repo to define the new backbone. I’ll be working on this in a Google Colab environment.

Could someone please clarify:
What are the exact steps to properly replace the backbone?
Do I need to write a custom MobileNetV3 module in PyTorch, or is there an existing one compatible with YOLOv8’s structure?
Are there any additional changes needed in the training or model loading scripts?
Is there any official or community guide/tutorial available on this?

[UltralyticsAssistant]

👋 Hello @eternalsummer01, thank you for your interest in Ultralytics 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧. Prefer in-depth discussions? Check out Discourse. Or dive into threads on our Subreddit to share knowledge with the community.

Upgrade

Upgrade to the latest ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8 to verify your issue is not already resolved in the latest version:

pip install -U ultralytics

Environments

YOLO may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

This is an automated response. An Ultralytics engineer will review your request and assist you soon! 😊

[glenn-jocher]

Hi @eternalsummer01! For replacing the YOLOv8 backbone with MobileNetV3, you'll need to create a custom model configuration. Here's the approach:

You'll need to define a custom YAML configuration that specifies MobileNetV3 as the backbone - this typically involves implementing the MobileNetV3 architecture in the model definition since it's not natively supported. The backbone replacement requires modifying both the model architecture and ensuring proper feature map outputs that match YOLO's neck expectations.

For the implementation, you can reference MobileNetV3 from torchvision and adapt it to work with YOLO's architecture, ensuring the output channels and feature map sizes align properly with the neck and head components.

I'd recommend checking the custom model training guide and looking at existing model configurations in the ultralytics/cfg/models/ directory for reference on how architectures are defined. You may also want to explore the model customization documentation for insights into extending the framework.

This is a fairly advanced modification that requires understanding both architectures - feel free to share your progress or specific implementation challenges as you work through it!

[Y-T-G]

You can find MobileNet configs here:

https://github.com/Y-T-G/community/tree/main/cfg/detect

[eternalsummer01]

Hi, I’m working with your mobilenet_v3_small-fpn.yaml model config from the community repo. I’d like to run inference on the COCO128 dataset or a similar multi-class dataset.

Could you please clarify the full steps to:

Adjust or configure the YAML file for multi-class detection (e.g., COCO128 with 80 classes) instead of nc: 1.

Load the model properly in Google Colab.

Run inference on test images and interpret the output classes.

[glenn-jocher]

Hi @eternalsummer01! To use the MobileNet config for multi-class detection with COCO128 (80 classes):

1. Modify the YAML: Change nc: 1 to nc: 80 in the mobilenet_v3_small-fpn.yaml file to match COCO's 80 classes.

2. Load and train the model:

from ultralytics import YOLO

# Load model with custom config
model = YOLO('path/to/mobilenet_v3_small-fpn.yaml')
# Train on COCO128
model.train(data='coco128.yaml', epochs=100)

3. Run inference:

# After training, or load pre-trained weights
results = model('path/to/image.jpg')
# View results
results[0].show()
# Access class names and predictions
print(results[0].names)  # Class names dictionary

The model will automatically handle the 80 COCO classes once you've updated the nc parameter and trained on the COCO128 dataset. The output will include bounding boxes, confidence scores, and class predictions for all 80 COCO classes.

[Y-T-G]

@eternalsummer01 The model isn't trained. It's a config that you use to train a new model. So running inference on COCO128 wouldn't return any detections if you don't train it first.