Unexpected Results: Higher cls Weight Leads to Worse Classification – Why?

[Lingpheng]

Hi, I am currently training a model to detect multiple objects. The exact positions of the bounding boxes are not very important to me — I care more about the accurate classification of different objects. So, I tried increasing cls loss weight, while keeping the weights for box and dfl unchanged.

I compared two values for the cls weight: on the left, cls=0.8, and on the right, cls=1.5. All other training settings were the same.

Here are the training and validation results:

And here are the corresponding test results:

From these results, it seems that cls=0.8 performs better — the final classification loss is 0.75 when cls=0.8, compared to 1.5 when cls=1.5.

So I have two questions:

1. Isn’t it supposed to be that a higher cls weight makes the model focus more on improving classification performance? Why does cls=0.8 
    perform better than cls=1.5 in my case?

2. How should I best set the weights for cls, box, and dfl if my main goal is to train a model that emphasizes classification accuracy more 
    than localization?

[UltralyticsAssistant]

👋 Hello @Lingpheng, thank you for sharing your detailed observations and results with Ultralytics 🚀! For users interested in optimizing loss weights, we recommend reviewing the Docs, including Python and CLI usage examples. These resources cover many of the most common questions about custom training and model configuration.

If this is a 🐛 Bug Report, please include a minimum reproducible example to help us investigate further.

For custom training ❓ questions like yours, providing as much context as possible—including dataset samples, relevant config files, and complete training logs—will help our team assist you effectively. Also, please review our Tips for Best Training Results to ensure you're following best practices.

Join the Ultralytics community where it suits you best! For real-time chat, visit Discord 🎧. For thoughtful discussions, check out Discourse. Or connect with others on our Subreddit.

Upgrade

To make sure you are working with the latest features and fixes, upgrade to the newest ultralytics package and all requirements in a Python>=3.8 environment with PyTorch>=1.8:

pip install -U ultralytics

Environments

YOLO can be run in any of these verified environments (with dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM: GCP Quickstart Guide
• Amazon Deep Learning AMI: AWS Quickstart Guide
• Docker Image: Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are passing. CI tests verify correct operation of all YOLO Modes and Tasks across macOS, Windows, and Ubuntu every 24 hours and on every commit.

This is an automated response 🤖. An Ultralytics engineer will review your discussion and assist you soon!

[glenn-jocher]

Higher classification weights don't always improve classification performance because loss components work together - over-weighting cls can actually hurt localization quality, which in turn degrades classification accuracy since the model needs good bounding box predictions to classify objects effectively.

The issue is that when cls=1.5 forces the model to over-prioritize classification loss, it may sacrifice localization quality, leading to worse overall performance. A balanced approach typically works better.

For classification-focused training, try gradually increasing cls weight (start with 1.0-1.2) while slightly reducing box weight, and monitor validation metrics. You might also consider using a smaller model size to naturally emphasize classification over precise localization, or experiment with confidence thresholds during inference to filter predictions.

[Lingpheng]

Thanks! I see—it's important to balance the three parameters. I'll try using the n model and start experimenting with cls=1.0. Appreciate the help!

[glenn-jocher]

You're welcome! That sounds like a solid plan - the n model is perfect for testing and cls=1.0 is a great starting point. Good luck with your experiments, and feel free to share your results!

[Y-T-G]

The loss would be higher because the weights are multiplied with the actual loss value. That's what you see in the log. Higher weights equals higher value of loss. They are not comparable without normalizing first with respect to the weights

[Lingpheng]

Thanks for your answer — you're right!

I looked into the code and found the v8DetectionLoss class in loss.py, where I saw the following lines:

loss[0] *= self.hyp.box  # box gain  
loss[1] *= self.hyp.cls  # cls gain  
loss[2] *= self.hyp.dfl  # dfl gain

So the final loss values are actually the raw computed losses multiplied by the weights we set for box, cls, and dfl during training. A larger cls weight leads to a higher classification loss value, which in turn results in a larger gradient for classification during backpropagation.

That means it's not meaningful to directly compare the loss values in the training logs of different models, since those values have already been scaled by the respective hyperparameters.

This makes things much clearer now!

[glenn-jocher]

Exactly right! You've identified the key insight - the logged loss values are the weighted losses, not the raw unscaled losses. This is why comparing loss values across different weight configurations can be misleading.

For meaningful comparison, you'd want to look at the actual performance metrics like mAP, precision, and recall on your validation set, which reflect the model's true classification and detection abilities rather than the scaled loss components used for gradient computation.

[Lingpheng]

So would box=7.5, cls=0.5, dfl=1.5 and box=15, cls=1, dfl=3 behave the same, since the relative weights are the same? As long as the ratios between the losses stay constant, the optimization direction shouldn't change, right?

[glenn-jocher]

Theoretically yes, those two configurations should behave very similarly since the ratios are identical (15:1:3). The optimization direction should be essentially the same because the relative importance between box, cls, and dfl losses remains constant.

However, there might be subtle differences due to the absolute gradient magnitudes being 2x larger in the second case, which could interact slightly differently with your learning rate and optimizer dynamics. In practice, the first configuration (smaller absolute values) is generally preferred to avoid potential numerical issues.