interpreting the validation result for YOLOv9

[wangp22]

Hi,
After running a Python script containing model.val() for validating a model on a test dataset, I get metrics that compose of 6 numbers printed at the terminal. Like:

all 1372 1586 0.857 0.733 0.815 0.754
class1 8 9 0.985 1 0.995 0.917
class2 4 5 0.943 0.8 0.817 0.671

and so on...

I would like to think they are the number of images, number of instances, precision, recall, map50, and map95 respectively. But then I'm not sure how to interpret map50 and map95 by the class, since I thought those are the average across all classes.

Also, for the precision and recall, those can vary based on the iou (intersection over union) used to match ground truth with the detection. I think a range of iou values is used from 0.5 to 0.95, in increments of 0.05. So I'm not sure how to interpret the single precision/recall shown. Do they correspond to a particular iou value?

Please clarify my confusion.
Thanks

[UltralyticsAssistant]

👋 Hello @wangp22, thank you for reaching out and using Ultralytics 🚀! We appreciate your thoughtful questions about interpreting validation metrics in YOLOv9. For new users, we recommend checking out the Docs where you’ll find Python and CLI examples, plus detailed explanations of results and metrics.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us investigate further.

If this is a custom training ❓ Question, please include as many details as possible, such as dataset sample images, training logs, and confirm you’re following our Tips for Best Training Results.

Join the Ultralytics community for support and discussion:

• For chat, visit Discord 🎧
• For in-depth posts, head to Discourse
• To share and discover tips, check out our Subreddit

Upgrade

Please ensure you are running the latest ultralytics package with all requirements in a Python>=3.8 environment with PyTorch>=1.8:

pip install -U ultralytics

Environments

You can run YOLO in any of these verified environments (all dependencies, CUDA/CUDNN, Python, PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks across macOS, Windows, and Ubuntu every 24 hours and on every commit.

---

This is an automated response 🦾. An Ultralytics engineer will review your question and assist you soon!

[glenn-jocher]

Your interpretation of the validation output is correct! Let me clarify the specifics:

The six columns are indeed: Images, Instances, Precision (P), Recall (R), mAP50, and mAP50-95.

For your questions:

mAP per class: mAP50 and mAP50-95 can be calculated both per-class and overall. The "all" row shows the mean across all classes, while each class row shows the mAP specifically for that class. This helps you understand which classes perform better or worse.

Precision and Recall IoU: The P and R values shown are calculated across all IoU thresholds from 0.5 to 0.95, averaged at the optimal confidence threshold that maximizes the F1 score. They represent the overall precision and recall performance rather than values at a single IoU threshold.

For a deeper understanding of these metrics and how they're calculated, check out our Performance Metrics Deep Dive guide which explains each metric in detail, including their significance and interpretation.

[wangp22]

Thanks for the response.

In the documentation page you shared, it mentions that:

_Average Precision (AP): AP computes the area under the precision-recall curve, providing a single value that encapsulates the model's precision and recall performance.

Mean Average Precision (mAP): mAP extends the concept of AP by calculating the average AP values across multiple object classes._

So it seems that each class rows shows the AP for that class, not the MAP. Because according to the above definition, MAP is the average AP values across multiple object classes. And that was the source of my confusion.

Your explanation on per-class P and R really helps a lot. Since IoU threshold can affect P and R, I have a follow up question on a related topic, specifically regarding the PR-curve generated at the end of the validation (PR_curve.png). The way I understand it, the PR-curve is generated by varying the conf parameter from 0 to 1. When performing validation model.val(), the default for conf is 0.01. Setting a different value for conf is allowed but I was told it's not a good idea. That's because the x-values for PR-curve are always set from 0 to 100 in 1000 steps, so if one set a 'conf' threshold, then that might cause the plot to be skewed. Also, is the P and R values for the PR-curve also calculated based on all IoU threshold from 0.5 to 0.95 and taking the average?

Thanks

[glenn-jocher]

You're absolutely correct on the terminology! Each class row shows AP (Average Precision) for that specific class, while the "all" row shows mAP (Mean Average Precision), which is the mean of all individual class AP values. Thank you for catching that distinction.

Regarding the PR curve generation, let me clarify the process:

The PR curve is generated independently of the conf parameter you set during validation. The validation process uses all detections (regardless of your conf setting) and internally varies confidence thresholds from 0 to 1 to plot the precision-recall relationship. Your conf parameter only affects which detections are counted as valid for the final metrics, not the curve generation itself.

The PR curve is typically calculated at IoU=0.5 (corresponding to mAP@0.5), not averaged across all IoU thresholds from 0.5 to 0.95. This gives you the precision-recall trade-off specifically at the 0.5 IoU threshold, which is the most commonly used standard for object detection evaluation.

So changing your validation conf parameter won't skew the PR curve - the curve generation process is separate and uses its own threshold sweeping mechanism internally.

[wangp22]

Thanks for the response. I would think the PR curve is generated independently of the conf parameter set during validation. However, I have multiple validation runs where the only difference is the conf setting and the PR_curve.png look different. I'm really curious why.
Thanks

[glenn-jocher]

You're absolutely right to question this - I need to correct my previous explanation. The conf parameter does affect the PR curve generation, which explains why you're seeing different curves with different confidence settings.

The validation documentation states that conf is "used during validation to compute precision-recall curves." When you set a higher confidence threshold, it filters out lower-confidence detections before the PR curve calculation, which changes the pool of detections used to generate the curve.

This means the PR curve you see reflects the precision-recall relationship for detections above your specified confidence threshold, not for all possible detections. A higher conf setting will typically result in higher precision values but potentially lower recall, shifting the curve accordingly.

So your observation is correct - different conf values during validation will indeed produce different PR curves because the confidence threshold acts as a pre-filter for the detections that go into the curve calculation.