Fix confusion matrix update when no predictions are made

[jbutle55]

This pull request is in response to #8729. All of the below confusion matrices were created using yolov5l.pt and coco128.yaml with val.py.

If an image has labelled objects within it, but the model makes zero predictions on the image, then each of these missed objects is a False Negative and should be recorded as such in the confusion matrix (a prediction of background where the object exists). The way val.py handles when an image has zero predictions results in the confusion matrix not being updated accordingly.

If YOLO were to make zero predictions on every image, simulated by the following temporary change to val.py before this pull request is implemented:

        # Metrics
        for si, pred in enumerate(out):
            pred = torch.Tensor() # Simulates zero predictions made for an image
            labels = targets[targets[:, 0] == si, 1:]
            nl, npr = labels.shape[0], pred.shape[0]  # number of labels, predictions
            path, shape = Path(paths[si]), shapes[si][0]
            correct = torch.zeros(npr, niou, dtype=torch.bool, device=device)  # init
            seen += 1

then the following confusion matrix is generated:

Following this pull request, val.py would generate the following confusion matrix with the same simulated zero predictions:

If the temporary simulation of zero predictions is removed, on the coco128.yaml data then the model produces a confusion matrix as follows with the implemented code changes.

This is compared to the below image, the confusion matrix generated by YOLOv5l without the implemented pull request on coco128:

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Improved handling of edge cases in confusion matrix computation.

📊 Key Changes

• Added a check for None in detections to handle cases with no detections.
• Updated the confusion matrix to increment false negatives when no detections are present.

🎯 Purpose & Impact

• 🎯 Purpose: The change ensures that the confusion matrix accurately represents situations where the model fails to detect any objects (false negatives).
• 🔍 Impact: Users will experience a more accurate performance evaluation of their models, especially when models occasionally detect nothing where there should be detections. This leads to better understanding and optimization of model behavior.

[github-actions]

👋 Hello @jbutle55, thank you for submitting a YOLOv5 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:

• ✅ Verify your PR is up-to-date with upstream/master. If your PR is behind upstream/master an automatic GitHub Actions merge may be attempted by writing /rebase in a new comment, or by running the following code, replacing 'feature' with the name of your local branch:

git remote add upstream https://github.com/ultralytics/yolov5.git
git fetch upstream
# git checkout feature  # <--- replace 'feature' with local branch name
git merge upstream/master
git push -u origin -f

• ✅ Verify all Continuous Integration (CI) checks are passing.
• ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." -Bruce Lee

[glenn-jocher]

@jbutle55 thanks for the PR! This looks good, but can you simply add an if statement to process_batch(), i.e. pass proccess_batch(detections=None, ...) in val.py and then:

if detections is None:
    [code from your new function here]
    return

[jbutle55]

I agree that's a cleaner way to do it. This has been changed now.

[glenn-jocher]

@jbutle55 PR is merged. Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐