Incorrect IntersectionOverUnion calculations when predictions or targets contain empty boxes

[yurithefury]

🐛 Bug

IntersectionOverUnion appears to mishandle cases where targets or predictions contain empty bounding boxes

To Reproduce

targets = [
    {
       "boxes": torch.FloatTensor([[0.4, 0.4, 0.5, 0.5], [0.6, 0.6, 0.7, 0.7]]),
        "labels": torch.LongTensor([1, 2]),
    },
    {
        "boxes": torch.FloatTensor([[0.1, 0.1, 0.2, 0.2], [0.3, 0.3, 0.4, 0.4]]),
        "labels": torch.LongTensor([1, 2]),
    },
]

preds = [
    {
        "boxes": torch.FloatTensor([[0.1, 0.1, 0.2, 0.2], [0.3, 0.3, 0.4, 0.4]]),
        "labels": torch.LongTensor([1, 2]),
        "scores": torch.FloatTensor([0.9, 0.8]),
    },
    {
        "boxes": torch.FloatTensor([[0.1, 0.1, 0.2, 0.2], [0.3, 0.3, 0.4, 0.4]]),
        "labels": torch.LongTensor([1, 2]),
        "scores": torch.FloatTensor([0.9, 0.8]),
    },
]
iou = IntersectionOverUnion()
iou.update(preds, targets)
result = iou.compute()

The code above correctly returns IoU of 0.5, but if the first targets item contains empty bounding boxes, the result unexpectedly changes to 1:

targets = [
    {
        "boxes": torch.empty(size=(0, 4), dtype=torch.float32),
        "labels": torch.tensor([], dtype=torch.long),
    },
    {
        "boxes": torch.FloatTensor([[0.1, 0.1, 0.2, 0.2], [0.3, 0.3, 0.4, 0.4]]),
        "labels": torch.LongTensor([1, 2]),
    },
]

preds = [
    {
        "boxes": torch.FloatTensor([[0.1, 0.1, 0.2, 0.2], [0.3, 0.3, 0.4, 0.4]]),
        "labels": torch.LongTensor([1, 2]),
        "scores": torch.FloatTensor([0.9, 0.8]),
    },
    {
        "boxes": torch.FloatTensor([[0.1, 0.1, 0.2, 0.2], [0.3, 0.3, 0.4, 0.4]]),
        "labels": torch.LongTensor([1, 2]),
        "scores": torch.FloatTensor([0.9, 0.8]),
    },
]
iou = IntersectionOverUnion()
iou.update(preds, targets)
result = iou.compute()

This seems incorrect, as the model made inaccurate predictions in both cases. The same issue arises when targets and predictions are swapped.

Expected behavior

The IoU value should be 0.5 when predictions or targets contain empty boxes.

Environment

• torchmetrics=1.5.1
• python=3.12, pytorch=2.5.0
• Windows 11

[github-actions]

Hi! thanks for your contribution!, great first issue!