Difference between functional and not?

[shatz01]

I seem to not understand why there is a torchmetrics.Accuracy and torchmetrics.functional.accuracy, can someone help me understand?

I would especially like to understand the difference in this particular example: when preds is a matrix and target is a vector, only torchmetrics.functional.accuracy works:

import torch
import torchmetrics

# simulate a classification problem
preds = torch.randn(10, 5).softmax(dim=-1)
target = torch.randint(5, (10,))

acc = torchmetrics.functional.accuracy(preds, target)

... which works fine, but if instead we finished with:
acc = torchmetrics.Accuracy(preds, target)
... we would get a RuntimeError: Boolean value of Tensor with more than one value is ambiguous

Most likely this is the intended behavior (otherwise I can fix it)... But could someone explain to me what the intention is?

[shatz01]

... bump

[SkafteNicki]

Hi @smartdanny, sorry for not getting back to you before now.
The difference between the functional and modular interface is that the modular interface is intended for accumulating over multiple batches, whereas the functional interface is meant for only evaluating a single batch (similar to sklearn)
So functional interface would be:

acc = torchmetrics.functional.accuracy(preds, target)  # this is the accuracy of this batch of data

whereas modular would normally be used as:

acc_metric = torchmetrics.Accuracy(num_classes=5)
for batch in dataloader:
    data, target = batch
    preds = model(data)
    acc_metric(preds, target)
acc = acc_metric.compute() # this is the accuracy over the hole dataloader

[shatz01]

ah that makes sense, thanks for explaining so clearly :)