What's the "right" way to use metrics?

[olliethomas]

Hi, this isn't really that important, but I figured I'd ask....

In the torchmetrics docs, the way to call a metric is

acc = metric(preds, target)
print(f"Accuracy on batch {i}: {acc}")

but, in the lightning-flash source code for a task

if isinstance(metric, torchmetrics.metric.Metric):
    metric(y_hat, y)
    logs[name] = metric  # log the metric itself if it is of type Metric

is there a performance reason to choose one approach over the other? Is it a matter of taste, of history? I'm a bit confused.

[ethanwharris]

Hi @olliethomas I think the recommended approach is to log the metric object. AFAIK, it enables using the on_epoch flag to have the metric automatically reduced (with compute) at the end of the epoch. The docs here may be helpful: https://torchmetrics.readthedocs.io/en/latest/pages/lightning.html#logging-torchmetrics

Hope that helps 😃

[olliethomas]

cheers for that