Accuracy no longer works with logits

[shenberg]

🐛 Bug

Thresholds are now limited to the range (0,1), so I cannot set a threshold that makes sense for logits (e.g. 0)

To Reproduce

Steps to reproduce the behavior:

Accuracy(threshold=0.)

Code sample

import torchmetrics
torchmetrics.Accuracy(threshold=0.0)

Expected behavior

A threshold of 0 should be valid for binary and multilabel classification.

Additional context

It makes sense to accept logits into accuracy, as often the training step has the logit->probability and probability->loss calculation in one layer for numerical reasons (e.g. nn.BCEWithLogitsLoss), and forcing recalculation of the sigmoid is unnecessary, as it's monotonic.

The documentation for Accuracy states:

threshold (float) – Threshold for transforming probability or logit predictions to binary (0,1) predictions, in the case of binary or multi-label inputs. Default value of 0.5 corresponds to input being probabilities.

[github-actions]

Hi! thanks for your contribution!, great first issue!

[Borda]

@shenberg what version are you using? this shall be fixed on master now #351
feel free to reopen if you still have this issue... 🐰

[shenberg]

@Borda, unfortunately, Accuracy has its own check, see accuracy.py, lines 208-209 on current commit - the PR you linked to fixed the general case in stats_scores.py but left this specific class non-functional as it has a bespoke test.

[cnut1648]

Agreed, can confirm that in the latest version 0.4.1 the threshold can't be set to 0, which makes it unable to handle output logit of binary classification model (where there is no sigmoid)