Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ._update_count and ._update_called public #1363

Closed
faroit opened this issue Nov 29, 2022 · 2 comments · Fixed by #1370
Closed

Make ._update_count and ._update_called public #1363

faroit opened this issue Nov 29, 2022 · 2 comments · Fixed by #1370
Labels
enhancement New feature or request

Comments

@faroit
Copy link

faroit commented Nov 29, 2022

🚀 Feature

running compute on a metric that hasn't been updated can lead to unforseen consequences. Currently a warning is raised in

https://github.com/Lightning-AI/metrics/blob/ed2249d1e250dbfc4f2e654b455f43f3d29abfb1/src/torchmetrics/metric.py#L510-L516

but in some cases a user want's to check the metric's state first to avoid running into issues.
Torchmetrics should offer the state as a public property to the users and callbacks.

Motivation

Running compute() before update() from lightning's ...epoch_ends callbacks is difficult to implement with multiple dataloaders and multiple metrics: its difficult to keep track of which metrics have been updated already.

Pitch

torchmetrics already has an api to check this, which is used by lightning internally.

https://github.com/Lightning-AI/metrics/blob/7cab17c680a3970a49d8a765878e28263e6dc302/src/torchmetrics/metric.py#L142-L145

This api is indicated as private, so I would propose to make this public to maintain support.

@faroit faroit added the enhancement New feature or request label Nov 29, 2022
@github-actions
Copy link

Hi! thanks for your contribution!, great first issue!

@faroit
Copy link
Author

faroit commented Dec 6, 2022

@SkafteNicki thanks for doing this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant