Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update torchmetrics requirement from <=0.7.2,>=0.4.1 to >=0.4.1,<0.9.1 in /requirements #13206

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Jun 2, 2022

Updates the requirements on torchmetrics to permit the latest version.

Release notes

Sourced from torchmetrics's releases.

Faster forward

Highligths

TorchMetrics v0.9 is now out, and it brings significant changes to how the forward method works. This blog post goes over these improvements and how they affect both users of TorchMetrics and users that implement custom metrics. TorchMetrics v0.9 also includes several new metrics and bug fixes.

Blog: TorchMetrics v0.9 — Faster forward

The Story of the Forward Method

Since the beginning of TorchMetrics, Forward has served the dual purpose of calculating the metric on the current batch and accumulating in a global state. Internally, this was achieved by calling update twice: one for each purpose, which meant repeating the same computation. However, for many metrics, calling update twice is unnecessary to achieve both the local batch statistics and accumulating globally because the global statistics are simple reductions of the local batch states.

In v0.9, we have finally implemented a logic that can take advantage of this and will only call update once before making a simple reduction. As you can see in the figure below, this can lead to a single call of forward being 2x faster in v0.9 compared to v0.8 of the same metric.

With the improvements to forward, many metrics have become significantly faster (up to 2x) It should be noted that this change mainly benefits metrics (for example, confusionmatrix) where calling update is expensive.

We went through all existing metrics in TorchMetrics and enabled this feature for all appropriate metrics, which was almost 95% of all metrics. We want to stress that if you are using metrics from TorchMetrics, nothing has changed to the API, and no code changes are necessary.

[0.9.0] - 2022-05-31

Added

  • Added RetrievalPrecisionRecallCurve and RetrievalRecallAtFixedPrecision to retrieval package (#951)
  • Added class property full_state_update that determines forward should call update once or twice (#984,#1033)
  • Added support for nested metric collections (#1003)
  • Added Dice to classification package (#1021)
  • Added support to segmentation type segm as IOU for mean average precision (#822)

Changed

  • Renamed reduction argument to average in Jaccard score and added additional options (#874)

Removed

Fixed

  • Fixed non-empty state dict for a few metrics (#1012)
  • Fixed bug when comparing states while finding compute groups (#1022)
  • Fixed torch.double support in stat score metrics (#1023)
  • Fixed FID calculation for non-equal size real and fake input (#1028)
  • Fixed case where KLDivergence could output Nan (#1030)
  • Fixed deterministic for PyTorch<1.8 (#1035)
  • Fixed default value for mdmc_average in Accuracy (#1036)
  • Fixed missing copy of property when using compute groups in MetricCollection (#1052)

Contributors

... (truncated)

Changelog

Sourced from torchmetrics's changelog.

[0.9.0] - 2022-05-30

Added

  • Added RetrievalPrecisionRecallCurve and RetrievalRecallAtFixedPrecision to retrieval package (#951)
  • Added class property full_state_update that determines forward should call update once or twice ( #984, #1033)
  • Added support for nested metric collections (#1003)
  • Added Dice to classification package (#1021)
  • Added support to segmentation type segm as IOU for mean average precision (#822)

Changed

  • Renamed reduction argument to average in Jaccard score and added additional options (#874)

Removed

Fixed

  • Fixed non-empty state dict for a few metrics (#1012)
  • Fixed bug when comparing states while finding compute groups (#1022)
  • Fixed torch.double support in stat score metrics (#1023)
  • Fixed FID calculation for non-equal size real and fake input (#1028)
  • Fixed case where KLDivergence could output Nan (#1030)
  • Fixed deterministic for PyTorch<1.8 (#1035)
  • Fixed default value for mdmc_average in Accuracy (#1036)
  • Fixed missing copy of property when using compute groups in MetricCollection (#1052)

[0.8.2] - 2022-05-06

Fixed

  • Fixed multi device aggregation in PearsonCorrCoef (#998)
  • Fixed MAP metric when using custom list of thresholds (#995)
  • Fixed compatibility between compute groups in MetricCollection and prefix/postfix arg (#1007)

... (truncated)

Commits

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Updates the requirements on [torchmetrics](https://github.com/PyTorchLightning/metrics) to permit the latest version.
- [Release notes](https://github.com/PyTorchLightning/metrics/releases)
- [Changelog](https://github.com/PyTorchLightning/metrics/blob/master/CHANGELOG.md)
- [Commits](Lightning-AI/torchmetrics@v0.4.1...v0.9.0)

---
updated-dependencies:
- dependency-name: torchmetrics
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added the ci Continuous Integration label Jun 2, 2022
@dependabot dependabot bot requested a review from a team June 2, 2022 01:01
@dependabot @github
Copy link
Contributor Author

dependabot bot commented on behalf of github Jun 13, 2022

Superseded by #13275.

@dependabot dependabot bot closed this Jun 13, 2022
@dependabot dependabot bot deleted the dependabot-pip-requirements-torchmetrics-gte-0.4.1-and-lt-0.9.1 branch June 13, 2022 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Continuous Integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants