Maintain float32 precision at minimum in ResultMetric #18686

awaelchli · 2023-10-02T01:25:29Z

What does this PR do?

This PR ensures that reduction of floating point values happens in float32 precision at least (the user can also choose to operate in higher (double) precision). The PR ensures that if you pass in float16/bfloat16 values, they get upcast to float32 for calculations inside the metric, even if torch.get_default_dtype() is lower precision.

Before submitting

Was this discussed/agreed via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--18686.org.readthedocs.build/en/18686/

cc @Borda @carmocca @Blaizzy

for more information, see https://pre-commit.ci

…fix/float16-logging

github-actions · 2023-10-02T11:29:39Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0)	success	✅

These checks are required after the changes to src/lightning/pytorch/trainer/connectors/logger_connector/result.py, tests/tests_pytorch/core/test_metric_result_integration.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
[pytorch-lightning (GPUs) (testing Lightning	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=177281&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e)	success
[pytorch-lightning (GPUs) (testing PyTorch	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=177281&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f)	success

These checks are required after the changes to src/lightning/pytorch/trainer/connectors/logger_connector/result.py, tests/tests_pytorch/core/test_metric_result_integration.py.

🟢 pytorch_lightning: Benchmarks

Check ID	Status
lightning.Benchmarks	success	✅

These checks are required after the changes to src/lightning/pytorch/trainer/connectors/logger_connector/result.py.

🟢 pytorch_lightning: Docs

Check ID	Status
docs-make (pytorch, doctest)	success	✅
docs-make (pytorch, html)	success	✅

These checks are required after the changes to src/lightning/pytorch/trainer/connectors/logger_connector/result.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/pytorch/trainer/connectors/logger_connector/result.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.11)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.11)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.11)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.11)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.11)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.11)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.11)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.11)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.11)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.11)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.11)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.11)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.11)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.11)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.11)	success	✅

These checks are required after the changes to src/lightning/pytorch/trainer/connectors/logger_connector/result.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

src/lightning/pytorch/trainer/connectors/logger_connector/result.py

codecov · 2023-10-02T12:27:19Z

Codecov Report

Merging #18686 (134fab4) into master (9d9220c) will decrease coverage by 33%.
The diff coverage is 100%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #18686      +/-   ##
==========================================
- Coverage      83%      50%     -33%     
==========================================
  Files         428      420       -8     
  Lines       33546    33405     -141     
==========================================
- Hits        27985    16824   -11161     
- Misses       5561    16581   +11020

awaelchli added 5 commits October 2, 2023 02:41

convert dtype

27bfa23

test

61bfd22

test

6591a54

update

4072ae1

docstring

5508179

github-actions bot added the pl Generic label for PyTorch Lightning package label Oct 2, 2023

[pre-commit.ci] auto fixes from pre-commit.com hooks

a097178

for more information, see https://pre-commit.ci

awaelchli added bug Something isn't working logging Related to the `LoggerConnector` and `log()` labels Oct 2, 2023

awaelchli added this to the 2.1 milestone Oct 2, 2023

awaelchli changed the title ~~Maintain float32 precision or higher in ResultMetric~~ Maintain float32 precision at minimum in ResultMetric Oct 2, 2023

awaelchli added 2 commits October 2, 2023 13:28

changelog

7ca7ade

Merge remote-tracking branch 'origin/bugfix/float16-logging' into bug…

cf4a0fb

…fix/float16-logging

awaelchli marked this pull request as ready for review October 2, 2023 11:28

awaelchli requested review from carmocca, justusschock, Borda and williamFalcon as code owners October 2, 2023 11:28

add remark

2c2615a

carmocca approved these changes Oct 2, 2023

View reviewed changes

src/lightning/pytorch/trainer/connectors/logger_connector/result.py Outdated Show resolved Hide resolved

awaelchli added 2 commits October 2, 2023 14:41

future proof

32d2d5a

consistent

5f7732e

Borda approved these changes Oct 2, 2023

View reviewed changes

mergify bot added the ready PRs ready to be merged label Oct 2, 2023

awaelchli added 2 commits October 2, 2023 11:25

Merge branch 'master' into bugfix/float16-logging

27a9380

Merge branch 'master' into bugfix/float16-logging

134fab4

awaelchli merged commit b69f3c6 into master Oct 3, 2023
88 of 89 checks passed

awaelchli deleted the bugfix/float16-logging branch October 3, 2023 17:19

MF-FOOM mentioned this pull request Nov 10, 2023

cast to float32 or float64 tensor when passing scalar to self.log #18984

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Maintain float32 precision at minimum in ResultMetric #18686

Maintain float32 precision at minimum in ResultMetric #18686

awaelchli commented Oct 2, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Oct 2, 2023 •

edited

Loading

codecov bot commented Oct 2, 2023 •

edited

Loading

Maintain float32 precision at minimum in ResultMetric #18686

Maintain float32 precision at minimum in ResultMetric #18686

Conversation

awaelchli commented Oct 2, 2023 • edited by github-actions bot Loading

What does this PR do?

PR review

github-actions bot commented Oct 2, 2023 • edited Loading

⚡ Required checks status: All passing 🟢

Groups summary

codecov bot commented Oct 2, 2023 • edited Loading

Codecov Report

awaelchli commented Oct 2, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Oct 2, 2023 •

edited

Loading

codecov bot commented Oct 2, 2023 •

edited

Loading