Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix OOM in pearson metric #380

Merged
merged 35 commits into from
Jul 28, 2021
Merged

Fix OOM in pearson metric #380

merged 35 commits into from
Jul 28, 2021

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Jul 16, 2021

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Fixes #373
Instead of storing all predictions and target in buffer for pearson correlation, we instead now iteratively update just 6 statistics which will save alot of memory.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@SkafteNicki SkafteNicki added the enhancement New feature or request label Jul 16, 2021
@pep8speaks
Copy link

pep8speaks commented Jul 16, 2021

Hello @SkafteNicki! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-07-28 14:47:48 UTC

Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work !

@Borda Borda added the bug / fix Something isn't working label Jul 16, 2021
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mind check the typing added?

Comment on lines 25 to 30
mx: Tensor,
my: Tensor,
vx: Tensor,
vy: Tensor,
cxy: Tensor,
n: Tensor,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a meaning to these vars?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried updating the variable names :]

@mergify mergify bot removed the has conflicts label Jul 17, 2021
@codecov
Copy link

codecov bot commented Jul 17, 2021

Codecov Report

Merging #380 (da43c3a) into master (7151833) will increase coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #380      +/-   ##
==========================================
+ Coverage   96.34%   96.39%   +0.05%     
==========================================
  Files         120      120              
  Lines        3771     3801      +30     
==========================================
+ Hits         3633     3664      +31     
+ Misses        138      137       -1     
Flag Coverage Δ
Linux 76.45% <74.54%> (-0.16%) ⬇️
Windows 76.45% <74.54%> (-0.16%) ⬇️
cpu 96.39% <100.00%> (+0.05%) ⬆️
macOS 96.39% <100.00%> (+0.05%) ⬆️
pytest 96.39% <100.00%> (+0.05%) ⬆️
python3.6 95.58% <100.00%> (+0.06%) ⬆️
python3.8 96.39% <100.00%> (+0.05%) ⬆️
python3.9 96.29% <100.00%> (+0.05%) ⬆️
torch1.3.1 95.58% <100.00%> (+0.06%) ⬆️
torch1.4.0 95.65% <100.00%> (+0.06%) ⬆️
torch1.9.0 96.29% <100.00%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
torchmetrics/functional/regression/pearson.py 100.00% <100.00%> (+4.00%) ⬆️
torchmetrics/regression/pearson.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7151833...da43c3a. Read the comment docs.

@Borda
Copy link
Member

Borda commented Jul 20, 2021

@SkafteNicki seems that most of the tests are still failing...

@Borda Borda enabled auto-merge (squash) July 24, 2021 16:33
@mergify mergify bot added the ready label Jul 28, 2021
@Borda Borda disabled auto-merge July 28, 2021 16:06
@Borda Borda merged commit e00e3ab into master Jul 28, 2021
@Borda Borda deleted the pearson_update branch July 28, 2021 16:07
@Borda Borda added this to the v0.5 milestone Aug 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug / fix Something isn't working enhancement New feature or request ready
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PearsonCorrcoef Causing OOM
4 participants