Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved the performance of RecallAtFixedPrecision for large batch sizes #2042

Merged
merged 6 commits into from
Sep 1, 2023

Conversation

GlavitsBalazs
Copy link
Contributor

@GlavitsBalazs GlavitsBalazs commented Sep 1, 2023

What does this PR do?

Addresses #2041.

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--2042.org.readthedocs.build/en/2042/

@Borda
Copy link
Member

Borda commented Sep 1, 2023

@GlavitsBalazs nice, do you have a benchmark just to give an idea about the speedup? Also pls add entry in changelog

@codecov
Copy link

codecov bot commented Sep 1, 2023

Codecov Report

Merging #2042 (2f1b332) into master (23c4e62) will decrease coverage by 38%.
The diff coverage is 95%.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #2042     +/-   ##
========================================
- Coverage      87%     49%    -38%     
========================================
  Files         279     279             
  Lines       15814   15828     +14     
========================================
- Hits        13718    7743   -5975     
- Misses       2096    8085   +5989     

@GlavitsBalazs
Copy link
Contributor Author

@GlavitsBalazs nice, do you have a benchmark just to give an idea about the speedup? Also pls add entry in changelog

Ok. I've added the changelog entry.

My issue submission contains some basic measurements illustrating the speedup. Please let me know if these aren't enough. I'll create more benchmarks if necessary.

@SkafteNicki
Copy link
Member

@GlavitsBalazs nice, do you have a benchmark just to give an idea about the speedup? Also pls add entry in changelog

Ok. I've added the changelog entry.

My issue submission contains some basic measurements illustrating the speedup. Please let me know if these aren't enough. I'll create more benchmarks if necessary.

@GlavitsBalazs I think the tests you done are fine, I am reposting here for future reference:

Batch Size Before PR After PR
100 ~9 ~8
1000 ~20 ~9
2000 ~30 ~10
5000 ~70 ~14
10000 ~124 ~19
100000 ~1200 ~110

GlavitsBalazs and others added 2 commits September 1, 2023 12:22
Fixes mypy return-value error.

Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
@Borda Borda enabled auto-merge (squash) September 1, 2023 13:49
@mergify mergify bot added the ready label Sep 1, 2023
@Borda Borda merged commit 694399a into Lightning-AI:master Sep 1, 2023
@GlavitsBalazs
Copy link
Contributor Author

Closes #2041

Borda pushed a commit that referenced this pull request Sep 9, 2023
…zes (#2042)

Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
(cherry picked from commit 694399a)
Borda pushed a commit that referenced this pull request Sep 11, 2023
…zes (#2042)

Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
(cherry picked from commit 694399a)
matsumotosan pushed a commit to matsumotosan/metrics that referenced this pull request Sep 19, 2023
…zes (Lightning-AI#2042)

Co-authored-by: Nicki Skafte Detlefsen <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants