Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flakyness in tests related to torch.unique with dim=None #2650

Merged
merged 10 commits into from
Sep 2, 2024

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Jul 23, 2024

What does this PR do?

One of test seems to be flaky. See for example this CI run:
https://github.com/Lightning-AI/torchmetrics/actions/runs/10037067157/job/27736179139?pr=2636#step:16:839
Trying to debug why this is happening

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--2650.org.readthedocs.build/en/2650/

@SkafteNicki SkafteNicki added the test / CI testing or CI label Jul 23, 2024
@SkafteNicki SkafteNicki added this to the v1.4.x milestone Jul 23, 2024
Copy link

codecov bot commented Jul 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41%. Comparing base (ef22554) to head (c5fedd2).
Report is 107 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (ef22554) and HEAD (c5fedd2). Click for more details.

HEAD has 152 uploads less than BASE
Flag BASE (ef22554) HEAD (c5fedd2)
Windows 8 3
python3.8 8 3
cpu 58 20
torch1.13.1+cpu 9 3
macOS 11 4
python3.10 15 5
torch2.0.1 6 2
torch2.0.1+cpu 9 3
torch1.12.1+cpu 3 1
Linux 39 13
python3.9 27 9
torch1.10.2+cpu 3 1
torch1.11.0+cpu 3 1
python3.11 8 3
torch2.4.0 3 1
torch2.3.1+cpu 6 2
torch2.1.2+cpu 3 1
torch2.2.2+cpu 6 2
torch2.4.0+cu121 3 1
torch2.4.0+cpu 2 1
torch1.13.1 2 1
Additional details and impacted files
@@           Coverage Diff            @@
##           master   #2650     +/-   ##
========================================
- Coverage      69%     41%    -27%     
========================================
  Files         316     316             
  Lines       17745   17745             
========================================
- Hits        12185    7307   -4878     
- Misses       5560   10438   +4878     

@Borda Borda changed the title Trying debug flaky test Trying debug flaky test [wip] Jul 23, 2024
@Borda
Copy link
Member

Borda commented Aug 8, 2024

seems like added retry helped but it would still be worse to know what is happening there...

@SkafteNicki SkafteNicki changed the title Trying debug flaky test [wip] Trying debug flaky test Aug 27, 2024
@SkafteNicki
Copy link
Member Author

@Borda should be good now, found that the issue had to do with how torch.unique deals with multidim tensors

Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remane the PR to fit the fix...

@mergify mergify bot added the ready label Aug 27, 2024
@SkafteNicki SkafteNicki changed the title Trying debug flaky test Fix flakyness in tests related to torch.unique with dim=None Aug 28, 2024
@Borda Borda merged commit 0313c6b into master Sep 2, 2024
61 checks passed
@Borda Borda deleted the tests/flaky_integer_test branch September 2, 2024 09:21
Borda pushed a commit that referenced this pull request Sep 11, 2024
* correct check
* found the problem

---------

Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: jirka <[email protected]>
(cherry picked from commit 0313c6b)
Borda pushed a commit that referenced this pull request Sep 13, 2024
* correct check
* found the problem

---------

Co-authored-by: Jirka Borovec <[email protected]>
Co-authored-by: jirka <[email protected]>
(cherry picked from commit 0313c6b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants