Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for SQ & RQ as well as per-class metrics #2381

Merged
merged 40 commits into from
Mar 25, 2024
Merged

Add support for SQ & RQ as well as per-class metrics #2381

merged 40 commits into from
Mar 25, 2024

Conversation

ChristophReich1996
Copy link
Contributor

@ChristophReich1996 ChristophReich1996 commented Feb 14, 2024

What does this PR do?

Fixes #2380

This pull request adds support for also computing the Segmentation and Recognition Quality alongside the Panoptic Quality. I also added a flag to return the per-class metrics. Additional tests probably need to be added, right?

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--2381.org.readthedocs.build/en/2381/

Copy link

codecov bot commented Feb 14, 2024

Codecov Report

Merging #2381 (63419ce) into master (78cbbf8) will decrease coverage by 33%.
Report is 1 commits behind head on master.
The diff coverage is 64%.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #2381     +/-   ##
========================================
- Coverage      69%     36%    -33%     
========================================
  Files         307     307             
  Lines       17357   17396     +39     
========================================
- Hits        11957    6301   -5656     
- Misses       5400   11095   +5695     

@SkafteNicki
Copy link
Member

@ChristophReich1996 thanks for sending the pull request!
Overall I think it looks good. I have a suggestions:

  • Should we rename per_class to return_per_class?
  • I would prefer if we try to keep one output type. So what I suggest is
    • If return_sq_and_rq=True return a tensor of length 3 e.g. torch.tensor([pq, sq, rq])
    • If return_per_class=True return a tensor of length classes but along the first dimension such that it is a [1, C] tensor
    • If both are True return a tensor with shape [3, C]

What do you think?

@ChristophReich1996
Copy link
Contributor Author

@SkafteNicki Yeah having the same return time is useful and renaming the per_class flag is as well. I updated the code. Let me know what you think!

@Borda Borda added this to the v1.4.0 milestone Feb 19, 2024
@Borda
Copy link
Member

Borda commented Feb 27, 2024

@SkafteNicki how is it going here, anything missing?

@mergify mergify bot added ready and removed ready labels Mar 14, 2024
mergify bot and others added 10 commits March 14, 2024 12:15
…irements (#2445)

build(deps): update fire requirement in /requirements

Updates the requirements on [fire](https://github.com/google/python-fire) to permit the latest version.
- [Release notes](https://github.com/google/python-fire/releases)
- [Commits](google/python-fire@v0.1.0...v0.6.0)

---
updated-dependencies:
- dependency-name: fire
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#2444)

Bumps [pytest-timeout](https://github.com/pytest-dev/pytest-timeout) from 2.2.0 to 2.3.1.
- [Commits](pytest-dev/pytest-timeout@2.2.0...2.3.1)

---
updated-dependencies:
- dependency-name: pytest-timeout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@Borda Borda force-pushed the master branch 2 times, most recently from 306bb3d to 4ed43e6 Compare March 14, 2024 12:36
@mergify mergify bot added ready and removed ready labels Mar 19, 2024
@mergify mergify bot added the ready label Mar 22, 2024
@Borda Borda merged commit 6dcb61c into Lightning-AI:master Mar 25, 2024
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Support for SQ and RQ in Panoptic Quality
4 participants