Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential Unsupported version of DICE. #2323

Closed
aymuos15 opened this issue Jan 24, 2024 · 5 comments · Fixed by #2725
Closed

Potential Unsupported version of DICE. #2323

aymuos15 opened this issue Jan 24, 2024 · 5 comments · Fixed by #2725
Assignees
Labels
bug / fix Something isn't working v1.3.x
Milestone

Comments

@aymuos15
Copy link

aymuos15 commented Jan 24, 2024

Bug description

https://lightning.ai/docs/torchmetrics/stable/classification/dice.html

Here, under average parameter of the dice score, there is the option of 'weighted' mentioned. But that triggers an error while using it.

How to reproduce the bug

!pip install torchmetrics
from torch import tensor
from torchmetrics.classification import Dice

preds  = tensor([2, 0, 2, 1])
target = tensor([1, 1, 2, 0])
dice = Dice(average='weighted')
dice(preds, target)

Error messages and logs

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-4-16f87eeaf06b>](https://localhost:8080/#) in <cell line: 6>()
      4 preds  = tensor([2, 0, 2, 1])
      5 target = tensor([1, 1, 2, 0])
----> 6 dice = Dice(average='weighted')
      7 dice(preds, target)

[/usr/local/lib/python3.10/dist-packages/torchmetrics/classification/dice.py](https://localhost:8080/#) in __init__(self, zero_division, num_classes, threshold, average, mdmc_average, ignore_index, top_k, multiclass, **kwargs)
    158         allowed_average = ("micro", "macro", "samples", "none", None)
    159         if average not in allowed_average:
--> 160             raise ValueError(f"The `average` has to be one of {allowed_average}, got {average}.")
    161 
    162         _reduce_options = (AverageMethod.WEIGHTED, AverageMethod.NONE, None)

ValueError: The `average` has to be one of ('micro', 'macro', 'samples', 'none', None), got weighted.

Environment

Google Colab -> (Hence the pip install)

More info

https://colab.research.google.com/drive/1GuBr6kI0ypj0sUDaqGQ8dN0Td-sbYWGI?usp=sharing

@awaelchli awaelchli transferred this issue from Lightning-AI/pytorch-lightning Jan 24, 2024
Copy link

Hi! thanks for your contribution!, great first issue!

@SuperSecureHuman
Copy link

I see the issue

Its in the doc string, but not within the code. I guess weighted is not supported (?)

- ``'weighted'``: Calculate the metric for each class separately, and average the

allowed_average = ("micro", "macro", "samples", "none", None)

Looks like most of the weighted code was removed

2509448

@SkafteNicki SkafteNicki added the bug / fix Something isn't working label Jan 29, 2024
@SkafteNicki SkafteNicki added this to the v1.4.0 milestone Jan 29, 2024
@Borda Borda added the v1.3.x label Jan 29, 2024
@aymuos15
Copy link
Author

Great Find. Thank you. Will keep the issue open for the time-being.

@Borda
Copy link
Member

Borda commented Jan 31, 2024

I think this shall be fixed in the docs, @SkafteNicki thoughts?

@adizhol-str
Copy link

same for average='none', I think:

if average not in ["micro", "macro", "samples"]:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug / fix Something isn't working v1.3.x
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants