Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metrics] Class reduction similar to sklearn #3322

Merged
merged 30 commits into from
Sep 15, 2020

Conversation

SkafteNicki
Copy link
Member

What does this PR do?

Fixes #3035
Changes the reduction for accuracy, precision, recall, fbeta, f1 metrics to be similar to sklearn: default will be 'micro' (global measure) with the option of 'macro' (our old standard, unweighted mean of individual scores), 'weighted' (weigted mean of individual scores) and 'none' (return score for each class).

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@pep8speaks
Copy link

pep8speaks commented Sep 2, 2020

Hello @SkafteNicki! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-09-15 09:45:22 UTC

@mergify mergify bot requested a review from a team September 2, 2020 11:39
@mergify
Copy link
Contributor

mergify bot commented Sep 3, 2020

This pull request is now in conflict... :(

@codecov
Copy link

codecov bot commented Sep 4, 2020

Codecov Report

Merging #3322 into master will increase coverage by 0%.
The diff coverage is 98%.

@@          Coverage Diff           @@
##           master   #3322   +/-   ##
======================================
  Coverage      90%     90%           
======================================
  Files         107     107           
  Lines        8149    8082   -67     
======================================
- Hits         7332    7307   -25     
+ Misses        817     775   -42     

@mergify mergify bot requested a review from a team September 4, 2020 15:56
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pytorch_lightning/metrics/functional/reduction.py Outdated Show resolved Hide resolved
@mergify mergify bot requested a review from a team September 4, 2020 17:05
@mergify mergify bot requested a review from a team September 5, 2020 18:07
pytorch_lightning/metrics/classification.py Outdated Show resolved Hide resolved
pytorch_lightning/metrics/functional/reduction.py Outdated Show resolved Hide resolved
@Borda Borda added the ready PRs ready to be merged label Sep 8, 2020
@mergify
Copy link
Contributor

mergify bot commented Sep 11, 2020

This pull request is now in conflict... :(

@Borda
Copy link
Member

Borda commented Sep 14, 2020

just extended the timeout, but not the tests are running super long, about month or two ago we hand testing in 8min, now 30 is not enough... :/

@SkafteNicki
Copy link
Member Author

@Borda it seems on master that since commit 9e33a72 there is always 9 checks that is being cancelled because they take too long time (not that commit seems to influence this in any way)

@mergify
Copy link
Contributor

mergify bot commented Sep 14, 2020

This pull request is now in conflict... :(

@SkafteNicki SkafteNicki merged commit 28af34b into Lightning-AI:master Sep 15, 2020
@SkafteNicki SkafteNicki deleted the metrics/new_class_reduce branch October 8, 2020 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready PRs ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect Precision/Recall/F1 score compared to sklearn
7 participants