Rework of Sklearn Metrics #1327

justusschock · 2020-04-01T15:05:00Z

What does this PR do?

Fixes #1305 and is a rework of #1320 based on #1326
Fixes #1295
Fixes #1296
Fixes #1302

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

pep8speaks · 2020-04-01T15:05:05Z

Hello @justusschock! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-06-09 22:46:37 UTC

pytorch_lightning/metrics/sklearn.py

environment.yml

mergify · 2020-05-05T18:13:13Z

This pull request is now in conflict... :(

Borda · 2020-05-05T19:02:25Z

@williamFalcon why did you close the metrics PR? @justusschock?

mergify · 2020-05-18T19:46:33Z

This pull request is now in conflict... :(

justusschock · 2020-05-25T09:05:17Z

@Borda any idea, why scipy building fails?

Borda · 2020-05-25T22:04:24Z

@justusschock there is also problem with installing the minimal requirements, mind check?
@awaelchli will check the docs, thx

awaelchli · 2020-05-25T22:30:50Z

I'm done fixing the docs errors but can't push, permission denied. Is it a setting on this branch?
Otherwise please merge this branch
https://github.com/awaelchli/pytorch-lightning/tree/new_sklearn_metric

Co-authored-by: Nicki Skafte <[email protected]>

tullie

This is looking really good, just one question about the forward implementations before I accept!

tullie · 2020-06-09T22:23:42Z

pytorch_lightning/metrics/sklearn.py

+            Accuracy Score
+
+        """
+        return super().forward(y_pred=y_pred, y_true=y_true, sample_weight=sample_weight)


Do you need any of these forward functions in this file? It seems like you're implementing what inheritance will already do, i.e. call the superfunction if it's not overriden.

@justusschock ^^

@tullie you're right. But the superfunction takes args and kwargs as arguments. I have explicitly implemented these for documentation and type hinting purposes. But we can also remove them.

But I simply dod not find a better solution to document these arguments

pytorch_lightning/metrics/utils.py

codecov · 2020-06-09T22:31:37Z

Codecov Report

Merging #1327 into master will decrease coverage by 0%.
The diff coverage is 74%.

@@           Coverage Diff           @@
##           master   #1327    +/-   ##
=======================================
- Coverage      86%     86%    -0%     
=======================================
  Files          75      78     +3     
  Lines        4788    4917   +129     
=======================================
+ Hits         4134    4229    +95     
- Misses        654     688    +34

Co-authored-by: Tullie Murrell <[email protected]>

tullie

Cool makes sense with the forward functions

* Create utils.py * Create __init__.py * redo sklearn metrics * add some more metrics * add sklearn metrics * Create __init__.py * redo sklearn metrics * New metric classes (#1326) * Create metrics package * Create metric.py * Create utils.py * Create __init__.py * add tests for metric utils * add docstrings for metrics utils * add function to recursively apply other function to collection * add tests for this function * update test * Update pytorch_lightning/metrics/metric.py Co-Authored-By: Jirka Borovec <[email protected]> * update metric name * remove example docs * fix tests * add metric tests * fix to tensor conversion * fix apply to collection * Update CHANGELOG.md * Update pytorch_lightning/metrics/metric.py Co-Authored-By: Jirka Borovec <[email protected]> * remove tests from init * add missing type annotations * rename utils to convertors * Create metrics.rst * Update index.rst * Update index.rst * Update pytorch_lightning/metrics/convertors.py Co-Authored-By: Jirka Borovec <[email protected]> * Update pytorch_lightning/metrics/convertors.py Co-Authored-By: Jirka Borovec <[email protected]> * add doctest example * rename file and fix imports * added parametrized test * replace lambda with inlined function * rename apply_to_collection to apply_func * Separated class description from init args * Apply suggestions from code review Co-Authored-By: Jirka Borovec <[email protected]> * adjust random values * suppress output when seeding * remove gpu from doctest * Add requested changes and add ellipsis for doctest * forgot to push these files... * add explicit check for dtype to convert to * fix ddp tests * remove explicit ddp destruction Co-authored-by: Jirka Borovec <[email protected]> * add sklearn metrics * start adding sklearn tests * fix typo * return x and y only for curves * fix typo * add missing tests for sklearn funcs * imports * __all__ * imports * fix sklearn arguments * fix imports * update requirements * Update CHANGELOG.md * Update test_sklearn_metrics.py * formatting * formatting * format * fix all warnings and formatting problems * Update environment.yml * Update requirements-extra.txt * Update environment.yml * Update requirements-extra.txt * fix all warnings and formatting problems * Update CHANGELOG.md * docs * inherit * docs inherit. * docs * Apply suggestions from code review Co-authored-by: Nicki Skafte <[email protected]> * docs * req * min * Apply suggestions from code review Co-authored-by: Tullie Murrell <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Jirka <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Nicki Skafte <[email protected]> Co-authored-by: Tullie Murrell <[email protected]> (cherry picked from commit bd49b07)

* Create utils.py * Create __init__.py * redo sklearn metrics * add some more metrics * add sklearn metrics * Create __init__.py * redo sklearn metrics * New metric classes (#1326) * Create metrics package * Create metric.py * Create utils.py * Create __init__.py * add tests for metric utils * add docstrings for metrics utils * add function to recursively apply other function to collection * add tests for this function * update test * Update pytorch_lightning/metrics/metric.py Co-Authored-By: Jirka Borovec <[email protected]> * update metric name * remove example docs * fix tests * add metric tests * fix to tensor conversion * fix apply to collection * Update CHANGELOG.md * Update pytorch_lightning/metrics/metric.py Co-Authored-By: Jirka Borovec <[email protected]> * remove tests from init * add missing type annotations * rename utils to convertors * Create metrics.rst * Update index.rst * Update index.rst * Update pytorch_lightning/metrics/convertors.py Co-Authored-By: Jirka Borovec <[email protected]> * Update pytorch_lightning/metrics/convertors.py Co-Authored-By: Jirka Borovec <[email protected]> * add doctest example * rename file and fix imports * added parametrized test * replace lambda with inlined function * rename apply_to_collection to apply_func * Separated class description from init args * Apply suggestions from code review Co-Authored-By: Jirka Borovec <[email protected]> * adjust random values * suppress output when seeding * remove gpu from doctest * Add requested changes and add ellipsis for doctest * forgot to push these files... * add explicit check for dtype to convert to * fix ddp tests * remove explicit ddp destruction Co-authored-by: Jirka Borovec <[email protected]> * add sklearn metrics * start adding sklearn tests * fix typo * return x and y only for curves * fix typo * add missing tests for sklearn funcs * imports * __all__ * imports * fix sklearn arguments * fix imports * update requirements * Update CHANGELOG.md * Update test_sklearn_metrics.py * formatting * formatting * format * fix all warnings and formatting problems * Update environment.yml * Update requirements-extra.txt * Update environment.yml * Update requirements-extra.txt * fix all warnings and formatting problems * Update CHANGELOG.md * docs * inherit * docs inherit. * docs * Apply suggestions from code review Co-authored-by: Nicki Skafte <[email protected]> * docs * req * min * Apply suggestions from code review Co-authored-by: Tullie Murrell <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Jirka <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Nicki Skafte <[email protected]> Co-authored-by: Tullie Murrell <[email protected]>

justusschock added feature Is an improvement or enhancement help wanted Open to be worked on labels Apr 1, 2020

justusschock self-assigned this Apr 1, 2020

mergify bot requested a review from a team April 1, 2020 15:05

jeremyjordan reviewed Apr 2, 2020

View reviewed changes

pytorch_lightning/metrics/sklearn.py Outdated Show resolved Hide resolved

Borda requested a review from a team April 3, 2020 08:06

Borda added this to the 0.8.0 milestone Apr 3, 2020

Borda force-pushed the new_sklearn_metric branch from 36c88c1 to 588a805 Compare April 17, 2020 15:55

Borda reviewed Apr 17, 2020

View reviewed changes

environment.yml Outdated Show resolved Hide resolved

mergify bot requested a review from a team April 17, 2020 15:56

justusschock mentioned this pull request Apr 27, 2020

Metrics: AUC #1296

Closed

justusschock marked this pull request as ready for review April 28, 2020 07:32

williamFalcon closed this May 5, 2020

Borda reopened this May 5, 2020

justusschock changed the base branch from metrics to master May 19, 2020 15:23

justusschock force-pushed the new_sklearn_metric branch from 2ce24ab to b2e727c Compare May 25, 2020 06:35

Borda requested a review from jeremyjordan May 25, 2020 16:23

Borda approved these changes May 25, 2020

View reviewed changes

mergify bot requested a review from a team May 25, 2020 19:30

Borda mentioned this pull request May 25, 2020

fix all warnings and formatting problems #1950

Merged

Borda and others added 16 commits June 9, 2020 23:22

format

c9908a1

fix all warnings and formatting problems

41e5971

Update environment.yml

82781e5

Update requirements-extra.txt

01c3e57

Update environment.yml

6a674b6

Update requirements-extra.txt

c96e3d7

Update CHANGELOG.md

b779e37

docs

f6e6cec

inherit

533bb7c

fix all warnings and formatting problems

ca117e4

docs inherit.

36fa04c

docs

0a3a31a

Apply suggestions from code review

c8d6cac

Co-authored-by: Nicki Skafte <[email protected]>

docs

eeede87

req

db8e724

min

d49298a

Borda force-pushed the new_sklearn_metric branch from b7dfd1f to d49298a Compare June 9, 2020 21:24

Borda removed the help wanted Open to be worked on label Jun 9, 2020

tullie reviewed Jun 9, 2020

View reviewed changes

mergify bot requested a review from a team June 9, 2020 22:31

Apply suggestions from code review

363bd64

Co-authored-by: Tullie Murrell <[email protected]>

Borda requested a review from tullie June 10, 2020 08:42

tullie approved these changes Jun 10, 2020

View reviewed changes

Borda merged commit bd49b07 into master Jun 10, 2020

Borda deleted the new_sklearn_metric branch June 10, 2020 13:43

Borda mentioned this pull request Jun 10, 2020

Native torch metrics #1488

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework of Sklearn Metrics #1327

Rework of Sklearn Metrics #1327

justusschock commented Apr 1, 2020 •

edited by Borda

Loading

pep8speaks commented Apr 1, 2020 •

edited

Loading

mergify bot commented May 5, 2020

Borda commented May 5, 2020

mergify bot commented May 18, 2020

justusschock commented May 25, 2020

Borda commented May 25, 2020

awaelchli commented May 25, 2020

tullie left a comment

tullie Jun 9, 2020

Borda Jun 9, 2020

justusschock Jun 10, 2020 •

edited

Loading

codecov bot commented Jun 9, 2020

tullie left a comment

Rework of Sklearn Metrics #1327

Rework of Sklearn Metrics #1327

Conversation

justusschock commented Apr 1, 2020 • edited by Borda Loading

What does this PR do?

PR review

Did you have fun?

pep8speaks commented Apr 1, 2020 • edited Loading

Comment last updated at 2020-06-09 22:46:37 UTC

mergify bot commented May 5, 2020

Borda commented May 5, 2020

mergify bot commented May 18, 2020

justusschock commented May 25, 2020

Borda commented May 25, 2020

awaelchli commented May 25, 2020

tullie left a comment

Choose a reason for hiding this comment

tullie Jun 9, 2020

Choose a reason for hiding this comment

Borda Jun 9, 2020

Choose a reason for hiding this comment

justusschock Jun 10, 2020 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Jun 9, 2020

Codecov Report

tullie left a comment

Choose a reason for hiding this comment

justusschock commented Apr 1, 2020 •

edited by Borda

Loading

pep8speaks commented Apr 1, 2020 •

edited

Loading

justusschock Jun 10, 2020 •

edited

Loading