-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disable pl optimizer temporarily to fix AMP issues #5163
Conversation
btw, there are quite many failing tests... |
I've added a todo and a fix (hopefully it builds fine now), seems like there is an underlying internal issue. We wrap the model in the lightning optimizer internally and never unwrap it. If a user specifies that they'd not like to wrap their model in the pl optimizer, not sure why we never unwrap cc @tchaton |
Codecov Report
@@ Coverage Diff @@
## master #5163 +/- ##
======================================
Coverage 93% 93%
======================================
Files 134 134
Lines 9928 9928
======================================
Hits 9242 9242
Misses 686 686 |
* Disable pl optimizer temporarily to fix AMP issues * Add todo and enable pl optimizer in the test
* Disable pl optimizer temporarily to fix AMP issues * Add todo and enable pl optimizer in the test
* Disable pl optimizer temporarily to fix AMP issues * Add todo and enable pl optimizer in the test
* Disable pl optimizer temporarily to fix AMP issues * Add todo and enable pl optimizer in the test
* Disable pl optimizer temporarily to fix AMP issues * Add todo and enable pl optimizer in the test
* Disable pl optimizer temporarily to fix AMP issues * Add todo and enable pl optimizer in the test
What does this PR do?
Temporary hotfix to address issues of #5159.
It seems in this part of the code:
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/plugins/native_amp.py#L54-L55
PL lightning optimizer results in different outputs. Not entirely sure yet but need to debug and track in the associated issue.
Fixes # (issue)
Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:
Did you have fun?
Make sure you had fun coding 🙃