Fix LightningDeprecationWarning: DataModule.setup has already been called #9943 #9953

Programmer-RD-AI · 2021-10-15T15:41:19Z

What does this PR do?

Fix LightningDeprecationWarning: DataModule.setup has already been called #9943

Does your PR introduce any breaking changes? If yes, please list them.

None

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[] Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃
Yes

Co-authored-by: ananthsub <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]>

* add on_exception callback hook * deprecate on_keyboard_interrupt * Apply suggestions from code review * raise keyboard interrupt * Delete cluster * update changelog Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]>

…9279) * Avoid deprecation warnings being called when hooks are not implemented * Update tests & changelog * Apply suggestions from code review Co-authored-by: Rohit Gupta <[email protected]>

* replace dev debugger in early stopping * remove unused imports

* reset progress updates * update docs * add test Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update parsing.py * add todo (for single arg) * unblock non container single arg * init test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update CHANGELOG.md * pep8 line length * Update pytorch_lightning/utilities/parsing.py * remove dict namespace conversion * add omegaconf support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add dict test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add omegaconf test * Update CHANGELOG.md * Update pytorch_lightning/utilities/parsing.py Co-authored-by: Jirka Borovec <[email protected]> * Update pytorch_lightning/utilities/parsing.py Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jirka Borovec <[email protected]>

Co-authored-by: Carlos Mocholí <[email protected]>

* CI: precommit - docformatter * fix deprecated Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Without clearing this reference, the loss tensor stays live through the next training step. This can be a problem for memory intensive models that produce very deep backward graphs such as neural ODEs. For these models, keeping the backward graph of the previous loss in memory can lead to OOM errors in the next training step even though the step might have succeeded if we had cleared (and thus GC'd) the previous backward graph. Co-authored-by: tchaton <[email protected]> Co-authored-by: Carlos Mocholi <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]>

Co-authored-by: Jirka Borovec <[email protected]>

Co-authored-by: Adrian Wälchli <[email protected]>

Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: thomas chaton <[email protected]>

Co-authored-by: Carlos Mocholí <[email protected]>

…is overridden (#9360)

#9367)

Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: thomas chaton <[email protected]>

Co-authored-by: ananthsub <[email protected]>

* Fix logging of nan parameters

Co-authored-by: Adrian Wälchli <[email protected]>

for more information, see https://pre-commit.ci

Programmer-RD-AI · 2021-10-17T04:42:21Z

#9970 is the new pull request

EricWiener and others added 30 commits September 4, 2021 23:33

Support infinite training (#8877)

cf1a589

Co-authored-by: ananthsub <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]>

Update ci badge (#9339)

ff1e691

Avoid optional Tracker attributes and enable mypy (#9320)

49c0485

Remove deprecation warnings being called for on_{task}_dataloader (#…

dc3391b

…9279) * Avoid deprecation warnings being called when hooks are not implemented * Update tests & changelog * Apply suggestions from code review Co-authored-by: Rohit Gupta <[email protected]>

remove early stopping tracking from internal debugger (#9327)

f9132e8

* replace dev debugger in early stopping * remove unused imports

fix progress bar restart with fault-tolerant training enabled (#9310)

50198d7

* reset progress updates * update docs * add test Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Add typing for ResultCollection [3/3] (#9271)

73fca23

[bugfix] Resolve PyTorch Profiling for Manual Optimization (#9316)

9149b64

Co-authored-by: Carlos Mocholí <[email protected]>

Fix mypy typing errors in optimizer loop (#9317)

9a14f04

Remove unnecessary TrainingEpochLoop return (#9298)

05ff1b2

Update requirements, update test (#9345)

72bb018

CI: precommit - docformatter (#8584)

6e124e7

* CI: precommit - docformatter * fix deprecated Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Add test assertion (#9309)

392c577

Add CITATION.cff (#9139)

54e95d3

Co-authored-by: Jirka Borovec <[email protected]>

Remove some incorrect comments in ddp.py (#9319)

0135a4b

Remove todo, ensure we only check rank 0 for deepspeed warning (#9311)

d49709e

Run plugin closure before on_before_optimizer_step [1/2] (#9288)

6892d53

Co-authored-by: Adrian Wälchli <[email protected]>

Add a warning to deepspeed when inferring batch size (#9221)

a79c351

Add ManualOptimization loop (#9266)

ca679cd

Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: thomas chaton <[email protected]>

Remove checkpoint tracking from internal debugger (#9326)

91ce0d0

Co-authored-by: Carlos Mocholí <[email protected]>

Enforce that the optimizer closure is executed when optimizer_step …

15d9430

…is overridden (#9360)

Fix replace_sampler missing the batch size under specific conditions (

f239b96

#9367)

Keep hidden state in the optimization loops (#9368)

8407238

rewrite and improve tests for truncated back-propagation (#9369)

25af4b1

Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: thomas chaton <[email protected]>

Enable inference mode for testing and predicting (#8813)

a079d7f

Co-authored-by: ananthsub <[email protected]>

Fix logging of nan parameters (#9364)

41ba639

* Fix logging of nan parameters

Fix hiddens type annotation (#9377)

3070a9e

Programmer-RD-AI and others added 12 commits October 11, 2021 07:28

Update pytorch_lightning/callbacks/model_checkpoint.py

83f379e

Co-authored-by: Adrian Wälchli <[email protected]>

Update CHANGELOG.md

fecfbc8

Merge branch 'PyTorchLightning:master' into master

7086701

Update test_model_checkpoint.py

3dc3337

[pre-commit.ci] auto fixes from pre-commit.com hooks

a3e7581

for more information, see https://pre-commit.ci

Update model_checkpoint.py

acf6e28

[pre-commit.ci] auto fixes from pre-commit.com hooks

73c0ae9

for more information, see https://pre-commit.ci

update

c56bcb8

update

ad6259d

chlog update

4a51af5

Update datamodule.py

3975616

Update CHANGELOG.md

63be67d

Programmer-RD-AI requested review from ananthsub, awaelchli, Borda, carmocca, edenlightning, justusschock, kaushikb11, SeanNaren, SkafteNicki, tchaton and williamFalcon as code owners October 15, 2021 15:41

This was referenced Oct 15, 2021

LightningDeprecationWarning: DataModule.setup has already been called #9943

Closed

Fix LightningDeprecationWarning: DataModule.setup has already been called #9943 #9945

Closed

mergify bot added the has conflicts label Oct 15, 2021

Programmer-RD-AI added 2 commits October 16, 2021 16:57

Update model_checkpoint.py

291c83d

Update CHANGELOG.md

f289882

Programmer-RD-AI closed this Oct 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix LightningDeprecationWarning: DataModule.setup has already been called #9943 #9953

Fix LightningDeprecationWarning: DataModule.setup has already been called #9943 #9953

Programmer-RD-AI commented Oct 15, 2021

Programmer-RD-AI commented Oct 17, 2021

Fix LightningDeprecationWarning: DataModule.setup has already been called #9943 #9953

Fix LightningDeprecationWarning: DataModule.setup has already been called #9943 #9953

Conversation

Programmer-RD-AI commented Oct 15, 2021

What does this PR do?

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

PR review

Did you have fun?

Programmer-RD-AI commented Oct 17, 2021