Set `num_nodes` and `sync_batchnorm` From Trainer for Manually Passed Training Type Plugin #7026

shuyingsunshine21 · 2021-04-15T01:17:25Z

What does this PR do?

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

…lightning pull latest code

…oint_consolidate Update test_all_gather_grad.py

This reverts commit 9d4a2b8.

…1-checkpoint_consolidate" This reverts commit c5053da, reversing changes made to 0d23d75.

This reverts commit 0d23d75.

This reverts commit 70fe5da.

This reverts commit a9aae99.

This reverts commit ea74906.

This reverts commit bf70e43.

This reverts commit f172101.

This reverts commit 536c132.

This reverts commit 3a9fde9.

This reverts commit 7a369f4.

…lightning

This reverts commit 8222dc9.

This reverts commit 6c095b2.

This reverts commit 250d0aa.

This reverts commit 8651d54.

This reverts commit dcdcd29.

…-lightning

…lightning into training_type_plugin_consolidate

shuyingsunshine21 · 2021-05-04T18:50:21Z

added deprecation tests.

pytorch_lightning/trainer/connectors/accelerator_connector.py

awaelchli

great!

pytorch_lightning/plugins/training_type/ddp.py

for more information, see https://pre-commit.ci

Shuying Sun and others added 30 commits March 23, 2021 12:06

Fix some test errors

89f284d

Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:

Merge branch 'master' of https://github.com/PyTorchLightning/pytorch-…

80cfbff

…lightning pull latest code

checkpoint consolidation

536c132

Update ddp_spawn.py

f172101

Update test_metric_result_integration.py

bf70e43

Update test_results.py

ea74906

Update utils.py

a9aae99

Update utils.py

70fe5da

Update test_all_gather_grad.py

0d23d75

Update test_all_gather_grad.py

ca6f98b

Merge pull request #1 from shuyingsunshine21/shuyingsunshine21-checkp…

c5053da

…oint_consolidate Update test_all_gather_grad.py

Update test_results.py

9d4a2b8

Revert "Update test_results.py"

7635b4f

This reverts commit 9d4a2b8.

Revert "Merge pull request #1 from shuyingsunshine21/shuyingsunshine2…

d64f90c

…1-checkpoint_consolidate" This reverts commit c5053da, reversing changes made to 0d23d75.

Revert "Update test_all_gather_grad.py"

dcdcd29

This reverts commit 0d23d75.

Revert "Update utils.py"

8651d54

This reverts commit 70fe5da.

Revert "Update utils.py"

15f4b9e

This reverts commit a9aae99.

Revert "Update test_results.py"

250d0aa

This reverts commit ea74906.

Revert "Update test_metric_result_integration.py"

6c095b2

This reverts commit bf70e43.

Revert "Update ddp_spawn.py"

8222dc9

This reverts commit f172101.

Revert "checkpoint consolidation"

3a9fde9

This reverts commit 536c132.

Revert "Revert "checkpoint consolidation""

7a369f4

This reverts commit 3a9fde9.

Revert "Revert "Revert "checkpoint consolidation"""

b4a0b9e

This reverts commit 7a369f4.

Merge branch 'master' of https://github.com/PyTorchLightning/pytorch-…

5cf1db1

…lightning

Revert "Revert "Update ddp_spawn.py""

0ce7e05

This reverts commit 8222dc9.

Revert "Revert "Update test_metric_result_integration.py""

fe9736d

This reverts commit 6c095b2.

Revert "Revert "Update test_results.py""

c314ef6

This reverts commit 250d0aa.

Revert "Revert "Update utils.py""

c3feda0

This reverts commit 8651d54.

Revert "Revert "Update test_all_gather_grad.py""

c759477

This reverts commit dcdcd29.

Merge branch 'master' of https://github.com/shuyingsunshine21/pytorch…

7a8e540

…-lightning

Shuying Sun added 2 commits May 4, 2021 03:27

comments

16858be

pull rebase

e8a110b

mergify bot removed the has conflicts label May 4, 2021

Shuying Sun added 4 commits May 4, 2021 03:31

remove extra in change.md

60580be

correct in change.md

20d59a4

fix test and flake8

0ab7147

Merge branch 'master' of https://github.com/PyTorchLightning/pytorch-…

9381117

…lightning into training_type_plugin_consolidate

shuyingsunshine21 requested a review from carmocca May 4, 2021 18:50

carmocca added 2 commits May 4, 2021 22:16

Merge branch 'master' into training_type_plugin_consolidate

a35fdc3

pre-commit

9fdde94

carmocca approved these changes May 4, 2021

View reviewed changes

ananthsub mentioned this pull request May 5, 2021

Allow obtaining num_nodes from ClusterEnvironment #7361

Open

awaelchli self-assigned this May 6, 2021

awaelchli reviewed May 7, 2021

View reviewed changes

pytorch_lightning/trainer/connectors/accelerator_connector.py Show resolved Hide resolved

mergify bot added the has conflicts label May 7, 2021

awaelchli approved these changes May 8, 2021

View reviewed changes

pytorch_lightning/plugins/training_type/ddp.py Outdated Show resolved Hide resolved

Merge branch 'master' into training_type_plugin_consolidate

6680b0d

mergify bot removed the has conflicts label May 8, 2021

awaelchli and others added 2 commits May 8, 2021 12:58

whitespace standardization

621bfc8

[pre-commit.ci] auto fixes from pre-commit.com hooks

29f720b

for more information, see https://pre-commit.ci

awaelchli changed the title ~~[RFC] Set num_nodes and sync_batchnorm From Trainer for Manually Passed Training Type Plugin~~ Set num_nodes and sync_batchnorm From Trainer for Manually Passed Training Type Plugin May 8, 2021

awaelchli enabled auto-merge (squash) May 8, 2021 11:02

awaelchli merged commit 987530c into Lightning-AI:master May 8, 2021

shuyingsunshine21 deleted the training_type_plugin_consolidate branch May 11, 2021 18:22

awaelchli mentioned this pull request May 14, 2021

1.3.x cherries 🍒 #7467

Closed

carmocca modified the milestones: v1.4, v1.3.x May 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set `num_nodes` and `sync_batchnorm` From Trainer for Manually Passed Training Type Plugin #7026

Set `num_nodes` and `sync_batchnorm` From Trainer for Manually Passed Training Type Plugin #7026

shuyingsunshine21 commented Apr 15, 2021 •

edited by awaelchli

Loading

shuyingsunshine21 commented May 4, 2021

awaelchli left a comment

Set num_nodes and sync_batchnorm From Trainer for Manually Passed Training Type Plugin #7026

Set num_nodes and sync_batchnorm From Trainer for Manually Passed Training Type Plugin #7026

Conversation

shuyingsunshine21 commented Apr 15, 2021 • edited by awaelchli Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

shuyingsunshine21 commented May 4, 2021

awaelchli left a comment

Choose a reason for hiding this comment

Set `num_nodes` and `sync_batchnorm` From Trainer for Manually Passed Training Type Plugin #7026

Set `num_nodes` and `sync_batchnorm` From Trainer for Manually Passed Training Type Plugin #7026

shuyingsunshine21 commented Apr 15, 2021 •

edited by awaelchli

Loading