Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up environment access in plugins #6941

Merged
merged 66 commits into from
Apr 13, 2021
Merged
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
c87cf9a
initial draft
awaelchli Apr 9, 2021
903e673
x
awaelchli Apr 9, 2021
bdf4eaa
x
awaelchli Apr 9, 2021
f9478c8
x
awaelchli Apr 9, 2021
a2f67bd
x
awaelchli Apr 9, 2021
64c8c36
x
awaelchli Apr 9, 2021
bb2acdc
torchelastic
awaelchli Apr 9, 2021
03fe590
x
awaelchli Apr 9, 2021
d4060e2
x
awaelchli Apr 9, 2021
1ac0a69
rename
awaelchli Apr 9, 2021
ab00577
spawn
awaelchli Apr 10, 2021
7a7f4ac
init ddp
awaelchli Apr 10, 2021
8d33db9
x
awaelchli Apr 10, 2021
3fa1264
horovod
awaelchli Apr 10, 2021
22b1ebf
horovod
awaelchli Apr 10, 2021
cd7327a
ranks for DP
awaelchli Apr 10, 2021
405734f
slurm variables
awaelchli Apr 10, 2021
39f2961
fix test
awaelchli Apr 10, 2021
ca64092
slurm env vars in test
awaelchli Apr 10, 2021
b17a6b7
fix test
awaelchli Apr 10, 2021
06fec87
rank
awaelchli Apr 10, 2021
c2744e6
fix test
awaelchli Apr 10, 2021
93a0538
TYPO
awaelchli Apr 10, 2021
ee3f7f8
cpu_te
awaelchli Apr 10, 2021
ca6ee97
slurm environment tests
awaelchli Apr 10, 2021
90f1d37
rpc
awaelchli Apr 10, 2021
391624d
clean up
awaelchli Apr 10, 2021
77af73d
added new tests
awaelchli Apr 10, 2021
5764ef5
None check
awaelchli Apr 10, 2021
d6b2f7c
add test description
awaelchli Apr 10, 2021
15324de
add comments
awaelchli Apr 10, 2021
0048ae5
add more plugins to test
awaelchli Apr 10, 2021
d9310a5
add tests for automatic plugin selection
awaelchli Apr 10, 2021
1c64dfa
include local world size for elastic
awaelchli Apr 10, 2021
5992563
clean up
awaelchli Apr 10, 2021
d968744
add torch elastic local world size to env variable in test
awaelchli Apr 10, 2021
c99043d
make changes to tpu plugin
awaelchli Apr 12, 2021
11c00da
format tests
awaelchli Apr 12, 2021
1b57674
isort
awaelchli Apr 12, 2021
50638f8
Merge branch 'master' into bugfix/elastic-world-size
awaelchli Apr 12, 2021
de8454a
redundant init
awaelchli Apr 12, 2021
4e4afcd
move helper function
awaelchli Apr 12, 2021
be33371
fix test
awaelchli Apr 12, 2021
eb143a7
type hint
awaelchli Apr 12, 2021
e892e57
typing rank properties
awaelchli Apr 12, 2021
d8d9e8b
typo
awaelchli Apr 12, 2021
7897e41
Merge branch 'master' into bugfix/elastic-world-size
awaelchli Apr 12, 2021
b2b705b
missing env var
awaelchli Apr 12, 2021
8d3dbcf
flake
awaelchli Apr 12, 2021
1a7af2e
changelog
awaelchli Apr 13, 2021
d809f2d
redundant init
awaelchli Apr 13, 2021
1bb6e9d
redundant init
awaelchli Apr 13, 2021
9161f4f
Merge branch 'master' into bugfix/elastic-world-size
awaelchli Apr 13, 2021
354c901
Apply suggestions from code review
awaelchli Apr 13, 2021
2218fac
Update pytorch_lightning/plugins/environments/lightning_environment.py
awaelchli Apr 13, 2021
7005b4e
add ddp2 test and fix
awaelchli Apr 13, 2021
5bbfe17
Merge remote-tracking branch 'origin/bugfix/elastic-world-size' into …
awaelchli Apr 13, 2021
a6d0f5d
Update pytorch_lightning/plugins/environments/lightning_environment.py
awaelchli Apr 13, 2021
c3b9db4
test for ddp_cpu, ddp_spawn
awaelchli Apr 13, 2021
246384d
set ranks in ddp_spawn
awaelchli Apr 13, 2021
dae1d73
fix signature
awaelchli Apr 13, 2021
01eb6de
patch xla
awaelchli Apr 13, 2021
45e9f78
world size defaults to 1
awaelchli Apr 13, 2021
9490ce9
deprecation docs
awaelchli Apr 13, 2021
a0a53b7
rename test file
awaelchli Apr 13, 2021
7d39a92
log debug setter
awaelchli Apr 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
horovod
awaelchli committed Apr 10, 2021
commit 3fa12641204c27f5416d3626adfeb4c1ac8e5463
8 changes: 4 additions & 4 deletions pytorch_lightning/plugins/training_type/parallel.py
Original file line number Diff line number Diff line change
@@ -52,19 +52,19 @@ def lightning_module(self):

@property
def global_rank(self):
return self.cluster_environment.global_rank()
return self.cluster_environment.global_rank() if self.cluster_environment is not None else 0

@property
def local_rank(self):
return self.cluster_environment.local_rank()
return self.cluster_environment.local_rank() if self.cluster_environment is not None else 0

@property
def node_rank(self):
return self.cluster_environment.node_rank()
return self.cluster_environment.node_rank() if self.cluster_environment is not None else 0

@property
def world_size(self):
return self.cluster_environment.world_size()
return self.cluster_environment.world_size() if self.cluster_environment is not None else 1

@property
def is_global_zero(self) -> bool: