-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Let TorchCollective works on the torch.distributed
WORLD process group by default
#16995
Let TorchCollective works on the torch.distributed
WORLD process group by default
#16995
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #16995 +/- ##
=========================================
- Coverage 82% 59% -23%
=========================================
Files 436 413 -23
Lines 31765 31486 -279
=========================================
- Hits 26009 18687 -7322
- Misses 5756 12799 +7043 |
torch.distributed
WORLD process group by default
tests/tests_fabric/plugins/collectives/test_torch_collective.py
Outdated
Show resolved
Hide resolved
tests/tests_fabric/plugins/collectives/test_torch_collective.py
Outdated
Show resolved
Hide resolved
tests/tests_fabric/plugins/collectives/test_torch_collective.py
Outdated
Show resolved
Hide resolved
I've seen also that from pytorch 2.0 the if _TORCH_GREATER_EQUAL_2_0:
from torch.distributed.distributed_c10d import _world
pg_map = _world.pg_map
else:
pg_map = dist.distributed_c10d._pg_map at the beginning and refers to |
…oup by default (#16995) Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> (cherry picked from commit bb861cb)
…oup by default (#16995) Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> (cherry picked from commit bb861cb)
…oup by default (#16995) Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> (cherry picked from commit bb861cb)
What does this PR do?
With this PR
TorchCollective
works by default on thetorch.distributed.group.WORLD
process group, without the need to first create a new group.Fixes #16922
Before submitting
PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:
Reviewer checklist