Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for specifying process group backend to relevant distributed strategies #11745

Merged
merged 19 commits into from
Mar 18, 2022

Conversation

ananthsub
Copy link
Contributor

@ananthsub ananthsub commented Feb 4, 2022

Fixes #11725

What does this PR do?

Does your PR introduce any breaking changes? If yes, please list them.

No

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

@ananthsub ananthsub force-pushed the feat/support-fairring branch 2 times, most recently from 6774abe to bfe8418 Compare February 7, 2022 08:36
@ananthsub ananthsub added strategy: ddp DistributedDataParallel strategy: ddp spawn strategy: deepspeed strategy: fairscale fsdp (removed) Fully Sharded Data Parallel strategy: fairscale sharded (removed) Sharded Data Parallel distributed Generic distributed-related topic deprecation Includes a deprecation labels Feb 7, 2022
@ananthsub ananthsub added this to the 1.6 milestone Feb 7, 2022
@ananthsub ananthsub force-pushed the feat/support-fairring branch from 0b645e3 to 85f257a Compare February 7, 2022 17:58
@ananthsub ananthsub marked this pull request as ready for review February 7, 2022 20:42
@mergify mergify bot added the has conflicts label Feb 7, 2022
@ananthsub ananthsub force-pushed the feat/support-fairring branch from 85f257a to 6ab83d5 Compare February 10, 2022 23:52
@mergify mergify bot removed the has conflicts label Feb 10, 2022
CHANGELOG.md Show resolved Hide resolved
pytorch_lightning/strategies/parallel.py Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Feb 16, 2022

Codecov Report

Merging #11745 (751094d) into master (73bda54) will decrease coverage by 4%.
The diff coverage is 73%.

❗ Current head 751094d differs from pull request most recent head 8296901. Consider uploading reports for the commit 8296901 to get more accurate results

@@           Coverage Diff            @@
##           master   #11745    +/-   ##
========================================
- Coverage      92%      88%    -4%     
========================================
  Files         206      198     -8     
  Lines       17585    17387   -198     
========================================
- Hits        16127    15233   -894     
- Misses       1458     2154   +696     

@mergify mergify bot removed the has conflicts label Mar 10, 2022
pytorch_lightning/utilities/distributed.py Outdated Show resolved Hide resolved
@mergify mergify bot added the ready PRs ready to be merged label Mar 10, 2022
@ananthsub ananthsub force-pushed the feat/support-fairring branch from 1a81f5b to 39b7cb0 Compare March 18, 2022 05:45
@ananthsub ananthsub merged commit 4277845 into Lightning-AI:master Mar 18, 2022
@ananthsub ananthsub deleted the feat/support-fairring branch March 18, 2022 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deprecation Includes a deprecation distributed Generic distributed-related topic ready PRs ready to be merged strategy: ddp DistributedDataParallel strategy: deepspeed strategy: fairscale fsdp (removed) Fully Sharded Data Parallel strategy: fairscale sharded (removed) Sharded Data Parallel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support custom process group backends
6 participants