Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove the FairScale integration #16400

Merged
merged 4 commits into from
Jan 23, 2023
Merged

Remove the FairScale integration #16400

merged 4 commits into from
Jan 23, 2023

Conversation

carmocca
Copy link
Contributor

@carmocca carmocca commented Jan 17, 2023 •
edited by github-actions bot
Loading

What does this PR do?

See title

In a follow-up I will replace the fsdp_native key for fsdp

Closes #13339
Closes #11207
Closes #13431
Closes #7217

Does your PR introduce any breaking changes? If yes, please list them.

Removes deprecated code

cc @justusschock @awaelchli @Borda

@carmocca carmocca added refactor breaking change Includes a breaking change strategy: fairscale fsdp (removed) Fully Sharded Data Parallel strategy: fairscale sharded (removed) Sharded Data Parallel pl Generic label for PyTorch Lightning package labels Jan 17, 2023
@carmocca carmocca self-assigned this Jan 17, 2023
@github-actions github-actions bot removed the pl Generic label for PyTorch Lightning package label Jan 17, 2023
@carmocca carmocca marked this pull request as ready for review January 17, 2023 13:40
@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2023 •
edited
Loading

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, pytorch, 3.8, 1.11) success ✅
pl-cpu (macOS-11, pytorch, 3.9, 1.12) success ✅
pl-cpu (macOS-11, pytorch, 3.10, 1.13) success ✅
pl-cpu (macOS-11, pytorch, 3.8, 1.10, oldest) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.10) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.9, 1.11) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.12) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.13) success ✅
pl-cpu (ubuntu-20.04, pytorch, 3.7, 1.10, oldest) success ✅
pl-cpu (windows-2022, pytorch, 3.9, 1.11) success ✅
pl-cpu (windows-2022, pytorch, 3.10, 1.12) success ✅
pl-cpu (windows-2022, pytorch, 3.10, 1.13) success ✅
pl-cpu (windows-2022, pytorch, 3.7, 1.10, oldest) success ✅
pl-cpu (slow, macOS-11, pytorch, 3.7, 1.11) success ✅
pl-cpu (slow, ubuntu-20.04, pytorch, 3.7, 1.11) success ✅
pl-cpu (slow, windows-2022, pytorch, 3.7, 1.11) success ✅
pl-cpu (macOS-11, lightning, 3.8, 1.13) success ✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.13) success ✅
pl-cpu (windows-2022, lightning, 3.8, 1.13) success ✅

These checks are required after the changes to requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt, src/pytorch_lightning/callbacks/stochastic_weight_avg.py, src/pytorch_lightning/overrides/fairscale.py, src/pytorch_lightning/plugins/__init__.py, src/pytorch_lightning/plugins/precision/__init__.py, src/pytorch_lightning/plugins/precision/fully_sharded_native_amp.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/serve/servable_module_validator.py, src/pytorch_lightning/strategies/__init__.py, src/pytorch_lightning/strategies/ddp.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/sharded.py, src/pytorch_lightning/strategies/sharded_spawn.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/helpers/runif.py, tests/tests_pytorch/plugins/precision/test_sharded_precision.py, tests/tests_pytorch/plugins/test_cluster_integration.py, tests/tests_pytorch/strategies/test_ddp_fully_sharded_with_full_state_dict.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_sharded_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/test_imports.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
pytorch-lightning (GPUs) success ✅

These checks are required after the changes to requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt, src/pytorch_lightning/callbacks/stochastic_weight_avg.py, src/pytorch_lightning/overrides/fairscale.py, src/pytorch_lightning/plugins/__init__.py, src/pytorch_lightning/plugins/precision/__init__.py, src/pytorch_lightning/plugins/precision/fully_sharded_native_amp.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/serve/servable_module_validator.py, src/pytorch_lightning/strategies/__init__.py, src/pytorch_lightning/strategies/ddp.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/sharded.py, src/pytorch_lightning/strategies/sharded_spawn.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/helpers/runif.py, tests/tests_pytorch/plugins/precision/test_sharded_precision.py, tests/tests_pytorch/plugins/test_cluster_integration.py, tests/tests_pytorch/strategies/test_ddp_fully_sharded_with_full_state_dict.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_sharded_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/test_imports.py.

🟢 pytorch_lightning: Benchmarks
Check ID Status
pytorch-lightning.Benchmark success ✅

These checks are required after the changes to requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt.

🟢 pytorch_lightning: Azure HPU
Check ID Status
pytorch-lightning (HPUs) success ✅

These checks are required after the changes to requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt, src/pytorch_lightning/callbacks/stochastic_weight_avg.py, src/pytorch_lightning/overrides/fairscale.py, src/pytorch_lightning/plugins/__init__.py, src/pytorch_lightning/plugins/precision/__init__.py, src/pytorch_lightning/plugins/precision/fully_sharded_native_amp.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/serve/servable_module_validator.py, src/pytorch_lightning/strategies/__init__.py, src/pytorch_lightning/strategies/ddp.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/sharded.py, src/pytorch_lightning/strategies/sharded_spawn.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/helpers/runif.py, tests/tests_pytorch/plugins/precision/test_sharded_precision.py, tests/tests_pytorch/plugins/test_cluster_integration.py, tests/tests_pytorch/strategies/test_ddp_fully_sharded_with_full_state_dict.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_sharded_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/test_imports.py.

🟢 pytorch_lightning: Azure IPU
Check ID Status
pytorch-lightning (IPUs) success ✅

These checks are required after the changes to requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt, src/pytorch_lightning/callbacks/stochastic_weight_avg.py, src/pytorch_lightning/overrides/fairscale.py, src/pytorch_lightning/plugins/__init__.py, src/pytorch_lightning/plugins/precision/__init__.py, src/pytorch_lightning/plugins/precision/fully_sharded_native_amp.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/serve/servable_module_validator.py, src/pytorch_lightning/strategies/__init__.py, src/pytorch_lightning/strategies/ddp.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/sharded.py, src/pytorch_lightning/strategies/sharded_spawn.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, tests/tests_pytorch/helpers/runif.py, tests/tests_pytorch/plugins/precision/test_sharded_precision.py, tests/tests_pytorch/plugins/test_cluster_integration.py, tests/tests_pytorch/strategies/test_ddp_fully_sharded_with_full_state_dict.py, tests/tests_pytorch/strategies/test_registry.py, tests/tests_pytorch/strategies/test_sharded_strategy.py, tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/test_imports.py.

🟢 pytorch_lightning: Docs
Check ID Status
make-doctest (pytorch) success ✅
make-html (pytorch) success ✅

These checks are required after the changes to src/pytorch_lightning/callbacks/stochastic_weight_avg.py, src/pytorch_lightning/overrides/fairscale.py, src/pytorch_lightning/plugins/__init__.py, src/pytorch_lightning/plugins/precision/__init__.py, src/pytorch_lightning/plugins/precision/fully_sharded_native_amp.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/serve/servable_module_validator.py, src/pytorch_lightning/strategies/__init__.py, src/pytorch_lightning/strategies/ddp.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/sharded.py, src/pytorch_lightning/strategies/sharded_spawn.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, docs/source-pytorch/api_references.rst, docs/source-pytorch/conf.py, docs/source-pytorch/extensions/plugins.rst, docs/source-pytorch/guides/speed.rst, requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt.

🟢 pytorch_lightning: Docker
Check ID Status
build-cuda (3.9, 1.10, 11.3.1) success ✅
build-cuda (3.9, 1.11, 11.3.1) success ✅
build-cuda (3.9, 1.12, 11.6.1) success ✅
build-cuda (3.9, 1.13, 11.7.1) success ✅
build-hpu (1.5.0, 1.11.0) success ✅
build-ipu (3.9, 1.10) success ✅
build-NGC success ✅
build-pl (3.9, 1.10, 11.3.1) success ✅
build-pl (3.9, 1.11, 11.3.1) success ✅
build-pl (3.9, 1.12, 11.6.1) success ✅
build-pl (3.9, 1.13, 11.7.1) success ✅
build-xla (3.7, 1.12) success ✅

These checks are required after the changes to requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt.

🟢 lightning_app: Tests workflow
Check ID Status
app-pytest (macOS-11, app, 3.8, latest) success ✅
app-pytest (macOS-11, app, 3.8, oldest) success ✅
app-pytest (macOS-11, lightning, 3.9, latest) success ✅
app-pytest (ubuntu-20.04, app, 3.8, latest) success ✅
app-pytest (ubuntu-20.04, app, 3.8, oldest) success ✅
app-pytest (ubuntu-20.04, lightning, 3.9, latest) success ✅
app-pytest (windows-2022, app, 3.8, latest) success ✅
app-pytest (windows-2022, app, 3.8, oldest) success ✅
app-pytest (windows-2022, lightning, 3.8, latest) success ✅

These checks are required after the changes to src/lightning_app/components/multi_node/trainer.py, tests/tests_app/components/multi_node/test_trainer.py.

🟢 lightning_app: Examples
Check ID Status
app-examples (macOS-11, app, 3.9, latest) success ✅
app-examples (macOS-11, app, 3.9, oldest) success ✅
app-examples (macOS-11, lightning, 3.9, latest) success ✅
app-examples (ubuntu-20.04, app, 3.9, latest) success ✅
app-examples (ubuntu-20.04, app, 3.9, oldest) success ✅
app-examples (ubuntu-20.04, lightning, 3.9, latest) success ✅
app-examples (windows-2022, app, 3.9, latest) success ✅
app-examples (windows-2022, app, 3.9, oldest) success ✅
app-examples (windows-2022, lightning, 3.9, latest) success ✅

These checks are required after the changes to src/lightning_app/components/multi_node/trainer.py.

🟢 lightning_app: Azure
Check ID Status
App.cloud-e2e success ✅

These checks are required after the changes to src/lightning_app/components/multi_node/trainer.py.

🟢 lightning_app: Docs
Check ID Status
make-doctest (app) success ✅
make-html (app) success ✅

These checks are required after the changes to src/lightning_app/components/multi_node/trainer.py.

🟢 mypy
Check ID Status
mypy success ✅

These checks are required after the changes to requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt, src/lightning_app/components/multi_node/trainer.py, src/pytorch_lightning/callbacks/stochastic_weight_avg.py, src/pytorch_lightning/overrides/fairscale.py, src/pytorch_lightning/plugins/__init__.py, src/pytorch_lightning/plugins/precision/__init__.py, src/pytorch_lightning/plugins/precision/fully_sharded_native_amp.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/serve/servable_module_validator.py, src/pytorch_lightning/strategies/__init__.py, src/pytorch_lightning/strategies/ddp.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/sharded.py, src/pytorch_lightning/strategies/sharded_spawn.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.7) success ✅
install-pkg (ubuntu-22.04, app, 3.10) success ✅
install-pkg (ubuntu-22.04, fabric, 3.7) success ✅
install-pkg (ubuntu-22.04, fabric, 3.10) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.7) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.10) success ✅
install-pkg (ubuntu-22.04, lightning, 3.7) success ✅
install-pkg (ubuntu-22.04, lightning, 3.10) success ✅
install-pkg (ubuntu-22.04, notset, 3.7) success ✅
install-pkg (ubuntu-22.04, notset, 3.10) success ✅
install-pkg (macOS-12, app, 3.7) success ✅
install-pkg (macOS-12, app, 3.10) success ✅
install-pkg (macOS-12, fabric, 3.7) success ✅
install-pkg (macOS-12, fabric, 3.10) success ✅
install-pkg (macOS-12, pytorch, 3.7) success ✅
install-pkg (macOS-12, pytorch, 3.10) success ✅
install-pkg (macOS-12, lightning, 3.7) success ✅
install-pkg (macOS-12, lightning, 3.10) success ✅
install-pkg (macOS-12, notset, 3.7) success ✅
install-pkg (macOS-12, notset, 3.10) success ✅
install-pkg (windows-2022, app, 3.7) success ✅
install-pkg (windows-2022, app, 3.10) success ✅
install-pkg (windows-2022, fabric, 3.7) success ✅
install-pkg (windows-2022, fabric, 3.10) success ✅
install-pkg (windows-2022, pytorch, 3.7) success ✅
install-pkg (windows-2022, pytorch, 3.10) success ✅
install-pkg (windows-2022, lightning, 3.7) success ✅
install-pkg (windows-2022, lightning, 3.10) success ✅
install-pkg (windows-2022, notset, 3.7) success ✅
install-pkg (windows-2022, notset, 3.10) success ✅

These checks are required after the changes to src/lightning_app/components/multi_node/trainer.py, src/pytorch_lightning/callbacks/stochastic_weight_avg.py, src/pytorch_lightning/overrides/fairscale.py, src/pytorch_lightning/plugins/__init__.py, src/pytorch_lightning/plugins/precision/__init__.py, src/pytorch_lightning/plugins/precision/fully_sharded_native_amp.py, src/pytorch_lightning/plugins/precision/sharded_native_amp.py, src/pytorch_lightning/serve/servable_module_validator.py, src/pytorch_lightning/strategies/__init__.py, src/pytorch_lightning/strategies/ddp.py, src/pytorch_lightning/strategies/fully_sharded.py, src/pytorch_lightning/strategies/sharded.py, src/pytorch_lightning/strategies/sharded_spawn.py, src/pytorch_lightning/trainer/connectors/accelerator_connector.py, requirements/pytorch/check-avail-strategies.py, requirements/pytorch/strategies.txt.

🟢 link-check
Check ID Status
markdown-link-check success ✅

These checks are required after the changes to src/pytorch_lightning/CHANGELOG.md.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

@mergify mergify bot added the ready PRs ready to be merged label Jan 17, 2023
@carmocca
Copy link
Contributor Author

Merge blocked by Lightning-Universe/lightning-quick-start#27

@mergify mergify bot added has conflicts and removed ready PRs ready to be merged labels Jan 18, 2023
@carmocca carmocca force-pushed the lite/debug-remove-fairscale branch from 3fddf7d to 5f1ea77 Compare January 18, 2023 15:06
@carmocca carmocca enabled auto-merge (squash) January 18, 2023 15:06
@mergify mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels Jan 18, 2023
@carmocca carmocca force-pushed the lite/debug-remove-fairscale branch from 5f1ea77 to f5f7b20 Compare January 19, 2023 17:13
@carmocca carmocca disabled auto-merge January 19, 2023 17:16
@carmocca carmocca force-pushed the lite/debug-remove-fairscale branch from f5f7b20 to 6ef051c Compare January 19, 2023 17:44
@carmocca carmocca changed the base branch from lite/debug to master January 19, 2023 17:44
@carmocca carmocca enabled auto-merge (squash) January 23, 2023 13:03
@carmocca carmocca merged commit 39b7cb8 into master Jan 23, 2023
@carmocca carmocca deleted the lite/debug-remove-fairscale branch January 23, 2023 13:39
@SpirinEgor SpirinEgor mentioned this pull request Jan 30, 2023
Closed
@Borda Borda mentioned this pull request Feb 3, 2023
Merged
8 tasks
@awaelchli awaelchli added this to the 2.0 milestone Feb 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@awaelchli awaelchli awaelchli approved these changes

@akihironitta akihironitta akihironitta approved these changes

@justusschock justusschock justusschock approved these changes

@Borda Borda Borda approved these changes

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

@lantiga lantiga Awaiting requested review from lantiga lantiga is a code owner

@hhsecond hhsecond Awaiting requested review from hhsecond

@ethanwharris ethanwharris Awaiting requested review from ethanwharris ethanwharris is a code owner

@williamFalcon williamFalcon Awaiting requested review from williamFalcon

@edenlightning edenlightning Awaiting requested review from edenlightning

Assignees

@carmocca carmocca

Labels
breaking change Includes a breaking change ready PRs ready to be merged refactor strategy: fairscale fsdp (removed) Fully Sharded Data Parallel strategy: fairscale sharded (removed) Sharded Data Parallel
Projects
None yet
Milestone
2.0
5 participants
@carmocca @awaelchli @Borda @justusschock @akihironitta