Remove truncated backpropagation from loops #16172

awaelchli · 2022-12-22T08:40:17Z

What does this PR do?

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

I made sure I had fun coding 🙃

cc @Borda

carmocca

Looks good so far. Pushed a commit with a few fixes. Also added links in the docstring

github-actions · 2022-12-24T01:48:33Z

⛈️ Required checks status: Has failure 🔴

Warning
This job will need to be re-run to merge your PR. If you do not have write access to the repository, you can ask Lightning-AI/lai-frameworks to re-run it. If you push a new commit, all of CI will re-trigger.

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, pytorch, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, pytorch, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, pytorch, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.10, oldest)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.10)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.9, 1.11)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.12)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.7, 1.10, oldest)	success	✅
pl-cpu (windows-2022, pytorch, 3.9, 1.11)	success	✅
pl-cpu (windows-2022, pytorch, 3.10, 1.12)	success	✅
pl-cpu (windows-2022, pytorch, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.7, 1.10, oldest)	success	✅
pl-cpu (slow, macOS-11, pytorch, 3.7, 1.11)	success	✅
pl-cpu (slow, ubuntu-20.04, pytorch, 3.7, 1.11)	success	✅
pl-cpu (slow, windows-2022, pytorch, 3.7, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.13)	success	✅

These checks are required after the changes to src/pytorch_lightning/callbacks/progress/base.py, src/pytorch_lightning/core/module.py, src/pytorch_lightning/loops/__init__.py, src/pytorch_lightning/loops/batch/__init__.py, src/pytorch_lightning/loops/batch/training_batch_loop.py, src/pytorch_lightning/loops/epoch/training_epoch_loop.py, src/pytorch_lightning/loops/fit_loop.py, src/pytorch_lightning/loops/loop.py, src/pytorch_lightning/loops/optimization/manual_loop.py, src/pytorch_lightning/loops/optimization/optimizer_loop.py, src/pytorch_lightning/loops/utilities.py, src/pytorch_lightning/trainer/configuration_validator.py, src/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/tuner/batch_size_scaling.py, src/pytorch_lightning/utilities/migration/migration.py, src/pytorch_lightning/utilities/migration/utils.py, tests/tests_pytorch/callbacks/progress/test_tqdm_progress_bar.py, tests/tests_pytorch/loops/batch/__init__.py, tests/tests_pytorch/loops/batch/test_truncated_bptt.py, tests/tests_pytorch/loops/epoch/test_training_epoch_loop.py, tests/tests_pytorch/loops/test_evaluation_loop_flow.py, tests/tests_pytorch/loops/test_loop_state_dict.py, tests/tests_pytorch/loops/test_loops.py, tests/tests_pytorch/loops/test_training_loop.py, tests/tests_pytorch/loops/test_training_loop_flow_scalar.py, tests/tests_pytorch/loops/test_utilities.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/migration/test_migration.py, tests/tests_pytorch/utilities/test_fetching.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs)	success	✅

These checks are required after the changes to src/pytorch_lightning/callbacks/progress/base.py, src/pytorch_lightning/core/module.py, src/pytorch_lightning/loops/__init__.py, src/pytorch_lightning/loops/batch/__init__.py, src/pytorch_lightning/loops/batch/training_batch_loop.py, src/pytorch_lightning/loops/epoch/training_epoch_loop.py, src/pytorch_lightning/loops/fit_loop.py, src/pytorch_lightning/loops/loop.py, src/pytorch_lightning/loops/optimization/manual_loop.py, src/pytorch_lightning/loops/optimization/optimizer_loop.py, src/pytorch_lightning/loops/utilities.py, src/pytorch_lightning/trainer/configuration_validator.py, src/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/tuner/batch_size_scaling.py, src/pytorch_lightning/utilities/migration/migration.py, src/pytorch_lightning/utilities/migration/utils.py, tests/tests_pytorch/callbacks/progress/test_tqdm_progress_bar.py, tests/tests_pytorch/loops/batch/__init__.py, tests/tests_pytorch/loops/batch/test_truncated_bptt.py, tests/tests_pytorch/loops/epoch/test_training_epoch_loop.py, tests/tests_pytorch/loops/test_evaluation_loop_flow.py, tests/tests_pytorch/loops/test_loop_state_dict.py, tests/tests_pytorch/loops/test_loops.py, tests/tests_pytorch/loops/test_training_loop.py, tests/tests_pytorch/loops/test_training_loop_flow_scalar.py, tests/tests_pytorch/loops/test_utilities.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/migration/test_migration.py, tests/tests_pytorch/utilities/test_fetching.py.

🟢 pytorch_lightning: Azure HPU

Check ID	Status
pytorch-lightning (HPUs)	success	✅

These checks are required after the changes to src/pytorch_lightning/callbacks/progress/base.py, src/pytorch_lightning/core/module.py, src/pytorch_lightning/loops/__init__.py, src/pytorch_lightning/loops/batch/__init__.py, src/pytorch_lightning/loops/batch/training_batch_loop.py, src/pytorch_lightning/loops/epoch/training_epoch_loop.py, src/pytorch_lightning/loops/fit_loop.py, src/pytorch_lightning/loops/loop.py, src/pytorch_lightning/loops/optimization/manual_loop.py, src/pytorch_lightning/loops/optimization/optimizer_loop.py, src/pytorch_lightning/loops/utilities.py, src/pytorch_lightning/trainer/configuration_validator.py, src/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/tuner/batch_size_scaling.py, src/pytorch_lightning/utilities/migration/migration.py, src/pytorch_lightning/utilities/migration/utils.py, tests/tests_pytorch/callbacks/progress/test_tqdm_progress_bar.py, tests/tests_pytorch/loops/batch/__init__.py, tests/tests_pytorch/loops/batch/test_truncated_bptt.py, tests/tests_pytorch/loops/epoch/test_training_epoch_loop.py, tests/tests_pytorch/loops/test_evaluation_loop_flow.py, tests/tests_pytorch/loops/test_loop_state_dict.py, tests/tests_pytorch/loops/test_loops.py, tests/tests_pytorch/loops/test_training_loop.py, tests/tests_pytorch/loops/test_training_loop_flow_scalar.py, tests/tests_pytorch/loops/test_utilities.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/migration/test_migration.py, tests/tests_pytorch/utilities/test_fetching.py.

🟢 pytorch_lightning: Azure IPU

Check ID	Status
pytorch-lightning (IPUs)	success	✅

These checks are required after the changes to src/pytorch_lightning/callbacks/progress/base.py, src/pytorch_lightning/core/module.py, src/pytorch_lightning/loops/__init__.py, src/pytorch_lightning/loops/batch/__init__.py, src/pytorch_lightning/loops/batch/training_batch_loop.py, src/pytorch_lightning/loops/epoch/training_epoch_loop.py, src/pytorch_lightning/loops/fit_loop.py, src/pytorch_lightning/loops/loop.py, src/pytorch_lightning/loops/optimization/manual_loop.py, src/pytorch_lightning/loops/optimization/optimizer_loop.py, src/pytorch_lightning/loops/utilities.py, src/pytorch_lightning/trainer/configuration_validator.py, src/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/tuner/batch_size_scaling.py, src/pytorch_lightning/utilities/migration/migration.py, src/pytorch_lightning/utilities/migration/utils.py, tests/tests_pytorch/callbacks/progress/test_tqdm_progress_bar.py, tests/tests_pytorch/loops/batch/__init__.py, tests/tests_pytorch/loops/batch/test_truncated_bptt.py, tests/tests_pytorch/loops/epoch/test_training_epoch_loop.py, tests/tests_pytorch/loops/test_evaluation_loop_flow.py, tests/tests_pytorch/loops/test_loop_state_dict.py, tests/tests_pytorch/loops/test_loops.py, tests/tests_pytorch/loops/test_training_loop.py, tests/tests_pytorch/loops/test_training_loop_flow_scalar.py, tests/tests_pytorch/loops/test_utilities.py, tests/tests_pytorch/trainer/test_trainer.py, tests/tests_pytorch/utilities/migration/test_migration.py, tests/tests_pytorch/utilities/test_fetching.py.

🟢 pytorch_lightning: Docs

Check ID	Status
make-doctest (pytorch)	success	✅
make-html (pytorch)	success	✅

These checks are required after the changes to src/pytorch_lightning/callbacks/progress/base.py, src/pytorch_lightning/core/module.py, src/pytorch_lightning/loops/__init__.py, src/pytorch_lightning/loops/batch/__init__.py, src/pytorch_lightning/loops/batch/training_batch_loop.py, src/pytorch_lightning/loops/epoch/training_epoch_loop.py, src/pytorch_lightning/loops/fit_loop.py, src/pytorch_lightning/loops/loop.py, src/pytorch_lightning/loops/optimization/manual_loop.py, src/pytorch_lightning/loops/optimization/optimizer_loop.py, src/pytorch_lightning/loops/utilities.py, src/pytorch_lightning/trainer/configuration_validator.py, src/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/tuner/batch_size_scaling.py, src/pytorch_lightning/utilities/migration/migration.py, src/pytorch_lightning/utilities/migration/utils.py, docs/source-pytorch/api_references.rst, docs/source-pytorch/common/lightning_module.rst, docs/source-pytorch/common/trainer.rst, docs/source-pytorch/extensions/loops.rst, docs/source-pytorch/guides/data.rst.

🔴 lightning_app: Tests workflow

Check ID	Status
app-pytest (macOS-11, app, 3.8, latest)	cancelled	🚫
app-pytest (macOS-11, app, 3.8, oldest)	cancelled	🚫
app-pytest (macOS-11, lightning, 3.9, latest)	success	✅
app-pytest (ubuntu-20.04, app, 3.8, latest)	failure	❌
app-pytest (ubuntu-20.04, app, 3.8, oldest)	failure	❌
app-pytest (ubuntu-20.04, lightning, 3.9, latest)	success	✅
app-pytest (windows-2022, app, 3.8, latest)	failure	❌
app-pytest (windows-2022, app, 3.8, oldest)	failure	❌
app-pytest (windows-2022, lightning, 3.8, latest)	success	✅

These checks are required after the changes to src/lightning_app/utilities/introspection.py, tests/tests_app/core/scripts/lightning_overrides.py, tests/tests_app/utilities/test_introspection.py.

🟢 lightning_app: Examples

Check ID	Status
app-examples (macOS-11, app, 3.9, latest)	success	✅
app-examples (macOS-11, app, 3.9, oldest)	success	✅
app-examples (macOS-11, lightning, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, app, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, app, 3.9, oldest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, latest)	success	✅
app-examples (windows-2022, app, 3.9, latest)	success	✅
app-examples (windows-2022, app, 3.9, oldest)	success	✅
app-examples (windows-2022, lightning, 3.9, latest)	success	✅

These checks are required after the changes to src/lightning_app/utilities/introspection.py.

🔴 lightning_app: Azure

Check ID	Status
App.cloud-e2e	failure	❌

These checks are required after the changes to src/lightning_app/utilities/introspection.py.

🟢 lightning_app: Docs

Check ID	Status
make-doctest (app)	success	✅
make-html (app)	success	✅

These checks are required after the changes to src/lightning_app/utilities/introspection.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning_app/utilities/introspection.py, src/pytorch_lightning/callbacks/progress/base.py, src/pytorch_lightning/core/module.py, src/pytorch_lightning/loops/__init__.py, src/pytorch_lightning/loops/batch/__init__.py, src/pytorch_lightning/loops/batch/training_batch_loop.py, src/pytorch_lightning/loops/epoch/training_epoch_loop.py, src/pytorch_lightning/loops/fit_loop.py, src/pytorch_lightning/loops/loop.py, src/pytorch_lightning/loops/optimization/manual_loop.py, src/pytorch_lightning/loops/optimization/optimizer_loop.py, src/pytorch_lightning/loops/utilities.py, src/pytorch_lightning/trainer/configuration_validator.py, src/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/tuner/batch_size_scaling.py, src/pytorch_lightning/utilities/migration/migration.py, src/pytorch_lightning/utilities/migration/utils.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.7)	success	✅
install-pkg (ubuntu-22.04, app, 3.10)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.7)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.10)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.7)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.10)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.7)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.10)	success	✅
install-pkg (ubuntu-22.04, notset, 3.7)	success	✅
install-pkg (ubuntu-22.04, notset, 3.10)	success	✅
install-pkg (macOS-12, app, 3.7)	success	✅
install-pkg (macOS-12, app, 3.10)	success	✅
install-pkg (macOS-12, fabric, 3.7)	success	✅
install-pkg (macOS-12, fabric, 3.10)	success	✅
install-pkg (macOS-12, pytorch, 3.7)	success	✅
install-pkg (macOS-12, pytorch, 3.10)	success	✅
install-pkg (macOS-12, lightning, 3.7)	success	✅
install-pkg (macOS-12, lightning, 3.10)	success	✅
install-pkg (macOS-12, notset, 3.7)	success	✅
install-pkg (macOS-12, notset, 3.10)	success	✅
install-pkg (windows-2022, app, 3.7)	success	✅
install-pkg (windows-2022, app, 3.10)	success	✅
install-pkg (windows-2022, fabric, 3.7)	success	✅
install-pkg (windows-2022, fabric, 3.10)	success	✅
install-pkg (windows-2022, pytorch, 3.7)	success	✅
install-pkg (windows-2022, pytorch, 3.10)	success	✅
install-pkg (windows-2022, lightning, 3.7)	success	✅
install-pkg (windows-2022, lightning, 3.10)	success	✅
install-pkg (windows-2022, notset, 3.7)	success	✅
install-pkg (windows-2022, notset, 3.10)	success	✅

These checks are required after the changes to src/lightning_app/utilities/introspection.py, src/pytorch_lightning/callbacks/progress/base.py, src/pytorch_lightning/core/module.py, src/pytorch_lightning/loops/__init__.py, src/pytorch_lightning/loops/batch/__init__.py, src/pytorch_lightning/loops/batch/training_batch_loop.py, src/pytorch_lightning/loops/epoch/training_epoch_loop.py, src/pytorch_lightning/loops/fit_loop.py, src/pytorch_lightning/loops/loop.py, src/pytorch_lightning/loops/optimization/manual_loop.py, src/pytorch_lightning/loops/optimization/optimizer_loop.py, src/pytorch_lightning/loops/utilities.py, src/pytorch_lightning/trainer/configuration_validator.py, src/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py, src/pytorch_lightning/trainer/trainer.py, src/pytorch_lightning/tuner/batch_size_scaling.py, src/pytorch_lightning/utilities/migration/migration.py, src/pytorch_lightning/utilities/migration/utils.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

carmocca

LGTM. But let's wait until the fabric rename lands before merging this

src/pytorch_lightning/utilities/migration/utils.py

awaelchli · 2023-01-05T23:25:02Z

How on earth am I going to rebase this.

for more information, see https://pre-commit.ci

tchaton

LGTM !

src/pytorch_lightning/core/module.py

src/pytorch_lightning/utilities/migration/migration.py

src/pytorch_lightning/loops/optimization/manual_loop.py

Co-authored-by: Carlos Mocholí <[email protected]>

for more information, see https://pre-commit.ci

Borda · 2023-01-10T12:18:53Z

@awaelchli this shall be fixed with #16302

awaelchli · 2023-01-10T12:22:47Z

@Borda Nice! Thank you for jumping onto it

github-actions bot added pl Generic label for PyTorch Lightning package and removed pl Generic label for PyTorch Lightning package labels Dec 22, 2022

carmocca reviewed Dec 22, 2022

View reviewed changes

awaelchli changed the title ~~WIP: Remove truncated backpropagation from loops~~ Remove truncated backpropagation from loops Dec 24, 2022

awaelchli marked this pull request as ready for review December 24, 2022 01:48

awaelchli requested review from tchaton, lantiga, hhsecond, ethanwharris, williamFalcon, justusschock, edenlightning and Borda as code owners December 24, 2022 01:48

Borda approved these changes Dec 24, 2022

View reviewed changes

mergify bot added has conflicts and removed has conflicts labels Dec 24, 2022

awaelchli requested a review from carmocca January 3, 2023 09:56

justusschock approved these changes Jan 3, 2023

View reviewed changes

mergify bot added the ready PRs ready to be merged label Jan 3, 2023

awaelchli mentioned this pull request Jan 3, 2023

Improvements to checkpoint migration #16233

Merged

11 tasks

carmocca reviewed Jan 4, 2023

View reviewed changes

src/pytorch_lightning/utilities/migration/utils.py Outdated Show resolved Hide resolved

mergify bot added has conflicts and removed ready PRs ready to be merged labels Jan 4, 2023

lantiga deleted the branch lite/debug January 5, 2023 07:07

lantiga closed this Jan 5, 2023

mergify bot added ready PRs ready to be merged and removed ready PRs ready to be merged labels Jan 5, 2023

mergify bot removed the ready PRs ready to be merged label Jan 5, 2023

awaelchli force-pushed the lite/debug-remove-tbptt branch from 12318ac to 4baf805 Compare January 6, 2023 00:41

reapply changes

3e112bf

mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels Jan 6, 2023

awaelchli force-pushed the lite/debug-remove-tbptt branch from 4baf805 to 3e112bf Compare January 6, 2023 00:42

mergify bot added the ready PRs ready to be merged label Jan 6, 2023

pre-commit-ci bot and others added 3 commits January 6, 2023 00:42

[pre-commit.ci] auto fixes from pre-commit.com hooks

a2ca664

for more information, see https://pre-commit.ci

reviewer suggestion

f67da12

fix

6955c68

awaelchli requested a review from carmocca January 6, 2023 20:49

fixes

8ee7408

tchaton approved these changes Jan 9, 2023

View reviewed changes

src/pytorch_lightning/core/module.py Show resolved Hide resolved

src/pytorch_lightning/utilities/migration/migration.py Show resolved Hide resolved

carmocca approved these changes Jan 9, 2023

View reviewed changes

src/pytorch_lightning/loops/optimization/manual_loop.py Outdated Show resolved Hide resolved

Update src/pytorch_lightning/loops/optimization/manual_loop.py

2bbd2a3

Co-authored-by: Carlos Mocholí <[email protected]>

awaelchli enabled auto-merge (squash) January 9, 2023 12:08

carmocca and others added 4 commits January 9, 2023 14:33

One more

2c1a478

trigger ci

036f5d7

[pre-commit.ci] auto fixes from pre-commit.com hooks

b8877c7

for more information, see https://pre-commit.ci

Merge branch 'lite/debug' into lite/debug-remove-tbptt

b92299f

awaelchli merged commit 90a7b58 into lite/debug Jan 11, 2023

awaelchli deleted the lite/debug-remove-tbptt branch January 11, 2023 17:28

carmocca restored the lite/debug-remove-tbptt branch January 11, 2023 17:39

carmocca mentioned this pull request Jan 11, 2023

Remove truncated backpropagation from loops #16337

Merged

carmocca deleted the lite/debug-remove-tbptt branch January 11, 2023 17:49

carmocca added this to the 2.0 milestone Jan 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove truncated backpropagation from loops #16172

Remove truncated backpropagation from loops #16172

awaelchli commented Dec 22, 2022 •

edited by github-actions bot

Loading

carmocca left a comment

github-actions bot commented Dec 24, 2022 •

edited

Loading

carmocca left a comment

awaelchli commented Jan 5, 2023

tchaton left a comment

Borda commented Jan 10, 2023

awaelchli commented Jan 10, 2023

Remove truncated backpropagation from loops #16172

Remove truncated backpropagation from loops #16172

Conversation

awaelchli commented Dec 22, 2022 • edited by github-actions bot Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

carmocca left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 24, 2022 • edited Loading

⛈️ Required checks status: Has failure 🔴

Groups summary

carmocca left a comment

Choose a reason for hiding this comment

awaelchli commented Jan 5, 2023

tchaton left a comment

Choose a reason for hiding this comment

Borda commented Jan 10, 2023

awaelchli commented Jan 10, 2023

awaelchli commented Dec 22, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Dec 24, 2022 •

edited

Loading