Add `Fabric.save(filter=...)` #17845

carmocca · 2023-06-16T02:12:49Z

What does this PR do?

Fixes #17724

cc @Borda @awaelchli @carmocca @justusschock

…/lightning into carmocca/fabric-save-filter

github-actions · 2023-06-16T04:39:09Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py.

🟢 pytorch_lightning: Benchmarks

Check ID	Status
lightning.Benchmarks	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py.

🟢 fabric: Docs

Check ID	Status
make-doctest (fabric)	success	✅
make-html (fabric)	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py, docs/source-fabric/guide/checkpoint.rst.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11)	success	✅
fabric-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13)	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py, tests/tests_fabric/strategies/test_deepspeed.py, tests/tests_fabric/strategies/test_fsdp_integration.py, tests/tests_fabric/test_fabric.py.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs)	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py, tests/tests_fabric/strategies/test_deepspeed.py, tests/tests_fabric/strategies/test_fsdp_integration.py, tests/tests_fabric/test_fabric.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.10)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.10)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.10)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.10)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.10)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.10)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.10)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.10)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.10)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.10)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.10)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.10)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.10)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.10)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.10)	success	✅

These checks are required after the changes to src/lightning/fabric/fabric.py, src/lightning/fabric/strategies/deepspeed.py, src/lightning/fabric/strategies/fsdp.py, src/lightning/fabric/strategies/strategy.py, src/lightning/fabric/strategies/xla.py.

🟢 link-check

Check ID	Status
check-md-links / markdown-link-check	success	✅

These checks are required after the changes to src/lightning/fabric/CHANGELOG.md.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

tests/tests_fabric/test_fabric.py

docs/source-fabric/guide/checkpoint.rst

src/lightning/fabric/fabric.py

Co-authored-by: Adrian Wälchli <[email protected]>

codecov · 2023-06-19T19:05:16Z

Codecov Report

Merging #17845 (0ec443f) into master (59c7fad) will decrease coverage by 22%.
The diff coverage is 67%.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #17845     +/-   ##
=========================================
- Coverage      83%      61%    -22%     
=========================================
  Files         425      420      -5     
  Lines       32004    31938     -66     
=========================================
- Hits        26603    19492   -7111     
- Misses       5401    12446   +7045

justusschock

LGTM in general. I just think that in 99% of the cases, you'd filter out model parameters, so I'd also add a shortcut where filter=my_callable would be equivalent to filter={"model":my_callable}

src/lightning/fabric/fabric.py

awaelchli

Great! This will be very useful for finetuning with PEFT and in general.

src/lightning/fabric/fabric.py

tests/tests_fabric/strategies/test_deepspeed_integration.py

src/lightning/fabric/strategies/strategy.py

carmocca added 2 commits June 16, 2023 04:10

Add fabric.save(filter=...)

fa52a5f

always empty

bc3454d

carmocca added feature Is an improvement or enhancement checkpointing Related to checkpointing fabric lightning.fabric.Fabric labels Jun 16, 2023

carmocca added this to the 2.1 milestone Jun 16, 2023

carmocca self-assigned this Jun 16, 2023

carmocca added 2 commits June 16, 2023 04:13

CHANGELOG

494e3ee

fsdp fix and mypy

7994d4b

carmocca mentioned this pull request Jun 16, 2023

Use FSDP everywhere Lightning-AI/litgpt#118

Merged

FSDP test

360aef6

carmocca marked this pull request as ready for review June 16, 2023 04:37

carmocca requested review from awaelchli, justusschock, edenlightning, lantiga and Borda as code owners June 16, 2023 04:37

carmocca added 2 commits June 16, 2023 06:38

Merge branch 'master' into carmocca/fabric-save-filter

76d934a

Merge branch 'carmocca/fabric-save-filter' of github.com:Lightning-AI…

c360be5

…/lightning into carmocca/fabric-save-filter

awaelchli reviewed Jun 19, 2023

View reviewed changes

Apply suggestions from code review

eea6824

Co-authored-by: Adrian Wälchli <[email protected]>

carmocca added 4 commits June 20, 2023 16:19

explicit filter keys

fc67ea2

Merge branch 'master' into carmocca/fabric-save-filter

62824bb

comments

5f3196f

Update

7594eed

justusschock approved these changes Jun 20, 2023

View reviewed changes

src/lightning/fabric/fabric.py Show resolved Hide resolved

awaelchli approved these changes Jun 20, 2023

View reviewed changes

src/lightning/fabric/fabric.py Outdated Show resolved Hide resolved

tests/tests_fabric/strategies/test_deepspeed_integration.py Outdated Show resolved Hide resolved

src/lightning/fabric/strategies/strategy.py Outdated Show resolved Hide resolved

mergify bot added the ready PRs ready to be merged label Jun 20, 2023

carmocca added 2 commits June 20, 2023 17:26

Review comments

69c0d6c

Separate deepspeed test

5fee3b7

carmocca enabled auto-merge (squash) June 20, 2023 17:21

carmocca added 2 commits June 20, 2023 19:26

Fix import

3f27f0b

runif

0ec443f

carmocca merged commit 58d2387 into master Jun 20, 2023

carmocca deleted the carmocca/fabric-save-filter branch June 20, 2023 18:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `Fabric.save(filter=...)` #17845

Add `Fabric.save(filter=...)` #17845

carmocca commented Jun 16, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Jun 16, 2023 •

edited

Loading

codecov bot commented Jun 19, 2023 •

edited

Loading

justusschock left a comment

awaelchli left a comment •

edited

Loading

Add Fabric.save(filter=...) #17845

Add Fabric.save(filter=...) #17845

Conversation

carmocca commented Jun 16, 2023 • edited by github-actions bot Loading

What does this PR do?

github-actions bot commented Jun 16, 2023 • edited Loading

⚡ Required checks status: All passing 🟢

Groups summary

codecov bot commented Jun 19, 2023 • edited Loading

Codecov Report

justusschock left a comment

Choose a reason for hiding this comment

awaelchli left a comment • edited Loading

Choose a reason for hiding this comment

Add `Fabric.save(filter=...)` #17845

Add `Fabric.save(filter=...)` #17845

carmocca commented Jun 16, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Jun 16, 2023 •

edited

Loading

codecov bot commented Jun 19, 2023 •

edited

Loading

awaelchli left a comment •

edited

Loading