Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System customization syncing for jobs run #16932

Merged
merged 14 commits into from
Mar 9, 2023
Merged

System customization syncing for jobs run #16932

merged 14 commits into from
Mar 9, 2023

Conversation

dmitsf
Copy link
Member

@dmitsf dmitsf commented Mar 2, 2023 •
edited by github-actions bot
Loading

What does this PR do?

Allows to copy system and conda environment sync files to an app tarball (to restore the environment before the app is running). This is a feature within the scope of work on running jobs directly from CloudSpaces.

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @Borda

@github-actions github-actions bot added the app (removed) Generic label for Lightning App package label Mar 2, 2023
ethanwharris
ethanwharris reviewed Mar 3, 2023
View reviewed changes
Copy link
Member

@ethanwharris ethanwharris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good

src/lightning/app/runners/cloud.py Outdated Show resolved Hide resolved
@dmitsf dmitsf changed the title [WIP] System customization syncing for jobs run System customization syncing for jobs run Mar 8, 2023
@dmitsf dmitsf requested a review from ethanwharris March 8, 2023 20:11
@dmitsf dmitsf marked this pull request as ready for review March 8, 2023 20:11
@github-actions
Copy link
Contributor

github-actions bot commented Mar 8, 2023 •
edited
Loading

⚡ Required checks status: All passing 🟢

Groups summary

🟢 lightning_app: Tests workflow
Check ID Status
app-pytest (macOS-11, lightning, 3.8, latest) success ✅
app-pytest (macOS-11, lightning, 3.8, oldest) success ✅
app-pytest (macOS-11, app, 3.9, latest) success ✅
app-pytest (ubuntu-20.04, lightning, 3.8, latest) success ✅
app-pytest (ubuntu-20.04, lightning, 3.8, oldest) success ✅
app-pytest (ubuntu-20.04, app, 3.9, latest) success ✅
app-pytest (windows-2022, lightning, 3.8, latest) success ✅
app-pytest (windows-2022, lightning, 3.8, oldest) success ✅
app-pytest (windows-2022, app, 3.8, latest) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/runners/cloud.py, src/lightning/app/source_code/local.py, tests/tests_app/runners/test_cloud.py.

🟢 lightning_app: Examples
Check ID Status
app-examples (macOS-11, lightning, 3.9, latest) success ✅
app-examples (macOS-11, lightning, 3.9, oldest) success ✅
app-examples (macOS-11, app, 3.9, latest) success ✅
app-examples (ubuntu-20.04, lightning, 3.9, latest) success ✅
app-examples (ubuntu-20.04, lightning, 3.9, oldest) success ✅
app-examples (ubuntu-20.04, app, 3.9, latest) success ✅
app-examples (windows-2022, lightning, 3.9, latest) success ✅
app-examples (windows-2022, lightning, 3.9, oldest) success ✅
app-examples (windows-2022, app, 3.9, latest) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/runners/cloud.py, src/lightning/app/source_code/local.py.

🟢 lightning_app: Azure
Check ID Status
App.cloud-e2e success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/runners/cloud.py, src/lightning/app/source_code/local.py.

🟢 lightning_app: Docs
Check ID Status
make-doctest (app) success ✅
make-html (app) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/runners/cloud.py, src/lightning/app/source_code/local.py.

🟢 mypy
Check ID Status
mypy success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/runners/cloud.py, src/lightning/app/source_code/local.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.8) success ✅
install-pkg (ubuntu-22.04, app, 3.10) success ✅
install-pkg (ubuntu-22.04, fabric, 3.8) success ✅
install-pkg (ubuntu-22.04, fabric, 3.10) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.8) success ✅
install-pkg (ubuntu-22.04, pytorch, 3.10) success ✅
install-pkg (ubuntu-22.04, lightning, 3.8) success ✅
install-pkg (ubuntu-22.04, lightning, 3.10) success ✅
install-pkg (ubuntu-22.04, notset, 3.8) success ✅
install-pkg (ubuntu-22.04, notset, 3.10) success ✅
install-pkg (macOS-12, app, 3.8) success ✅
install-pkg (macOS-12, app, 3.10) success ✅
install-pkg (macOS-12, fabric, 3.8) success ✅
install-pkg (macOS-12, fabric, 3.10) success ✅
install-pkg (macOS-12, pytorch, 3.8) success ✅
install-pkg (macOS-12, pytorch, 3.10) success ✅
install-pkg (macOS-12, lightning, 3.8) success ✅
install-pkg (macOS-12, lightning, 3.10) success ✅
install-pkg (macOS-12, notset, 3.8) success ✅
install-pkg (macOS-12, notset, 3.10) success ✅
install-pkg (windows-2022, app, 3.8) success ✅
install-pkg (windows-2022, app, 3.10) success ✅
install-pkg (windows-2022, fabric, 3.8) success ✅
install-pkg (windows-2022, fabric, 3.10) success ✅
install-pkg (windows-2022, pytorch, 3.8) success ✅
install-pkg (windows-2022, pytorch, 3.10) success ✅
install-pkg (windows-2022, lightning, 3.8) success ✅
install-pkg (windows-2022, lightning, 3.10) success ✅
install-pkg (windows-2022, notset, 3.8) success ✅
install-pkg (windows-2022, notset, 3.10) success ✅

These checks are required after the changes to src/lightning/app/core/constants.py, src/lightning/app/runners/cloud.py, src/lightning/app/source_code/local.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

ethanwharris
ethanwharris approved these changes Mar 8, 2023
View reviewed changes
Copy link
Member

@ethanwharris ethanwharris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 😃

@dmitsf dmitsf self-assigned this Mar 8, 2023
awaelchli
awaelchli reviewed Mar 8, 2023
View reviewed changes
src/lightning/app/core/constants.py Show resolved Hide resolved
tests/tests_app/runners/test_cloud.py Show resolved Hide resolved
src/lightning/app/runners/cloud.py Outdated Show resolved Hide resolved
@dmitsf dmitsf requested a review from awaelchli March 8, 2023 22:52
@dmitsf dmitsf force-pushed the dmitsf/sync_env_for_jobs branch from 692afe2 to 2c515a3 Compare March 8, 2023 23:07
@awaelchli awaelchli added this to the v1.9.x milestone Mar 9, 2023
@mergify mergify bot added the ready PRs ready to be merged label Mar 9, 2023
@dmitsf dmitsf enabled auto-merge (squash) March 9, 2023 19:35
@dmitsf dmitsf merged commit 4a531b2 into master Mar 9, 2023
@dmitsf dmitsf deleted the dmitsf/sync_env_for_jobs branch March 9, 2023 19:35
Borda pushed a commit that referenced this pull request Mar 30, 2023
@dmitsf @Borda
System customization syncing for jobs run (#16932)
e5dd401 
* System customization syncing for jobs run

* Constants moved

* Moving files

* Update src/lightning/app/runners/cloud.py

Co-authored-by: Adrian Wälchli <[email protected]>

* Update test_cloud.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test_cloud.py

---------

Co-authored-by: Adrian Wälchli <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

(cherry picked from commit 4a531b2)
lantiga pushed a commit that referenced this pull request Apr 3, 2023
@dmitsf @lantiga
System customization syncing for jobs run (#16932)
94af83b 
* System customization syncing for jobs run

* Constants moved

* Moving files

* Update src/lightning/app/runners/cloud.py

Co-authored-by: Adrian Wälchli <[email protected]>

* Update test_cloud.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test_cloud.py

---------

Co-authored-by: Adrian Wälchli <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

(cherry picked from commit 4a531b2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ethanwharris ethanwharris ethanwharris approved these changes

@awaelchli awaelchli awaelchli approved these changes

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

@lantiga lantiga Awaiting requested review from lantiga lantiga is a code owner

@hhsecond hhsecond Awaiting requested review from hhsecond

Assignees

@dmitsf dmitsf

Labels
app (removed) Generic label for Lightning App package ready PRs ready to be merged
Projects
None yet
Milestone
v1.9.x
Development

Successfully merging this pull request may close these issues.
3 participants
@dmitsf @awaelchli @ethanwharris