Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support preloading container images into Bottlerocket data volumes with Karpenter #587

Merged
merged 29 commits into from
Aug 21, 2024

Conversation

lindarr915
Copy link
Contributor

@lindarr915 lindarr915 commented Jul 16, 2024

What does this PR do?

🛑 Please open an issue first to discuss any significant work and flesh out details/direction - we would hate for your time to be wasted.
Consult the CONTRIBUTING guide for submitting pull-requests.

Motivation

Reduce the cold start time for ray worker pods.
By preloading container images into the bottlerocket OS data volume, ~10 mins can be saved for container startup time as it is not necessary to download and extract the container images layers.

#559

More

  • Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
  • Mandatory for new blueprints. Yes, I have added a example to support my blueprint PR
  • Mandatory for new blueprints. Yes, I have updated the website/docs or website/blog section for this feature
  • Yes, I ran pre-commit run -a with this PR. Link for installing pre-commit locally

For Moderators

  • E2E Test successfully complete before merge?

Additional Notes

@lindarr915 lindarr915 changed the title Support Preloading Container Images into Bottlerocket Data Disks with Karpenter feat: support preloading container images into Bottlerocket data volumes with Karpenter Jul 16, 2024
@lindarr915 lindarr915 changed the title feat: support preloading container images into Bottlerocket data volumes with Karpenter feat: Support preloading container images into Bottlerocket data volumes with Karpenter Jul 17, 2024
Copy link
Collaborator

@askulkarni2 askulkarni2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lindarr915 thanks for the PR. Approach looks good but lets fix the karpenter helm chart to accept the snapshot_id instead of introducing a breaking change this way.

* feat: run GPU node with BR and EBS snapshot with container image cache

* refactor: remove kubectl_manifest of karpenter custom resources

* feat: locust file fo load testing

* feat: End-to-end deployment of Bottlerocket nodes with container image cache
@lindarr915
Copy link
Contributor Author

Move to draft

@lindarr915 lindarr915 marked this pull request as draft August 2, 2024 14:33
@lindarr915 lindarr915 marked this pull request as ready for review August 13, 2024 08:09
@lindarr915 lindarr915 requested a review from askulkarni2 August 13, 2024 08:11
ai-ml/jark-stack/preload-container-image-ami/README.md Outdated Show resolved Hide resolved
ai-ml/jark-stack/terraform/README.md Outdated Show resolved Hide resolved
ai-ml/jark-stack/terraform/addons.tf Outdated Show resolved Hide resolved
ai-ml/jark-stack/terraform/install.sh Outdated Show resolved Hide resolved
ai-ml/jark-stack/terraform/karpenter.tf Outdated Show resolved Hide resolved
website/docs/gen-ai/inference/stablediffusion-gpus.md Outdated Show resolved Hide resolved
@lindarr915 lindarr915 requested a review from askulkarni2 August 19, 2024 15:40
@lindarr915
Copy link
Contributor Author

@askulkarni2 Feel free to to review again

Copy link
Collaborator

@askulkarni2 askulkarni2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🥳

@askulkarni2 askulkarni2 merged commit 4631cfe into awslabs:main Aug 21, 2024
36 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants