Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: Symlink blobs from base image instead of copying over. #505

Closed
thesayyn opened this issue Feb 20, 2024 · 4 comments · Fixed by #560
Closed

FR: Symlink blobs from base image instead of copying over. #505

thesayyn opened this issue Feb 20, 2024 · 4 comments · Fixed by #560
Labels
enhancement New feature or request performance
Milestone

Comments

@thesayyn
Copy link
Collaborator

Currently when an oci_image depends on another one, all the blobs from the base oci_image is copied over. This was done for two reasons;

  • Avoid cascading builds.
  • Bazel did not allow symlinks inside treeartifacts that pointed outside of the treeartifact.

With bazelbuild/bazel#21263 fixed, now we can have symlinks inside treeartifacts pointing outside.

We expect this to significantly improve build time of oci_image.

@thesayyn
Copy link
Collaborator Author

thesayyn commented Feb 20, 2024

This will close: #365, #439

@sgammelmark
Copy link

It would be great, if the provided tar-files could be symlinked as well.

I have an aspect the creates individual layers for each external python dependency. Incrementally updating the individual layers on a code change is very fast (less than a second), but updating the image can be time consuming for large images (more than 60 s). In my case, the individual blobs are exactly the same file as the tar-file, so a symlink would be far more efficient than copying the layers.

A missing part of this is maybe that the digest of each layer needs to be computed as well? Having a rule or another mechanism for pre-computing that layer digest so layers can be re-used in different images would be great.

Not sure if it fits this issue exactly, but the symlinking part seems to be relevant.

@thesayyn
Copy link
Collaborator Author

It would be great, if the provided tar-files could be symlinked as well.

We have separate tracking issue for that; #508

@thesayyn
Copy link
Collaborator Author

thesayyn commented May 8, 2024

fixed by #560

@thesayyn thesayyn closed this as completed May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance
Projects
None yet
2 participants