Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: don't produce big tar file as default output of oci_tarball #548

Merged
merged 9 commits into from
May 2, 2024

Conversation

alexeagle
Copy link
Collaborator

@alexeagle alexeagle commented Apr 26, 2024

This is expected to be a big performance improvement because it doesn't overload the remote cache with these files, that only should be needed for local dev workflow of load'ing the image into a container runtime.

Outputs

The default output is an mtree specification file.
This is because producing the tarball in bazel build is expensive, and should typically not be an input to any other build actions,
so producing it only creates unnecessary load on the action cache.

If needed, the tarball output group allows you to depend on the tar output from another rule.

On the command line, bazel build //path/to:my_tarball --output_groups=tarball

or in a BUILD file:

oci_tarball(
    name = "my_tarball",
    ...
)
filegroup(
    name = "my_tarball.tar",
    srcs = [":my_tarball"],
    output_group = "tarball",
)

This is expected to be a big performance improvement because it doesn't overload the remote cache with these files, that only should be needed for local dev workflow of load'ing the image into a container runtime.
@alexeagle
Copy link
Collaborator Author

@thesayyn we have to figure out whether we want to use tags=manual and if so, whether there's any point splitting the mtree action from the tar action

Copy link
Contributor

@illicitonion illicitonion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems pretty reasonable to me!

That forces the big .tar file to be an action output, which is not good
for caching. Instead the docker driver that can load the image is
preferred.
Copy link
Collaborator

@thesayyn thesayyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small nit, looks great!

Instead stream the output of tar as the stdin of docker load
@alexeagle alexeagle merged commit 560e017 into 2.x May 2, 2024
19 of 20 checks passed
@alexeagle alexeagle deleted the bazel-lib_tar branch May 2, 2024 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants