Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add config data in image manifest #757

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

chenhunter
Copy link

This PR adds the data field to the config of the manifest in the image of oci_image.

Fixes #756

@thesayyn
Copy link
Collaborator

thesayyn commented Jan 20, 2025

I think the right behavior is to just drop that field as its not in the spec.

EDIT: Ha, it is actually in the spec! https://github.com/opencontainers/image-spec/blob/main/descriptor.md#properties

However, i am still not sure if we should keep the data field, we could just drop it and it would work fine.

@chenhunter
Copy link
Author

chenhunter commented Feb 6, 2025

I originally thought it wasn't in the spec either, but had to look again as well. Turns out it's been in the spec for year!

Dropping the field definitely is a solution, but considering it's in the spec, and the work is already done in the pull request, I don't see a good reason to NOT just add it? Unless the implementation is wrong or doesn't follow best practices. If so, please advise.

@thesayyn
Copy link
Collaborator

thesayyn commented Feb 6, 2025

Still, though, i think we should only inline the config if we see that the base has it too.

plobsing added a commit to plobsing/rules_oci that referenced this pull request Feb 11, 2025
The [OCI spec](https://github.com/opencontainers/image-spec/blob/main/descriptor.md#properties)
provides a `data` field in order to embed a copy of an image's config
directly into that image's manifest:

> data string
>
> This OPTIONAL property contains an embedded representation of the referenced content. Values MUST conform to the Base 64 encoding, as defined in RFC 4648. The decoded data MUST be identical to the referenced content and SHOULD be verified against the digest and size fields by content consumers. See Embedded Content for when this is appropriate.

This is [intended as an optimization](https://github.com/opencontainers/image-spec/blob/main/descriptor.md#embedded-content),
in order to reduce the number of network round-trips incurred to pull a
container from a remote repository.

However, as highlighted in the release announcement for the new
[`data` field](https://opencontainers.org/posts/blog/2024-03-13-image-and-distribution-1-1/#data-field),
the contexts in which an embedded config is beneficial can be a
subtle determination. Further, in the same announcement, the spec
clarified that registry implementations usually enforce a
[maximum size](https://opencontainers.org/posts/blog/2024-03-13-image-and-distribution-1-1/#manifest-maximum-size)
on manifests, an obscure limit which incautious use of the `data` field
can trigger.

Given the trade-offs dropping any embedded data (e.g. that might have
been propagated from a base image) seems the preferable option. The
alternative — updating embedded data to match the config on every
change, as proposed by bazel-contrib#757 — is
less universally correct and only sometimes more performant.
Either option would satisfy bazel-contrib#756 .
@plobsing
Copy link
Contributor

plobsing commented Feb 11, 2025

Just ran into this issue today. IMO the data field should be dropped by default, my rationale is provided in the description of #774 where I propose this behaviour. I think it should not be too difficult to post-process an oci_image to add an embedded config in cases where users have determined that this is a correct and desirable optimization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

oci_image inherits data field from config in base image
3 participants