-
-
Notifications
You must be signed in to change notification settings - Fork 338
Support OCI index images (such as produced by buildkit cache exports) #227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Hello, thank you for using my project 😄 Hum, interesting, so I should add this header here docker-registry-ui/src/scripts/docker-image.js Lines 132 to 136 in db6b74a
|
Yes I belive so. The manigests are a bit different, but just for extracting the sha from the header to support delete should work the same I think. |
I am also looking forward to have support for cache manifest (view and delete) |
Hi there, I tried Buildkit with Docker Registry UI. It seems like you are using this kind of command with buildkit:
The use of You can compare with How to fix your issue ? I suggest you to use the option
Is this possible for you or do you have any limitations that prevent you from using |
Those are two different things. I'm using the --output as well as for the actual image that I use for creating containers when running the apps. The --export-cache is for the build cache which I need exported to have a cluster of build agents reuse the build cache. The inline build bache option does not include the cache for all intermediate build stages in a multi stage build as far as I can tell. So thats why we're using the separe cache image. Not sure what you mean with the images being incompatible with the docker registry. The registry backend seems to handle them just fine and buildkit can pull the cache down again when building. |
In the UI, in order to retrieve all information for a image I need two endpoints from the Docker Registry API. GET /v2/<image name>/manifests/<tag name>
Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json As you can see, I already accept manifest schema from docker Click to expand Manifest Result{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"digest": "sha256:4e459fee8dd03c413580dfc6e9caf6f5d294dc1d1cd857a183b696daff15e81c",
"size": 10038
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3",
"size": 2818413
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:8d6ba530f6489d12676d7f61628427d067243ba4a3a512c3e28813b977cb3b0e",
"size": 7341319
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:5288d7ad7a7f84bdd19c1e8f0abb8684b5338f3da86fe9ae1d7f0e9bc2de6595",
"size": 601
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:39e51c61c033442d00c40a30b2a9ed01f40205875fbd8664c50b4dc3e99ad5cf",
"size": 894
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:ee6f71c6f4a82b2afd01f92bdf6be0079364d03020e8a2c569062e1c06d3822b",
"size": 665
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:f2303c6c88653b9a6739d50f611c170b9d97d161c6432409c680f6b46a5f112f",
"size": 1390
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
"size": 32
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:30ad6d18caaa2d3ca5af9e5426c56d0cf65c16f89944c7722da7e7373a9577a3",
"size": 975
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:890316ae144ef726d6ddfa31eb691f01aaec354aa8f4f74a0350e8189f8da8dc",
"size": 843
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:e02b12b1d987d9b8755c313e846d29866f62185b88f2cab907517531581b6b84",
"size": 874351
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:829b142c0a44d7553c30f1fdb476c074be70181e441e909de0a3f9497ffdbb57",
"size": 18338
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"digest": "sha256:9bf51e297d370f9436b6866ea120a1e4bd2962786017edb5875315eaa4066d97",
"size": 896185
}
]
} From this JSON I will sum GET /v2/<image name>/blobs/<config.digest string>
Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json This blob contains important metadata such as creation date for the tag list page and all the information to create the history page (Labels, Environments, Exposed Port, Image Stages....). Without this object, my UI can't work properly. You can see a response example below. Click to expand Blob Result{
"architecture": "amd64",
"config": {
"ExposedPorts": {
"80/tcp": {}
},
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"NGINX_VERSION=1.21.6",
"NJS_VERSION=0.7.2",
"PKG_RELEASE=1",
"NGINX_PROXY_HEADER_Host=$http_host",
"NGINX_LISTEN_PORT=80"
],
"Entrypoint": [
"/docker-entrypoint.sh"
],
"Cmd": [
"nginx",
"-g",
"daemon off;"
],
"WorkingDir": "/usr/share/nginx/html/",
"Labels": {
"maintainer": "Jones MAGLOIRE @Joxit"
},
"StopSignal": "SIGQUIT",
"OnBuild": null
},
"created": "2022-03-13T08:13:28.458764628+01:00",
"history": [
{
"created": "2021-11-24T20:19:40.199700946Z",
"created_by": "/bin/sh -c #(nop) ADD file:9233f6f2237d79659a9521f7e390df217cec49f1a8aa3a12147bbca1956acdb9 in / "
},
{
"created": "2021-11-24T20:19:40.483367546Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
"empty_layer": true
},
{
"created": "2021-12-29T19:29:04.16538557Z",
"created_by": "/bin/sh -c #(nop) LABEL maintainer=NGINX Docker Maintainers <[email protected]>",
"empty_layer": true
},
{
"created": "2022-01-25T20:38:57.564847523Z",
"created_by": "/bin/sh -c #(nop) ENV NGINX_VERSION=1.21.6",
"empty_layer": true
},
{
"created": "2022-01-25T20:38:57.780944464Z",
"created_by": "/bin/sh -c #(nop) ENV NJS_VERSION=0.7.2",
"empty_layer": true
},
{
"created": "2022-01-25T20:38:58.001318895Z",
"created_by": "/bin/sh -c #(nop) ENV PKG_RELEASE=1",
"empty_layer": true
},
{
"created": "2022-01-25T20:39:07.536319062Z",
"created_by": "/bin/sh -c set -x ...."
},
{
"created": "2022-01-25T20:39:08.151512909Z",
"created_by": "/bin/sh -c #(nop) COPY file:65504f71f5855ca017fb64d502ce873a31b2e0decd75297a8fb0a287f97acf92 in / "
},
{
"created": "2022-01-25T20:39:08.618949282Z",
"created_by": "/bin/sh -c #(nop) COPY file:0b866ff3fc1ef5b03c4e6c8c513ae014f691fb05d530257dfffd07035c1b75da in /docker-entrypoint.d "
},
{
"created": "2022-01-25T20:39:09.304128562Z",
"created_by": "/bin/sh -c #(nop) COPY file:0fd5fca330dcd6a7de297435e32af634f29f7132ed0550d342cad9fd20158258 in /docker-entrypoint.d "
},
{
"created": "2022-01-25T20:39:09.813218835Z",
"created_by": "/bin/sh -c #(nop) COPY file:09a214a3e07c919af2fb2d7c749ccbc446b8c10eb217366e5a65640ee9edcc25 in /docker-entrypoint.d "
},
{
"created": "2022-01-25T20:39:10.15290576Z",
"created_by": "/bin/sh -c #(nop) ENTRYPOINT [\"/docker-entrypoint.sh\"]",
"empty_layer": true
},
{
"created": "2022-01-25T20:39:10.498199739Z",
"created_by": "/bin/sh -c #(nop) EXPOSE 80",
"empty_layer": true
},
{
"created": "2022-01-25T20:39:10.860347544Z",
"created_by": "/bin/sh -c #(nop) STOPSIGNAL SIGQUIT",
"empty_layer": true
},
{
"created": "2022-01-25T20:39:11.21594189Z",
"created_by": "/bin/sh -c #(nop) CMD [\"nginx\" \"-g\" \"daemon off;\"]",
"empty_layer": true
},
{
"created": "2022-03-13T08:13:28.237041716+01:00",
"created_by": "LABEL maintainer=Jones MAGLOIRE @Joxit",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2022-03-13T08:13:28.237041716+01:00",
"created_by": "WORKDIR /usr/share/nginx/html/",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2022-03-13T08:13:28.237041716+01:00",
"created_by": "ENV NGINX_PROXY_HEADER_Host=$http_host",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2022-03-13T08:13:28.237041716+01:00",
"created_by": "ENV NGINX_LISTEN_PORT=80",
"comment": "buildkit.dockerfile.v0",
"empty_layer": true
},
{
"created": "2022-03-13T08:13:28.292012782+01:00",
"created_by": "COPY nginx/default.conf /etc/nginx/conf.d/default.conf # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2022-03-13T08:13:28.309117311+01:00",
"created_by": "COPY bin/entrypoint /docker-entrypoint.d/90-docker-registry-ui.sh # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2022-03-13T08:13:28.336973324+01:00",
"created_by": "COPY dist/ /usr/share/nginx/html/ # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2022-03-13T08:13:28.35325107+01:00",
"created_by": "COPY favicon.ico /usr/share/nginx/html/ # buildkit",
"comment": "buildkit.dockerfile.v0"
},
{
"created": "2022-03-13T08:13:28.458764628+01:00",
"created_by": "RUN /bin/sh -c chown -R nginx:nginx /etc/nginx/ /usr/share/nginx/html/ /var/cache/nginx # buildkit",
"comment": "buildkit.dockerfile.v0"
}
],
"moby.buildkit.buildinfo.v1": "eyJmcm9udGVuZCI6ImRvY2tlcmZpbGUudjAiLCJzb3VyY2VzIjpbeyJ0eXBlIjoiZG9ja2VyLWltYWdlIiwicmVmIjoiZG9ja2VyLmlvL2xpYnJhcnkvbmdpbng6YWxwaW5lIiwicGluIjoic2hhMjU2OmRhOWM5NGJlYzFkYTgyOWViZDUyNDMxYTg0NTAyZWM0NzFjOGU1NDhmZmIyY2VkYmYzNjI2MGZkOWJkMWQ0ZDMifV19",
"os": "linux",
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:8d3ac3489996423f53d6087c81180006263b79f206d3fdec9e66f0e27ceb8759",
"sha256:67bae81de3dc5ccf5101dd4973e8e4b10ee3d5681feb9c30119e6acee1e12dd5",
"sha256:89f4d03665cee1ca943d0b55c086ea7577f4658c2cabe69eaf640f1dc47129f9",
"sha256:318191938fd7801d80ad617482e4312f0ced71e6c1d4bcb02e12d90a9d65eb11",
"sha256:a770f8eba3cb3840d7579e2c4925c9fa275813e8719fe5435905b887a9ee7674",
"sha256:6fda88393b8b02872c87c6626807e7bfac1f49e63f47a5ddba0cd11fae99b575",
"sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
"sha256:16eb951e9ea9c0ab11e5af087c762528f85e6b216a7c56ef003805439ea5f90f",
"sha256:1a1db89726971b65895b495442ed10b5df0d4c7e90cbb943d28b4fce770257cb",
"sha256:5c8eff90fb43c7dbe872fdf130dc51ab8828cd9d461af9944c3ad8ba4286301a",
"sha256:18749a2b685b525bef320c94d198d7b40fc83a08bc1d951cc1e3d3b121e2ff92",
"sha256:254c4dc7aefcff4cca8b682e73b9297707f6d8874626a167a5cb8ac23149d12e"
]
}
} Now, when I add the GET /v2/<image name>/manifests/<tag name>
Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json This new JSON do not have the Click to expand OCI Index Result{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:30ad6d18caaa2d3ca5af9e5426c56d0cf65c16f89944c7722da7e7373a9577a3",
"size": 975,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:28.292012782+01:00",
"containerd.io/uncompressed": "sha256:16eb951e9ea9c0ab11e5af087c762528f85e6b216a7c56ef003805439ea5f90f"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:39e51c61c033442d00c40a30b2a9ed01f40205875fbd8664c50b4dc3e99ad5cf",
"size": 894,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:25.930661922+01:00",
"containerd.io/uncompressed": "sha256:318191938fd7801d80ad617482e4312f0ced71e6c1d4bcb02e12d90a9d65eb11"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
"size": 32,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:28.237041716+01:00",
"containerd.io/uncompressed": "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:5288d7ad7a7f84bdd19c1e8f0abb8684b5338f3da86fe9ae1d7f0e9bc2de6595",
"size": 601,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:25.922233753+01:00",
"containerd.io/uncompressed": "sha256:89f4d03665cee1ca943d0b55c086ea7577f4658c2cabe69eaf640f1dc47129f9"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3",
"size": 2818413,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:25.907648369+01:00",
"containerd.io/uncompressed": "sha256:8d3ac3489996423f53d6087c81180006263b79f206d3fdec9e66f0e27ceb8759"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:829b142c0a44d7553c30f1fdb476c074be70181e441e909de0a3f9497ffdbb57",
"size": 18338,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:28.35325107+01:00",
"containerd.io/uncompressed": "sha256:18749a2b685b525bef320c94d198d7b40fc83a08bc1d951cc1e3d3b121e2ff92"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:890316ae144ef726d6ddfa31eb691f01aaec354aa8f4f74a0350e8189f8da8dc",
"size": 843,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:28.309117311+01:00",
"containerd.io/uncompressed": "sha256:1a1db89726971b65895b495442ed10b5df0d4c7e90cbb943d28b4fce770257cb"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:8d6ba530f6489d12676d7f61628427d067243ba4a3a512c3e28813b977cb3b0e",
"size": 7341319,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:25.913689097+01:00",
"containerd.io/uncompressed": "sha256:67bae81de3dc5ccf5101dd4973e8e4b10ee3d5681feb9c30119e6acee1e12dd5"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:9bf51e297d370f9436b6866ea120a1e4bd2962786017edb5875315eaa4066d97",
"size": 896185,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:28.458764628+01:00",
"containerd.io/uncompressed": "sha256:254c4dc7aefcff4cca8b682e73b9297707f6d8874626a167a5cb8ac23149d12e"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:e02b12b1d987d9b8755c313e846d29866f62185b88f2cab907517531581b6b84",
"size": 874351,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:28.336973324+01:00",
"containerd.io/uncompressed": "sha256:5c8eff90fb43c7dbe872fdf130dc51ab8828cd9d461af9944c3ad8ba4286301a"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:ee6f71c6f4a82b2afd01f92bdf6be0079364d03020e8a2c569062e1c06d3822b",
"size": 665,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:25.938244858+01:00",
"containerd.io/uncompressed": "sha256:a770f8eba3cb3840d7579e2c4925c9fa275813e8719fe5435905b887a9ee7674"
}
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:f2303c6c88653b9a6739d50f611c170b9d97d161c6432409c680f6b46a5f112f",
"size": 1390,
"annotations": {
"buildkit/createdat": "2022-03-13T08:13:25.946044285+01:00",
"containerd.io/uncompressed": "sha256:6fda88393b8b02872c87c6626807e7bfac1f49e63f47a5ddba0cd11fae99b575"
}
},
{
"mediaType": "application/vnd.buildkit.cacheconfig.v0",
"digest": "sha256:6482ebe3fa00365be4c1b86a89c6620624e5a6af53dc8aa845237f3f23143e4a",
"size": 2809
}
]
} It totally makes sense because the FYI, So if you think having the interface without a creation date or history for this kind of images is fine with you, I can do it anyway. Otherwise, I would say that it is simply not compatible with my project. |
Thanks for the detailed investigation! I think I understand now, and as you say most of the data under the history view does not apply to this kind of "image". If that could be supported then that would be great! :) |
Fair enough, I will add the support then 😄 |
Hi there, this will be available in the next release and is already available with the |
I have configured a buildkit daemon to use our private docker registry as a cache registry. The pushed cache images does not appear corrent in the GUI. See screenshot.
I have used curl to try and list the maifest and then I got the following response
"OCI index found, but accept header does not support OCI indexes"
This error comes from here:
https://github.com/distribution/distribution/blob/5f1974ab8b537b5bbce467971b05ee62c441fd7a/registry/handlers/manifests.go#L174
From this page:
https://github.com/opencontainers/image-spec/blob/main/media-types.md#oci-image-media-types
I concluded that I must use "application/vnd.oci.image.index.v1+json" in the Accept header.
With this value appended to the curl command I was able to list the image and also use the 'Docker-Content-Digest' response header to delete the image using curl.
It would be great if that header can be added to this UI to support these image types as well.
The text was updated successfully, but these errors were encountered: