Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal to add Encrypted Layer Mediatype #775

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

lumjjb
Copy link

@lumjjb lumjjb commented Apr 27, 2019

This is a first draft of modifications to the OCI spec on Encrypted Container Images. Work around this has been on-going for quite a while (since the start of the issue). The current design is based on the discussions we've had in the issue and with the runtime implementation in containerd.

Abstract

We would like to propose a new media type for encrypted layers of a container image. This addition would facilitate the ecosystem for encrypted container images. This allows users with stricter trust requirements to be able ensure end-to-end encryption from build to runtime. In addition, it allows users to use a centralized managed repository (i.e. Docker Hub) without any risk of their images being compromised.

Issue Details

Based on the discussion of #747

Implementations

The specification and libraries to perform encryption/decryption

The implementation to perform encryption and decryption of this spec is in:

  • https://github.com/containers/ocicrypt
    • func EncryptLayer(ec *config.EncryptConfig, encOrPlainLayerReader io.Reader, desc ocispec.Descriptor) (io.Reader, EncryptLayerFinalizer, error)
    • func DecryptLayer(dc *config.DecryptConfig, encLayerReader io.Reader, desc ocispec.Descriptor, unwrapOnly bool) (io.Reader, digest.Digest, error)

Container Runtimes

Implemented and upstreamed for containerd stack

Implemented and upstreamed for RedHat stack

Registry

Current docker distribution supports: https://github.com/docker/distribution. Tested with v2.7.1.

Build toolchain

The plan for this is to have docker CLI, skopeo and buildah. In the meantime, we also support encryption of images in containerd via imgcrypt ctr client.

Current implementations for RedHat stack

Current implementations for containerd stack

Kubernetes Integration

It is important to note that kubernetes integration is not required to use encrypted container images. i.e. in cri-o you are able to specify a directory containing keys.

A way to do this is via an operator k8s-enc-image-operator

Signed-off-by: Brandon Lum [email protected]

layer.md Outdated

When using the `+enc` mediatype, the layer data blobs are encrypted with a symmetric encryption algorithm (i.e. AES_128_GCM, AES_128_CBC, etc.) according to the [IANA Registry](https://www.iana.org/assignments/aead-parameters/aead-parameters.xhtml).

Details of the algorithms and protocols used in the encryption of the data blob are defined in a JSON object below. We note that:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm understanding this correctly, this structure ends up in the annotations below. It might be helpful to describe the annotations first, then their contents.

I also think for annotations, they need to be added to the annotations.md.

Copy link
Author

@lumjjb lumjjb Jun 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EDIT: Updated to reflect introduction of annotations with link to annotations.md before metadata explanation.

Agreed on adding to annotations.md.

@mrunalp
Copy link
Contributor

mrunalp commented Sep 12, 2019

@vbatts ptal

@lumjjb
Copy link
Author

lumjjb commented Sep 17, 2019

@vbatts As per our conversation a couple days ago, Based on the RFC6838 and RFC6839.

A specification like nondistributable layers would work:

  • application/vnd.oci.image.layer.enc.tar.v1
  • application/vnd.oci.image.layer.enc.tar.gzip.v1

The other way to do this is to have one encrypted layer mediatype application/vnd.oci.image.layer.enc.v1 and specify the original underlying mediatype in an annotation. But i'm unsure if this may make handling on the runtimes a bit more difficult.

cc: @estesp @dmcgowan @crosbymichael @mrunalp @mtrmac @vrothberg

@dmcgowan
Copy link
Member

I don't think reproducing what was done for nondistributable layers is a good idea. Runtimes end up having to special case these since they are hard to interpret. For example vnd.oci.image.layer. may indicate that an object should be treated as a layer, but then .tar.v1 indicates it is a tar. What is sandwiched in between those is some important metadata. In the nondistributable case it is just a hint of how an artifact is distributed, in the encrypted case, it is a requirement for how the tar must be processed. I think we really need to format the content types in a way that indicates how they should be processed, ensuring that modifications on those types are wrapping. Personally I like using + to indicate encoding (whether that is encryption or formatting). I think we should discuss at an upcoming OCI weekly meeting about these types and their relation to RFC6838.

@lumjjb lumjjb force-pushed the encryption branch 2 times, most recently from b433f02 to fbdd040 Compare September 30, 2019 20:10
@lumjjb
Copy link
Author

lumjjb commented Oct 1, 2019

Updated based on discussions from OCI weekly dev discussion to move from +enc to +encrypted. We agreed on using suffixes on mediatypes and having these defined on top of artifacts.

@lumjjb
Copy link
Author

lumjjb commented Oct 9, 2019

@mikebrow Responding to #791 (comment) in this thread instead since it is more specific to encryption details.

The main integration points for Encrypted Container Images are:

  • The specification and libraries to perform encryption/decryption
  • Container runtimes
  • The build toolchain
  • Registry
  • Kubernetes Integration

Do let me know if there's something specific you are looking for and I can provide a direct link.

The specification and libraries to perform encryption/decryption

The definition of encryption indication and how encryption metadata is defined in:

The implementation to perform encryption and decryption of this spec is in:

  • https://github.com/containers/ocicrypt
    • func EncryptLayer(ec *config.EncryptConfig, encOrPlainLayerReader io.Reader, desc ocispec.Descriptor) (io.Reader, EncryptLayerFinalizer, error)
    • func DecryptLayer(dc *config.DecryptConfig, encLayerReader io.Reader, desc ocispec.Descriptor, unwrapOnly bool) (io.Reader, digest.Digest, error)

Container Runtimes

Implemented and upstreamed for containerd stack

Implemented and upstreamed for RedHat stack

Registry

Current docker distribution supports: https://github.com/docker/distribution. Tested with v2.7.1.

Build toolchain

The plan for this is to have docker CLI, skopeo and buildah. In the meantime, we also support encryption of images in containerd via imgcrypt ctr client.

Current implementations for RedHat stack

Current implementations for containerd stack

Kubernetes Integration

It is important to note that kubernetes integration is not required to use encrypted container images. i.e. in cri-o you are able to specify a directory containing keys.

cc: @stefanberger, @harche

@vbatts
Copy link
Member

vbatts commented Mar 4, 2020

@lumjjb ok, finally discussing this in the weekly call.
Since this encrypted layer discussion has begun, the https://github.com/opencontainers/artifacts has come along. To a large degree, this is another mimetype/mediaType that would be documented there, and allowed as an extension to the image-spec.

There is a lot of good work here in the PR, that should transfer to something like a README for these mime-types in the artifacts repo.
Does this make sense?

@lumjjb
Copy link
Author

lumjjb commented Mar 5, 2020

@lumjjb ok, finally discussing this in the weekly call.

Awesome, sounds good.. I will come by the next OCI call on Wednesday.

There is a lot of good work here in the PR, that should transfer to something like a README for these mime-types in the artifacts repo.
Does this make sense?

Yup! I will create an encryption.md PR to the artifacts repo, and I suppose it would be an edit pointing to that file from opencontainers/artifacts#13 ?

@AkihiroSuda
Copy link
Member

What's current status of this?

@vbatts
Copy link
Member

vbatts commented Feb 5, 2021

@AkihiroSuda it seems to be stalled on opencontainers/artifacts#15

@lumjjb
Copy link
Author

lumjjb commented Feb 8, 2021

As a quick update on this, I've updated the list of links in the comments above.

The encryption support is now part of upstreamed containerd, cri-o, buildah, docker registry and skopeo tools.

@npmccallum
Copy link

@vbatts I'm a bit late to the party. But this has recently come into scope for me.

It isn't clear to me from the various pieces of documentation strewn about whether the integrity protection for a given layer also provides integrity protection for all the layers beneath it.

For example, you have an image with multiple layers:

  • Layer 1
  • Layer 2
  • Layer 3 (encrypted)

Does the integrity protection of layer 3 also include measurements of layers 2 and 1?

If not, it would be possible to modify one of the intermediate layers in order to gain backdoor access to the plaintext.

@lumjjb
Copy link
Author

lumjjb commented Mar 3, 2021

Hi @npmccallum The encryption is only targeting confidentiality, although it has some side effects of integrity on a per layer basis. This was designed to be used in conjunction with Docker Content Trust / Notary / RH Simple signing, they should be able to work hand in hand.

@npmccallum
Copy link

npmccallum commented Mar 3, 2021

@lumjjb If you don't have integrity of the whole, you don't have confidentiality of the layer.

@lumjjb
Copy link
Author

lumjjb commented Mar 3, 2021

If i understand your statement correctly, I believe those signing technologies ensure the integrity of the image manifest, and the integrity of the layers are ensured as being part of the signed manifest hash. This is done as a side effect of layers being content hashes.

@npmccallum
Copy link

@lumjjb Okay. So this is working off the presumption that you have integrity protection for the whole manifest which contains measurements of all the layers. Therefore, there is integrity protection for the whole and confidentiality protection for the parts.

During deployment, do you ensure that the manifest is validly signed and all the layers match their measurements before you decrypt? Or is it possible to decrypt without validating a signature of the manifest and validating the measurement of the layers?

@stefanberger
Copy link

I suspect there is a desire to allow the layering of encrypted layers on top of unencrypted layers. It might be enough to codify:

1. Unencrypted layers `MUST NOT` be built on top of encrypted layers.

2. Encrypted layers `MAY` be built on top of unencrypted layers.

3. All encrypted layers `MUST` use the same encryption key.

4. An encrypted layer `MUST` include the measurements of all previous layers in its AEAD.

@npmccallum Would it not be sufficient to require that the last layer must be encrypted for each key that was used? At that layer the accumulation of the hashes of all previous layers, encrypted or plain, can be enforced.

@lumjjb
Copy link
Author

lumjjb commented Mar 5, 2021

Have not forgotten about this - have allocated some time next week to go through the previous discussions and synthesize them.

@lumjjb
Copy link
Author

lumjjb commented Mar 8, 2021

Below are the goals and usecases/scenarios from the initial proposal

Content encryption of OCI artifacts (Protection at rest from build -> runtime)

All artifacts have their content kept confidential from the build phase to the runtime or whenever the artifact needs to be consumed. This attempts to address the attack vectors between build and runtime. The threat model is that the producer and consumers of the image are trusted. The proposed PR is opencontainers/artifacts#15.

The discussions of this was surrounding the work going around Docker Content Trust / Notary around that time. It was the intention that DCT or other image manifest signing technologies together with the artifact design of being content addressible would be used to attest the integrity and provenance of the payload.

Initial Scope

The initial focus of the ocicrypt project was on the container images. The idea was to focus on the encryption of the content aspects. This consists of the config file and the layers.

The work was later scoped down to only layer artifacts. This was due to the implementation scope as a POC in containerd. The mechanics of introducing this into configs were much more involved than the layers, resulting in the initial scope.

Geofencing execution

Another scenario was towards providing geofencing capabilities. Encrypting content to tie it to key material which can then be securely distributed to an authorized platform. The specific motivator of this usecase was geofencing and compliance, and export control/DRM were other examples.

Keys would be used or distributed with platform attestation-backed authorization. Thus with key management, The images would only be usable if they were on nodes that are authorized. This is talked about a bit more in https://kccnceu20.sched.com/event/Zepc.

Container layer flexibility

One of the main themes for the design on layers was to try to preserve characteristics of container images that people like. This includes ability to build on top of other images, layer reuse, external image scanning.

The decision for encrypting each layer with a generated DEK, and wrapping the key was to be able to provide layer re-use. The wrapped metadata is therefore stored within the annotations and not the layer itself, as authorizing a new consumer would have changed the hash of the layer.

Having the wrapped keys as annotations (these wrapped keys could be artifacts on their own pointing to the layers), would also allow approved image scanners or trusted CI/CD tooling to perform analysis on the layers. Thus one could encrypt an image for both its runtime and a trusted party.

Another notable discussion point was that of multiple owners of different parts of the stack. The idea is one that is difficult to do with current tooling due to the way the builders construct images, but not impossible. For example, if the middleware team maintains the middleware layer and the application team is responsible for the application. An encrypted image could be created by the layering of the middleware encrypted layer, and the application could be
added on top without accessing the middleware, based on a file based contract provided between teams. The runtime could then run the container image only if it is authorized to use components of both teams.

@stefanberger

@@ -40,6 +40,10 @@ This specification defines the following annotation keys, intended for but not l
```
* **org.opencontainers.image.title** Human-readable title of the image (string)
* **org.opencontainers.image.description** Human-readable description of the software packaged in the image (string)
* **org.opencontainers.image.org.opencontainers.image.enc.keys.pkcs7** Base64 comma separated [PKCS7(RFC2315)](https://tools.ietf.org/html/rfc2315) encrypted messages that contain wrapped PrivateLayerBlockCipherOptions JSON objects for [encryption](layer.md#layer-encryption)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* **org.opencontainers.image.org.opencontainers.image.enc.keys.pkcs7** Base64 comma separated [PKCS7(RFC2315)](https://tools.ietf.org/html/rfc2315) encrypted messages that contain wrapped PrivateLayerBlockCipherOptions JSON objects for [encryption](layer.md#layer-encryption)
* **org.opencontainers.image.enc.keys.pkcs7** Base64 comma separated [PKCS7(RFC2315)](https://tools.ietf.org/html/rfc2315) encrypted messages that contain wrapped PrivateLayerBlockCipherOptions JSON objects for [encryption](layer.md#layer-encryption)

@vbatts
Copy link
Member

vbatts commented Jul 9, 2021

@lumjjb would like to bubble this work up. What is the current situation and your approach? has it evolved since you first opened this PR? I'm thinking this discussion could start up again.

@lumjjb
Copy link
Author

lumjjb commented Jul 21, 2021

Hey @vbatts ! Certainly , for some reason this was buried in my inbox and surfaced when i was searching for something! Would like to get back to this discussion, where shall be a good place to do this? Should we chat on the sidebar first or bring this up at one of the weekly oci calls?

@vbatts
Copy link
Member

vbatts commented Aug 23, 2021 via email

@lumjjb
Copy link
Author

lumjjb commented Aug 23, 2021

Awesome - i should be able to come to this week's or next week's meeting.

@AkihiroSuda
Copy link
Member

needs rebase

Signed-off-by: Brandon Lum <[email protected]>
@lumjjb lumjjb force-pushed the encryption branch 2 times, most recently from 2ea9e17 to 3d8cfbe Compare October 27, 2021 13:35
layer.md Outdated Show resolved Hide resolved
Signed-off-by: Brandon Lum <[email protected]>
@@ -4,7 +4,7 @@ This document describes how to serialize a filesystem and filesystem changes lik
One or more layers are applied on top of each other to create a complete filesystem.
This document will use a concrete example to illustrate how to create and consume these filesystem layers.

This section defines the `application/vnd.oci.image.layer.v1.tar`, `application/vnd.oci.image.layer.v1.tar+gzip`, `application/vnd.oci.image.layer.v1.tar+zstd`, `application/vnd.oci.image.layer.nondistributable.v1.tar`, `application/vnd.oci.image.layer.nondistributable.v1.tar+gzip`, and `application/vnd.oci.image.layer.nondistributable.v1.tar+zstd` [media types](media-types.md).
This section defines the `application/vnd.oci.image.layer.v1.tar`, `application/vnd.oci.image.layer.v1.tar+gzip`, `application/vnd.oci.image.layer.v1.tar+zstd`, `application/vnd.oci.image.layer.nondistributable.v1.tar`, `application/vnd.oci.image.layer.nondistributable.v1.tar+gzip`, `application/vnd.oci.image.layer.nondistributable.v1.tar+zstd`, `application/vnd.oci.image.layer.v1.tar+encrypted`, `application/vnd.oci.image.layer.v1.tar+gzip+encrypted`, `application/vnd.oci.image.layer.nondistributable.v1.tar+encrypted` and `application/vnd.oci.image.layer.nondistributable.v1.tar+gzip+encrypted` [media types](media-types.md).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask a question why tar+ encrypted + zstd or tar + zstd + encrypted is not supported? Thanks.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at the time, there wasn't +zstd yet, but yes it should support. +encrypted should be treated as a suffix and thus should be handled on all mediatypes that end with +encrypted

@lumjjb
Copy link
Author

lumjjb commented Sep 21, 2022 via email

@zvonkok
Copy link

zvonkok commented Oct 28, 2022

/cc @zvonkok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.