[RFC] Allow mounts to be defined for all steps on docker build CLI

[sudo-bmitch]

This is an alternate take on #1209. I'd like a way to inject mounts into my build that apply to all RUN steps that do not appear in the final image. E.g. I would like the ability to say:

docker build --mount type=bind,target=/etc/ssl/certs/ca-certificates.crt,source=ca.crt ...

This would allow me to inject local build configurations, like a custom CA certificate along with HTTPS proxy settings, that do not apply to other build environments and therefore shouldn't be included in the Dockerfile. I'm looking to use this functionality to enable reproducible builds for specific use cases, MitM https requests and replaying a previous builds responses.

As a second advantage, it would allow me to leverage buildkit specific performance improvements without modifying the Dockerfile. That allows me to distribute a single Dockerfile with my projects and build unmodified 3rd party Dockerfiles with these features.

[rittneje]

What defines "the final image"? If you essentially mean the target stage, then that would cause a problem if you ever intentionally target some intermediate stage (perhaps to debug an issue) and suddenly it builds completely differently. Or did you mean "mounts that don't appear in the final image" and not "RUN steps that do not appear in the final image"?

For env vars, why not useARG for this instead? BTW, assuming buildkit follows the docker spec properly, HTTP_PROXY can get forwarded fairly easily. https://docs.docker.com/engine/reference/builder/#predefined-args

[sudo-bmitch]

I mean that the mounts do not appear in the image, effectively the same as adding a --mount to every run step in the build.

How would you modify the trusted root certificates with a build arg in a way that does not modify the filesystem and does not require changes to the Dockerfile? The proxy settings themselves I would change with the build arg, but if you intercept https traffic and MitM it, you must inject custom root certs.

[rittneje]

Yes, the need for the custom root of trust does seem like a blind spot in docker's HTTPS_PROXY magic. I'm not sure how people have been using it historically, other than maybe just COPYing over top of /etc/ssl/certs/ca-certificates.crt or whatever.

I will also note that recently, support was added to create a secret from an environment variable in the buildctl build command. And (I think?) support is coming to automatically mount a secret directly into an environment variable (#2122). So that probably covers things like HTTPS_PROXY better.

As a hypothetical:

buildctl build \
    --secret id=https_proxy,env=HTTPS_PROXY \
    --automount type=secret,id=https_proxy,env=HTTPS_PROXY \
    --automount type=bind,source=proxy.crt,target=/etc/ssl/certs/ca-certificates.crt

So very similar to your original proposal. Although I'm not certain if allowing auto-mounted bind mounts is the right approach, or if we should just make that a mounted secret too (even though it technically isn't secret). It is also worth considering whether this mechanism should be flexible enough to allow for options other than --mount, even though I don't think any exist today.

[sudo-bmitch]

Others either don't MitM the https traffic, or they are building internal images and add the certificate. My use case is a bit different than normal because I'm making a proxy per build that shouldn't be used outside of the build.

The data I'm adding isn't really a secret, but I'm sure others will have that requirement. I can't think of any other options I'd want to set this way, but if we wanted to be more generic, I'd be okay with:

docker build --runext flag=mount,type=bind,target=/etc/ssl/certs/ca-certificates.crt,source=ca.crt ...

Or something similar, that just needs to ensure no flags define the flag field. Given the options, I'd lean towards --mount and let other settings get their own flag if and when needed.

[sudo-bmitch]

Related: opencontainers/image-spec#843

[AkihiroSuda]

This is also covered by:

• Proposal: hooks for RUN instructions (use cases: reproducible builds, cross-compilation, malware detection, ...) #4576