Dockerfile: Allow mounting secrets directly into env vars #2122
Comments
|
I thought this was easy and added to milestone but actually this can't be done in Dockerfile frontend as envs with the secret values would leak to build cache. So only way to add this is to add a special option to |
|
@tonistiigi Can you expand upon this? Why is this different with respect to the cache than the existing secret file mounting feature? Or do you mean to say the existing feature leverages a special Also, what is the specific nature of the "non-backward compatible change"? Is it something between buildctl (the command line) and buildkitd (the daemon)? Or something else? |
|
Yes, secret mount is a special mount type in
When we can make a change that only touches the frontend it means that older versions of buildkit will still be able to build Dockerfiles with new syntax. With LLB changes Dockerfile will not build unless BuildKit has been updated to support new LLB feature. |
|
Wouldn't this kind of compatibility issue inevitably arise when trying to use any new kind of mount, not just this change in particular? For example, if By the way, if it would make things simpler in the implementation, I personally would be fine having this new feature be something like |
Yes, I'm not saying that we can never add this. Just that it is more complicated than what I initially assumed. |
|
Another example is Go's |
|
LLB side of this is implemented in #2579 |
Can you share this boilerplate please? |
|
@kevcube It's in the example I shared. RUN --mount=type=secret,id=pip-index-url \
PIP_INDEX_URL="$(cat /run/secrets/pip-index-url)" && \
export PIP_INDEX_URL && \
pip3 install ... |
tonistiigi
changed the title
|
@tonistiigi What still needs to be done for this feature to be fully realized? |
|
ping @tonistiigi |
Dockerfile frontend would need to be able to parse it from the flag and do the correct LLB calls. For testing all this can be done with an external frontend image. |
|
@tonistiigi any update on this? |
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome.
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
|
@rittneje The change implementing support for this has been merged. |
This adds the syntax to Dockerfile frontend. I purposedly chose to use a simple format for this as it's likely going to be debated. As implemented, the following format is supported: ``` RUN --mount=type=secret,id=MYSECRET,env ``` or, more explicitly: ``` RUN --mount=type=secret,id=MYSECRET,env=true ``` will mount the secret with id MYSECRET as a new environment variable with the same name. Using 'target', it's possible to create a different environment variable: ``` RUN --mount=type=secret,id=mysecret,target=MY_SECRET,env ``` will mount 'mysecret' secret as MY_SECRET environment variable. Any suggestions on making it more ergonomic are welcome. Signed-off-by: a-palchikov <[email protected]>
This is a followup to the discussion touched on back in #1703. Currently buildctl supports pulling secrets from env vars via
--secret id=xyz,env=ENV_VAR_NAME. However, in order to populate the corresponding env var withinRUN, we have to doRUN --mount=type=secret,id=xyz ENV_VAR_NAME="$(cat /run/secrets/xyz)" ....Not only is having to run cat every time annoyingly repetitive, it doesn't really work well in conjunction with the
required=falsemount option, sincecatwill fail due to the file not existing, so even more complex boilerplate is needed.In addition, if multiple commands need the environment variable, we must explicitly export it, which is a pain because it requires even more boilerplate to be properly safe:
I propose adding an
envoption to--mount, which is mutually exclusive withdst. If provided, it contains the name of the environment variable that will be set to the value of the secret in question (and no file will be mounted at /run/secrets). If the mount is not required and the secret does not exist, then the environment variable will not be set at all (rather than having an empty value).For example:
RUN --mount=type=secret,id=pip-index-url,env=PIP_INDEX_URL pip3 install ...The text was updated successfully, but these errors were encountered: