Skip to content

Conversation

@MisterDA
Copy link
Contributor

The environment variables were not correctly quoted which introduce unnecessary and sometimes buggy escape sequences in the variable value.
Piggy-back a switch from Fmt.strf to Fmt.str as the former is a deprecated alias to the later.

@MisterDA MisterDA mentioned this pull request Jun 16, 2021
15 tasks
@avsm
Copy link
Member

avsm commented Jun 17, 2021

I'm not sure how to evaluate this PR. Could I get an example of what broke, and an explanation of how this fixes it please?

@MisterDA
Copy link
Contributor Author

MisterDA commented Jun 17, 2021

Sure. Currently, environment variables are quoted with %S:

https://ocaml.org/releases/4.12/api/Printf.html

S: convert a string argument to OCaml syntax (double quotes, escapes).

Docker ENV states:

The value will be interpreted for other environment variables, so quote characters will be removed if they are not escaped. Like command line parsing, quotes and backslashes can be used to include spaces within values.

However the Docker documentation is wrong: the character used to escape the value is not always a backslash, it's the character set by the escape parser directive which on Windows is usually set to ` (backtick).

My solution is to fetch the escape character, always enclose values in double-quotes, and escape inside the value the escape character and double-quotes.

Without the patch applied:

Dockerfile.(parser_directive (`Escape '`') @@
            env [("VAL1", {|quick "fox"|}); ("VAL2", {|la`zy \dog|})] |> string_of_t) |> print_string;;
(*
# escape=`
ENV VAL1="quick \"fox\"" VAL2="la`zy \\dog"
*)
Dockerfile.(parser_directive (`Escape '\\') @@
            env [("VAL1", {|quick "fox"|}); ("VAL2", {|la`zy \dog|})] |> string_of_t) |> print_string;;
(*
# escape=\
ENV VAL1="quick \"fox\"" VAL2="la`zy \\dog"
*)

With the patch applied:

Dockerfile.(parser_directive (`Escape '`') @@
            env [("VAL1", {|quick "fox"|}); ("VAL2", {|la`zy \dog|})] |> string_of_t) |> print_string;;
(*
# escape=`
ENV VAL1="quick `"fox`"" VAL2="la``zy \dog"
*)
Dockerfile.(parser_directive (`Escape '\\') @@
            env [("VAL1", {|quick "fox"|}); ("VAL2", {|la`zy \dog|})] |> string_of_t) |> print_string;;
(*
# escape=\
ENV VAL1="quick \"fox\"" VAL2="la`zy \\dog"
*)

Without the patch, on Windows with #escape=`:

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM mcr.microsoft.com/windows/nanoserver:20H2
 ---> 67ae078d1460
Step 2/3 : ENV VAL1="quick \"fox\"" VAL2="la`zy \\dog"
 ---> Using cache
 ---> 3b5b781e04c9
Step 3/3 : RUN echo %VAL1% && echo %VAL2%
 ---> Running in d3992da5a7b7
quick \fox\
la`zy \\dog
Removing intermediate container d3992da5a7b7
 ---> 3d5be9509bd7
Successfully built 3d5be9509bd7

With the patch, on Windows with #escape=`:

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM mcr.microsoft.com/windows/nanoserver:20H2
 ---> 67ae078d1460
Step 2/3 : ENV VAL1="quick `"fox`"" VAL2="la``zy \dog"
 ---> Running in 71774ece9441
Removing intermediate container 71774ece9441
 ---> 00b5151e0109
Step 3/3 : RUN echo %VAL1% && echo %VAL2%
 ---> Running in 2b3d038b10d0
quick "fox"
la`zy \dog
Removing intermediate container 2b3d038b10d0
 ---> 0ac57d847e36
Successfully built 0ac57d847e36

You get the original values only with the patch.

MisterDA added 2 commits June 18, 2021 08:39
Docker uses the character defined by the #escape parser directive to
escape characters in strings. If we surround the environment variable
value with double-quotes, we only need to escape double-quotes and the
escape character itself in the value.

Using the OCaml quoting %S introduces wrong changes and doesn't work
when Docker escape character is changed.
@MisterDA
Copy link
Contributor Author

The force-push is a little simplification in the code.
I've tested this PR yesterday and it works as expected.

@avsm
Copy link
Member

avsm commented Jun 18, 2021

Thanks for the explanation! @djs55 @justincormack, just copying you about the incorrect Docker documentation referenced above in case you want to get that fixed: #53 (comment)

@MisterDA
Copy link
Contributor Author

MisterDA commented Jun 18, 2021

The escape character problem can also be seen with this, when escaping a single value containing a space without double-quotes:

$ cat << 'EOF' | docker build --no-cache -
# escape=`
FROM mcr.microsoft.com/windows/nanoserver:20H2
ENV VAR=foo\ bar
RUN echo %VAR%
EOF
Sending build context to Docker daemon  2.048kB
Error response from daemon: failed to parse Dockerfile: Syntax error - can't find = in "bar". Must be of the form: name=value

$ cat << 'EOF' | docker build --no-cache -
# escape=`
FROM mcr.microsoft.com/windows/nanoserver:20H2
ENV VAR=foo` bar
RUN echo %VAR%
EOF
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM mcr.microsoft.com/windows/nanoserver:20H2
 ---> 67ae078d1460
Step 2/3 : ENV VAR=foo` bar
 ---> Running in 4769f1cc508c
Removing intermediate container 4769f1cc508c
 ---> e5f3bcb89cab
Step 3/3 : RUN echo %VAR%
 ---> Running in 8912b40a1146
foo bar
Removing intermediate container 8912b40a1146
 ---> c9c5ac5359df
Successfully built c9c5ac5359df

@avsm avsm merged commit 695e9f9 into ocurrent:master Jun 18, 2021
@avsm
Copy link
Member

avsm commented Jun 18, 2021

Looks good, merging!

@MisterDA MisterDA deleted the windows branch June 19, 2021 09:04
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Sep 23, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Sep 23, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Sep 24, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Sep 30, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Oct 1, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Oct 4, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Oct 4, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Oct 8, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Oct 21, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Oct 27, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Nov 22, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Nov 23, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Nov 30, 2021
MisterDA added a commit to MisterDA/obuilder that referenced this pull request Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants