Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildctl: add dump #5

Merged
merged 1 commit into from
Jun 13, 2017
Merged

buildctl: add dump #5

merged 1 commit into from
Jun 13, 2017

Conversation

AkihiroSuda
Copy link
Member

The last PR today 😄

Example output:

$ go run examples/llboput/example.go > a.llb
$ buildctl dump < a.llb
2017/06/12 09:51:52 ([]main.llbOp) (len=4 cap=4) {
 (main.llbOp) {
  Op: (pb.Op) &Op{Inputs:[],Op:&Op_Source{Source:&SourceOp{Identifier:docker-image://docker.io/library/busybox:latest,},},},
  Digest: (digest.Digest) (len=71) sha256:74ecb5d790c33a213ba00f8f43d62947df1d4dea72f5ec13353a9b943f469f0f
 },
 (main.llbOp) {
  Op: (pb.Op) &Op{Inputs:[&Input{Digest:sha256:74ecb5d790c33a213ba00f8f43d62947df1d4dea72f5ec13353a9b943f469f0f,Index:0,}],Op:&Op_Exec{Exec:&ExecOp{Meta:&Meta{Args:[/bin/sleep 1],Env:[],Cwd:/,},Mounts:[&Mount{Input:0,Selector:,Dest:/,Output:0,}],},},},
  Digest: (digest.Digest) (len=71) sha256:1de5701b93648843bf2cf6ebf982b4f63ea55dcb9259e73a5bd6737505660d06
 },
 (main.llbOp) {
  Op: (pb.Op) &Op{Inputs:[&Input{Digest:sha256:1de5701b93648843bf2cf6ebf982b4f63ea55dcb9259e73a5bd6737505660d06,Index:0,}],Op:&Op_Exec{Exec:&ExecOp{Meta:&Meta{Args:[/bin/sh -c echo foo > /bar],Env:[],Cwd:/,},Mounts:[&Mount{Input:0,Selector:,Dest:/,Output:0,}],},},},
  Digest: (digest.Digest) (len=71) sha256:dcfef0f97703c4df76cba8f40891d2a851ae78d3bedce839e43e2e505e691341
 },
 (main.llbOp) {
  Op: (pb.Op) &Op{Inputs:[&Input{Digest:sha256:dcfef0f97703c4df76cba8f40891d2a851ae78d3bedce839e43e2e505e691341,Index:0,}],Op:&Op_Exec{Exec:&ExecOp{Meta:&Meta{Args:[/bin/ls -l /],Env:[],Cwd:/,},Mounts:[&Mount{Input:0,Selector:,Dest:/,Output:0,}],},},},
  Digest: (digest.Digest) (len=71) sha256:cec673a11a7de1687f9626850bfbd802ee8fe964e9b07f2ebbb365ae650fab09
 }
}

I'm also planning graphviz output (buildctl dump --mode dot) in another PR.

Signed-off-by: Akihiro Suda [email protected]

Signed-off-by: Akihiro Suda <[email protected]>
@tonistiigi
Copy link
Member

Thanks. As a follow up can you group this behind a debug command? I'm not sure if we want the end user to ever realize that this format exists.

@tonistiigi tonistiigi merged commit 5eb3c53 into moby:master Jun 13, 2017
@tonistiigi
Copy link
Member

tonistiigi commented Jun 13, 2017

The actual visualization of graph could be something like ./buildctl build --dry-run --graph=dot , so it would be returned from the daemon with the information about the cache hits etc. and would also work with Dockerfiles and other frontends.

tiborvass pushed a commit to tiborvass/buildkit that referenced this pull request Sep 29, 2018
…stomName

Example Dockerfile:

 FROM busybox
 ARG foo=abc
 ENV bar=def
 RUN echo $foo $bar $baz

Output without fix:

 moby#5 [2/2] RUN echo  def
 moby#5       digest: sha256:647df948c9689163efaf92b24b38779bc9f8b350482ecc3de3533f86a544c196
 moby#5         name: "[2/2] RUN echo  def "
 moby#5      started: 2018-09-29 19:01:11.376665368 +0000 UTC
 moby#5 0.764 abc def
 moby#5    completed: 2018-09-29 19:01:12.320141054 +0000 UTC
 moby#5     duration: 943.475686ms

Output with fix:

 moby#5 [2/2] RUN echo abc def
 moby#5       digest: sha256:647df948c9689163efaf92b24b38779bc9f8b350482ecc3de3533f86a544c196
 moby#5         name: "[2/2] RUN echo abc def "
 moby#5      started: 2018-09-29 19:01:11.376665368 +0000 UTC
 moby#5 0.764 abc def
 moby#5    completed: 2018-09-29 19:01:12.320141054 +0000 UTC
 moby#5     duration: 943.475686ms

Signed-off-by: Tibor Vass <[email protected]>
thaJeztah referenced this pull request in thaJeztah/buildkit Apr 9, 2020
Currently, `docker build --progress=plain` truncates output, which makes it
difficult to debug problems during build (the step that failed may not be the
cause of the failure).

For example, in the following build, the output of the `RUN echo ...` is truncated:

    DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t foo -<<EOF
    FROM busybox
    RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
    RUN echo "something went wrong"; exit 1
    EOF

    #5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit....
    #5 DONE 0.2s

    #6 [3/3] RUN echo "something went wrong"; exit 1
    #6 0.211 something went wrong
    #6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
    ------
     > [3/3] RUN echo "something went wrong"; exit 1:
    ------
    failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully

While there is an existing `PROGRESS_NO_TRUNC` environment variable, I think that
this should be the default if the user opted to use `--progress=plain` (or in
situations where no TTY is attached, which could be in CI).

This patch changes the default to disable truncating in those situations, but
allowing users to opt-out by setting `PROGRESS_NO_TRUNC=0`

With this change the same build looks like this:

    #5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
    #5 sha256:b2f0f47d63911ee55e7cf17c81007310e28190b5be84aa1a7869ba90786d5cee
    #5 DONE 0.2s

    #6 [3/3] RUN echo "something went wrong"; exit 1
    #6 sha256:c037b34bb998ae7d30572b489286da14df87e1478adf6d0c5c71c79b84b11bcc
    #6 0.293 something went wrong
    #6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
    ------
     > [3/3] RUN echo "something went wrong"; exit 1:
    ------
    failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully

Signed-off-by: Sebastiaan van Stijn <[email protected]>
alexcb pushed a commit to alexcb/buildkit that referenced this pull request Nov 6, 2020
tianon added a commit to infosiftr/buildkit that referenced this pull request Aug 30, 2022
This edge case was already handled for `CANCELED` or `ERROR`, but not `DONE` (or `CACHED`, which shouldn't have logs, but 🤷).

Before:

```console
$ docker buildx build --progress=plain --no-cache - <<<$'FROM bash\nRUN echo -n no newline'
...
moby#5 [2/2] RUN echo -n no newline
moby#5 0.268 no newline#5 DONE 0.3s
...
```

After:

```console
$ docker buildx build --progress=plain --no-cache - <<<$'FROM bash\nRUN echo -n no newline'
...
moby#5 [2/2] RUN echo -n no newline
moby#5 0.268 no newline
moby#5 DONE 0.3s
...
```

Signed-off-by: Tianon Gravi <[email protected]>
profnandaa added a commit to profnandaa/buildkit that referenced this pull request Aug 22, 2024
`shlex` in was treating single backslash "\\" in string as
escape sequence, hence stripping them off. Modify `processCmdEnv`
to double escape `\\`.

This was in turn affecting the vertex `Name`, and mostly pronounced
on Windows where backslashes are prevalent in path names. For example
`C:\hello\world\path` was being rendered as `C:helloworldpath`, hence
confusing users.

Fixes moby#5250

Build progress before fix:
```
    moby#5 [2/4] RUN echo C:helloworldpath
    moby#5 1.359 C:\hello\world\path
    moby#5 DONE 1.7s

    moby#6 [3/4] RUN echo C:\hello\escaped\path
    moby#6 1.734 C:\\hello\\escaped\\path
    moby#6 DONE 2.1s

    moby#7 [4/4] RUN echo "C:\hello\quoted\path"
    moby#7 1.765 "C:\hello\quoted\path"
    moby#7 DONE 2.1s
```

Build progress after fix:
```
	moby#5 [2/4] RUN echo C:\hello\world\path
	moby#5 1.458 C:\hello\world\path
	moby#5 DONE 1.8s

	moby#6 [3/4] RUN echo C:\\hello\\escaped\\path
	moby#6 1.730 C:\\hello\\escaped\\path
	moby#6 DONE 2.1s

	moby#7 [4/4] RUN echo "C:\hello\quoted\path"
	moby#7 1.702 "C:\hello\quoted\path"
	moby#7 DONE 2.1s
```

You can also see that now paths in the step/vertex names now
match those in the log lines right after the name.

Signed-off-by: Anthony Nandaa <[email protected]>
profnandaa added a commit to profnandaa/buildkit that referenced this pull request Aug 22, 2024
`shlex` in was treating single backslash "\\" in string as
escape sequence, hence stripping them off. Modify `processCmdEnv`
to double escape `\\`.

This was in turn affecting the vertex `Name`, and mostly pronounced
on Windows where backslashes are prevalent in path names. For example
`C:\hello\world\path` was being rendered as `C:helloworldpath`, hence
confusing users.

Fixes moby#5250

Build progress before fix:
```
	moby#5 [2/4] RUN echo C:helloworldpath
	moby#5 1.359 C:\hello\world\path
	moby#5 DONE 1.7s

	moby#6 [3/4] RUN echo C:\hello\escaped\path
	moby#6 1.734 C:\\hello\\escaped\\path
	moby#6 DONE 2.1s

	moby#7 [4/4] RUN echo "C:\hello\quoted\path"
	moby#7 1.765 "C:\hello\quoted\path"
	moby#7 DONE 2.1s
```

Build progress after fix:
```
	moby#5 [2/4] RUN echo C:\hello\world\path
	moby#5 1.458 C:\hello\world\path
	moby#5 DONE 1.8s

	moby#6 [3/4] RUN echo C:\\hello\\escaped\\path
	moby#6 1.730 C:\\hello\\escaped\\path
	moby#6 DONE 2.1s

	moby#7 [4/4] RUN echo "C:\hello\quoted\path"
	moby#7 1.702 "C:\hello\quoted\path"
	moby#7 DONE 2.1s
```

You can also see that now paths in the step/vertex names now
match those in the log lines right after the name.

Signed-off-by: Anthony Nandaa <[email protected]>
profnandaa added a commit to profnandaa/buildkit that referenced this pull request Sep 9, 2024
`shlex` in was treating single backslash "\\" in string as
escape sequence, hence stripping them off. Modify `processCmdEnv`
to double escape `\\`.

This was in turn affecting the vertex `Name`, and mostly pronounced
on Windows where backslashes are prevalent in path names. For example
`C:\hello\world\path` was being rendered as `C:helloworldpath`, hence
confusing users.

Fixes moby#5250

Build progress before fix:
```
	moby#5 [2/4] RUN echo C:helloworldpath
	moby#5 1.359 C:\hello\world\path
	moby#5 DONE 1.7s

	moby#6 [3/4] RUN echo C:\hello\escaped\path
	moby#6 1.734 C:\\hello\\escaped\\path
	moby#6 DONE 2.1s

	moby#7 [4/4] RUN echo "C:\hello\quoted\path"
	moby#7 1.765 "C:\hello\quoted\path"
	moby#7 DONE 2.1s
```

Build progress after fix:
```
	moby#5 [2/4] RUN echo C:\hello\world\path
	moby#5 1.458 C:\hello\world\path
	moby#5 DONE 1.8s

	moby#6 [3/4] RUN echo C:\\hello\\escaped\\path
	moby#6 1.730 C:\\hello\\escaped\\path
	moby#6 DONE 2.1s

	moby#7 [4/4] RUN echo "C:\hello\quoted\path"
	moby#7 1.702 "C:\hello\quoted\path"
	moby#7 DONE 2.1s
```

You can also see that now paths in the step/vertex names now
match those in the log lines right after the name.

Signed-off-by: Anthony Nandaa <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants