Cache downloaded Go dependencies and Atlantis compilation in Docker #2738
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This has the potential to speed up rebuilds of the Atlantis container significantly, especially for non-x86 platforms which are emulated when done in GitHub Actions. It works by using cache mounts: https://docs.docker.com/engine/reference/builder/#run---mounttypecache. They mean we can have specific paths set that are carried over from one image build to the next, regardless of which files are changed in the project. The files are however at the same time not part of the final image, which reduces the generated "builder" image from 1.66 GB to just 386 MB on the current main branch. In my local tests, which doesn't make use of emulation, I saw a drop for the "go build" step going from taking just over 40 seconds with an empty cache to around 1 second when I make a one character change to a Go source file. That is otherwise a change that prior to this would cause a complete rebuild of the Docker image layer.
|
I just tried the same quick comparison locally when doing cross-compiling. I'm on an arm64 machine, so I added With a reused download cache, but where it had to do all the cross-compiling, the |
nitrocode
reviewed
Dec 1, 2022
nitrocode
reviewed
Dec 2, 2022
| @@ -3,7 +3,9 @@ FROM golang:1.19.3-alpine AS builder | |||
|
|
|||
| WORKDIR /app | |||
| COPY . /app | |||
| RUN CGO_ENABLED=0 go build -trimpath -ldflags "-s -w" -v -o atlantis . | |||
| RUN --mount=type=cache,target=/go/pkg/mod \ | |||
| --mount=type=cache,target=/root/.cache/go-build \ | |||
There was a problem hiding this comment.
Have you been able to test this in your fork to confirm if this works as intended?
There was a problem hiding this comment.
I have tested it locally and use the same pattern in a bunch of other repositories which also build using GitHub Actions.
There was a problem hiding this comment.
For this to have any effect in CI here, though, I believe it needs the caching options from my other PR, otherwise the cache won't be shared between the runs.
nitrocode
approved these changes
Dec 5, 2022
krrrr38
pushed a commit
to krrrr38/atlantis
that referenced
this pull request
Dec 16, 2022
…unatlantis#2738) This has the potential to speed up rebuilds of the Atlantis container significantly, especially for non-x86 platforms which are emulated when done in GitHub Actions. It works by using cache mounts: https://docs.docker.com/engine/reference/builder/#run---mounttypecache. They mean we can have specific paths set that are carried over from one image build to the next, regardless of which files are changed in the project. The files are however at the same time not part of the final image, which reduces the generated "builder" image from 1.66 GB to just 386 MB on the current main branch. In my local tests, which doesn't make use of emulation, I saw a drop for the "go build" step going from taking just over 40 seconds with an empty cache to around 1 second when I make a one character change to a Go source file. That is otherwise a change that prior to this would cause a complete rebuild of the Docker image layer. Co-authored-by: nitrocode <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
what
This has the potential to speed up rebuilds of the Atlantis container significantly, especially for non-x86 platforms which are emulated when done in GitHub Actions.
It works by using cache mounts:
https://docs.docker.com/engine/reference/builder/#run---mounttypecache. They mean we can have specific paths set that are carried over from one image build to the next, regardless of which files are changed in the project. The files are however at the same time not part of the final image, which reduces the generated "builder" image from 1.66 GB to just 386 MB on the current main branch.
why
In my local tests, which doesn't make use of emulation, I saw a drop for the "go build" step going from taking just over 40 seconds with an empty cache to around 1 second when I make a one character change to a Go source file. That is otherwise a change that prior to this would cause a complete rebuild of the Docker image layer.
references