[v2.11] Make RPMs and DEBs reproducible#6359
Merged
derekcollison merged 2 commits intonats-io:mainfrom Jan 23, 2025
Merged
Conversation
Member
|
Is this still in-flight? |
neilalexander
changed the title
Signed-off-by: Alex Bozhenko <alexbozhenko@gmail.com>
alexbozhenko
force-pushed
the
reproducible_rpms
branch
from
b1b0023 to
3ed94c2
Compare
Member
Author
|
@neilalexander it's ready for review now. |
wallyqs
changed the title
derekcollison
added a commit
that referenced
this pull request
Feb 20, 2025
## Details We have reproducible binaries, [rpms/debs](#6359) and [archives](#6299). And `goreleaser.yml` holds [all the knobs](https://goreleaser.com/customization/builds/go/) that can be tuned to build the binary. The only thing that affects the build and is not defined in the goreleaser config is the go toolchain version. Currently, we set the version of the go toolchain that is used for releases in Travis: <https://github.com/nats-io/nats-server/blob/5e6017135b4b1d333b435ff55b720275a39bb7af/.travis.yml#L11-L12> <https://github.com/nats-io/nats-server/blob/5e6017135b4b1d333b435ff55b720275a39bb7af/.travis.yml#L67> I spent some time [trying to understand the behavior of go and toolchain directives](https://alexbozhenko.github.io/posts/2024-12-19-understand-go-toolchain-directive-or-your-money-back/). I think setting the toolchain used for releases in .goreleser.yml would make it more explicit and future-proof. With this change, any human or script who has `go>1.21.0` and goreleaser installed can checkout the repo at any commit, run one command, and get binaries that will be _exactly_ as if we were cutting a release on that commit. ``` goreleaser build --snapshot --clean --single-target ``` Several places would benefit from not having to worry about keeping toolchain and all the go build flags in sync: * get-nats.io(already uses gorelaser, but toolchain used to build depends on the build host): <https://github.com/ConnectEverything/client-tools/blob/eba999ac9a1e107205fbf89ba230df3f80458028/build-nightlies.sh#L184-L188> * <redacted> number of private repos If we land this, we can update the above places to use `goreleaser build`, thus making sure we use _exactly the same_ binary everywhere, and forever forget about managing/updating go versions in other places that need to build the binary ## Reproducible test plan ### Before Behavior is not hermetic. We depend on toolchain that happens to be installed on the build host. 1. <details> <summary>Local go(`1.21.0`) < one that is specified in go.mod. Go will download and use toolchain from go.mod:</summary> ``` # go version go version go1.21.0 linux/amd64 # TARGET='linux_amd64' goreleaser build --snapshot --clean --single-target # go version -m dist/nats-server_linux_amd64_v1/nats-server | grep go1 dist/nats-server_linux_amd64_v1/nats-server: go1.22.8 ``` </details> 2. <details> <summary>Local go(`1.23.3`) > one that is specified in go.mod. Local toolchain will be used:</summary> ``` # wget https://go.dev/dl/go1.23.3.linux-amd64.tar.gz # sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.23.3.linux-amd64.tar.gz # go version go version go1.23.3 linux/amd64 # TARGET='linux_amd64' goreleaser build --snapshot --clean --single-target # go version -m dist/nats-server_linux_amd64_v1/nats-server | grep go1 dist/nats-server_linux_amd64_v1/nats-server: go1.23.3 ``` </details> ### After 1. <details> <summary>Local go version (1.23.4) did not affect what was used for the binary</summary> It will download (just like it [downloads all the modules](https://youtu.be/KqTySYYhPUE?feature=shared&t=229)) the version of the toolchain specified in goreleaer config, and use it for building the binary. ``` # go version go version go1.23.4 linux/amd64 # TARGET='linux_amd64' goreleaser build --snapshot --clean --single-target # go version -m dist/nats-server_linux_amd64_v1/nats-server | grep go1 dist/nats-server_linux_amd64_v1/nats-server: go1.23.5 # sha256sum dist/nats-server_linux_amd64_v1/nats-server 95d52ed8656f74abd0c0576d8d9be50fcc00562c8e18215d1f1f04b8c0b6fc3d dist/nats-server_linux_amd64_v1/nats-server ``` </details> 2. <details> <summary>Any go >= go1.21.0 (released 2023-08-08) will build exactly the same binary:</summary> ``` # wget <https://go.dev/dl/go1.21.0.linux-amd64.tar.gz> # sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.21.0.linux-amd64.tar.gz # go version go version go1.21.0 linux/amd64 # TARGET='linux_amd64' goreleaser build --snapshot --clean --single-target # go version -m dist/nats-server_linux_amd64_v1/nats-server | grep go1 dist/nats-server_linux_amd64_v1/nats-server: go1.23.5 # sha256sum dist/nats-server_linux_amd64_v1/nats-server 95d52ed8656f74abd0c0576d8d9be50fcc00562c8e18215d1f1f04b8c0b6fc3d dist/nats-server_linux_amd64_v1/nats-server ``` </details> 3. To build using specific toolchain one would set the `GORELEASER_TOOLCHAIN` env variable: ``` # GORELEASER_TOOLCHAIN="go1.22.8" TARGET='linux_amd64' goreleaser build --snapshot --clean --single-target # go version -m dist/nats-server_linux_amd64_v1/nats-server | grep go1 dist/nats-server_linux_amd64_v1/nats-server: go1.22.8 ``` Note that starting from go1.24.0 we can use the [tool directive](https://tip.golang.org/doc/modules/managing-dependencies#tools ) in go.mod, and even version goreleaser itself. It will look something like this: ``` go tool goreleaser build --snapshot --clean --single-target ```
neilalexander
added a commit
that referenced
this pull request
Sep 16, 2025
Follow-up to #6359 and #6701 Since 2.11.2 version of goreleaser, it is allowed to set buildhost for RPMs https://github.com/orgs/goreleaser/discussions/5662 I believe with this all the rpms, debs and tars will be reproducible(the binaries have been reproducible for long time) ## Test plan: ``` # go clean -cache && goreleaser release --skip=announce,publish,validate --clean -f .goreleaser.yml ``` ``` # rpm -qip nats-server-v2.12.0-RC.3-s390x.rpm | grep -i 'Build Host' Build Host : synadia.com ``` [skip ci] Signed-off-by: Alex Bozhenko <alexbozhenko@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Similar to #6299
Make packages set mtime to
commitdate.Upgrade goreleaser to the latest 2.6.1, which includes the feature used in this PR: goreleaser/goreleaser#5392
Signed-off-by: Alex Bozhenko alex@synadia.com