x/build/cmd/relui: update for new distribution archive builds #58659
Assignees
Labels
Builders
x/build issues (builders, bots, dashboards)
early-in-cycle
A change that should be done early in the 3 month dev cycle.
FrozenDueToAge
NeedsFix
The path to resolution is known, but the work has not been done.
release-blocker
Milestone
Comments
rsc
added
the
early-in-cycle
thanm
added
the
NeedsFix
|
Change https://go.dev/cl/470755 mentions this issue: |
|
One question is how to test that things stay reproducible. One possible answer would be to use the trusted Linux machine as source of truth but then also build everything on Windows and require getting bit-for-bit identical outputs before blessing those files as part of the release. The Windows machine does not have to be as trusted since it is only a sanity check on the trusted builder. The two OSes Windows and Linux should be sufficiently different to serve as a good check. |
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Feb 23, 2023
Relui replaced release since Go 1.18 or so. We kept it around a bit longer while there were remaining Go+BoringCrypto releases that still used the release command. Those are no more, so to avoid confusion, remove cmd/release code given it's no longer maintained or supported. Updates golang/go#40279. Updates golang/go#58659. Fixes golang/go#45893. Change-Id: Id0d641bee49c9584c52e5616322f0656b89cd851 Reviewed-on: https://go-review.googlesource.com/c/build/+/470755 Reviewed-by: Dmitri Shuralyov <[email protected]> Reviewed-by: Heschi Kreinick <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> Reviewed-by: Carlos Amedee <[email protected]> Run-TryBot: Dmitri Shuralyov <[email protected]>
|
@rsc once the CL is merged we'll prioritize this. Looking forward to it. |
|
Change https://go.dev/cl/478158 mentions this issue: |
|
Change https://go.dev/cl/482996 mentions this issue: |
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Apr 18, 2023
In 1.21, make.bash has a new argument, -distpack, which builds the release archives reproducibly from any host architecture. See the related bug for more details. This converts relui to use -distpack to build everything. Mostly that's as simple as adding an alternate build mode for the source and binary archives. There are/were some wrinkles to work out: - We need to generate the timestamp at the top level of the workflow so that the source consistency check reuses that same timestamp. - Since we now get a zip out of the Windows build, we comically have to convert that back to a tarball to push it to a buildlet. Or at least I thought that was better than running unzip. - I haven't figured out what to do about the module files yet. I think I'd prefer to recreate them in relui -- this code is really not happy about getting multiple files back from a build step. - I didn't make a clear distinction between host and target builders in the BuildletStep. Maybe I should. But the distpack codepaths are so much smaller that it was easy to verify they didn't use the wrong config anywhere. For golang/go#58659. Change-Id: I4c4cf5b5450046a62d062d7c0bbfe94157ee9446 Reviewed-on: https://go-review.googlesource.com/c/build/+/478158 Run-TryBot: Heschi Kreinick <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Auto-Submit: Heschi Kreinick <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Carlos Amedee <[email protected]>
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Apr 21, 2023
Building on relui's new support for distpack builds, publish the toolchain module files it generates. This turns out to be quite a bit more complicated than I had hoped. First, save the entire distpack archive, not just the binary distribution. Extract the binary distribution for use in the normal workflow. Then extract the modules files (really just the .zip, the .mod and .info are tiny) to a separate artifact. Publish those to the CDN alongside the normal release artifacts. For non-distpack builds, copy the dl2mod code in from CL 475920 and generate the module files as a postprocessing step on the binary distribution. The really annoying part is macOS, where we need to distribute the signed binaries in the module, not the ones we built. For that, take the module zip generated by distpack and replace the binaries with the ones from the signed binary distribution. This is a fairly picky process, but hopefully means we don't risk drifting out of sync with the canonical distpacks. Finally, the testing changes: the fake make.bash needs to emulate distpack too, which wasn't too bad. And then the new requirement to process the results of the macOS signing process means that the testing fakes can't just produce cute text files -- they have to contain the real distribution. Change the fake macOS packaging tools to deal in prefixed tarballs. For golang/go#58659. Change-Id: I9cca8af2cb562fb39ef805a044109877b4ab7fb8 Reviewed-on: https://go-review.googlesource.com/c/build/+/482996 TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Run-TryBot: Dmitri Shuralyov <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>
|
Change https://go.dev/cl/487795 mentions this issue: |
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Apr 24, 2023
The intermediate PKG file is no longer a single line of plain text, it has binary data (a release tarball) appended as of CL 482996. There is a good chance said binary data will have a byte sequence that can't be parsed as UTF-8 or some other character encoding without an error. sed in turn fails with "sed: RE error: illegal byte sequence" on macOS. Avoid this problem by using sed on the text and not the binary part of the file. For golang/go#58659. Change-Id: If6acfed4c1f41c7ce3366679808614fecb98ec11 Reviewed-on: https://go-review.googlesource.com/c/build/+/487795 TryBot-Result: Gopher Robot <[email protected]> Auto-Submit: Dmitri Shuralyov <[email protected]> Run-TryBot: Dmitri Shuralyov <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Reviewed-by: Heschi Kreinick <[email protected]>
|
The bulk of this is done; it would be good to do the reproducibility testing described in #58659 (comment). |
|
Oh, I also intended to prefetch the module files from the module proxy to make sure that automatic toolchain updates started working immediately. |
|
Change https://go.dev/cl/489275 mentions this issue: |
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Apr 26, 2023
Creating the module zips is surprisingly CPU-intensive, and we end up doing it many times in parallel, resulting in it taking more than 10 minutes to finish and triggering the watchdog timer. Disable the watchdog for this task, and give relui more CPU so that it's less of a problem in the first place. (Collateral damage to other tasks is still a concern.) For golang/go#58659. Change-Id: I65b5c47f9fdc011885c0d5f886f4050551d55f61 Reviewed-on: https://go-review.googlesource.com/c/build/+/489275 Reviewed-by: Dmitri Shuralyov <[email protected]> Auto-Submit: Heschi Kreinick <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Run-TryBot: Heschi Kreinick <[email protected]>
|
Change https://go.dev/cl/489415 mentions this issue: |
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Apr 26, 2023
At some point the modules will be a key part of the release, just like the website entries are now. Make sure that they're available on proxy.golang.org before the release is announced. For golang/go#58659. Change-Id: I744c067f6efa06f3ca4cdbaf40a351f583331c16 Reviewed-on: https://go-review.googlesource.com/c/build/+/489415 Run-TryBot: Heschi Kreinick <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Auto-Submit: Heschi Kreinick <[email protected]> Reviewed-by: Carlos Amedee <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>
|
Change https://go.dev/cl/490615 mentions this issue: |
dmitshur
changed the title
|
Change https://go.dev/cl/502537 mentions this issue: |
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Jun 13, 2023
Try to shake out any reproducibility bugs by building each distpack on two different host types. For golang/go#58659 Change-Id: Ibe858909bf8469b71ba5c25c1588286f459101e8 Reviewed-on: https://go-review.googlesource.com/c/build/+/502537 TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Reviewed-by: Russ Cox <[email protected]> Auto-Submit: Heschi Kreinick <[email protected]> Run-TryBot: Heschi Kreinick <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
For #24904, I have mailed CL 470676, which adds a new
make.bash -distpackflag that generates distribution archives.On the release infrastructure side, x/build needs to start using
make.bash -distpackin Go 1.21 and later to produce the distribution archives (including the source archives), instead of the code it currently uses to generate them. The-distpackmode also writes the module files that we need to serve for GOTOOLCHAIN support (#57001). Those need to start being served from go.dev/dl like the other archives.When writing a VERSION file for Go 1.21 or later, the file should be of the form
That time stamp is used as the modification time in the archives (for reproducibility). This means something in the x/build metadata will have to record the time to use for each release and then write that time into the VERSION file.
The reproducibility work enables building distribution archives on any kind of machine. In particular it should be possible to build everything on an appropriately trusted+locked down linux/amd64 system. You use
GOOS=goos GOARCH=goarch ./make.bash -distpack.Note that if we move the generation of the linux/arm distribution archives to linux/amd64, we need to set
GOARM=6as well, to mimic the environment on linux-arm-aws. SoGOOS=linux GOARCH=arm GOARM=6 ./make.bash -distpack.It doesn't matter what goos/goarch is used to generate the source archives (you get the same result on every system, as one would expect), but it does need to be generated by
make.bash -distpackfor reproducibility.Once the release infrastructure is updated, we should issue a go1.21alpha1 (built from a master branch is fine, no need for a release branch) to exercise the infrastructure and to test that external users can reproduce the archives (#57120).
/cc @golang/release
The text was updated successfully, but these errors were encountered: