x/build: configure dashboard builders to test the same configurations that will be shipped in binary releases #46900
Labels
Builders
x/build issues (builders, bots, dashboards)
NeedsInvestigation
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Comments
bcmills
added
Builders
|
Change https://golang.org/cl/330252 mentions this issue: |
gopherbot
pushed a commit
that referenced
this issue
Jun 25, 2021
TestStdlib was failing after running rm -r $(go env GOROOT)/pkg/*/cmd as the builders do when building binary releases.¹ For users who write programs that depend on go/types, it should be reasonable to run the tests for go/types as part of 'go test all', and those tests should pass even if they installed Go from a binary release. I had originally drafted this as a fallback to import from source only if the affected packages can't be imported by the default export-data importer. Unfortunately, I realized that we don't currently have a builder that tests the actual release (#46900), so it is quite likely that the fallback path would bit-rot and produce unexpected test regressions. So instead, we now unconditionally import from source in TestStdlib. That makes the test substantially slower (~15s instead of ~5s on my workstation), but with less risk of regression, and TestStdlib is skipped in short mode already so short-mode test time is unaffected. If we change the builders to test the actual release configuration, we can consider restoring the faster path when export data is available. ¹https://github.com/golang/build/blob/df58bbac082bc87c4a3cdfe336d1ffe60bbaa916/cmd/release/release.go#L533-L545 For #43232 Change-Id: I764ec56926c104053bb2ef23cf258c8f0f773290 Reviewed-on: https://go-review.googlesource.com/c/go/+/330252 Trust: Bryan C. Mills <[email protected]> Trust: Robert Griesemer <[email protected]> Run-TryBot: Bryan C. Mills <[email protected]> TryBot-Result: Go Bot <[email protected]> Reviewed-by: Robert Griesemer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
#43232 notes a case in which
go teston a package instdfails for a Go toolchain installed from our supported binary distributions, but passes for a Go toolchain built from source.Even if we fix that issue, we don't currently have a good way to prevent regressions, and it makes me wonder if there are other cases where we're shipping packages and/or tests that work locally but fail for users who use our provided installers.
For the platforms for which we ship official binaries, I think we should have dashboard builders that test the actual bytes that we would ship in a release — either by applying the same transformations that we will apply at release time, or by eliminating differences between the release process and
make.bash.(Related: #29205, #29239.)
CC @golang/release
The text was updated successfully, but these errors were encountered: