goreleaser: add arm64 build for each OS #1073

ee7 · 2022-11-28T15:06:06Z

The latest release of the Exercism CLI (3.1.0, 2022-10-04) includes these assets, via GoReleaser:

exercism-3.1.0-darwin-x86_64.tar.gz
exercism-3.1.0-freebsd-i386.tar.gz
exercism-3.1.0-freebsd-x86_64.tar.gz
exercism-3.1.0-linux-armv5.tar.gz
exercism-3.1.0-linux-armv6.tar.gz
exercism-3.1.0-linux-i386.tar.gz
exercism-3.1.0-linux-ppc64.tar.gz
exercism-3.1.0-linux-x86_64.tar.gz
exercism-3.1.0-openbsd-i386.tar.gz
exercism-3.1.0-openbsd-x86_64.tar.gz
exercism-3.1.0-windows-armv5.zip
exercism-3.1.0-windows-armv6.zip
exercism-3.1.0-windows-i386.zip
exercism-3.1.0-windows-x86_64.zip
exercism_checksums.txt
exercism_checksums.txt.sig

For the next release, make GoReleaser include an arm64 (AKA aarch64) build for each of the existing OSes. This means we will have separate x86_64 and arm64 release assets for macOS; previously macOS users on arm64 had to run the x86_64 binary via Rosetta (or build the CLI themselves).

At least for now, let's avoid adding a fat binary (which contains both x86_64 and arm64 executables). But note that GoReleaser does support it.

Closes: #966

I am not a GoReleaser expert - please review this carefully.

Note for Go 1.19.3, the first class targets are:

$ go tool dist list -json \
    | jq -r '.[] | select(.FirstClass) | [.GOOS, .GOARCH] | @tsv'
darwin	amd64
darwin	arm64
linux	386
linux	amd64
linux	arm
linux	arm64
windows	386
windows	amd64

so Go itself does not declare arm64 as "first class" on Windows, FreeBSD, and OpenBSD. But the Exercism CLI already provides lots of releases that aren't in the above list.

GitHub does not yet provide arm64 machines for their hosted runners, so we can't run integration tests of the new arm64 binaries on the native architecture without adding a new CI provider.

ErikSchierboom · 2022-11-30T10:07:10Z

@ekingery Could you look at the GoReleaser part of the code?

ekingery

Fair warning - I'm also no goreleaser expert, but this seems totally fine and should be easy enough to validate / confirm. This may have already been done, but if not you can build off the branch to test before merging.

kytrinyx · 2022-12-05T22:42:32Z

I tried building this on my machine and got the following failure:

  ⨯ release failed after 23s                 error=failed to build for windows_arm64: exit status 2: cmd/go: unsupported GOOS/GOARCH pair windows/arm64

I'm not sure if this means that I need to do something on my machine to support the pair, or if it's just unsupported in general.

It did successfully create all of the following:

    • building                                       binary=dist/exercism_freebsd_arm64/exercism
    • building                                       binary=dist/exercism_linux_386/exercism
    • building                                       binary=dist/exercism_linux_arm_6/exercism
    • building                                       binary=dist/exercism_linux_arm_5/exercism
    • building                                       binary=dist/exercism_windows_386/exercism.exe
    • building                                       binary=dist/exercism_freebsd_amd64_v1/exercism
    • building                                       binary=dist/exercism_darwin_arm64/exercism
    • building                                       binary=dist/exercism_windows_arm64/exercism.exe
    • building                                       binary=dist/exercism_linux_ppc64/exercism
    • building                                       binary=dist/exercism_linux_arm64/exercism
    • building                                       binary=dist/exercism_darwin_amd64_v1/exercism
    • building                                       binary=dist/exercism_windows_amd64_v1/exercism.exe
    • building                                       binary=dist/exercism_windows_arm_6/exercism.exe
    • building                                       binary=dist/exercism_linux_amd64_v1/exercism
    • building                                       binary=dist/exercism_windows_arm_5/exercism.exe
    • building                                       binary=dist/exercism_freebsd_386/exercism
    • building                                       binary=dist/exercism_openbsd_amd64_v1/exercism
    • building                                       binary=dist/exercism_openbsd_386/exercism
    • building                                       binary=dist/exercism_openbsd_arm64/exercism

ErikSchierboom · 2022-12-06T05:51:44Z

@ee7 Do you know if Windows ARM64 is supported?

ee7 · 2022-12-06T08:08:14Z

Do you know if Windows ARM64 is supported?

I think it should be. windows/arm64 appears in the output of go tool dist list:

And there seems to be plenty of software that both uses GoReleaser, and has windows/arm64 releases. For example, the GitHub CLI:

ee7 · 2022-12-06T08:29:09Z

And building windows/arm64 seems to work for me locally:

$ go version
go version go1.19.3 linux/amd64
$ goreleaser --version
goreleaser version 1.12.3
goos: linux
goarch: amd64

https://goreleaser.com
$ GOARCH=arm64 GOOS=windows goreleaser build --single-target --snapshot --rm-dist
  • starting build...
  • loading config file                              file=.goreleaser.yml
  • building only for windows/arm64                  reason=single target is enabled
  • loading environment variables
  • getting and validating git state
    • building...                                    commit=bda6cd2700acf05f3511abe6f957367d8482fe91 latest tag=v3.1.0
    • pipe skipped                                   reason=disabled during snapshot mode
  • parsing tag
  • setting defaults
  • snapshotting
    • building snapshot...                           version=3.1.0-SNAPSHOT-bda6cd2
  • checking distribution directory
    • --rm-dist is set, cleaning it up
  • loading go mod information
  • build prerequisites
  • writing effective config file
    • writing                                        config=dist/config.yaml
  • building binaries
    • building                                       binary=dist/exercism_windows_arm64/exercism.exe
  • storing release metadata
    • writing                                        file=dist/artifacts.json
    • writing                                        file=dist/metadata.json
  • build succeeded after 0s
$ file dist/exercism_windows_arm64/exercism.exe
dist/exercism_windows_arm64/exercism.exe: PE32+ executable (console) Aarch64 (stripped to external PDB), for MS Windows

ee7 · 2022-12-06T08:34:55Z

@kytrinyx What version of Go and GoReleaser did you use?

Maybe the build failure was due to https://goreleaser.com/deprecations/#builds-for-windowsarm64? Does the windows/arm64 build still fail if you use the latest releases of Go and GoReleaser?

kytrinyx · 2022-12-06T08:44:21Z

I used

go version go1.16 darwin/amd64
goreleaser version 1.11.5

I'll upgrade both and try again.

ee7 · 2022-12-06T11:36:04Z

For me, every build succeeds apart from freebsd_arm64 - that fails unless I bump golang.org/x/sys. It seems like a separate issue. Can anybody reproduce that failure?

Currently the version is:

cli/go.mod

Line 25 in e0dbcd1

golang.org/x/sys v0.0.0-20170803140359-d8f5ea21b929

So I think the options are:

Add a commit to bump that - committing the output of something like go get -u golang.org/x/sys.
Add a commit to ignore at least the freebsd_arm64 build for now.

Any thoughts?

kytrinyx · 2022-12-06T11:58:39Z

I'm unable to reproduce the failure on freebsd_arm64.

ee7 · 2022-12-06T12:06:05Z

Thanks. So this command succeeds for you?

GOARCH=arm64 GOOS=freebsd goreleaser build --rm-dist --snapshot --single-target

I'm asking because you wrote that windows_arm64 failed, but also:

It did successfully create all of the following:
   • building                                       binary=dist/exercism_windows_arm64/exercism.exe

My understanding is that goreleaser builds in parallel (by default using all CPUs), prints the "building" line when beginning each build, and reports only the first build failure. So a "building" line does not imply that the printed arch/os combination was successfully built.

Edit: And if the command to build freebsd_arm64 does work, does it still work after running go clean -modcache? goreleaser ends up using the minimum version of golang.org/x/sys that satisfies go.mod and exists locally, right? Maybe you've already got a more recent version?

kytrinyx · 2022-12-06T15:59:19Z

GOARCH=arm64 GOOS=freebsd goreleaser build --rm-dist --snapshot --single-target

Ah, no. That fails for me. Sorry about that.

Maybe you've already got a more recent version?

Maybe? I doubt it. I will take a proper look again tomorrow when I can give this my full attention.

kytrinyx · 2022-12-07T10:54:05Z

I upgraded go and goreleaser:

$ go version
go version go1.19.4 darwin/amd64

$ goreleaser --version
goreleaser version 1.13.1

I tried running the following command, which failed (details collapsed):

$ goreleaser --skip-publish --snapshot --rm-dist

Output

  • starting release...
  • loading config file                              file=.goreleaser.yml
  • loading environment variables
  • getting and validating git state
    • building...                                    commit=bda6cd2700acf05f3511abe6f957367d8482fe91 latest tag=v3.1.0
    • pipe skipped                                   reason=disabled during snapshot mode
  • parsing tag
  • setting defaults
  • snapshotting
    • building snapshot...                           version=3.1.0-SNAPSHOT-bda6cd2
  • checking distribution directory
    • --rm-dist is set, cleaning it up
  • loading go mod information
  • build prerequisites
  • writing effective config file
    • writing                                        config=dist/config.yaml
  • building binaries
    • building                                       binary=dist/exercism_windows_arm64/exercism.exe
    • building                                       binary=dist/exercism_linux_arm64/exercism
    • building                                       binary=dist/exercism_windows_386/exercism.exe
    • building                                       binary=dist/exercism_freebsd_amd64_v1/exercism
    • building                                       binary=dist/exercism_linux_ppc64/exercism
    • building                                       binary=dist/exercism_freebsd_386/exercism
    • building                                       binary=dist/exercism_freebsd_arm64/exercism
    • building                                       binary=dist/exercism_linux_amd64_v1/exercism
    • building                                       binary=dist/exercism_darwin_arm64/exercism
    • building                                       binary=dist/exercism_linux_386/exercism
    • building                                       binary=dist/exercism_windows_arm_6/exercism.exe
    • building                                       binary=dist/exercism_windows_arm_5/exercism.exe
    • building                                       binary=dist/exercism_linux_arm_5/exercism
    • building                                       binary=dist/exercism_darwin_amd64_v1/exercism
    • building                                       binary=dist/exercism_linux_arm_6/exercism
    • building                                       binary=dist/exercism_windows_amd64_v1/exercism.exe
    • building                                       binary=dist/exercism_openbsd_amd64_v1/exercism
    • building                                       binary=dist/exercism_openbsd_386/exercism
    • building                                       binary=dist/exercism_openbsd_arm64/exercism
    • took: 1m15s
  ⨯ release failed after 1m15s               error=failed to build for freebsd_arm64: exit status 2: # golang.org/x/sys/unix
../../go/pkg/mod/golang.org/x/[email protected]/unix/sockcmsg_unix.go:43:9: undefined: Cmsghdr
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_freebsd.go:26:9: undefined: RawSockaddrDatalink
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:142:38: undefined: _Socklen
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:148:7: undefined: RawSockaddrInet4
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:155:9: undefined: RawSockaddrInet6
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:160:7: undefined: RawSockaddrUnix
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:18:23: undefined: CAP_RIGHTS_VERSION_00
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:19:23: undefined: CAP_RIGHTS_VERSION_00
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:46:21: undefined: CapRights
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:50:24: undefined: CapRights
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:50:24: too many errors

error=failed to build for freebsd_arm64: exit status 2: # golang.org/x/sys/unix

Here's the version where I just run the single target:
$ GOARCH=arm64 GOOS=freebsd goreleaser build --rm-dist --snapshot --single-target

Output

  • starting build...
  • loading config file                              file=.goreleaser.yml
  • building only for freebsd/arm64                  reason=single target is enabled
  • loading environment variables
  • getting and validating git state
    • building...                                    commit=bda6cd2700acf05f3511abe6f957367d8482fe91 latest tag=v3.1.0
    • pipe skipped                                   reason=disabled during snapshot mode
  • parsing tag
  • setting defaults
  • snapshotting
    • building snapshot...                           version=3.1.0-SNAPSHOT-bda6cd2
  • checking distribution directory
    • --rm-dist is set, cleaning it up
  • loading go mod information
  • build prerequisites
  • writing effective config file
    • writing                                        config=dist/config.yaml
  • building binaries
    • building                                       binary=dist/exercism_freebsd_arm64/exercism
  ⨯ build failed after 0s                    error=failed to build for freebsd_arm64: exit status 2: # golang.org/x/sys/unix
../../go/pkg/mod/golang.org/x/[email protected]/unix/sockcmsg_unix.go:43:9: undefined: Cmsghdr
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_freebsd.go:26:9: undefined: RawSockaddrDatalink
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:142:38: undefined: _Socklen
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:148:7: undefined: RawSockaddrInet4
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:155:9: undefined: RawSockaddrInet6
../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:160:7: undefined: RawSockaddrUnix
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:18:23: undefined: CAP_RIGHTS_VERSION_00
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:19:23: undefined: CAP_RIGHTS_VERSION_00
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:46:21: undefined: CapRights
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:50:24: undefined: CapRights
../../go/pkg/mod/golang.org/x/[email protected]/unix/cap_freebsd.go:50:24: too many errors

ee7 · 2022-12-07T11:48:37Z

I upgraded go and goreleaser:

Thanks. It looks like that fixed the windows_arm64 failure, then.

Here's the version where I just run the single target:

Great - that's what I see too. Bumping golang.org/x/sys like a2cc8a4 makes every build succeed for me locally. But it fails in CI, since it requires a more recent Go version. So I'll bump that dependency a little less aggressively than I just did, then hopefully everything will build both locally and in CI.

ee7 · 2022-12-07T12:21:59Z

OK - I did a binary search to determine the minimum golang.org/x/sys bump to fix the freebsd build. Happily, that version does not produce the CI failure that occurs when bumping to the latest golang.org/x/sys release. ~~Every build still succeeds for me locally.~~

@kytrinyx ~~Does every build now succeed locally for you with the latest state of this PR?~~

Edit: now darwin_arm64 fails. I'll see if there's a version that fixes that without making CI fail.

kytrinyx · 2022-12-07T12:35:26Z

For me openbsd_arm64 now fails.

ee7 · 2022-12-07T12:58:32Z

For me openbsd_arm64 now fails.

I've reproduced that failure too. I spoke too soon - sorry for the noise.

But every build really does succeed for me locally with 5aa8768. Does that fix everything for you too?

kytrinyx · 2022-12-07T15:24:00Z

This now succeeds for me as well!

The latest release of the Exercism CLI (3.1.0, 2022-10-04) includes these assets, via GoReleaser: exercism-3.1.0-darwin-x86_64.tar.gz exercism-3.1.0-freebsd-i386.tar.gz exercism-3.1.0-freebsd-x86_64.tar.gz exercism-3.1.0-linux-armv5.tar.gz exercism-3.1.0-linux-armv6.tar.gz exercism-3.1.0-linux-i386.tar.gz exercism-3.1.0-linux-ppc64.tar.gz exercism-3.1.0-linux-x86_64.tar.gz exercism-3.1.0-openbsd-i386.tar.gz exercism-3.1.0-openbsd-x86_64.tar.gz exercism-3.1.0-windows-armv5.zip exercism-3.1.0-windows-armv6.zip exercism-3.1.0-windows-i386.zip exercism-3.1.0-windows-x86_64.zip exercism_checksums.txt exercism_checksums.txt.sig For future releases, this commit configures GoReleaser to include an arm64 (also known as aarch64) release asset for each of the existing OSes. Note that we will have separate x86_64 and arm64 release assets for macOS; previously macOS users on arm64 had to run the x86_64 binary via Rosetta (or build the CLI themselves). At least for now, let's avoid adding a fat binary. But note that GoReleaser does support it [1]. [1] https://goreleaser.com/customization/universalbinaries/ Closes: exercism#966

This commits the result of running go get golang.org/x/[email protected] to fix a build failure for freebsd_arm64. This seems to be the minimum bump to fix that failure without introducing a build failure for darwin_arm64 and openbsd_arm64. We cannot bump to the latest release (0.3.0, 2022-12-03) because that causes the Linux and macOS CI jobs to fail with: Error: ../../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall.go:83:16: undefined: unsafe.Slice Error: ../../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_darwin.go:95:8: undefined: unsafe.Slice Error: ../../../go/pkg/mod/golang.org/x/[email protected]/unix/syscall_unix.go:118:7: undefined: unsafe.Slice Error: ../../../go/pkg/mod/golang.org/x/[email protected]/unix/sysvshm_unix.go:33:7: undefined: unsafe.Slice note: module requires Go 1.17 and we need other changes in order to bump the minimum Go version.

ee7 · 2022-12-08T12:38:13Z

This now succeeds for me as well!

Great. Thanks again for testing.

This PR seems OK to me now. For the future, it'd be nice if we could try to bump some dependency versions in this repo - this PR was trickier than I hoped.

In case somebody wanted to merge with a merge commit: I've rebased on main, removed the obsolete commits, and tidied the commit messages. Squashing is also fine by me.

I'd only say that I'd prefer to avoid "rebase and merge" here, since it means that we'd introduce a commit on main for which goreleaser can't build everything. But we could swap the commit order if we really wanted to merge like that.

kytrinyx · 2022-12-09T08:04:34Z

Thank you so much for working through this so carefully. I agree that getting some dependencies bumped would be a Good Thing(tm)!

ErikSchierboom · 2022-12-09T08:49:13Z

Thanks @ee7!

ErikSchierboom requested a review from ekingery November 30, 2022 10:06

ekingery approved these changes Nov 30, 2022

View reviewed changes

ee7 added 2 commits December 8, 2022 13:12

ee7 force-pushed the goreleaser-add-arm64 branch from 5aa8768 to ba91c32 Compare December 8, 2022 12:13

kytrinyx merged commit b29b91d into exercism:main Dec 9, 2022

ee7 deleted the goreleaser-add-arm64 branch December 9, 2022 09:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

goreleaser: add arm64 build for each OS #1073

goreleaser: add arm64 build for each OS #1073

ee7 commented Nov 28, 2022 •

edited

Loading

ErikSchierboom commented Nov 30, 2022

ekingery left a comment

kytrinyx commented Dec 5, 2022

ErikSchierboom commented Dec 6, 2022

ee7 commented Dec 6, 2022 •

edited

Loading

ee7 commented Dec 6, 2022

ee7 commented Dec 6, 2022 •

edited

Loading

kytrinyx commented Dec 6, 2022

ee7 commented Dec 6, 2022 •

edited

Loading

kytrinyx commented Dec 6, 2022

ee7 commented Dec 6, 2022 •

edited

Loading

kytrinyx commented Dec 6, 2022

kytrinyx commented Dec 7, 2022 •

edited

Loading

ee7 commented Dec 7, 2022

ee7 commented Dec 7, 2022 •

edited

Loading

kytrinyx commented Dec 7, 2022

ee7 commented Dec 7, 2022 •

edited

Loading

kytrinyx commented Dec 7, 2022

ee7 commented Dec 8, 2022

kytrinyx commented Dec 9, 2022

ErikSchierboom commented Dec 9, 2022

goreleaser: add arm64 build for each OS #1073

goreleaser: add arm64 build for each OS #1073

Conversation

ee7 commented Nov 28, 2022 • edited Loading

ErikSchierboom commented Nov 30, 2022

ekingery left a comment

Choose a reason for hiding this comment

kytrinyx commented Dec 5, 2022

ErikSchierboom commented Dec 6, 2022

ee7 commented Dec 6, 2022 • edited Loading

ee7 commented Dec 6, 2022

ee7 commented Dec 6, 2022 • edited Loading

kytrinyx commented Dec 6, 2022

ee7 commented Dec 6, 2022 • edited Loading

kytrinyx commented Dec 6, 2022

ee7 commented Dec 6, 2022 • edited Loading

kytrinyx commented Dec 6, 2022

kytrinyx commented Dec 7, 2022 • edited Loading

ee7 commented Dec 7, 2022

ee7 commented Dec 7, 2022 • edited Loading

kytrinyx commented Dec 7, 2022

ee7 commented Dec 7, 2022 • edited Loading

kytrinyx commented Dec 7, 2022

ee7 commented Dec 8, 2022

kytrinyx commented Dec 9, 2022

ErikSchierboom commented Dec 9, 2022

ee7 commented Nov 28, 2022 •

edited

Loading

ee7 commented Dec 6, 2022 •

edited

Loading

ee7 commented Dec 6, 2022 •

edited

Loading

ee7 commented Dec 6, 2022 •

edited

Loading

ee7 commented Dec 6, 2022 •

edited

Loading

kytrinyx commented Dec 7, 2022 •

edited

Loading

ee7 commented Dec 7, 2022 •

edited

Loading

ee7 commented Dec 7, 2022 •

edited

Loading