cmd/go: explicitly specify macOS base SDK version when compiling go binary

[andybons]

This is needed to ensure that Apple’s notarization service can check that the go binary is using (at minimum) the 10.9 SDK.

$ codesign -dvv /usr/local/go/bin/go
Executable=/usr/local/go/bin/go
Identifier=go
Format=Mach-O thin (x86_64)
CodeDirectory v=20200 size=116482 flags=0x0(none) hashes=3636+2 location=embedded
Library validation warning=OS X SDK version before 10.9 does not support Library Validation
Signature size=9042
Authority=Developer ID Application: Google, Inc. (EQHXZ8M8AV)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=Oct 31, 2019 at 7:23:46 PM
Info.plist=not bound
TeamIdentifier=EQHXZ8M8AV
Sealed Resources=none
Internal requirements count=1 size=164

Notice Library validation warning=OS X SDK version before 10.9 does not support Library Validation

The SDK is not properly specified in the go binary:

$ otool -l /usr/local/go/bin/go | grep -B1 -A3 MIN_MACOS
Load command 5
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.10
      sdk n/a

All other binaries built with the toolchain have the correct values:

$ otool -l /usr/local/go/pkg/tool/darwin_amd64/vet | grep -B1 -A3 MIN_MACOS
Load command 5
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.9
      sdk 10.9

The fix for this will likely need to be backported. Hopefully it’s simple. 🤞

Related issues:

• cmd/go: MacOS binaries invalid for eventual Apple Notary #30488
• cmd/link: keep MacOS binaries compatible with Apple Notary #31918
• all: ensure that Go toolchain meets Apple’s notarization requirements #34986

[andybons]

It seems that we want to specify the -mmacosx-version-min when compiling the go binary.

#18400 is related.

[andybons]

The fact that the SDK isn’t specified in the Mach-O headers makes me wonder if the xcode toolchain we’re using to compile cgo binaries at release time may need to be updated. When I build the toolchain on a darwin-amd64-10_11 machine, I get the same results, while when I build the toolchain locally, I get:

$ otool -l ../bin/go | grep -B1 -A3 MIN_MAC
Load command 5
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.10
      sdk 10.15

When running xcode-select -p on the gomote builder I get /Library/Developer/CommandLineTools while when I run it locally, I get /Applications/Xcode.app/Contents/Developer. It may be that without the full Xcode install, we won’t get the SDK version set because we don’t have the SDK installed. Yay.

@cagedmantis @toothrot @dmitshur.

[networkimprov]

cc @eliasnaur @odeke-em

[eliasnaur]

If the Go linker detects a load command in an external .o file, it will use that. We can't just pass -mmacosx-version-min to CC, because the load commands for macOS, iOS etc. are mutually exclusive.

In internal linking mode, the Go linker sets a default LC_VERSION_MIN_MACOSX version. The version was bumped to 10.9 in https://go-review.googlesource.com/c/go/+/175918/.

Perhaps it's enough to update/fix the toolchain used to build Go release. However, we could also expand the linker check above ("machoPlatform == 0") to trigger when the version and sdk are less than 10.9 in external linking mode. If you do that, please keep my https://go-review.googlesource.com/c/go/+/206337 in mind to only use the macOS load command for macOS binaries.

[andybons]

@ianlancetaylor for cgo.

It may be OK to specify the MACOSX_DEPLOYMENT_TARGET env var when building the toolchain only and not for every cgo binary.

[networkimprov]

cc @thanm

[gopherbot]

Change https://golang.org/cl/208268 mentions this issue: cmd/release: use macOS 10.15 (Catalina) for releases

[andybons]

Confirmed that a toolchain built using the new 10.15 builder with Xcode installed and no changes to the min deployment target gets rid of the warning from codesign when validating the go command.

$ codesign -dvv go/bin/go
Executable=/private/tmp/go/bin/go
Identifier=go
Format=Mach-O thin (x86_64)
CodeDirectory v=20500 size=116526 flags=0x10000(runtime) hashes=3636+2 location=embedded
Signature size=9063
Authority=Developer ID Application: Andrew Bonventre (USS93J4HN5)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=Nov 21, 2019 at 7:35:42 PM
Info.plist=not bound
TeamIdentifier=USS93J4HN5
Runtime Version=10.15.0
Sealed Resources=none
Internal requirements count=1 size=164

The issue may or may not affect notarization, since the min SDK version is still being set to 10.10 for the go command no matter what builder we’re using (I was misreading the Mach-O header values above) and the minimum for notarization is 10.9. Also, the notarization service is currently not reporting any errors or warnings with the go binary if it’s signed, has a secure timestamp, and has the hardened runtime enabled. Whether the notarization service would throw an error with the old SDK values sometime in the future is up for speculation, but it’s better to be safe than sorry.

It may be fine to leave the target SDK version at the default (10.10) and not set it explicitly, but I worry about defaults changing down the road.

[andybons]

Removing release-blocker label since the default value for the base SDK is 10.10 using the 10_15 Catalina builder. Keeping open to track setting this explicitly.

[toothrot]

/cc @dmitshur @cagedmantis

[luckymateo]

Is that the only way to solve the problem by updating the Mac system ? @andybons

[cagedmantis]

@huoxingyun We've isolated what the fix would be and will be implementing it soon.

[gopherbot]

Change https://golang.org/cl/215721 mentions this issue: cmd/go: add -d flag to mod_get_test

[gopherbot]

Change https://golang.org/cl/215957 mentions this issue: cmd/link: ensure cgo cflags do not leak into tvOS test

[gopherbot]

Change https://golang.org/cl/216177 mentions this issue: cmd/link: ensure cgo cflags do not leak into dwarf tests

[belloyang]

Is there a solution to this issue?
-- Feb. 18, 2020

[toothrot]

@cagedmantis What else is needed on this issue? Should the milestone be changed to 1.15?

[gopherbot]

Change https://golang.org/cl/218598 mentions this issue: [release-branch.go1.13] cmd/link: ensure cgo cflags do not leak into tvOS test

[gopherbot]

Change https://golang.org/cl/217057 mentions this issue: [release-branch.go1.13] cmd/link: ensure cgo cflags do not leak into dwarf tests

[gopherbot]

Change https://golang.org/cl/217059 mentions this issue: [release-branch.go1.13] cmd/go: fix cgo test when min macOS version is set

[gopherbot]

Change https://golang.org/cl/218597 mentions this issue: [release-branch.go1.13] cmd/go: add -d flag to mod_get_test

[gopherbot]

Change https://golang.org/cl/221100 mentions this issue: cmd/release: set the minimum macOS version supported by go 1.13

[ianlancetaylor]

What else needs to be done for this issue?

[jayconrod]

I believe this is resolved now, but if anyone knows differently, please reopen.

[cagedmantis]

Indeed. The work for this has been done. Thanks for closing the issue.

[gopherbot]

Change https://golang.org/cl/278435 mentions this issue: cmd/release: set macOS deployment target for darwin/arm64 too

[gopherbot]

Change https://golang.org/cl/322629 mentions this issue: cmd/go,cmd/link: do not check for staleness in most tests

[gopherbot]

Change https://golang.org/cl/381854 mentions this issue: cmd/go: rewrite TestScript/cgo_stale_precompiled to be agnostic to staleness