crypto/x509: Trust setting not inherited on darwin #30471
Comments
|
@FiloSottile In #24652 you requested to be tagged if the problem is not completely solved in Go 1.12. |
bcmills
added
OS-Darwin
NeedsInvestigation
|
I've run into the same issue, and while I can't share the output of the test in question. I can confirm that marking the intermediate ca as |
|
Thank you for the report. Is this a regression or was it broken in Go 1.11 as well? |
|
This was working for me in Go 1.11. I installed my companies root CA, and two intermediate CAs in the system keychain. At the time when I did this, it was required because Go didn't check the login keychain. That specific details appears to have changed, but I'm not 100% sure that was introduced in Go 1.12. I only had the root CA marked as After running into this issue today, I cleaned up the certs in my system keychain, and moved them to the login keychain. I tested this in both locations. When only the root CA is marked As an additional note, The ouput of I can confirm that a script that relies on the certificate being trusted works correctly when I change the intermediate CA it depends on to |
|
I'm going to include this tidbit just because it's slightly related I'm on macOS Mojave 10.14.3, and it appears security handles pointing login.keychain to login.keychain-db internally. I shasum'd the output of the commands with the different paths below it it's the same value. go/src/crypto/x509/root_darwin.go Lines 74 to 79 in 13d24b6 steel@computer:src$ls -la ~/Library/Keychains/
drwxr-xr-x 11 steel 000000000 352 Feb 28 19:36 .
drwx------@ 75 steel 000000000 2400 Feb 11 10:21 ..
-rw-r--r--@ 1 steel 000000000 6148 Jan 7 15:33 .DS_Store
-rw-r--r--@ 1 steel 000000000 736696 Feb 28 19:36 login.keychain-db
-rw------- 1 steel 000000000 30740 Feb 12 11:13 metadata.keychain-db
steel@computer:src$security find-certificate -a ~/Library/Keychains/login.keychain | shasum
a2b9a124e87200b0663f8b45df45c39584663bd9 -
steel@computer:src$security find-certificate -a ~/Library/Keychains/login.keychain-db | shasum
a2b9a124e87200b0663f8b45df45c39584663bd9 - |
|
@FiloSottile .. I sent you an email yesterday regarding this same issue as mentioned in 24652 still occurring. I'm trying to get past the issue now so if you want me to try some stuff to help out please let me know. golang 1.12.1. |
|
I employed the All Trust work around on three certs using system defaults, all the others in play were already trusted and still receive "x509: certificate signed by unknown authority" when calling urls from golang project. As mentioned by others pasting the url in chrome and it returns data as expected. |
|
Anyone? Bueller? Bueller? |
|
Could your issue be fixed by #30672 ? There's an open CL, could you try it? |
|
@adamdecaf .. how might I go about trying this since it hasn't been merged as far as I can tell? $>brew upgrade golang |
|
I'll try it, because I'm having a similar problem with the same error message (in this case, it's an IRC server's certificate; I don't remember why it's misconfigured). Patching 33e5da4, running on 10.12.6, using the Cherry-Pick option in Gerrit. Before: After: @kewilson You will need to build Go from source, downloading the repository via git and keeping it at HEAD. In Gerrit, there will be a button at the bottom right of the description that says DOWNLOAD; click it to get instructions on how to apply the Gerrit patch locally. |
|
Change https://golang.org/cl/178539 mentions this issue: |
|
This should be now fixed at tip. Please test it with https://golang.org/dl/gotip and report back. |
|
Can confirm the normal |
|
cannot confirm this to be working. Running it as my normal user, I get: Am I missing something? |
|
@tommyknows Please post the full (redacted) output of #30471 (comment) in a new issue and tag me. |
|
Change https://golang.org/cl/227037 mentions this issue: |
+----------------------------------------------------------------------+ | Hello, if you are reading this and run macOS, please test this code: | | | | $ GO111MODULE=on go get golang.org/dl/gotip@latest | | $ gotip download | | $ GODEBUG=x509roots=1 gotip test crypto/x509 -v -run TestSystemRoots | +----------------------------------------------------------------------+ We currently have two code paths to extract system roots on macOS: one uses cgo to invoke a maze of Security.framework APIs; the other is a horrible fallback that runs "/usr/bin/security verify-cert" on every root that has custom policies to check if it's trusted for SSL. The fallback is not only terrifying because it shells out to a binary, but also because it lets in certificates that are not trusted roots but are signed by trusted roots, and because it applies some filters (EKUs and expiration) only to roots with custom policies, as the others are not passed to verify-cert. The other code path, of course, requires cgo, so can't be used when cross-compiling and involves a large ball of C. It's all a mess, and it broke oh-so-many times (#14514, #16532, #19436, #20990, #21416, #24437, #24652, #25649, #26073, #27958, #28025, #28092, #29497, #30471, #30672, #30763, #30889, #32891, #38215, #38365, ...). Since macOS does not have a stable syscall ABI, we already dynamically link and invoke libSystem.dylib regardless of cgo availability (#17490). How that works is that functions in package syscall (like syscall.Open) take the address of assembly trampolines (like libc_open_trampoline) that jump to symbols imported with cgo_import_dynamic (like libc_open), and pass them along with arguments to syscall.syscall (which is implemented as runtime.syscall_syscall). syscall_syscall informs the scheduler and profiler, and then uses asmcgocall to switch to a system stack and invoke runtime.syscall. The latter is an assembly trampoline that unpacks the Go ABI arguments passed to syscall.syscall, finally calls the remote function, and puts the return value on the Go stack. (This last bit is the part that cgo compiles from a C wrapper.) We can do something similar to link and invoke Security.framework! The one difference is that runtime.syscall and friends check errors based on the errno convention, which Security doesn't follow, so I added runtime.syscallNoErr which just skips interpreting the return value. We only need a variant with six arguments because the calling convention is register-based, and extra arguments simply zero out some registers. That's plumbed through as crypto/x509/internal/macOS.syscall. The rest of that package is a set of wrappers for Security.framework and Core Foundation functions, like syscall is for libSystem. In theory, as long as macOS respects ABI backwards compatibility (a.k.a. as long as binaries built for a previous OS version keep running) this should be stable, as the final result is not different from what a C compiler would make. (One exception might be dictionary key strings, which we make our own copy of instead of using the dynamic symbol. If they change the value of those strings things might break. But why would they.) Finally, I rewrote the crypto/x509 cgo logic in Go using those wrappers. It works! I tried to make it match 1:1 the old logic, so that root_darwin_amd64.go can be reviewed by comparing it to root_cgo_darwin_amd64.go. The only difference is that we do proper error handling now, and assume that if there is no error the return values are there, while before we'd just check for nil pointers and move on. I kept the cgo logic to help with review and testing, but we should delete it once we are confident the new code works. The nocgo logic is gone and we shall never speak of it again. Fixes #32604 Fixes #19561 Fixes #38365 Awakens Cthulhu Change-Id: Id850962bad667f71e3af594bdfebbbb1edfbcbb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/227037 Reviewed-by: Katie Hockman <[email protected]>
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
$ go build
What did you expect to see?
Nothing (a successful build).
What did you see instead?
go: git.intern.migros.net/[email protected]: unrecognized import path "git.intern.migros.net/package/path" (https fetch: Get https://git.intern.migros.net/package/path?go-get=1: x509: certificate signed by unknown authority)
go: error loading module requirements
More details
This is basically just issue #24652 but with Go 1.12
The same fix (mark root ca as "Allways trust") as described in #24652 (comment)
makes the problem go away.
Curl and Browsers do accept the certificate. E.g
But running TestSystemRoots yields:
$ GODEBUG=x509roots=1 ../bin/go test -v -run TestSystemRoots crypto/x509 === RUN TestSystemRoots crypto/x509: 6 certs have a trust policy crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH crypto/x509: verify-cert approved CN=webproxy.dc.migros.ch,OU=Informatik,O=Migros,L=Zuerich,ST=Zuerich,C=CH crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH crypto/x509: verify-cert rejected CN=dlv-cert: "Cert Verify Result: Invalid Extended Key Usage for policy" crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH crypto/x509: verify-cert approved CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH crypto/x509: verify-cert approved CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH crypto/x509: ran security verify-cert 13 times Number of trusted certs = 2 Cert 0: Migros Root CA 2 Number of trust settings : 4 Trust Setting 0: Policy OID : SSL Policy String : autodiscover.migros.net Allowed Error : CSSMERR_TP_CERT_EXPIRED Result Type : kSecTrustSettingsResultTrustRoot Trust Setting 1: Policy OID : SSL Policy String : autodiscover.migros.net Allowed Error : Host name mismatch Result Type : kSecTrustSettingsResultTrustRoot Trust Setting 2: Policy OID : Apple X509 Basic Policy String : autodiscover.migros.net Allowed Error : CSSMERR_TP_CERT_EXPIRED Result Type : kSecTrustSettingsResultTrustRoot Trust Setting 3: Allowed Error : CSSMERR_TP_CERT_EXPIRED Result Type : kSecTrustSettingsResultTrustRoot Cert 1: macman.migros.net Number of trust settings : 2 Trust Setting 0: Policy OID : EAP Allowed Error : CSSMERR_TP_CERT_EXPIRED Result Type : kSecTrustSettingsResultTrustAsRoot Trust Setting 1: Policy OID : Apple X509 Basic Allowed Error : CSSMERR_TP_CERT_EXPIRED Result Type : kSecTrustSettingsResultTrustAsRoot Number of trusted certs = 5 Cert 0: Migros User CA 2 Number of trust settings : 1 Trust Setting 0: Result Type : kSecTrustSettingsResultTrustAsRoot Cert 1: QuoVadis Swiss Advanced CA G2 Number of trust settings : 1 Trust Setting 0: Result Type : kSecTrustSettingsResultTrustAsRoot Cert 2: webproxy.dc.migros.ch Number of trust settings : 0 Cert 3: dlv-cert Number of trust settings : 0 Cert 4: Migros Root CA 2 Number of trust settings : 0 --- FAIL: TestSystemRoots (0.31s) root_darwin_test.go:35: cgo sys roots: 56.90889ms root_darwin_test.go:36: non-cgo sys roots: 150.793582ms root_darwin_test.go:77: certificate only present in non-cgo pool: CN=Migros User CA 2,O=Migros,C=CH (verify error: x509: certificate signed by unknown authority) root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH root_darwin_test.go:77: certificate only present in non-cgo pool: CN=Migros Root CA 2,O=Migros,C=CH (verify error: x509: certificate signed by unknown authority) root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US root_darwin_test.go:99: off-EKU certificate only present in cgo pool (acceptable): CN=dlv-cert FAIL crypto/x509: kSecTrustSettingsResultInvalid = 0 crypto/x509: kSecTrustSettingsResultTrustRoot = 1 crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2 crypto/x509: kSecTrustSettingsResultDeny = 3 crypto/x509: kSecTrustSettingsResultUnspecified = 4 crypto/x509: Migros User CA 2 returned 4 crypto/x509: QuoVadis Swiss Advanced CA G2 returned 4 crypto/x509: webproxy.dc.migros.ch returned 1 crypto/x509: dlv-cert returned 1 crypto/x509: Migros Root CA 2 returned 4 crypto/x509: Migros Root CA 2 returned 4 crypto/x509: macman.migros.net returned 4 FAIL crypto/x509 0.317sThe text was updated successfully, but these errors were encountered: