CI builds are failing due to go get issues

[jakubgs]

Problem

If I try to build develop in CI right now using this job:
https://ci.status.im/job/status-go/job/manual/
The Android and iOS builds fail with the following error:

+ make statusgo-android
Building status-go for Android...
gomobile init
gomobile: go install golang.org/x/mobile/cmd/gobind failed: exit status 1
../../../golang.org/x/mobile/internal/importers/ast.go:37:2: cannot find package "golang.org/x/tools/go/packages" in any of:
/usr/local/go/src/golang.org/x/tools/go/packages (from $GOROOT)
/home/jenkins/workspace/status-go/platforms/android/src/golang.org/x/tools/go/packages (from $GOPATH)

Makefile:93: recipe for target 'statusgo-android' failed
make: *** [statusgo-android] Error 1

Not sure what changed but something did.

[jakubgs]

The last successful develop build is from 29th of July:
https://ci.status.im/job/status-go/job/manual/438/
https://ci.status.im/job/status-go/job/platforms/job/ios/120/
https://ci.status.im/job/status-go/job/platforms/job/android/124/
Which was done on the 21a62c7 commit, which bumped the version to 0.31.0.
I will see if I can identify which commit caused this.

[jakubgs]

So I made a bisect branch which points at 21a62c7 and ran a built on it:
https://ci.status.im/job/status-go/job/manual/457/
https://ci.status.im/job/status-go/job/platforms/job/ios/134/
https://ci.status.im/job/status-go/job/platforms/job/android/139/
And it fails the same way, so that would indicate that the issue is with the CI hosts and not with the repo.

[jakubgs]

Related gomobile issues:

• x/mobile/cmd/gobind: building error golang/go#13740
• x/mobile/cmd/gomobile: bind fails for cloud.google.com/go/trace golang/go#24882
• x/mobile/cmd/gomobile: gomobile bind fails a package importing gl golang/go#24941

And one of those states that:

Change https://golang.org/cl/108336 mentions this issue: cmd/gomobile: add CGO_ENABLED=1 to gobind runs
golang/go#24941 (comment)

[jakubgs]

According to docs:

The cgo tool is enabled by default for native builds on systems where it is expected to work. It is disabled by default when cross-compiling. You can control this by setting the CGO_ENABLED environment variable when running the go tool: set it to 1 to enable the use of cgo, and to 0 to disable it.
https://golang.org/cmd/cgo/

Not sure how that is supposed to help with a missing dependency.

[jakubgs]

Since we are using -u when installing gomobile:

status-go/Makefile

Lines 214 to 215 in 8383fee

	gomobile-install:
	go get -u golang.org/x/mobile/cmd/gomobile

It is possible that we upgraded to a broken version.

[jakubgs]

But the most recent commits in gomobile are from May 9th:
https://github.com/golang/mobile/commits/master/cmd/gomobile
So if last successful build on develop was on 29th of July that doesn't add up.

[jakubgs]

I managed to reproduce the issue by purging my ~/go/src and running first

make setup-build

And then

make release-android

And it does indeed fail with:

~/work/status-go develop*
 lilim > make statusgo-android                
Building status-go for Android...
gomobile init
gomobile: go install golang.org/x/mobile/cmd/gobind failed: exit status 1
../../go/src/golang.org/x/mobile/internal/importers/ast.go:37:2: cannot find package "golang.org/x/tools/go/packages" in any of:
	/usr/local/go/src/golang.org/x/tools/go/packages (from $GOROOT)
	/home/sochan/go/src/golang.org/x/tools/go/packages (from $GOPATH)

make: *** [Makefile:94: statusgo-android] Error 1

But even if I install the mentioned missing package:

~/work/status-go develop*
 lilim > go get golang.org/x/tools/go/packages
go: finding golang.org/x/tools/go/packages latest
go: finding golang.org/x/tools/go latest
go: finding golang.org/x/tools latest

It still fails the same way.

[jakubgs]

Okay, this makes no sense to me:

~/work/status-go develop*
 lilim > go get golang.org/x/tools/go/packages
go: finding golang.org/x/tools/go/packages latest
go: finding golang.org/x/tools/go latest
go: finding golang.org/x/tools latest

~/work/status-go develop*
 lilim > ls -l ~/go/src/golang.org/x/tools    
ls: cannot access '/home/sochan/go/src/golang.org/x/tools': No such file or directory

[jakubgs]

Running:

make modvendor-install

And:

make vendor

Didn't help at all.

[jakubgs]

Ohh, thanks to help from @adambabik it turns out that this:

GO111MODULE=off go get golang.org/x/tools/go/packages

Fixes the cannot find package "golang.org/x/tools/go/packages" error!
But generates a new one:

Building status-go for Android...
gomobile init
gomobile bind -target=android -ldflags="-s -w" -ldflags ' -X main.buildStamp=2019-08-19.16:42:37 -X github.com/status-im/status-go/params.Version=0.32.0-beta.0 -X github.com/status-im/status-go/params.GitCommit=8383feea -X github.com/status-im/status-go/vendor/github.com/ethereum/go-ethereum/metrics.EnabledStr=true' -o build/bin/statusgo.aar github.com/status-im/status-go/mobile
gomobile: /home/sochan/go/bin/gobind -lang=go,java -outdir=/tmp/gomobile-work-188115786 github.com/status-im/status-go/mobile failed: exit status 1
2019/08/19 12:42:44 -: go: finding github.com/status-im/status-protocol-go/zaputil latest
go build github.com/status-im/status-protocol-go/zaputil: no Go files in 
go build github.com/ethereum/go-ethereum/crypto/secp256k1: build constraints exclude all Go files in /home/sochan/go/pkg/mod/github.com/status-im/[email protected]/crypto/secp256k1

[cammellos]

cc @pombeirp any idea on how to fix this?

[adambabik]

@cammellos I just merged #1574 which should fix the problem in the CI.

[cammellos]

oh wow, thanks @adambabik and @jakubgs !

[jakubgs]

The build looks green to me: https://ci.status.im/job/status-go/job/manual/468/
I'm closing this.