cmd/go: go get <package> doesn't resolve build issues

[pauldthomson]

What version of Go are you using (go version)?

$ go version
go version go1.16 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/pthomson/Library/Caches/go-build"
GOENV="/Users/pthomson/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/pthomson/go/pkg/mod"
GONOPROXY="github.com/tyro-private"
GONOSUMDB="github.com/tyro-private"
GOOS="darwin"
GOPATH="/Users/pthomson/go"
GOPRIVATE="github.com/tyro-private"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.16/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.16/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/pthomson/go-1_16-issue/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1n/fg0y62lj3sz31db9vtvbym006nsw3r/T/go-build3382657949=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

1. Brand new module with a dependency to github.com/prometheus/client_golang/prometheus added

2. run go mod tidy

3. run go build .

4. prompt for missing dependency with suggestion to fix:

   λ go build .
   ../go/pkg/mod/github.com/prometheus/[email protected]/prometheus/internal/metric.go:19:2: missing go.sum entry for module providing package github.com/prometheus/client_model/go (imported by github.com/prometheus/client_golang/prometheus); to add:
       go get github.com/prometheus/client_golang/[email protected]
   
   

5. follow instructions:

   λ go get github.com/prometheus/client_golang/[email protected]
   

6. run go build . again:

   λ go build .
   ../go/pkg/mod/github.com/prometheus/[email protected]/prometheus/internal/metric.go:19:2: missing go.sum entry for module providing package github.com/prometheus/client_model/go (imported by github.com/prometheus/client_golang/prometheus); to add:
       go get github.com/prometheus/client_golang/[email protected]
   

Issue persists. Can be fixed by running go build -mod=mod .

What did you expect to see?

Build worked after running suggested go get

What did you see instead?

Build still failed

Sample repo: https://github.com/pauldthomson/go-1_16-issue

[davecheney]

@pauldthomson could you try

go mod tidy -mod=mod

[pauldthomson]

@davecheney that doesn't seem to be a valid flag?

[davecheney]

@pauldthomson my mistake, the original source of the advice is here, #44129 (comment)

maybe

go mod -mod=mod tidy

[pauldthomson]

@davecheney same thing, not a valid flag:

λ go mod -mod=mod tidy
go mod -mod=mod: unknown command
Run 'go help mod' for usage.

No flags for that command it seems:

λ go help mod
Go mod provides access to operations on modules.

Note that support for modules is built into all the go commands,
not just 'go mod'. For example, day-to-day adding, removing, upgrading,
and downgrading of dependencies should be done using 'go get'.
See 'go help modules' for an overview of module functionality.

Usage:

        go mod <command> [arguments]

The commands are:

        download    download modules to local cache
        edit        edit go.mod from tools or scripts
        graph       print module requirement graph
        init        initialize new module in current directory
        tidy        add missing and remove unused modules
        vendor      make vendored copy of dependencies
        verify      verify dependencies have expected content
        why         explain why packages or modules are needed

Use "go help mod <command>" for more information about a command.

And for the tidy subcommand, these seem to be the only flags:

λ go help mod tidy
usage: go mod tidy [-e] [-v]

Tidy makes sure go.mod matches the source code in the module.
It adds any missing modules necessary to build the current module's
packages and dependencies, and it removes unused modules that
don't provide any relevant packages. It also adds any missing entries
to go.sum and removes any unnecessary ones.

The -v flag causes tidy to print information about removed modules
to standard error.

The -e flag causes tidy to attempt to proceed despite errors
encountered while loading packages.

See https://golang.org/ref/mod#go-mod-tidy for more about 'go mod tidy'.

Also tried the env var as per #44129 (comment) (GOFLAGS=-mod=mod go mod tidy) but no change

[bcmills]

To clarify, go mod tidy and go build -mod=mod are two distinct alternatives. Either ought to work (and if either doesn't work, that's probably a bug that we need to fix.)

[bcmills]

@pauldthomson, I'm not able to reproduce this failure mode using https://golang.org/dl/go1.16 and a clean module cache.

I suspect that you are seeing the same issue as #44749.

go-1_16-issue$ git reset --hard
HEAD is now at ef3daa2 Initial commit

go-1_16-issue$ go1.16 build
../.gopath/pkg/mod/github.com/prometheus/[email protected]/prometheus/internal/metric.go:19:2: missing go.sum entry for module providing package github.com/prometheus/client_model/go (imported by github.com/prometheus/client_golang/prometheus); to add:
        go get github.com/prometheus/client_golang/[email protected]

go-1_16-issue$ go1.16 mod tidy

go-1_16-issue$ git diff
diff --git i/go.sum w/go.sum
index 55a3d61..94b6a13 100644
--- i/go.sum
+++ w/go.sum
@@ -212,6 +212,7 @@ github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:
 github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=
 github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
 github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=

go-1_16-issue$ go1.16 build

go-1_16-issue$ git reset --hard
HEAD is now at ef3daa2 Initial commit

go-1_16-issue$ go1.16 build
../.gopath/pkg/mod/github.com/prometheus/[email protected]/prometheus/internal/metric.go:19:2: missing go.sum entry for module providing package github.com/prometheus/client_model/go (imported by github.com/prometheus/client_golang/prometheus); to add:
        go get github.com/prometheus/client_golang/[email protected]

go-1_16-issue$ go1.16 get github.com/prometheus/client_golang/[email protected]

go-1_16-issue$ git diff
diff --git i/go.sum w/go.sum
index 55a3d61..94b6a13 100644
--- i/go.sum
+++ w/go.sum
@@ -212,6 +212,7 @@ github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:
 github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=
 github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
 github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=

go-1_16-issue$ go1.16 build

go-1_16-issue$

[jayconrod]

@pauldthomson I was also unable to reproduce this and suspect you're seeing the same issue as in #44749.

To confirm, could you check if this file is present:

/Users/pthomson/go/pkg/mod/cache/download/github.com/prometheus/client_model/@v/v0.2.0.ziphash

If that file exists, what other files are in the same directory?

[pauldthomson]

😬 I went for a scorched earth policy and blew away my ~/go/pkg directory as per @bcmills link to the other issue. Good news is it worked, bad news is I can't confirm if the missing sum was actually the cause in my case..
Happy to close this one for now since it seems to be related to another issue, or if you want to leave it for a bit I might see if I can reproduce the issue based on the assumptions about the cause above..

[jayconrod]

No worries, let's close this then. If you see this happen again, and the fix for #44749 doesn't work for you, please comment, and we'll re-open and investigate further.

[pauldthomson]

FWIW I seem to be able to reproduce the behaviour as per your comment on #44749.
If I remove the .ziphash from the cache and entry from my go.sum, go build . prompts me to go get the missing package, but the go get doesn't add the entry to the go.sum file and so the issue persists.