proposal: cmd/go: add GOPROXY=cache to rely only on module cache

[Merovius]

When I try to import a yet-unrequired module, e.g.

mero@hix ~/src/github.com/Merovius/foo$ cat foo.go 
package main

import "rsc.io/qr"

func main() {
	var _ qr.Code
}
mero@hix ~/src/github.com/Merovius/foo$ cat go.mod
module github.com/Merovius/foo

go 1.15

and I run go mod tidy, while not connected to the internet, I get an error

go: finding module for package rsc.io/qr
github.com/Merovius/foo imports
	rsc.io/qr: module rsc.io/qr: Get "https://proxy.golang.org/rsc.io/qr/@v/list": dial tcp: lookup proxy.golang.org on [::1]:53: read udp [::1]:57927->[::1]:53: read: connection refused

That makes sense - the proxy is not reachable. However, my module cache contains a usable version of this module:

mero@hix ~/src/github.com/Merovius/foo$ l -d ~/pkg/mod/rsc.io/[email protected] 
dr-x------. 5 mero mero 4,0K 14. Mai 2020  /home/mero/pkg/mod/rsc.io/[email protected]

So, it might be a reasonable course of action for me, to just use the version I have in the cache - whether it is the latest, or not. If I want to do that, as far as I can tell, either

a) I have to look up what versions I have in my cache and run go get rsc.io/[email protected], to update go.mod. It would be great, if that could be automated, or
b) set GOPROXY=file:///home/mero/pkg/mod/cache/download and run go mod tidy

Both of these can be considered somewhat clunky (and the latter is not very discoverable). I would like to suggest to make this easier. There are several ways this could happen:

1. Automatically fall back to the module cache when running go mod tidy or similar and a network error happens. IMO that would be the most convenient - it also has the downside of unexpectedly adding an outdated version of the module.
2. Add a value to the -mod flag that is equivalent to -mod=mod, but only touching the local cache.
3. Add a flag to go mod tidy (and similar), to use the local cache as a proxy (essentially automate setting the environment variable).

It's also fine, if this is deemed WAI, of course. I just thought I would suggest it after the case came up on golang-nuts.

[jayconrod]

cc @bcmills @matloob @rsc

Closely related: #32337

We've also talked about adding GOPROXY=cache, which would be similar to GOPROXY=file://$(go env GOMODCACHE)/cache/download, but would synthesize version lists and .info files.

[Merovius]

GOPROXY=cache would also address this and is, I think, a very logical place for that knob (and far easier to remember than the path).

[mvdan]

Just so others find this issue more easily via keywords on google: I was using "offline mode" in my search.

[dolmen]

Use case: for goeval (Go one-liners), when the same command is run multiple times, querying the proxy every time while everything is available locally is a waste of resources and has a huge performance impact (longer run time) for the user (note: I also understand that this is a security feature as it allows to block retracted versions).

So, as a workaround for that go get slowness I've implemented a shortcut: set GOPROXY=file://$(go env GOMODCACHE)/cache/download if all module versions are canonical and found in the cache.

However, it would be appreciated if go get had an built-in opt-in mode where dependencies would be resolved locally if possible, and the GOPROXY would be still be used but only as a fallback. Because my shortcut is limited: it is all or nothing: if just one version is non-canonical (ex: @latest), all dependencies resolution will hit the proxy.

[dolmen]

GOPROXY=off is documented at https://go.dev/ref/mod#environment-variables.

Does it fit the need from this proposal?

[mvdan]

@dolmen I don't think so, because multiple commands try to resolve @latest and that will often still try to use the network to fetch source from upstream's VCS with GOPROXY=off.

[dolmen]

The documentation about resolving packages to a module claims:

off indicates that no communication should be attempted.

[mvdan]

Right, I got slightly confused with GOPROXY=direct. But I still think that GOPROXY=off is not enough for an "offline mode" to work well enough with what's in the module cache. For example, go get module@latest simply fails with module lookup disabled by GOPROXY=off rather than finding the latest version that's already in the module cache.

[dolmen]

Ok, so in GOPROXY=cache mode $module@latest would be resolved using content of $GOMODCACHE/cache/download/$module/@v/list, isn't it?

[mvdan]

That's how I would imagine it, yes, and it's what I understood from #43646 (comment). It's worth noting that "latest" might be out of date in such a cache/offline mode, but that's by design.