"go get" fails in GOPATH mode

[nmiyake]

Running GO111MODULE=off go get github.com/mholt/archiver (or, more generally, running go get github.com/mholt/archiver in GOPATH mode) fails with the following output:

package github.com/pierrec/lz4/v3: cannot find package "github.com/pierrec/lz4/v3" in any of:
	/usr/local/go/src/github.com/pierrec/lz4/v3 (from $GOROOT)
	/go/src/github.com/pierrec/lz4/v3 (from $GOPATH)

This is due to an interaction between Go module semantic version imports and GOPATH mode not working well together in certain cases -- filed golang/go#35630 to track the underlying issue on the Golang side.

Even though it's not best practice, some clients appear to depend on "go get" to populate their GOPATH for development, and the current conflation of circumstances are such that there is no easy work-around for this.

The easiest short-term fix would probably be to revert #192 -- if the import path is restored to "github.com/pierrec/lz4", then this will continue to work (and the only downside of this is that module imports are not quite as nice).

[kounelios13]

Is there any way do go get it right now?

[awgh]

Revert change #192 on master immediately, only development branches should refer to other development branches!!! That's not only a best practice, it's common sense.

You just gave a serious WTF moment to all of your users over this change, for no real reason. The modules thing is not even the issue, and doesn't really solve the dependency management aspect of the problem here at all.

[nmiyake]

@awgh the change you referenced doesn't reference a development branch -- it adds a dependency for version 3.0.1, which is a release version of the referenced project: https://github.com/pierrec/lz4/tree/v3.0.1

The github.com/pierrec/lz4 project has taken an unconventional (but valid) approach to modules, where it seems that they only add the go.mod file on releases but don't maintain it on the development branch. This makes the "go get" operation fail due to interactions with GOPATH mode.

For module consumers (which should be the vast majority of consumers going forward), this change is strictly correct, as it properly declares the module dependency of an imported package. However, it does have the unfortunate side effect of negative impact for GOPATH mode due to the unconventional approach of the lz4 repo.

The comment on golang/go#35630 (comment) did provide a workaround for GOPATH mode -- I'll update this issue with the specific instructions.

There certainly is a trade-off here between module correctness and maintaining smoothness for GOPATH/go get consumers. @mholt is in a better position to evaluate the trade-off for this package and decide what he's comfortable with -- for example, this package itself could adopt the "major version subdirectory" mode of development and create a "v3" or "v4" subdirectory for the new module-based version (which would maintain GOPATH compatibility for the previous version), but that would be a major change to the repo structure.

[nmiyake]

Although not pretty, in GOPATH mode with a clean GOPATH, running the following command will allow "go get" or "go install" to succeed:

go get github.com/pierrec/lz4 && cd $GOPATH/src/github.com/pierrec/lz4 && git fetch && git checkout v3.0.1

Basically, if the copy of github.com/pierrec/lz4 in the local GOPATH includes the go.mod definition that declares the major version of the module, then the "go get" or "go install" of this package will succeed due to the minimal module compatibility code.

[mholt]

Even though it's not best practice, some clients appear to depend on "go get" to populate their GOPATH for development

Unfortunately, Go modules broke the behavior of go get for this, I also opened an issue about it: golang/go#31529

I think if a user wants to maintain a deprecated Go convention from earlier versions of Go while using a newer version of Go, that is their responsibility -- and the lack of a go.mod in the lz4 lib isn't a bug in this project.

I think it's important that we strictly maintain Go module compatibility going forward. I'm not really interested in maintaining old versions. Just the most recent version is enough work. 😅

[aprice2704]

Although not pretty, in GOPATH mode with a clean GOPATH, running the following command will allow "go get" or "go install" to succeed:

Thanks! This helped me out :)