Missing support for "minimal module-awareness"

[joshuarubin]

I know that modules are a touchy subject over here in dep land. I just wanted to bring up an issue that we are having while we wanted to support both dep and module dependency management.

Since Go 1.10.3 added "minimal module-awareness", we've started adding go.mod files in some of our libraries.

If a library has a major version 2, then it's module line in go.mod will be module github.com/foo/bar/v2 even if it is being fetched from github.com/foo/bar. Go, since 1.10.3, will build just fine when using imports like import bar "github.com/foo/bar/v2", but dep complains that the repo doesn't have a submodule v2.

As a result, we can't use dep at all if we depend on packages using go.mod.

I know we are bleeding edge here, but any help in reconciling this issue would be much appreciated.

Please note that this is an issue for the current production release of Go, and not limited to builds using the new module support in vgo or go1.11.beta2.

[sdboyer]

I know that modules are a touchy subject over here in dep land. I just wanted to bring up an issue that we are having while we wanted to support both dep and module dependency management.

i appreciate it 😄 but please know, this sorta thing is totally fine - we've no interest in making things harder for the community than it absolutely has to be.

Yeah, Russ brought this to my attention a little while ago. i think the implementation here should be reasonably straightforward, though i'm not sure i'll have time to get to it right away. If you think you have some time to possibly work on a fix, i can point you in the right direction.

[joshuarubin]

I’ve poked around the guts of dep a bit. If you have anything that could get me started, I might be able knock something out.

[sdboyer]

awesome!

So, this is one of the few cases where we actually want to modify pkgtree.ListPackages(). We need to make that parser go.mod-aware, and create some kind of virtual representation of the module's packages at the versioned import path.

Maybe this is something we make transparently available to the solver, or maybe we have to change the API a bit for figuring out if a given path exists. i think the former is probably preferable, because the "API" is currently a map lookup, and i'd rather KISS. That probably just means duplicating the packages into both the module-accessible path and the non-module-accessible path. (for now, i'm not worried about enforcing the invariant in the solver that module-based projects must use the former)

[lopezator]

Also ran into this and waiting eagerly for #1963 to get merged.

[bvisness]

From Russ Cox's post about Go modules:

"Users of dep or vendoring tools should be able to consume repositories using either convention [that is, major branch or major subdirectory]. Certainly we will make sure dep can."

And yet dep still does not understand what v2 means in an import path. Because of this, the "major branch" approach recommended by the Modules wiki page comes with the caveat that tools "such as dep" can have trouble dealing with packages using this approach. In fact, dep is completely unable to handle such a package even though the go compiler itself can.

Some people have reported success using a custom version of dep built from #1963, but that PR is stalled because the maintainers need funding to write tests (?!)

I know the Go team has long since moved on from dep. But this failing of dep makes it harder for libraries to migrate to modules, and as the "official experiment", I think it deserves just a little more attention.

I am tagging a few members of the Go team who may have some advice for how to proceed here. I apologize for yet another entry in your notifications.

@spf13 @bradfitz @bcmills

[mvdan]

Dep was officially deprecated earlier this year, and the proposal to archive this repository was accepted. As such, I'm closing outstanding issues before archiving the repository. For any further comments, please use the proposal thread on the Go issue tracker. Thanks!