Errors you may encounter when upgrading this dependency

[KateGo520]

(The purpose of this report is to alert gravitational/teleport to the possible problems when gravitational/teleport try to upgrade the following dependencies)

An error will happen when upgrading library prometheus/client_golang:

github.com/prometheus/client_golang

-Latest Version: V1.6.0 (Latest commit 6edbbd9 on 28 Apr)
[NOW, you used version v1.0.0]
-Where did you use it:
https://github.com/gravitational/teleport/search?q=prometheus%2Fclient_golang&unscoped_q=prometheus%2Fclient_golang
-Detail:

github.com/prometheus/client_golang/go.mod

module github.com/prometheus/client_golang
go 1.11
require (
	github.com/beorn7/perks v1.0.1
	github.com/cespare/xxhash/v2 v2.1.1
	…
)

github.com/prometheus/client_golang/prometheus/registry.go

package prometheus
import (
	"github.com/cespare/xxhash/v2"
	"github.com/golang/protobuf/proto"
	…
)

This problem has existed since version v1.2.0. If you try to get these version, you will get an error: no package exists at "github.com/cespare/xxhash/v2"

Why?

1. These dependencies all added Go modules in the last version.

2. They all comply with the specification of "Releasing Modules for v2 or higher" available in the Modules documentation. Quoting the specification:

A package that has opted in to modules must include the major version in the import path to import any v2+ modules. Such as: Repo github.com/my/module opted in to Modules on version v3.x.y. Then the import statements within the module to also use /v3 (e.g., import "github.com/my/module/v3/mypkg").

3. This "github.com/my/module/v3/mypkg" is not the physical path. So earlier versions of Go (including those that don't have minimal module awareness) plus all tooling(like dep, glide, govendor, etc) don't have minimal module awareness as of now and therefore don't handle import paths correctly See golang/dep#1962, golang/dep#2139.

Note: creating a new branch is not required. If instead you have been previously releasing on master and would prefer to tag v3.0.0 on master, that is a viable option. (However, be aware that introducing an incompatible API change in master can cause issues for non-modules users who issue a go get -u given the go tool is not aware of semver prior to Go 1.11 or when module mode is not enabled in Go 1.11+).
Pre-existing dependency management solutions such as dep currently can have problems consuming a v2+ module created in this way. See for example dep#1962.
https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Solution

1. Migrate to Go Modules.

Go Modules is the general trend of ecosystem, if you want a better upgrade package experience, migrating to Go Modules is a good choice.

Migrate to modules will be accompanied by the introduction of virtual paths(It was discussed above).

This "github.com/my/module/v3/mypkg" is not the physical path. So Go versions older than 1.9.7 and 1.10.3 plus all third-party dependency management tools (like dep, glide, govendor, etc) don't have minimal module awareness as of now and therefore don't handle import paths correctly.

Then the downstream projects might be negatively affected in their building if they are module-unaware (Go versions older than 1.9.7 and 1.10.3; Or use third-party dependency management tools, such as: Dep, glide, govendor…).

2. Maintaining v2+ libraries that use Go Modules in Vendor directories.

If gravitational/teleport want to keep using the dependency manage tools (like dep, glide, govendor, etc), and still want to upgrade the dependencies, can choose this fix strategy.
Manually download the dependencies into the vendor directory and do compatibility dispose(materialize the virtual path or delete the virtual part of the path). Avoid fetching the dependencies by virtual import paths. This may add some maintenance overhead compared to using modules.

There are 2 module users downstream, such as gravitational/oom, gravitational/gravity…)
https://github.com/search?q=gravitational%2Fteleport++filename%3Ago.mod

As the import paths have different meanings between the projects adopting module repos and the non-module repos, materialize the virtual path is a better way to solve the issue, while ensuring compatibility with downstream module users. A textbook example provided by repo github.com/moby/moby is here:
https://github.com/moby/moby/blob/master/VENDORING.md
https://github.com/moby/moby/blob/master/vendor.conf
In the vendor directory, github.com/moby/moby adds the /vN subdirectory in the corresponding dependencies.
This will help more downstream module users to work well with your package.

3. Request upstream to do compatibility processing.

The prometheus/client_golang have 1049 module-unaware users in github, such as: AndreaGreco/mqtt_sensor_exporter, seekplum/plum_exporter, arl/monitoring…
https://github.com/search?q=prometheus%2Fclient_golang+filename%3Avendor.conf+filename%3Avendor.json+filename%3Aglide.toml+filename%3AGodep.toml+filename%3AGodep.json&type=Code

Summary

You can make a choice when you meet this DM issues by balancing your own development schedules/mode against the affects on the downstream projects.

For this issue, Solution 1 can maximize your benefits and with minimal impacts to your downstream projects the ecosystem.

References

• https://github.com/golang/go/wiki/Modules#semantic-import-versioning
• https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning
• https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Do you plan to upgrade the libraries in near future?
Hope this issue report can help you ^_^
Thank you very much for your attention.

Best regards,
Kate

[KateGo520]

@klizhentas @kontsevoy

[awly]

Thanks for the report.
We plan to migrate to Go modules after a few large branches get merged into master (to avoid disruption for the authors).
I'll close this issue in favor of #3541