"zef install Some::Module" doesn't show the alternatives

[titsuki]

Context

"zef install Some::Module" doesn't show the alternatives when there are two modules that have the same name but authored by the different user:

$ zef --force-install install "Term::termios"
===> Searching for: Term::termios
===> Building: Term::termios:ver<0.0.1>:auth<github:titsuki>
===> Building [OK] for Term::termios:ver<0.0.1>:auth<github:titsuki>
===> Testing: Term::termios:ver<0.0.1>:auth<github:titsuki>
===> Testing [OK] for Term::termios:ver<0.0.1>:auth<github:titsuki>
===> Installing: Term::termios:ver<0.0.1>:auth<github:titsuki>

$ zef --force-install install "Term::termios:ver<0.0.1>:auth<github:krunen>"
===> Searching for: Term::termios:ver<0.0.1>:auth<github:krunen>
No candidates found matching identity: Term::termios:ver<0.0.1>:auth<github:krunen>

$ zef --force-install install "Term::termios:auth<github:krunen>"
===> Searching for: Term::termios:auth<github:krunen>
===> Testing: Term::termios:auth<github:krunen>
===> Testing [OK] for Term::termios:auth<github:krunen>
===> Installing: Term::termios:auth<github:krunen>

Expected Behavior

zef --force-install install "Term::termios" shows the alternatives to install and stops its install process temporarily.

Actual Behavior

zef --force-install install "Term::termios" ignores the author name field (i.e., auth) and selects the newer one (i.e., Term::termios:ver<0.0.1>:authgithub:titsuki).

Steps to Reproduce

I showed in the Context section.

Your Environment

• perl6 -v

$ perl6 -v
This is Rakudo version 2017.12-213-g4d5e097af built on MoarVM version 2017.12.1-25-g1e2a3fbe9
implementing Perl 6.c.

• zef list --installed

$ zef list --installed
===> Found via /home/itoyota/experimental-rakudo/install/share/perl6/site
App::Mi6:ver<0.1.2>
CPAN::Uploader::Tiny:ver<0.0.3>
Config::JSON:ver<1.001003>
Digest:ver<0.3.4>:auth<Lucien Grondin>
File::Directory::Tree:auth<labster>
File::Find:ver<0.1>
File::Temp:ver<0.0.6>
File::Which:ver<1.0.0>
HTTP::Tinyish:ver<0.1.0>
Inline::Perl5:ver<0.29>:auth<github:niner>
JSON::Class:ver<0.0.10>:auth<github:jonathanstowe>
JSON::Fast:ver<0.9.9>
JSON::Marshal:ver<0.0.14>:auth<github:jonathanstowe>
JSON::Name:ver<0.0.2>:auth<github:jonathanstowe>
JSON::Pretty:ver<0.1.0>:auth<github:FROGGS>
JSON::Tiny:ver<1.0>
JSON::Unmarshal:ver<0.08>
LibraryMake:ver<1.0.0>:auth<github:retupmoca>
META6:ver<0.0.19>:auth<github:jonathanstowe>
Pod::To::HTML:ver<0.3.13>
Pod::To::Markdown:ver<0.1.3>
Shell::Command
Temp::Path:ver<1.001003>
Term::termios:auth<github:krunen>
Term::termios:ver<0.0.1>:auth<github:titsuki>
Terminal::Getpass:ver<0.0.1>
Terminal::Getpass:ver<0.0.2>
Terminal::Getpass:ver<0.0.3>
Terminal::Getpass:ver<0.0.4>
Test::META:ver<0.0.13>:auth<github:jonathanstowe>
Test::When:ver<1.001008>
Testo:ver<1.003004>
Timer::Breakable:ver<0.1.0>
URI:ver<0.1.4>
zef:ver<0.1.32>:auth<github:ugexe>
===> Found via /home/itoyota/experimental-rakudo/install/share/perl6
CORE:ver<6.c>:auth<perl>

[ugexe]

To preface, first consider these points:

1. the behavior of explicitness of auth already exists, although in the non-default case ( zef install "Term::termios:auth<github:titsuki>" ).

2. zef emulates CompUnit::Repository in its resolution strategy.
   a) use Foo::Bar / "depends" : [ "Foo::Bar" ] / zef install Foo::Bar
   b) use Foo::Bar:auth<xxx> / "depends" : [ "Foo::Bar:auth<xxx>" ] / zef install Foo::Bar:auth<xxx>

---

So what should happen what someone does use Term::termios; and they have alternatives? We wouldn't want it to require interaction -- even failure is better than that. Instead there is some implicit responsibility for either/or:

1. whomever is managing the dependencies to keep the module repository in such a state that even if alternatives exist they will be chosen properly by use Term::termios
2. module authors start writing explicit use Term::termio:auth<github:titsuki> and "depends" : [ "Term::termios:auth<github:titsuki>" ]

While 1 is possible, 2 is the direction that e.g. production grade modules and the ecosystem need to move regardless. It kind of sucks that this means that inside a module like zef I should be pinning the version inside every-single-provides ala use Zef::Blorg:ver<0.2.1> ( or else installing a newer version means the older version, if loaded, might load some of itself from the newer version ). In perl6 we should probably create something like use-from-this-dist-only Zef::Blorg;, but that is neither here-nor-there.

Also consider that ultimately ecosystems themselves will play their own role in policy ( what gets chosen when a query has multiple matching distributions ). Policy essentially comes from the package list served by an ecosystem ( to oversimplify ), primarily by the packages included and the ordering of that list.

---

So while this might be a good idea it is only hiding problems that will be encountered ( for instance if the wrong Term::termios was pulled in as a dependency earlier, which can't always have user interaction and must choose a graph to traverse ). Currently it might be suggested to run zef info Foo::Bar to check what will be chosen or zef search Foo::Bar to get the list of full identifies to give to zef install ....

More broadly I think what is needed is a general utility/module to ask for user input based around meta data. Specifically I'm thinking of license auth version and api. But ultimately it is the module author's responsibility to declare everything appropriately, and the user to ask for exactly what they want ( so use search to find what you want, not count on install to present alternatives ).

[titsuki]

Thanks for your neat explanation!

So while this might be a good idea it is only hiding problems that will be encountered ( for instance if the wrong Term::termios was pulled in as a dependency earlier, which can't always have user interaction and must choose a graph to traverse ).

I understand that.
However, IMHO, zef can warn something like this even if it can't have user interaction:

Ambiguous dependencies are detected. The following are chosen:
- Term::termios
   - Term::termios:auth<github:titsuki>:ver<0.0.1>
...

In perl6 we should probably create something like use-from-this-dist-only Zef::Blorg;, but that is neither here-nor-there.

It sounds good to me.

[2colours]

zef emulates CompUnit::Repository in its resolution strategy.

I'm still not 100% I get this right but it sounds like zef ultimately installs (or at least pretends to install) compilation units (or maybe more accurately modules) as opposed to distributions.

Do I get it right? Because that sounds like a horrible idea, however it would explain the problem with the two Digest::HMACs on two different conceptual levels - one is a distribution while the other is only an exposed module.

It immediately illustrates the fundamental problem of that approach: a module that can have a perfectly legitimate reason to exist on that name, may concur with a standalone, independent, only semantically related distribution that can also have a perfectly legitimate reason to exist on that name.
Then, a dependency makes little to no sense to be a something essentially a compunit - as opposed to a distribution which is the real unit of releasing modules. zef should at least respect that when installing a distribution - not a lower-level module, mind you.

The bitter pill is that this in itself wouldn't solve the whole problem of the name collision: use does act on modules and without context, it still wouldn't know which one to choose. It sounds nice in theory that it should "obviously" resolve to the one in the dependency - however, I suspect that would be nowhere obvious to add that transformation to the installation step, let alone CompUnit::Registry...

Anyway, having said that, can we agree that dependencies and units of installation for zef are not modules but distributions?

[2colours]

I forgot to add the "inb4" of "point down the auth". Two problems (edit: in the meantime it turned into 3 lol) with that:

1. it basically makes the whole naming system kinda useless if one always must add the auth suffix - why not just enforce unique names, then?
2. as I said, it really seems the plot itself was legitimate, "nobody did anything wrong". It shouldn't be necessary to patch a conceptional gap with extra measures
3. this might be the least theoretical and more practically problematic point: if dozens of modules a couple of years ago, in a highly different ecosystem situation, did write use Digest::HMAC anyway, will we just accept that unless somebody walks around ringing the bells/creating PRs and hoping to get them merged, all of them break? Production level or not, this doesn't sound like an acceptable resolution. A good system wouldn't leave you in this situation, especially when "nobody did anything wrong".