Compiler lookup path separation

[alexcrichton]

Right now the compiler has one lookup path, defined by -L and RUST_PATH. This one lookup path is used for many different purposes, leading to a number of conflicts. In the compiler today there are a number of distinct concepts of lookup paths, including:

• A lookup path to find native libraries for inclusion in rlibs
• A lookup path to find extern crate dependencies
• A lookup path to find transitive dependencies of extern crate directives.

Conflating all of these concepts into one lookup path leads to bugs such as rust-lang/cargo#1037 and #16402 which would benefit greatly from distinct sets of lookup paths.

I would initially propose something like:

    -L [KIND=]PATH      Add a directory to the compiler's search paths. The
                        KIND provided may be omitted or one of `crate`,
                        `native`, or `dependency`. Each value adds the lookup
                        path to a separate lookup path in the compiler, and if
                        omitted the path is added to all lookup paths.

Using this, cargo itself would only pass -L dependency=..., build scripts would pass -L native=..., and normal uses of the compiler would continue just using -L. This behavior is also backwards compatible with today's semantics.

I'd like to nominate this issue, however, as rust-lang/cargo#1037 is quite a worrying issue which needs to be solved as part of Cargo's stability story.

[emberian]

This design seems sensible to me.

[codyps]

An alternative here is to use prioritization of libraries based on the path they're found in (as traditional linkers do) insead of failing when 2 of the same lib are found.

If we do impliment filesearch path categories, do they really eliminate all the duplicate lib conditions we could run into?

[alexcrichton]

I thought we would be able to do so, but I don't think that it will solve all of our problems. It would primarily not solve rust-lang/cargo#1037 as it wouldn't prevent the operation, and it would only solve #16402 if the compiler's rustlib directory had the highest priority (which seems a little backwards).

We could still choose to do what traditional linkers do with conflicting libs if we have path categories, however.

[pnkfelix]

P-backcompat-libs, 1.0.

[codyps]

Another thing to consider (gcc, ld, and probably others allow) is simply being able to specify the full path to a library when linking, rather than appending a path & library name and hoping that we manage to choose the one intended.

Doing this would also allow cargo (and others) to more accurately communicate their intent to rustc.

[alexcrichton]

That's largely what --extern can be used for, but we'd need to extend it to support native libraries as well as transitive dependencies as well.