Implement cargo print-source-root to expose where packages' sources live

[carols10cents]

Fixes #1098. This command will list all of this project's dependencies and where their source is on your file system.

The output is currently an alphabetical list of TOML-formatted dependency name/source location pairs, but that was just the first thing I could think of that would be useful to both people and programs. I'm totally open to changing the output to something else!

Looking forward to any other feedback yinz have!!! 🎉

[rust-highfive]

r? @huonw

(rust_highfive has picked a reviewer for you, use r? to override)

[carols10cents]

And of course I messed up the tests on windows :)

[alexcrichton]

I've been a little hesitant to push on this in the past because there's certainly a lot of design space here in terms of instrumenting Cargo to give back metadata about compilation. For example #1434 was a PR in the past to output even more metadata (but not the same set of data as this).

Originally Cargo was envisioned as being a set of disconnected subcommands that would sorta pipe output between one another to implement each step, and that way the internal plumbing could be reused externally as subcommands for tasks such as this (e.g. discovering where dependencies are located).

I think a good way to think about this may be to basically just take the function signatures of each of the functions in cargo::ops and have that be the output of a command like this. In theory we'd have a subcommand per layer, but we can always start off smaller! This is also just a vague idea about how we might approach metadata extraction from Cargo in a more principled manner.

Overall, on this PR specifically, my thoughts are:

• The output should probably be something implementing RustcEncodable and then printed on stdout. That way we can switch the encoding between TOML and JSON easily. This was one of the original visions of Cargo (lots of machine readable output in various easily read formats) but we never really quite pushed on it hard enough to make it a reality.
• I'm a little worried that you'll want more information than just the roots of all the dependencies. For example this is sorta the input to cargo::ops::cargo_rustc but there's a lot more contextual information in a Package which is left out here. I'd be hesitant to stabilize a command like this without the ability to extend it with more metadata in the future.
• A good deal of the plumbing in this subcommand seems duplicated with other functions in ops, perhaps these can be refactored into just one function here and there?

cc @brson, @wycats, you guys may also have thoughts on this.

[brson]

Not an opinion about this particular PR, but I have a use case just today where I need to know all the names of a crates dependencies so I can go cargo build -p {dependency} and build all the dependencies of a crate before building the crate itself. Is there such a way already? Seems like there might be overlap with this feature.

[alexcrichton]

I don't think there's currently a great way of doing that today short of parsing Cargo.lock unfortunately

[carols10cents]

Originally Cargo was envisioned as being a set of disconnected subcommands that would sorta pipe output between one another to implement each step, and that way the internal plumbing could be reused externally as subcommands for tasks such as this (e.g. discovering where dependencies are located).

I think a good way to think about this may be to basically just take the function signatures of each of the functions in cargo::ops and have that be the output of a command like this. In theory we'd have a subcommand per layer, but we can always start off smaller! This is also just a vague idea about how we might approach metadata extraction from Cargo in a more principled manner.

Ah I see! I'll take another look with this in mind... it feels like things are more tangled together than this right now, but I'll look for opportunities for different seams to make this more of a pipeline. I feel like I need to be able to understand and find all the pieces like:

• Input: Cargo.toml and registry, Output: Cargo.lock
• Input: Cargo.lock, Output: downloaded dependency sources into local locations
• Input: Locations of downloaded dependencies, Output: Compiled dependencies
• Input: Compiled dependencies, Output: rustc command for current package

I know this kind of nests and there are special cases in all of these, but reading through the open issues it sounds like there are use cases for being able to easily externally do each piece of this...

I'll work on understanding this better and potentially asking more specific questions/proposing specific, smaller changes to parts of this :)

The output should probably be something implementing RustcEncodable and then printed on stdout. That way we can switch the encoding between TOML and JSON easily.

OOOHH that would be cool! Definitely a piece I'm missing :)

A good deal of the plumbing in this subcommand seems duplicated with other functions in ops, perhaps these can be refactored into just one function here and there?

You caught me :) I definitely copy-pasted a bunch of stuff to get this to function. I'll look for more opportunities to reorganize and share code like in carols10cents@d1cd597.

I'm a little worried that you'll want more information than just the roots of all the dependencies. For example this is sorta the input to cargo::ops::cargo_rustc but there's a lot more contextual information in a Package which is left out here. I'd be hesitant to stabilize a command like this without the ability to extend it with more metadata in the future.

Yeah, I can see that. I'll keep that in mind working on the RustcEncodable part.

I think I'm going to close this PR for now and try making and submitting smaller refactoring changes to support this. Thank you so much for the comments and guidance, they're very helpful!!!

[alexcrichton]

Ok, sounds good to me, and thanks for taking the initiative on this! I think that your general listing of inputs/outputs sounds pretty good to me, the Cargo types in the backend will also help reflect what's going on and what metadata can be provided (e.g. PackageId vs Package)

[mdinger]

I think rustbyexample will be able to use something like the in the future. The rustbyexample TOC structure would be in the Cargo.toml and so cargo test would automatically test all examples. Then exporting the TOC via JSON or TOML to be input into the doc generator like gitbook or something similar would simplify both the build process and the be tested fairly simply.

Then the entire example set Cargo.toml contains would be of interest and exporting everything the Cargo.toml contains may end up being useful to someone.

[SimonSapin]

#2193 adds a cargo metadata sub-command that contains among other things per-"target" src_path and per-package manifest_path keys.