doc: RUSTDOCFLAGS and --target is not passed to rustdoc when documenting proc-macros with a --target set

[QuietMisdreavus]

Problem

On docs.rs, we've run into an issue (rust-lang/docs.rs#422) where proc-macros were not correctly being documented because of being passed an explicit --target flag.

Steps

1. Start with a proc-macro crate.
2. Run cargo doc --no-deps --target x86_64-unknown-linux-gnu (substituting your native target or some other target for which you have its std installed)

Expected outcome: Documentation is available in target/x86_64-unknown-linux-gnu/doc.

Actual outcome: Documentation is available in target/doc.

The situation gets worse when RUSTDOCFLAGS is involved; cargo doc does not pass RUSTDOCFLAGS to rustdoc when documenting a proc-macro with an explicit --target.

Possible Solution(s)

My preferred solution would be to always handle calls to rustdoc as if they were regular libraries, i.e. give them a --target, output them into the target-specific directory, etc. The lack of RUSTDOCFLAGS seems like a weird oversight, but given #4423 there may be some crossed wires with rustc and building things that run immediately on the host.

Notes

Docs.rs always runs the latest nightly, but the logs in rust-lang/rust#66796 (comment) were collected with cargo 1.41.0-nightly (750cb1482 2019-11-23).

[ehuss]

Hm, I'm having a hard time trying to figure out what the best behavior would be. I don't think Cargo can always pass --target, because then cargo doc wouldn't work on no_std targets. Cargo could link or copy the host copy of the proc_macro docs into the target directory, but that would be a lie because the docs were actually built for a different target.

[QuietMisdreavus]

I don't think Cargo can always pass --target, because then cargo doc wouldn't work on no_std targets.

Ohhhh, that's a good point. Would it still be feasible to allow for some kind of override to attempt to cross-compile (or cross-document, in this case) a proc-macro crate? For Docs.rs's purposes, we know we're going to be building for targets that have std available, but i'm not sure if there's a way to force cargo to build all of a proc-macro crate's dependencies for a cross target.

[jyn514]

For Docs.rs's purposes, we know we're going to be building for targets that have std available

Well, someone did ask on Discord if we could build more targets. At the time I told them it's feasible to build any target supported by rustup, is that not the case?

Why won't cargo doc --target ... work for no_std targets?

[ehuss]

Why won't cargo doc --target ... work for no_std targets?

Specifically, passing rustdoc --target for host dependencies (like proc_macro crates) would fail for no_std targets because they don't have proc_macro or std, so rustdoc would fail.

[jyn514]

Oh I see. no_std targets must be cross compiled if they have host dependencies that depend on std, since the target does not have std. It then makes no sense to compile a host dependency for that target because --target for a host dependency sets the host for the cross compile, not the target (runtime for proc-macros is when building a crate on the host).

I think I said that all right. Now going from there: I would expect that to behave the same as when compiling for a target that's not installed:

$ cargo build --target i686-pc-windows-msvc
   Compiling cargo-new v0.1.0 (/home/joshua/Documents/Programming/rust/cargo-new)
error[E0463]: can't find crate for `std`
  |
  = note: the `i686-pc-windows-msvc` target may not be installed

error: aborting due to previous error

[alexcrichton]

I think we're pretty unlikely to change Cargo's behavior here, but I think one possible fix for docs.rs would be to use --message-format=json from Cargo which should tell you all the locations of all the artifacts, including documentation. I forget if we already have a message for docs, but it would be quite easy to add one in if it doesn't already exist!

[jyn514]

I think we're pretty unlikely to change Cargo's behavior here, but I think one possible fix for docs.rs would be to use --message-format=json from Cargo which should tell you all the locations of all the artifacts, including documentation. I forget if we already have a message for docs, but it would be quite easy to add one in if it doesn't already exist!

This doesn't help with the biggest problem, which is that RUSTDOCFLAGS isn't passed. We can work around quirks in the output directory on our end.