Doctests don't work in bin targets

[kornelski]

It's surprising that doctests don't work binary targets:

cargo new foo --bin

/// ```rust
/// assert!(false);
/// ```
fn main() {
    println!("Hello, world!");
}

cargo test --all

I'd expect the above to fail, but it looks like the doc-comment code is never ran. Even if I explicitly enable doctests for the binary:

[[bin]]
name = "foo"
path = "src/main.rs"
doctest = true

they still aren't run.

[alexcrichton]

This is a limitation of rustdoc, it can't run doctests over binaries.

[huxi]

Could this limitation be addressed? I elaborated a bit why I consider this important over in the users forum.

[alexcrichton]

Heh everything can be addressed with enough elbow grease! It's a matter of design and problem solving now moreso than "let's fix the accidental omission"

[huxi]

I wasn't sure if your comment was meant in a "works as intended" or "it has to be like that" way so I wanted to add a bit more background info.

The issue description isn't 100% on point.

/// ```rust
/// assert!(false);
/// ```
fn foo() {
    println!("Hello, world!");
}

In the code above rustdoc won't execute the doctest, neither in case of lib nor bin, because it is not pub. This is also problematic.

In case of bin even pub doctests are ignored.

[alexcrichton]

Indeed that is correct! It may be best to discuss this at rust-lang/rust rather than Cargo, as this is all mostly remnants of rustdoc itself

[vi]

If it is tricky to get

/// ```

to execute always, maybe it can warn about it instead? Like "This doctest is not expected to be executed because of {it is a non-library crate, not a public item}".

[kornelski]

Moved issue to: rust-lang/rust#50784

[jyn514]

@kornelski this is now supported in rustdoc: rust-lang/rust#50784 (comment)
Can you reopen the issue?

[kriomant]

Is this issue blocked by anything or there's just no spare hands to fix it? Will you accept PR?

[epage]

@kriomant Looking at rust-lang/rust#50784, it looks like this is all in cargo's court and that we have most of the tools in place for this. I've gone ahead and marked this as "needs mentor". That doesn't mean this can't be worked on but that you likely won't be able to get much help from the cargo team.

For me, the main question is what happens today if you have a failing doctest in a bin and then upgrade (ie cargo test passes, you upgrade, cargo test fails). For example, we might not fail because we could be defaulting doctest = false on bins. However, we might fail for which we need to consider how much of an impact this will have and what we can do to lessen the impact.

[ehuss]

I'm still skeptical that it would be a good idea to move forward with this as-is. As mentioned in rust-lang/rust#50784 (comment), doctests would not have access to any of the symbols within the binary, and thus would only be able to show illustrations from a corresponding lib target. Since I suspect that is not what most people will expect it to do, I think it could add to confusion about things not working.

[epage]

Thanks for pointing out your comment! I missed that!

[m-ou-se]

[..] As mentioned in rust-lang/rust#50784 (comment), doctests would not have access to any of the symbols within the binary, [..]

It could still be useful to be able to run doctests for documentation in integration tests.

[weihanglo]

This is also tracked in rust-lang/testing-devex-team#5 (comment).