Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide indicator of doctest attributes with --persist-doctests #92597

Open
xd009642 opened this issue Jan 5, 2022 · 1 comment
Open

Provide indicator of doctest attributes with --persist-doctests #92597

xd009642 opened this issue Jan 5, 2022 · 1 comment
Labels
A-doctests Area: Documentation tests, run by rustdoc T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@xd009642
Copy link
Contributor

xd009642 commented Jan 5, 2022

Relevant tarpaulin issue: xd009642/tarpaulin#924

So in cargo-tarpaulin we use --persist-doctests in order to keep the doctest binaries around and run them for doctest coverage. In order to maintain the equivalent behaviour to cargo test, we need to keep track of the no_run and ignore attributes, as well as ensure the should_panic doctests exit with a non-zero exit code.

Currently the persisted doctests have a file name which is equivalent to the pah to the source file with the directory separator replaced with an underscore underscore separated integers for line number and also an index to prevent name collisions (likely due to the fact that src/foo/bar.rs and src/foo_bar.rs will normalise to same name when replacing directory separators).

The approach then taken is from the doctest name, it works out candidate files and locations, and then from this checks there for a doctest and looks for the attributes. However, this is just a heuristic and things like #![doc = include_str!("../README.md")] where the readme contains doctests using attributes cause the heuristic to fail.

Adding the attributes to the binary name would be a simple solution. Some way of querying metadata for a doctest and finding the binary name and tying it to a file location would also work. As far as I know there's no stability guarantee for the doc test binary naming - although I may be incorrect. Is there an openness to adapting the doc test binary name or metadata in order to make it simpler to take the persisted doctests and handle them the same way cargo test will?

@Nemo157 Nemo157 added A-doctests Area: Documentation tests, run by rustdoc T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jan 10, 2022
@sunshowers
Copy link
Contributor

I think the best way forward might be that --message-format=json exposes additional lines related to doctests, with file and line numbers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-doctests Area: Documentation tests, run by rustdoc T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
Status: No status
Development

No branches or pull requests

3 participants