Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"./x.py test --stage 0 --doc" does not run doc tests #73286

Closed
RalfJung opened this issue Jun 12, 2020 · 9 comments · Fixed by #73411
Closed

"./x.py test --stage 0 --doc" does not run doc tests #73286

RalfJung opened this issue Jun 12, 2020 · 9 comments · Fixed by #73411
Labels
A-doctests Area: Documentation tests, run by rustdoc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@RalfJung
Copy link
Member

I would expect the following command to run the libcore doc tests:

./x.py test --stage 0 src/libcore --doc

This used to work, but now all it does it build libcore and its test binaries. No tests are actually being run (which I verified by breaking one of them deliberately).

@Mark-Simulacrum any idea what is happening?
@RalfJung RalfJung mentioned this issue Jun 12, 2020
Merged
@jonas-schievink jonas-schievink added A-doctests Area: Documentation tests, run by rustdoc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jun 12, 2020
@ehuss
Copy link
Contributor

ehuss commented Jun 12, 2020

I can confirm, doctests are not being run for std. I think a bigger question is, have they ever worked? Cargo will skip doctests whenever --target is used, which AFAIK is how std is always built/tested.

@RalfJung
Copy link
Member Author

I am pretty sure that have used this command before and got test failures when I had screwed up. If you want, I can go back to older versions and verify.

@ehuss
Copy link
Contributor

ehuss commented Jun 12, 2020

I'll try to figure out what's going on.

I tried running with -Zdoctest-xcompile and fortunately nothing looks broken.

It's also very possible I am missing something, but I edited some doctests with errors, and a bare ./x.py test doesn't fail.

@ehuss
Copy link
Contributor

ehuss commented Jun 12, 2020 •
edited
Loading

I see what's going on. It's a consequence of rust-lang/cargo#6892. That means doctests haven't been running since 1.39. This was wrong, see below.

I'll try to put together a fix.

@RalfJung
Copy link
Member Author

That means doctests haven't been running since 1.39.

Ouch. Is there a way we can ensure this does not happen again in the future? Looks like our test suite needs a test suite...

@Mark-Simulacrum
Copy link
Member

It's pretty hard :/

I guess we can add a dummy doc test that writes some "well known" file and we can assert that the file is present -- perhaps do this per-crate.

@ehuss ehuss mentioned this issue Jun 14, 2020
Merged
@ehuss
Copy link
Contributor

ehuss commented Jun 14, 2020

OK, it wasn't as bad as I originally reported. This regressed via rust-lang/cargo#8167 which is only in 1.45 (beta). I have a fix started at rust-lang/cargo#8358. This will require various steps:

  1. Fixing cargo.
  2. Backport to beta.
  3. Update cargo on beta.
  4. Wait for new beta release.
  5. Update bootstrap on nightly.

@RalfJung
Copy link
Member Author

Well, but it does mean that currently on CI we do not run doctests, right? --stage 1 does not change this.

@ehuss
Copy link
Contributor

ehuss commented Jun 14, 2020

Right, I just mean it hasn't been that long (only about a week).

bors added a commit to rust-lang/cargo that referenced this issue Jun 15, 2020
@bors
Auto merge of #8358 - ehuss:fix-target-host-doctest, r=alexcrichton
4e14cc1 
Fix doctests not running with --target=HOST.

There was a regression in #8167 where `cargo test --target=$HOST` stopped running doctests. This caused doctests to silently stop running in rust-lang/rust (rust-lang/rust#73286).  This PR restores the original behavior where `--target=$HOST` behaves as-if it is a normal host test.

There was a discussion about this at #8167 (review), but I think I let it slip through the cracks.
@ehuss ehuss mentioned this issue Jun 16, 2020
Merged
Manishearth added a commit to Manishearth/rust that referenced this issue Jun 20, 2020
@Manishearth
Rollup merge of rust-lang#73411 - ehuss:bump-stage0, r=Mark-Simulacrum
ccef1f5 
Update bootstrap to rustc 1.45.0-beta.2 (1dc0f6d 2020-06-15)

Pulls in changes from rust-lang#73326.

Closes rust-lang#73286
Manishearth added a commit to Manishearth/rust that referenced this issue Jun 20, 2020
@Manishearth
Rollup merge of rust-lang#73411 - ehuss:bump-stage0, r=Mark-Simulacrum
abd4096 
Update bootstrap to rustc 1.45.0-beta.2 (1dc0f6d 2020-06-15)

Pulls in changes from rust-lang#73326.

Closes rust-lang#73286
@bors bors closed this as completed in 9a82736 Jun 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-doctests Area: Documentation tests, run by rustdoc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update bootstrap to rustc 1.45.0-beta.2 (1dc0f6d8e 2020-06-15) ehuss/rust
4 participants
@ehuss @RalfJung @jonas-schievink @Mark-Simulacrum