bootstrap: "test library/core --stage 2" does not actually test stage 2

[RalfJung]

./x.py test library/core --stage 1 builds the stage1 compiler, then uses that to build libcore, and then tests that. That's all working as intended, then only strange part is the output it prints:

Testing  {core} (stage1 -> stage2, x86_64-unknown-linux-gnu)

There's no stage 2 involved here, so this makes little sense to me.

However, ./x.py test library/core --stage 2 should be building the stage 2 compiler, use that to build libcore, and then test that. However that's not happening, I don't even get the second rustc build for this command:

$ ./x.py test library/core --stage 2
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.02s
Building stage0 library artifacts (x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.06s
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.10s
Creating a sysroot for stage1 compiler (use `rustup toolchain link 'name' build/host/stage1`)
Building stage1 library artifacts (x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.06s
Building tool rustdoc (stage0 -> stage1, x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.07s
Testing  {core} (stage1 -> stage2, x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.06s
     Running tests/lib.rs (build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/coretests-e9f28340ff8f3e51)

running 1647 tests

That's a problem because when I use this command I specifically want to test the libcore that was built by the rustc that was built by the current source code -- I want to go around the bootstrap loop.

Cc @rust-lang/bootstrap

[the8472]

You have to set full-bootstrap = true in config.toml to get the standard library rebuilt because normally the stage1 and stage2 libs should be identical since they get built by the same compiler version.

That's a problem because when I use this command I specifically want to test the libcore that was built by the rustc that was built by the current source code

Testing stage 1 should be sufficient for that unless you're using --keep-stage

[RalfJung]

Testing stage 1 should be sufficient for that

No it's not. There's a difference between "the libcore that was built by the current rustc sources" and "the libcore that was built by the rustc that was built by the current rustc". I was specifically looking for "segfaults during liballoc build" that might be caused by miscompilations of rustc itself, so I needed the extra bootstrap round.

You have to set full-bootstrap = true in config.toml to get the standard library rebuilt because normally the stage1 and stage2 libs should be identical since they get built by the same compiler version.

Oh I see. I guess that makes sense, though on the other hand when I explicitly ask for stage 2 I don't see why it would optimize that away. If I didn't want full bootstrap I would not ask for stage 2. So now that stage 1 is the default, what even is the point of full-bootstrap = false?

[the8472]

--stage 2 with full-bootstrap = false still does something, it rebuilds the compiler, which is necessary to get rid of the last "built by beta" artifact in the chain.

--stage0

• compiler is not built, it's the beta compiler
• std is built from current source with the beta compiler

--stage1

• compiler is built from current source, by the beta compiler but with current std from stage 0
• std is again built from current source, with the stage 1 compiler. so it's fully up to date

In other words both are from current source but the compiler is still built by the beta compiler.

--stage2 without full bootstrap

• compiler is built from current source, by the stage 1 compiler.
• std is copied over from stage 1 because it has already been built by a current compiler with current optimizations. it should be identical to rebuilding it unless the current-source-compiled-by-beta compiler (stage1) has a miscompilation bug that goes away by doing a current-source-built-by-current-source build.

At this point both are from current source, built by toolchain from current source.

--stage2 with full bootstrap

same as above, except that std does get rebuilt. it should produce identical code anyway.

As the documentation says:

This is only useful for verifying that rustc generates reproducible builds.

[RalfJung]

So full-bootstrap = true only affects stage 2 standard library builds? I always thought it did something with rustc builds.

My question still stands though, when would I write test library/core --stage 2 and not want to have the final library built "properly"? Skipping that final build shaves off less than a minute, the standard library builds pretty fast compared to building rustc twice which this command already does, so the timesave is relatively small.

[RalfJung]

The comment says

# Typically the build system will build the Rust compiler twice. The second
# compiler, however, will simply use its own libraries to link against. If you
# would rather to perform a full bootstrap, compiling the compiler three times,
# then you can set this option to true.

I don't want to compile the compiler three times. I just want the standard library built with the stage that I explicitly specified. So if I need to set this to "true" to fix this issue, then the docs are very misleading. They sound like when I set this, ./x.py test and ./x.py build and so on will build the compiler thrice.

---

Also, there's still a bug where it prints stage1 -> stage2 even when I set --stage 1. That's a bug, right?

[onur-ozkan]

I just want the standard library built with the stage that I explicitly specified.

This sounds reasonable I think. Will check it out in about a week or so.

Also, there's still a bug where it prints stage1 -> stage2 even when I set --stage 1. That's a bug, right?

Yes it is.