Bootstrap uses stage 1 artifacts in a stage 2 cross testing context

[tekbar11]

I'm building a cross toolchain hosted on x86_64 Linux targeting AArch64 Linux. In order to test this toolchain, I am using the remote-test-{client,server} tools to run tests on a remote board. Everything runs fine except for one issue.

I noticed that despite running the tests with --stage 2, bootstrap (I suppose it's bootstrap doing this) builds stage 1 artifacts and pushes those to the remote test server instead of the stage 2 artifacts.

Here is the output of the test command to illustrate:

❯ TEST_DEVICE_ADDR="localhost:50200" ./x --stage 2 test --force-rerun --no-fail-fast --target aarch64-unknown-linux-gnu library/std
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.05s
Building stage0 library artifacts (x86_64-unknown-linux-gnu) 
    Finished release [optimized] target(s) in 0.16s
Building compiler artifacts (stage0 -> stage1)
    Finished release [optimized] target(s) in 0.25s
Assembling stage1 compiler
Building stage1 library artifacts (x86_64-unknown-linux-gnu -> aarch64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.17s
REMOTE copy libs to emulator (aarch64-unknown-linux-gnu)
Building tool remote-test-server (stage1:x86_64-unknown-linux-gnu -> stage1:aarch64-unknown-linux-gnu)
   Compiling remote-test-server v0.1.0 (/home/tulisz/aarch64-linux/rust/src/tools/remote-test-server)
    Finished release [optimized] target(s) in 1.02s
Building tool remote-test-client (stage0)
   Compiling remote-test-client v0.1.0 (/home/tulisz/aarch64-linux/rust/src/tools/remote-test-client)
    Finished release [optimized] target(s) in 0.95s
Connecting to remote device localhost:50200 ...
done pushing "/home/tulisz/aarch64-linux/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd-668e19cdfe2bbd09.so"
done pushing "/home/tulisz/aarch64-linux/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/aarch64-unknown-linux-gnu/lib/libtest-b4d8c23afb91914c.so"
Building tool rustdoc (stage0 -> stage1)
    Finished release [optimized] target(s) in 0.18s
Testing {std} stage1 (x86_64-unknown-linux-gnu -> aarch64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.17s
     Running unittests src/lib.rs (build/x86_64-unknown-linux-gnu/stage1-std/aarch64-unknown-linux-gnu/release/deps/std-de648538d02ebbda)
uploaded "/home/tulisz/aarch64-linux/rust/build/x86_64-unknown-linux-gnu/stage1-std/aarch64-unknown-linux-gnu/release/deps/std-de648538d02ebbda", waiting for result

I know the localhost in the test device address is confusing, I'm using a SSH tunnel to access the remote board, it's still running on AArch64 Linux. 😅

On the server side I can see that it is indeed the stage 1 artifacts that are being pushed (hashes are the same):

starting test server
listening on 0.0.0.0:50200!
Received ping
push "/tmp/work/libstd-668e19cdfe2bbd09.so"
push "/tmp/work/libtest-b4d8c23afb91914c.so"
[...]

Thing is, I would like to validate the stage 2 artifacts. Any idea how to correct this behavior? Is this a bug in bootstrap (or any of the tools)?

[jyn514]

this is probably related to remote-test-harness specifically, i don't think this bug is present when testing natively (e.g. cross compiling from x32 to x64 Linux)

[onur-ozkan]

This is related with #118233

Temporary solution would be setting build.full-bootstrap = true in your config.toml.