Build caching issue: stage 0 fails in thiserror with "unresolved imports std::any::Demand, std::any::Provider"

[RalfJung]

Error message says it all:

$ ./x.py test ui
Building bootstrap
    Finished dev [unoptimized] target(s) in 0.02s
Building stage0 library artifacts (x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.07s
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
   Compiling parking_lot_core v0.9.8
   Compiling io-lifetimes v1.0.11
   Compiling parking_lot_core v0.8.6
   Compiling perf-event-open-sys v3.0.0
   Compiling memmap2 v0.2.3
   Compiling getrandom v0.2.10
   Compiling jobserver v0.1.26
   Compiling generic-array v0.14.7
   Compiling stacker v0.1.15
   Compiling indexmap v2.0.0
   Compiling tracing v0.1.37
   Compiling thiserror-core v1.0.38
   Compiling flate2 v1.0.26
   Compiling tinystr v0.7.1
   Compiling thiserror v1.0.40
   Compiling rustc_lexer v0.1.0 (/home/r/src/rust/rustc/compiler/rustc_lexer)
   Compiling type-map v0.4.0
   Compiling itoa v1.0.6
   Compiling self_cell v0.10.2
   Compiling ryu v1.0.13
error[E0432]: unresolved imports `std::any::Demand`, `std::any::Provider`
 --> /home/r/.cargo/registry/src/index.crates.io-6f17d22bba15001f/thiserror-1.0.40/src/provide.rs:1:16
  |
1 | use std::any::{Demand, Provider};
  |                ^^^^^^  ^^^^^^^^ no `Provider` in `any`
  |                |
  |                no `Demand` in `any`

For more information about this error, try `rustc --explain E0432`.
error: could not compile `thiserror` (lib) due to previous error

This is probably related to #113464. It goes away when I rm build/host/stage0-rustc -rf. Looks like the thiserror build script is caching its output too aggressively, so it doesn't notice that it needs to be rerun? Cc @dtolnay

[dtolnay]

As far as I understand, this is a bootstrap bug, not caching.

See 1st paragraph of dtolnay/thiserror#243 (review).

[dtolnay]

Although, sounds like I'm wrong because I wouldn't have expected rm build/host/stage0-rustc to help. ¯\_(ツ)_/¯

I don't know of anything the thiserror build script is doing wrong with regard to caching.

[RalfJung]

As far as I understand, this is a bootstrap bug, not caching.

I think you mean rust-lang/cargo#11138, which I'd say is a cargo limitation.
But indeed then clearing my build cache wouldn't have helped...

Here we don't have a proc macro dependency I think, so I don't think it is that problem.

[RalfJung]

I think what happens is that usually, when the rustc version changes, cargo will rerun build scripts automatically. But here the rustc version is always dev and so even though the output of running rustc on a particular file can change, cargo keeps using the old cache. thiserror would have to say "rerun if the output of running $RUSTC might change", but it doesn't have a way to do that. (The rustc binary didn't even change, the sysroot did.)

Maybe bootstrap should delete the cached build script output when the stage0 library artifacts change? Not sure if it even knows that this is the case though.
Cc @rust-lang/bootstrap

[onur-ozkan]

Maybe bootstrap should delete the cached build script output when the stage0 library artifacts change?

Will check what we can do on the bootstrap side by the coming weekend. Thanks for the ping.

[Mark-Simulacrum]

Hm, we usually rely on Cargo doing the rebuild as a result of depinfo containing references to the sysroot when it uses it (via -Zbinary-dep-depinfo), but in this case that doesn't get passed since we don't control the rustc invocation. I'm not sure we have a good way of recognizing when we need the rebuild and when we don't; we wouldn't want to always clear build scripts because that would cause many more spurious rebuilds...

[RalfJung]

Hm yeah, maybe the right solution is for cargo to provide a native way of doing "probe builds", that integrates with its caching...

[onur-ozkan]

Fixing this on bootstrap will most likely require implementing hacky tricks(like creating/moving some values as files between outputs and comparing them each time on the compilations) and that could make our complex flow even more confusing. And there's a good chance these things could cause unexpected problems that are hard to detect.

Hm yeah, maybe the right solution is for cargo to provide a native way of doing "probe builds", that integrates with its caching...

+1

[RalfJung]

FWIW for now thiserror is working around this via dtolnay/thiserror#248