bootstrap: Move some shared source files into a new shim_utils crate

[Zalathar]

Bootstrap's rustc and rustdoc shims are mostly self-contained, but they do share some code with each other or with other bootstrap tools.

Historically that sharing was achieved using #[path] attributes pointing to the shared files. That's unusual, making the code harder to navigate, and making dependencies and warnings harder to manage. It also causes problems for rust-analyzer, which currently struggles with source files outside the crate root.

This PR therefore takes three such source files, and moves them into a new shim_utils crate, which can then be used as a normal crate-path dependency. That new crate is separate from the existing build_helper crate, since that crate contains other code that is not of interest to the shims specifically.

Former paths of the files that have been moved:

• src/bootstrap/src/utils/proc_macro_deps.rs
• src/bootstrap/src/utils/shared_helpers.rs
• src/build_helper/src/arg_file_command.rs

---

Relevant PRs that created these shared source files:

• bootstrap: Don't apply -Ztls-model=initial-exec to deps of proc-macros #134865
• unify dylib and bin_helpers and create shared_helpers::parse_value_from_args #127108
• Enable Cargo's new build-dir layout #155439

---

r? Kobzol (or bootstrap)

[rustbot]

The list of allowed third-party dependencies may have been modified! You must ensure that any new dependencies have compatible licenses before merging.

cc @davidtwco, @BoxyUwU

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

Some changes occurred in src/tools/compiletest

cc @jieyouxu

[rustbot]

Kobzol is not on the review rotation at the moment.
They may take a while to respond.

[Zalathar]

The list of allowed third-party dependencies may have been modified! You must ensure that any new dependencies have compatible licenses before merging.

No changes to any of the actual lists or logic in src/tools/tidy/src/deps.rs, just an adjustment to the path of proc_macro_deps.rs.

[Zalathar]

Local benchmarks indicate no measurable change to bootstrap build times:

hyperfine --warmup=1 --runs=5 --ignore-failure --prepare="./x clean" "./x run cyclic-step" --show-output

before:
  Time (mean ± σ):     17.929 s ±  0.060 s    [User: 46.763 s, System: 5.325 s]
  Range (min … max):   17.890 s … 18.032 s    5 runs

after:
  Time (mean ± σ):     17.965 s ±  0.097 s    [User: 46.615 s, System: 5.323 s]
  Range (min … max):   17.887 s … 18.132 s    5 runs

[jieyouxu]

Yes! This tripped me up a few times in the past.

[Kobzol]

Thank you, that looks like a good cleanup, the #[path] handling was always a mess.

View changes since this review

[Kobzol]

Doesn't have to be in this PR, but I think that these tests should live directly within shim_utils, and we should create a new default test step for that crate in bootstrap.