Regen cost shape: one corpus read, rustfmt once per distinct input (memo + verified per-version cache), rustfmt_spawns on the receipt

[briansrls]

Regen cost shape: steps 1 and 2 of the round-cost follow-up (stacked on #9735, now merged)

Before/after by phase from the instrument #9735 landed (claim_executor --regen-round-cost --source-root dag --source-root src/v2), all on srv1, all at the fixed point (changed_paths=0), wall_ms.

Step 1 — one corpus read; rustfmt once per distinct input

• compile_stage0 takes the sources its caller already read (corpus_reload phase deleted from the model and the host).
• ResolvedFormatter gains a memo keyed by the raw input bytes (shared by the three consumers: write_emitted_tree, compare_generated_surfaces, tree_digest_from_map) and a per-rustfmt-version disk cache under target/stage0-regen-rustfmt-cache/<version>/, whose entries store the raw input beside the normalized output and are honoured only when the stored raw is byte-equal to the request (a digest names the location, never the identity). rustfmt_spawns is on the receipt (gunbc.regen_round_cost), so the fixed-point claim is checked against the count.

phase	before (75bc500, fixed point)	after, cache cold (8bc9d73, round 1)	after, fixed-point control (8bc9d73, round 2)
seed_build	140	371251	353662
corpus_load	52413	55557	59425
corpus_reload	24317	—	—
compile.frontend	19677	19975	20584
compile.normalize	498	503	534
compile.reconcile	32050	32074	35953
compile.analyses	562	539	585
compile.emit	253444	247815	276982
mirror_write	25124	24204	46
candidate_verify	0	0	0
adjudicate	24997	10	10
hand_verify	127	122	125
digest	47146	22686	74
install	0	0	0
rebuild_from_installed	146	404286	354852
diff	13	11	12
rustfmt_spawns	(not counted)	571	2
regen process (sum of regen./compile. phases)	480355	403485	394318

(seed_build/rebuild_from_installed in the two "after" columns are step 2's defect firing — see below; the regen process itself went 480 s → 394 s, and the three rustfmt passes 97 s → 0.13 s at the fixed point.)

Step 2 — why a one-crate release build cost 228–395 s inside every round

Not sccache, not the profile, not a discarded target/: src/v1/stage0/build.rs declares rerun-if-changed on .git/index (and HEAD/refs) and then runs git status --porcelain during its own run, which refreshes the index — so the build that follows any build sees a changed input, reruns the script and recompiles the whole v1-compiler crate with nothing changed. Measured on a quiescent separate worktree (srv1, tree fce29f50): cold 444 s → no-change build 382 s → after git status 0 s → dirty flips 365 s / 400 s → 0 s. Inside the instrument the same thing fired as seed_build_compiled_crates=1 rebuild_compiled_crates=1 at changed_paths=0 (rounds above: 371 s + 404 s, 354 s + 355 s).

Fix: git --no-optional-locks … in build.rs, so reading git state no longer writes the index the script watches. Proof on the same quiescent worktree at 27e9915c: after-fix build 393 s (real change) → no-change 0 s → after git status 0 s. The rebuilt-from-installed proof is unchanged: the rebuild still runs cargo build against the installed tree and GUNBC_BUILD_IDENTITY still flips on any real change (a dirty tree, a new commit).

After receipt (tree 27e9915c, srv1, fixed point):

regen-round-cost: producer=claim_executor --regen-round-cost host=srv1 tree=27e9915cd2b8b26ebfd788d868fab384011265e6 tree_dirty=false seed_build_compiled_crates=0 rebuild_compiled_crates=0 rustfmt_spawns=20
regen-round-cost: phase=seed_build wall_ms=130 cpu_ms=130
regen-round-cost: phase=corpus_load wall_ms=53687 cpu_ms=53540
regen-round-cost: phase=compile.frontend wall_ms=20201 cpu_ms=20150
regen-round-cost: phase=compile.normalize wall_ms=495 cpu_ms=500
regen-round-cost: phase=compile.reconcile wall_ms=32557 cpu_ms=32460
regen-round-cost: phase=compile.analyses wall_ms=540 cpu_ms=530
regen-round-cost: phase=compile.emit wall_ms=261326 cpu_ms=260490
regen-round-cost: phase=mirror_write wall_ms=3568 cpu_ms=3570
regen-round-cost: phase=candidate_verify wall_ms=0 cpu_ms=0
regen-round-cost: phase=adjudicate wall_ms=10 cpu_ms=0
regen-round-cost: phase=hand_verify wall_ms=121 cpu_ms=130
regen-round-cost: phase=digest wall_ms=2919 cpu_ms=2910
regen-round-cost: phase=install wall_ms=0 cpu_ms=0
regen-round-cost: phase=rebuild_from_installed wall_ms=145 cpu_ms=130
regen-round-cost: phase=diff wall_ms=12 cpu_ms=20
regen-round-cost: total wall_ms=375711 cpu_ms=374560
regen-round-cost: changed_paths=0 []

Round total on this host: 1103–1179 s (steps 1 + 2 defects) → 376 s, with the two cargo builds at 130 ms / 145 ms and rustfmt at 20 spawns (the merge from main brought 9 changed mirrors through the cache once).

What remains of a fixed-point round is the emit (261 s, single-threaded) and the corpus read (55–59 s); a round with a real edit adds one genuine crate compile (~6 min on srv1 under load, ~4 min on a 6-core runner) because every mirror is one crate.

Also in this PR (review 57570 remarks on #9735)

Counts are Nat; the stray .regen-probe-untracked swept in by the shared-filesystem experiment is removed.

Evidence: ten .dag witnesses green on srv1; both Rust lockstep tests green remotely on this head; clippy clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KGCnzm2zkb37mVHgrGCt9L