Skip to content

Regen normalization: memo fixed points, batch rustfmt per pass; seed build reads its compiled-crate count - #13481

Merged
gunbai-bot[bot] merged 5 commits into
mainfrom
session/jolly-moth-17-rustfmt
Oct 7, 2026
Merged

gunbai-bot[bot] merged 5 commits into
mainfrom
session/jolly-moth-17-rustfmt

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Fast-regen layer 3 follow-up: two cost findings from the first //gunbc/instruments:regen-round-cost measurement (instrument-dispatch run 37293307189, on #13370's head). Both were scoped by sharp-raven-357.

1. rustfmt spawns: a bare-minimum-cost defect (DESIGN §6)

That round made 654 rustfmt spawns for 163 mirrors, roughly 58 s of mirror_write + digest. Two causes:

  • Every distinct input paid its own fixed-point seek of at least two spawns: format, then confirm.
  • The committed side (whose bytes ARE the previous normalized output) re-ran a seek the emitted side had just finished.

Fix, in required_regen_host:

  • memo_normalized records a confirmed fixed point as itself. It's exact: the seek returns Y only once it has seen rustfmt(Y) == Y. A committed mirror equal to a normalized output then costs no spawn.
  • normalize_population runs the same fixed-point seek over a whole population with one rustfmt invocation per pass. write_emitted_tree and tree_digest_for_basenames prefetch through it.
  • It passes --config skip_children=true. Given a FILE, rustfmt follows mod declarations to sibling files, which stdin mode never does, and that refused a remote round with failed to resolve mod. On rustfmt 1.9.0-stable the option is accepted and the output is byte-equal to stdin. A rustfmt that ignored it would refuse loudly on the unresolved mod, never produce wrong bytes.
  • Control: batched_normalization_is_byte_identical_to_the_single_seek uses fresh formatters with no shared memo. Every member's batched bytes must equal the single stdin seek's, and each output must be memoized as its own fixed point. The inputs include a member declaring a module with no sibling file. Run remotely and green. Unit tests are off CI (rust_unit_tests_off_the_merge_path), so this executed run is its evidence.

2. Seed build reported compiled_crates=0 while spending 655 s of CPU

The cause is the parse, not the build. The workflows set CARGO_TERM_COLOR=always, the round's cargo child inherits it, and cargo's Compiling lines arrive with escape codes, so the count read 0. seed_cargo_build now passes --color never.

The build itself DID compile, at 655 s CPU in 203 s wall. The next dispatch will say which crates. My suspicion is unconfirmed: the job builds -p v1-compiler --bin gunbc while the round builds --bin claim_executor without -p, so cargo may resolve features differently. Changing that waits for the evidence.

Declared

The three new host declarations get gunbc.regen_rustfmt_batch_seed_growth, enrolled in seed_growth_admission.

Evidence

🤖 Generated with Claude Code

Brian Searls and others added 4 commits October 5, 2026 14:37
…build reads its compiled-crate count

Measured by //gunbc/instruments:regen-round-cost (run 37293307189): 654 rustfmt
spawns for 163 mirrors (~58 s of a ~12 min round), because every distinct
input paid its own two-spawn fixed-point seek and the committed side re-ran a
seek the emitted side had just completed.

- memo_normalized records a confirmed fixed point as itself, so committed
  mirrors equal to a normalized output cost no spawn.
- normalize_population runs the same fixed-point seek for a whole population
  with ONE rustfmt invocation per pass; write_emitted_tree and
  tree_digest_for_basenames prefetch through it.
- Control: batched_normalization_is_byte_identical_to_the_single_seek.

seed_cargo_build passes --color never: an inherited CARGO_TERM_COLOR=always
prefixed cargo's Compiling lines with escapes, so compiled_crates read 0 while
the build spent 655 s of CPU. The next dispatch says whether it was a rebuild.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t followed; control covers it

A remote round refused the batch with 'failed to resolve mod
extdeps_filesystem_rust_realization': given a FILE, rustfmt follows mod
declarations to siblings, which stdin mode never does. skip_children=true
formats each member alone (checked on rustfmt 1.9.0-stable: accepted, output
byte-equal to stdin). The byte-identity control now includes a member
declaring a module with no sibling file.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 6, 2026
Merged via the queue into main with commit 4d78cfd Oct 7, 2026
5 checks passed
@gunbai-bot
gunbai-bot Bot deleted the session/jolly-moth-17-rustfmt branch October 7, 2026 09:02
@briansrls
briansrls restored the session/jolly-moth-17-rustfmt branch October 7, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants