Repository navigation
Regen normalization: memo fixed points, batch rustfmt per pass; seed build reads its compiled-crate count - #13481
Merged
Merged
Conversation
…build reads its compiled-crate count Measured by //gunbc/instruments:regen-round-cost (run 37293307189): 654 rustfmt spawns for 163 mirrors (~58 s of a ~12 min round), because every distinct input paid its own two-spawn fixed-point seek and the committed side re-ran a seek the emitted side had just completed. - memo_normalized records a confirmed fixed point as itself, so committed mirrors equal to a normalized output cost no spawn. - normalize_population runs the same fixed-point seek for a whole population with ONE rustfmt invocation per pass; write_emitted_tree and tree_digest_for_basenames prefetch through it. - Control: batched_normalization_is_byte_identical_to_the_single_seek. seed_cargo_build passes --color never: an inherited CARGO_TERM_COLOR=always prefixed cargo's Compiling lines with escapes, so compiled_crates read 0 while the build spent 655 s of CPU. The next dispatch says whether it was a rebuild. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t followed; control covers it A remote round refused the batch with 'failed to resolve mod extdeps_filesystem_rust_realization': given a FILE, rustfmt follows mod declarations to siblings, which stdin mode never does. skip_children=true formats each member alone (checked on rustfmt 1.9.0-stable: accepted, output byte-equal to stdin). The byte-identity control now includes a member declaring a module with no sibling file. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fast-regen layer 3 follow-up: two cost findings from the first
//gunbc/instruments:regen-round-costmeasurement (instrument-dispatch run 37293307189, on #13370's head). Both were scoped by sharp-raven-357.1. rustfmt spawns: a bare-minimum-cost defect (DESIGN §6)
That round made 654 rustfmt spawns for 163 mirrors, roughly 58 s of
mirror_write+digest. Two causes:Fix, in
required_regen_host:memo_normalizedrecords a confirmed fixed point as itself. It's exact: the seek returns Y only once it has seenrustfmt(Y) == Y. A committed mirror equal to a normalized output then costs no spawn.normalize_populationruns the same fixed-point seek over a whole population with one rustfmt invocation per pass.write_emitted_treeandtree_digest_for_basenamesprefetch through it.--config skip_children=true. Given a FILE, rustfmt followsmoddeclarations to sibling files, which stdin mode never does, and that refused a remote round withfailed to resolve mod. On rustfmt 1.9.0-stable the option is accepted and the output is byte-equal to stdin. A rustfmt that ignored it would refuse loudly on the unresolvedmod, never produce wrong bytes.batched_normalization_is_byte_identical_to_the_single_seekuses fresh formatters with no shared memo. Every member's batched bytes must equal the single stdin seek's, and each output must be memoized as its own fixed point. The inputs include a member declaring a module with no sibling file. Run remotely and green. Unit tests are off CI (rust_unit_tests_off_the_merge_path), so this executed run is its evidence.2. Seed build reported
compiled_crates=0while spending 655 s of CPUThe cause is the parse, not the build. The workflows set
CARGO_TERM_COLOR=always, the round's cargo child inherits it, and cargo'sCompilinglines arrive with escape codes, so the count read 0.seed_cargo_buildnow passes--color never.The build itself DID compile, at 655 s CPU in 203 s wall. The next dispatch will say which crates. My suspicion is unconfirmed: the job builds
-p v1-compiler --bin gunbcwhile the round builds--bin claim_executorwithout-p, so cargo may resolve features differently. Changing that waits for the evidence.Declared
The three new host declarations get
gunbc.regen_rustfmt_batch_seed_growth, enrolled inseed_growth_admission.Evidence
cargo clippy -p v1-compiler --all-targets -D warnings: clean (remote, on the merged-with-main tree).regen-round-costdispatch. That runs after instrument-dispatch: admit //gunbc/instruments:regen-round-cost (priced regen round + one-mirror emit discriminator) #13370 lands, together with the two-mirror probe.🤖 Generated with Claude Code