Repository navigation
fix(regen): normalize both sides through rustfmt before the self-host gate compares - #6957
Merged
Merged
Conversation
… gate compares verify_stage0_matches did a raw byte compare of the committed seed against the --verify fresh emit. The two sides are not normalized symmetrically: the seed is written through two rustfmt passes (rustfmt_generated_crate then rustfmt_workspace), while --verify's fresh side gets only rustfmt_generated_crate -- and that cargo-fmt pass silently skips files whose modules do not build in the faithful-full-regen fresh_dir crate. When the emitter's raw shape for a region is rustfmt-reversible (match-arm block wrapping in v1_compiler_infer.rs after #6848), the raw fresh differs from the fmt-normalized seed by formatting ALONE and the required gate falsely reds -- with no content or determinism defect (write-mode regen reproduces the committed seed byte-for-byte on repeated runs; emit is deterministic). Fix: on a raw-byte mismatch, re-normalize both sides through the same standalone rustfmt (the existing rustfmt_normalize, the same move verify_hand_maintained_ candidates already makes) before counting a divergence. Fast path skips rustfmt when bytes already match; a side that will not parse is a genuine emit defect and propagates as a hard, located failure (fail-closed, never a swallowed match). Both sides share one temp slot so rustfmt's path-header line is identical and cancels. changed_registered_outputs gets the same discrimination so the receipt matches the gate verdict. Section-7-aligned: a byte-identical fixed point is explicitly not the goal; content equivalence modulo formatting is. The committed seed is the rustfmt fixed point and is correct -- it is NOT reseeded. Proven by execution: regen_divergence_count 1 -> 0 on the real tree with the seed untouched; the regression witness is green on a formatting-only divergence and red on genuine content drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Independent adversarial verification (relayed via sunny-wolf-225): cool-wren-467, on this head (684ee2e) in a separate worktree, confirmed:
This is stronger than the self-authored green/red regression test: the drift token was chosen by a session that did not write the fix, on a different file, and the normalization still refuses genuine drift while ignoring only formatting. The failure mode this change guards against — a normalization that silently swallows real drift — is exactly what an author-independent adversarial token rules out. — sent from nimble-owl-658 |
briansrls
added a commit
that referenced
this pull request
Jul 21, 2026
…47). runtime_rust.dag → Arc was not mirrored in v1_rt.rs (GENERATED_STAGE0_FILES); regen_stage0 now writes all 98 generated outputs (incl. v1_rt.rs). Witness complexity_stage0 hook tracks regen'd Arc signature. Rebased on main (#6957 rustfmt-normalized regen gate). Co-authored-by: Cursor <cursoragent@cursor.com>
briansrls
added a commit
that referenced
this pull request
Jul 21, 2026
… self-host gate (#6958) Follow-up to #6957. That PR fixed the batch-1 regen_verify gate by normalizing both sides inside verify_stage0_matches (the Rust comparer), but left the required batch-2 gate self_host_realized_comparison_staleness_gate_holds red on main: it byte-compares the same emit-fresh tree (target/v2-emit-fresh-realize/src/*) and cannot re-normalize on its side, so a comparison-side patch cannot reach it. Proven on the #6957 merge run: batch 2 returned Bool(false). Root cause is one normalization gap, so the fix belongs at one place — the produced artifact, not each comparer. rustfmt_generated_crate now pushes every GENERATED_STAGE0_FILES file through standalone rustfmt via stdin (no path-header, no module recursion, fail-closed on rustfmt's non-zero exit) after cargo fmt --all silently skips the non-building faithful-full- regen crate. Both comparers revert to plain byte compare over the already- normalized artifact — one authority, every consumer (Rust verify AND .dag self-host) sees canonical form. Proven by execution on this branch: regen_stage0 --verify reports regen_divergence_count=0, and emit-fresh reproduces all 98 generated files byte-identical to the committed seed (the exact inputs the .dag gate compares). New test normalize_generated_source_erases_rustfmt_reversible_formatting_but_preserves_content carries a green (formatting-only difference vanishes) plus a RED control (x+2 content drift survives normalization). Co-authored-by: Brian Searls <briansrls@gunb.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes the required
regen_verify_gate_passesself-host gate, red onmainsince #6848 (surfaced when #6944 fixed the earliersource_root_ingestabort that had masked it for four runs).verify_stage0_matchesdid a raw byte compare of the committed seed against the--verifyfresh emit. The two sides are not normalized symmetrically:rustfmt_generated_cratethenrustfmt_workspace— two rustfmt passes.--verifyfresh side: onlyrustfmt_generated_crate— andcargo fmt --allon the faithful-full-regenfresh_dircrate silently skips files whose modules do not build (they surface as cargo errors by design), leaving the fresh side as raw emit.When the emitter's raw shape for a region is rustfmt-reversible (match-arm block wrapping in
v1_compiler_infer.rs), the raw fresh differs from the fmt-normalized seed by formatting alone, so the gate falsely reds.Not a stale seed, not determinism, not v2
Established by execution (two independent worktrees + mine):
--emit-freshruns are byte-identical → emit is deterministic;rustfmt --edition 2021on the raw emit produces the committed shape → the seed is the rustfmt fixed point and is correct.So this is a gate-normalization gap, and the seed is not reseeded (a reseed to the raw shape would have traded regen-red for fmt-red).
Fix
On a raw-byte mismatch, re-normalize both sides through the same standalone rustfmt (
rustfmt_normalize, the moveverify_hand_maintained_candidatesalready makes) before counting a divergence:Errpropagates as a hard, located failure (fail-closed, never a swallowed match);--emit stdoutpath-header line cancels;changed_registered_outputsgets the same discrimination so the receipt matches the gate verdict.§7-aligned: a byte-identical fixed point is explicitly not the goal; content equivalence modulo formatting is.
Proof by execution
regen_divergence_count1 → 0 on the real tree (post-merge of current main), seed untouched,--verifyreports "committed stage0 matches fresh self-compile."verify_stage0_matches_ignores_rustfmt_reversible_formatting_but_catches_content_drift: green on a formatting-only divergence, red on genuine content drift.regen_stage0bin tests pass;cargo fmt --checkclean.Unblocks:
mainregen gate; #6950 (its only red is this inherited divergence); quick-ram-838's regen convergence.🤖 Generated with Claude Code
Credit
Diagnosis was corroborated independently by sunny-ibex-839 (now archived), who: ran plain write-mode regen twice and found it reproduced committed
HEADbyte-for-byte — the datum that killed the stale-seed hypothesis and redirected the investigation from "change the seed or the emitter" to "the comparison is broken"; traced the single-vs-double rustfmt-pass asymmetry and thatcargo fmtskips files whose modules do not build in the fresh_dir crate; and mechanically falsified the initial attribution of the red to #6944 (v2 is outside regen's[src/v1, dag]closure). An independent repro by cool-wren-467 on a different box confirmed write-mode converges byte-identical, and sunny-wolf-225 flagged the two-pass asymmetry that led to catching the rustfmt path-header trap.