Repository navigation
Model census heads in the authority and drop non-closure bodies after parsing - #11093
Merged
Merged
Conversation
`gunbc compile --source-root dag --source-root src/v2 --entry src/v2/workflow/floor2_prepared_subject.dag` peaks at 9.54 GiB RSS for an entry whose reference-derived closure is 16 files. Reproduced at c560821: VmHWM 10,006,804 KiB, wall ~230s (sleek-swift-789 measured 9,964,140 KiB). Half of that peak is one term. `parse_census_fill_sources` is handed the whole indexed pool minus the compile closure -- 5,399 modules the entry never references -- and built `prepared`, a list holding EVERY one of those modules' token streams, before parsing any of them. The fold then consumes each entry exactly once, in order, so module N's tokens stayed resident through the parse of modules N+1..5399 with no consumer. Measured by emptying the census fill behind a throwaway probe: 10,006,804 -> 5,027,748 KiB, so the fill accounts for 4.75 GiB -- ~79 bytes of resident compiler state per byte of unreferenced source. Fusing the tokenize into the fold frees each module's tokens before the next is read. This is DESIGN §2's "minimise the demand before materialising" applied to a RETENTION shape rather than to redundant work: same work, same order, same results, held only for the interval it is used. It is explicitly NOT a cache -- nothing is memoised, and the pool is not scoped to the closure. The pre-intern pass dissolves with `prepared` as a consequence, not a separate decision: it existed only because the token lists were already all in hand. Interning is idempotent and `parse_with_table_in_occurrence_scope` already threads an updated table through the fold, so pre-interning module N's tokens immediately before parsing module N reaches that parse with the same table contents. Only id assignment ORDER changes, which no consumer reads -- names resolve positionally via `authored_name_at`, never by intern id. Every module is still parsed with FULL BODIES and every parse diagnostic still surfaces. That is load-bearing and executed, not a formality: a body-level parse break in a non-closure module is invisible to the heads-only module index and is caught here and nowhere else. Evidence, same invocation: before VmHWM 10,006,804 KiB (9.54 GiB) wall ~230s after VmHWM 8,986,500 KiB (8.57 GiB) wall 163s Both: `resolved 16 sources ... 5399 indexed modules in the name census only`, 213 advisory diagnostics, 1 file emitted, exit 0. Discriminating RED (the wall this change risks deleting), planted in dag/std/access.dag, which is outside the entry's closure: fn body_probe(x: Int) -> Int { let = = = 9 } Balanced braces, so the heads-only module index does NOT catch it. Before and after, identically: exit 1, `refused at emit`, 2 hard diagnostics, `error[dag/std/access.dag]: expected name, found Eq`. A brace-unbalanced probe was tried first and rejected as a control -- it fires, but the module index refuses it before the census fill ever runs, so it does not discriminate. Positive control: `--source-root src/v1 --entry src/v1/compile.dag` produces 2487 hard diagnostics before and after -- the edit adds none. NOT claimed: the emitted dag-artifact.json is run-to-run nondeterministic on main (baseline binary, two identical runs, differing output), so artifact equality was not usable as an equivalence oracle and is not cited as one. Named, not done: - Retention of full-body ASTs for the same 5,399 modules. `fill.modules` reaches only `build_symbol_index_qualified_fill`, which reads declaration heads. A probe retaining `census_heads_module_node(node)` measured a further 8,986,140 -> 7,663,268 KiB (1.26 GiB). Deferred because the post-parse heads strip exists only in the seed (`cli_run::census_heads`), so landing it in the authority needs that projection declared in .dag first -- `02_parse.dag` already carries `census_heads_body_stand_in`, so the vocabulary is half there. - `build_both_closure_edge_index` computes reference and bare pull edges for all ~5.4k pool files, while an --entry BFS queries only the closure's. That work is not merely duplicated, it is never demanded. It sits inside the residual 4.79 GiB and is the brief's originally-named term. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdEbLY7W5T2jBtqXQLBgVE
Review 63932: the annotation added to parse_census_fill_sources carried two transcribed number pairs (VmHWM before/after, wall before/after). DESIGN §6 forbids exactly that -- a copied number is unreachable from the producer that owns it, so it rots without anyone touching either end. Naming the entry point was the right half and is kept; the digits are replaced by the invocation that re-derives them, plus why this entry is the discriminating one. The figures stay in the PR and commit message, which are bound to a revision and do not claim to stay true. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdEbLY7W5T2jBtqXQLBgVE
CI caught what the file's own comment denies: `v1_compiler_compile.rs` is NOT
hand-synced, it is regenerated from `compile.dag` and gated.
required-ci: generated-artifact population=stage0-mirrors
FAIL generated surface drift: v1_compiler_compile.rs
The drift was exactly, and only, the 8-line comment block I hand-added inside
`parse_census_fill_sources`. Diffing the committed file against
`target/stage0-regen-candidate/src/v1_compiler_compile.rs` produced 9 diff
lines, all of them that comment; the generated bytes are otherwise identical
to what I had written by hand, which is independent evidence that the manual
sync was semantically right and procedurally wrong.
The reasoning the comment carried already lives in `src/v1/compile.dag`, which
is the authority and the only correct home for it. A generated file cannot hold
commentary: the next regeneration deletes it, so it would rot by construction.
Produced by `claim_executor --regen-round-cost` (run twice -- the first pass
refuses because its own seed build replaces the running executable). The round
itself then refuses at its rebuild step with
`RebuildScopeRefused MirrorHasNoOwningPackage mirror=v1_compiler_compile.rs`
and restores its checkpoint, so the emitted candidate was installed for this
one path rather than via the round's install. Only that path was copied --
copying the candidate tree wholesale silently reverts hand-authored mirrors.
Re-verified after regeneration, not assumed from the earlier run: release build
clean, `cargo fmt --all --check` clean, and the discriminating RED still refuses
identically -- planted `fn body_probe(x: Int) -> Int { let = = = 9 }` in
dag/std/access.dag (outside the entry's closure) gives exit 1,
`error[dag/std/access.dag]: expected name, found Eq`, 2 hard diagnostics.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QdEbLY7W5T2jBtqXQLBgVE
…gunbc into session/valiant-badger-803
Contributor
Author
|
Operator handoff for
Implementation and evidence are complete. Per sunny-tern-813’s ruling, stopping here for the operator to land this PR; GitHub’s refused-floor check remains explicitly red. — sent from valiant-badger-803 |
6 tasks
briansrls
pushed a commit
that referenced
this pull request
Sep 12, 2026
…op, #11047 receipt separator)
This was referenced Sep 12, 2026
File the class: a change green in one repository breaks another that no gate compiles with it
#11100
Merged
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Sep 12, 2026
gunbc#11047 moved the separator into the fold -- `authored` is now `join(r.receipts, " ")` -- and deleted the per-receipt trailing-space convention this row was authored under. All eight of my receipts ended in a space, so after the merge every one of the seven boundaries would have rendered double-spaced. Stripped the trailing space from all eight; the diff is 8/8 and each changed line is a receipt terminator, not prose. The coupling is the RENDERER, not the import graph. This row imports only RecurringFailureMode and NonEmptyStr, neither touched by #11047, so an import-graph reading clears it and is wrong -- which is how #11100 was judged safe. Re-derived against the merged tree, with a pre-existing row on the same page as the control: - regen EXIT=0; my rendered paragraph 7426 chars, double-space count 0; control `absorbing_fallback` paragraph 3602 chars, double-space count 0. - all seven receipt boundaries render with exactly one space ("...by no one. WHY IT IS WORSE", "...a climb. THE SPECIMEN", and so on). - second regen byte-identical to the first (cmp clean). The mutation evidence is RE-DERIVED under the new fold rather than carried over from the old one: one `receipts` element replaced by 42 gives regen EXIT=1, a located resolve refusal at :10:5 "type mismatch: expected 'Primitive(String)', got 'Primitive(Int)'", and no projection written. Restoring the row reproduces the projection byte-for-byte. This merge also brings in #11093 (census heads modelled, non-closure bodies dropped after parsing), which is the cause of the MemoryStallRefusedPageThrash that refused the floor on the previous head. A rerun would have replayed the merge ref pinned before it; only a push re-measures the current memory shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NuEqKq5Jxcy4Wj3fYEwYFL
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Sep 12, 2026
The floor lane refused MemoryStallRefusedPageThrash on three attempts. All three compiled a base predating the fix: the run was created 02:50:42Z and gunbc#11093 -- which drops non-closure bodies from census heads, ~1.26 GiB less per compile -- merged at 04:23:22Z. A re-run reuses the original merge ref, so no amount of re-running could have picked it up. This reverses my earlier refusal to re-integrate. That refusal was correct when main carried no fix (merging more commits only enlarges the compile that thrashed); it is wrong now, because the content in main is the REDUCTION. Pushing also forces a synchronize so the merge ref is recomputed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013YcKjgpfRzhjNJrjaNZjJG
gunbai-bot Bot
added a commit
that referenced
this pull request
Sep 12, 2026
… adjudicate it (#11020) * File the class: routing a subject to an adjudicator never observed to adjudicate it One row under dag/gunbc/recurring_failure_mode/. The invalid state: a subject whose adjudication is transferred to an authority never OBSERVED to adjudicate it, with the transfer recorded as a routing rather than as a loss of coverage. The origin gate goes green on a typed, honest decline; the destination's silence is indistinguishable from coverage. Specimen: the near-miss of 2026-09-09/10 on the four native_decl_selection witnesses, the ruling that they be routed to required-v2-native, and clever-dove-148's out-of-scope check of the DESTINATION, which found it refusing at its own emit/resolve stage before adjudicating any witness. The row records what actually caught it -- an acceptance condition demanding the destination COMPLETE and NAME all four identities -- as the repair it points at. Bounded explicitly against shared_precondition_re_derived_once_per_claim_frame (cost, not destination), absorbing_fallback (widens, does not refuse), the inert lens (never consulted; here the destination is consulted and loud), and the escape hatch (proceeds as if no refusal; here the refusal fires and is located). Rung found at mitigatable; ceiling argued at mechanically preventable -- a decline arm required to cite a destination receipt naming the routed subject -- and explicitly NOT structural, since a receipt can be stale or name a run that reached no verdict. Trigger names the capability: a cross-gate adjudication receipt carrying destination lane, run, and the subject identities that run actually reached a verdict on. EVIDENCE, by execution against a release gunbc built from this tree, through tools.docs_projection_gate regen (gunbc.design_ledgers expected_design_failure_modes_md): - green: regen EXIT=0, writes docs/design-failure-modes.md 1563582 bytes, and the row's identity and prose render into it. - MUTATION, consumed field: one element of `receipts` replaced by `42`. regen EXIT=1, resolve refusal located at routing_a_subject_to_an_unverified_adjudicator.dag:10:5 "type mismatch: expected 'Primitive(String)', got 'Primitive(Int)'", and NO docs/design-failure-modes.md is written -- the fold reds and the text is gone. - CONTROL, unconsumed field: `evidence: 42` on the same row. regen EXIT=0 and the projection writes normally, reproducing exactly the green that #11015's author found and threw the resolve-only check out over. The pair is what makes the first result discriminating rather than incidental. - regen re-run after restore is byte-identical to the first generation (cmp clean), no drift; docs_projection_gate main EXIT=0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NuEqKq5Jxcy4Wj3fYEwYFL * Describe the sibling class by shape, not by an identity that resolves nowhere review 63417 (cursor/auto, REQUEST_CHANGES) is correct: the BOUNDARIES receipt cited `shared_precondition_re_derived_once_per_claim_frame` as a peer ledger identity, and that row is on neither tree -- empty on origin/main and empty in git ls-files at this head, so the only mention of the name in the corpus was my own citation of it. The named sibling is filed but unlanded. Repaired the way `unlanded_citation_indistinguishable_at_the_citing_end` prescribes: describe the sibling by SHAPE -- "the class filed by clever-dove-148 over the same specimen, whose subject is a shared precondition re-derived once per claim frame" -- and say in the same breath why the identity is withheld, so a later reader is not left to wonder whether the name rotted or never existed. The boundary the receipt draws is unchanged; only the handle is. Landing the sibling in this closure was the other admissible repair and is not mine to do: it belongs to that row's author and to its own PR. Every remaining backticked ledger identity in the row resolves against the directory: `absorbing_fallback` and `unlanded_citation_indistinguishable_at_the_citing_end`. Re-verified by execution: tools.docs_projection_gate regen EXIT=0, the reworded receipt renders into docs/design-failure-modes.md, and a second regen is byte-identical to the first (cmp clean). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NuEqKq5Jxcy4Wj3fYEwYFL * Migrate the row's receipts to the #11047 separator convention gunbc#11047 moved the separator into the fold -- `authored` is now `join(r.receipts, " ")` -- and deleted the per-receipt trailing-space convention this row was authored under. All eight of my receipts ended in a space, so after the merge every one of the seven boundaries would have rendered double-spaced. Stripped the trailing space from all eight; the diff is 8/8 and each changed line is a receipt terminator, not prose. The coupling is the RENDERER, not the import graph. This row imports only RecurringFailureMode and NonEmptyStr, neither touched by #11047, so an import-graph reading clears it and is wrong -- which is how #11100 was judged safe. Re-derived against the merged tree, with a pre-existing row on the same page as the control: - regen EXIT=0; my rendered paragraph 7426 chars, double-space count 0; control `absorbing_fallback` paragraph 3602 chars, double-space count 0. - all seven receipt boundaries render with exactly one space ("...by no one. WHY IT IS WORSE", "...a climb. THE SPECIMEN", and so on). - second regen byte-identical to the first (cmp clean). The mutation evidence is RE-DERIVED under the new fold rather than carried over from the old one: one `receipts` element replaced by 42 gives regen EXIT=1, a located resolve refusal at :10:5 "type mismatch: expected 'Primitive(String)', got 'Primitive(Int)'", and no projection written. Restoring the row reproduces the projection byte-for-byte. This merge also brings in #11093 (census heads modelled, non-closure bodies dropped after parsing), which is the cause of the MemoryStallRefusedPageThrash that refused the floor on the previous head. A rerun would have replayed the merge ref pinned before it; only a push re-measures the current memory shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NuEqKq5Jxcy4Wj3fYEwYFL * Cite the sibling class by identity: it landed, so the withholding clause rotted review 64357 (claude/opus[1m], REQUEST_CHANGES) is correct and I verified it rather than taking it: shared_precondition_re_derived_once_per_claim_frame is on this tree. It landed via #11015 (ec1571e), was appended by #11037 (ce76f34), and this PR's own merge of main brought it in -- so the file is present in the worktree and the identity resolves. The BOUNDARIES receipt was therefore printing a justification that was TRUE WHEN WRITTEN AND IS NOW FALSE, and a false one in the worst direction: it told a reader the name does not exist when it does, which inverts unlanded_citation_indistinguishable_at_the_citing_end rather than applying it. Restored the direct citation and deleted the withholding clause entirely; the boundary the receipt draws is unchanged, only the handle. Every backticked ledger identity in the row now resolves against the directory: absorbing_fallback and shared_precondition_re_derived_once_per_claim_frame. (namespace_graft is the specimen's error name, not a ledger identity.) Re-derived: regen EXIT=0, rendered paragraph 7096 chars with 0 double spaces, second regen byte-identical to the first (cmp clean), and the receipt trailing-space count stays 0 under the #11047 join convention. Worth naming, since this row is about exactly this: the withheld citation was a correct judgement about a tree that then moved underneath it, and nothing in the row re-read its own premise. That is the same shape as the class being filed -- a conclusion whose condition was checked once and never again -- which is why a reviewer reading the current tree caught it and the author who wrote it did not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NuEqKq5Jxcy4Wj3fYEwYFL --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Entry-scoped compilation still retained parsed bodies for every non-closure census module after #11072 streamed their tokens. This change models the declaration-head projection in
v1.compiler.compile, then retains that projection after full parsing and annotation admission. The seed delegates to the generated projection instead of maintaining a second strip implementation.The projection preserves declaration identity, names, kinds, signatures, imports, and spans. Functions retain the parser’s refusing body marker for function-kind discrimination; executable bodies, uses, match patterns, and expression data are discarded. Both census accumulator references share the projected parse result. Full-body parse errors still reach the compile refusal.
Only
compile_to_resolved_with_optionsselectsRetainDeclarationHeadsfor its non-closure sources. The existingparse_census_fill_sourcesAPI retains full nodes for namespace-wave admission, singleton CLI admission, entry resolution, and the diagnostics-only compile-clean caller. This avoids changing occurrence/namespace consumers for no meaningful singleton retention saving. Closure modules continue through the normal frontend. Frontier (b), the whole-pool edge index, is separate.Validation:
-p v1-compiler --bin gunbc --bin claim_executor -j 2) andcargo fmt --all --checkpass. Binaries were copied out of shared/cargo-targetimmediately after each build.f6581910ab2contains the authority projection and its generated mirror. Retention and seed delegation follow in812c542d491.claim_executor --required-regen --source-root dag --source-root src/v2passes at812c542d491:first_generation_equal=true, planned/executed/adjudicated 155/155/155, changed paths empty. Authority digestfnv1a64:1321de6fd99aff1b; committed/candidate generated digestfnv1a64:d70e079d37a101da.v1_compiler_compile.rscandidate was installed, using the parent-approved interim route for the installer gap tracked by Include host-shell mirror ownership in regen installation #11094. No generated Rust was hand-edited.claim_executor --required-regen-fixed-point --source-root dag --source-root src/v2passes:fixed_point_equal=true,referenced_first_generation_equal=true, both at812c542d491. CI build (includingcargo clippy --all-targets -- -D warnings) and generated-artifact regeneration pass. Dashboard review 64084 approves with no findings. The witness lane is infrastructure-refused; see below.Isolated before/after, same worktree corpus and invocation (16 closure sources, 5,429 census modules):
The measured positive saving is 1,177,196 KiB = 1.123 GiB (12.99%). This is smaller than bright-bear-236’s prior heads-retention probe (8,986,140 → 7,663,268 KiB, about 1.26 GiB); #11072’s streaming-only result was 8,986,500 KiB after 10,006,804 KiB. Those are predecessor measurements, not substituted for this session’s results. The admitted budget stays 16,106,127,360 bytes (15 GiB). No floor budget rows or parse witnesses changed, and no floor JOB verdict is claimed as proof.
Both positive runs report 0 blocking errors, 213 advisories, and one emitted file. The discriminating red appends
fn body_probe(x: Int) -> Int { let = = = 9 }to non-closuredag/std/access.dag, then restores the file. Both runs refuse at emit with two hard diagnostics and the identicalerror[dag/std/access.dag]: expected name, found Eq(span 2646–2647). Balanced braces make this invisible to the heads index; the full-body census parse must catch it. Artifact byte equality is not used because registry ordering is nondeterministic on the baseline.Reproduction (run the identical command against each snapshotted binary, polling
/proc/<pid>/statusfor VmHWM and timing with a monotonic clock):Baseline gunbc SHA-256:
c5c07879131d35731fec3f90c1524d8acdf128e9f8b9c13fae75142c33e918b3.Final gunbc SHA-256:
d0a38657f28295aaacdcba875557617896627741211dd45b11a305bce8edb375.CI disposition: witness job 103431616486 refused with
MemoryStallRefusedPageThrashwhile resolvingfloor_terminal_ledger_wireand typecheckingfloor2_prepared_subject: 282,224 major faults/minute, 4% CPU share over 34,140 ms, admitted budget unchanged at 15 GiB. The terminal ledger could not be rendered, so this is explicitly not a green floor. The run classified itselfclass=infra; it is a draw, and the isolated before/after measurements above remain the evidence for this change. GitHub is blocked pending disposition of that check.