Repository navigation
Scope construction: measure the split, then derive each module's contribution once per subject instead of once per scope - #9809
Merged
Conversation
…ers per term The required floor builds 660 claim scopes over a 2,249-module prepared corpus at a mean of 505.7 modules each (run 33374789700, `floor: 660 scope construction(s)`) -- ~334k per-scope module materializations, a ~148x membership duplication across scopes that overlap, with distinct_scopes == constructions so none of it is same-scope rebuild waste. The loader layer beneath it is already shared and memoized once per entry (`load_sources_for_entry_with_pool` -> `entry_closure_sources`), so the duplication is not the closure INPUT; it is the per-scope DERIVATION over that shared input. `[floor-scope-cost]` prices a scope in resident bytes and `floor: N scope construction(s)` counts them, and neither can say WHICH of the three constructions inside `claim_scope_for` the time is: the precedence ORDER walk (func_env.parents plus the transitive reference closure), the item-registry UNION over that order, or `build_scope_indexes_with_module_order`, which re-reads every item of every module in the order to rebuild fn_nodes, import bindings and service ops. Those are three different constructions and the view that replaces each is a different change, so choosing the view's shape from the resident-byte delta would be picking a remedy from a quantity that cannot distinguish the candidates. `ScopeBuildSplit` carries the three nanosecond terms out on `PreparedClaimScope`, where the floor runner already reads `module_count` and `ambiguous_bare_names`, and the fold sums them into one `[floor-scope-split]` line beside `[floor-scope-cost]`. Data on the carrier the constructor already returns, not a thread-local a second reader could read at the wrong time. No semantic change: same scopes, same order, same registry resolution, same claim population and verdicts. This is the instrument for the terminal correction the `[floor-scope-cost]` comment already names -- "a scope that is a view rather than a rebuild costs nothing to enter" -- and it dissolves with that correction, which removes the constructions it times. Seen and priced, not missed: the slice-1 entry-closure overlap probe (`measure_selected_entry_closure_overlap`, marker `cli_run_selected_closure_overlap_probe`) is INERT -- its scaffold header names `src/v1/stage0/src/bin/measure_selected_closure_overlap.rs` as its runner and that file is not in the tree, so its only callers are arithmetic unit tests. It measures module membership across floor ENTRIES, which is the layer already shared, so it is not the deciding instrument for this lane and is deliberately not revived here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BtVuh6N448SbFMQxCguJx7
…r scope The split the previous commit installed answered the question it was built for. From this PR's own required floor job (99457075331): [floor-scope-split] constructions=660 total_ms=75251 order_ms=18567 registry_ms=12198 indexes_ms=44486 mean_ms_per_scope=114.0 59% of scope construction is `build_scope_indexes_with_module_order`, which re-read every item of every module of every scope: 660 scopes over a mean of 427 modules, repaying `authored_name_at` per item, `item_kind` per item, the parsed import list per module, and a `format!` per qualified name once per scope the module appeared in. That derivation reads NOTHING BUT THE MODULE, so its result is the same in every scope containing it. What is not module-local is which module WINS a colliding bare name, and that stays in the per-scope fold: `ModuleScopeFragment` carries one module's entries in the module's own order, and the fold applies the scope's precedence to them exactly as before -- first-write-wins for bare slots under an order, unconditional writes for qualified slots, or_insert for import bindings, last-write-wins for file module paths and service ops. The two `fn_nodes` kinds stay INTERLEAVED in the fragment rather than grouped, because the original walk emitted a bare slot and then its qualified slot per item and grouping would change which write lands last if a bare name ever spells a qualified one. The memo is keyed by the prepared subject's own digest, the same key `REFERENCE_CLOSURE_INDEXES` uses and for the same reason: a fragment a scope consumes was derived from the graph that scope is over, by construction rather than by a module-name coincidence across two subjects. It is bounded by the same two-subject population, holds at most one fragment per module of that subject, and is in-process only -- nothing is persisted and nothing carries across runs. The cache is an ARGUMENT to the fold, not a property of it: `None` re-derives every fragment, which is what the entry-major `build_scope_indexes` path passes and what the equivalence control passes. One fold, two callers, no fork. EVIDENCE. `scope_fragment_memo_equivalence` builds every scope of a corpus whose two entries resolve one colliding bare name to DIFFERENT declarations, with the memo and without it, and asserts identical answers at identity grain: fn_nodes down to which declaration node each name resolves to (Rc address, not spelling), ambiguity set, file-module and import-binding tiers, service ops, item registry, precedence identity, module count. A second test asserts the fixture actually exercises order-dependent resolution, so the equivalence is not clean by absence; a third asserts two subjects declaring the same module name do not share one answer, which is the cross-subject leak the key exists to prevent. The order term (25%) and the registry term (16%) are untouched here. The order walk being 660 independent BFS traversals over shared subtrees is a second change with its own equivalence evidence, not a rider on this one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BtVuh6N448SbFMQxCguJx7
gunbai-bot
Bot
changed the title
Contributor
Author
Measured result of cut 2 (the fragment memo), from this PR's own floor jobsSame job shape, same corpus (2,123 modules, 660 scopes, mean 427.3 modules/scope):
Acceptance, compared line by line against the previous run
Wall, stated honestly: the fold wall went 371.9s → 337.8s and the floor job 31m47s → 20m49s. Neither is claimed as the effect — the instrument-attributable term is −18.7s; the rest is undecomposed, and one run against one run on shared runners cannot separate this change from queueing and neighbours. Not on the table, for the record: sharing the qualified — sent from proud-wolf-477 |
The order term the split isolated is 17.1s of the 56.5s that scope construction now costs (`[floor-scope-split]` `order_ms`). It was 660 traversals asking one question of every module they visited -- what does this module reach -- and recomputing the answer each time: `reference_targets_of` re-walks the module's reference set and re-runs longest-declared-prefix selection over the declarer index, and `parents_of` rebuilt a whole-subject map per scope to read one entry out of it. That answer depends on the module and the prepared subject, never on which scope is asking. `ScopeOrderIndex` holds it once per subject: parents eagerly in one pass, the reference-target-then-parents concatenation on first visit. WHAT IS DELIBERATELY NOT MEMOIZED IS THE WALK. The sequence a scope visits those lists in is a function of its entry, and that sequence IS the scope's identity -- it decides which module wins a colliding bare name, and `scope_identity` is a non-commutative fold over exactly it. So the traversal, its `seen` set and its frontier stay per scope; only the per-module answer they consume is shared, and it is the same list in the same sequence the unmemoized walk built. The control was widened with the memo, not left behind it: `claim_scope_for_without_memos` now withholds BOTH the fragment memo and the order memo, building a scope-private order index -- which reproduces the pre-memo walk exactly, since a module is visited at most once per scope and a scope-private memo therefore never hits. The equivalence witness compares `scope_identity` alongside the resolution fingerprint, so a memo that reproduced the SET while perturbing the SEQUENCE fails it rather than passing every count. And the witness is no longer vacuous by construction: a new test asserts the second scope of a subject adds to the SAME memos the first scope filled. Without it, a per-scope cache would answer identically to the control and the equivalence assertion would prove only that two unmemoized folds agree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BtVuh6N448SbFMQxCguJx7
Contributor
Author
Cut 2(a) — the order memo, measuredThree floor jobs on this PR, same instrument, in order:
Scope construction is down 39% end to end (75.3s → 46.2s) and each of the two cuts moved the term it targeted and left the others alone: Precedence did not move
Two numbers that moved, both named rather than left to be found
Job wall (31m47s → 20m49s → 21m53s) is deliberately not claimed as the effect: one run per configuration on shared runners cannot separate this change from queueing and neighbours. The instrument-attributable total is −29.1s. — sent from proud-wolf-477 |
…mposition main landed `[floor-fold-time]`, which partitions the fold into scope_build and frame_build; this branch landed `[floor-scope-split]`, which partitions ONE scope build into its order walk, registry union and index rebuild. Both are wanted -- they answer different questions, and frame_build is visible to neither of the other receipts -- but taken verbatim they printed the same scope total twice, differing only by the few instructions between a timer around `claim_scope_for` and the timers inside it. Two spellings of one number is the redundancy the split was built to avoid, so the split now carries no total and no overall mean and says in its own comment that its total is `scope_build` on the line above. `ScopeBuildSplit::total_nanos` went with it rather than staying as an accessor nothing reads. The three terms this branch measured are unchanged; only the total it stopped restating is gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BtVuh6N448SbFMQxCguJx7
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Aug 31, 2026
…s slots do not share one polarity The seed rebuilds a set of name-keyed maps for every claim scope it enters. `[floor-scope-cost]` already names the remedy beside the number it prices — "a scope that is a view rather than a rebuild costs nothing to enter" — and #9809 removed the module-local half of the derivation. What survives is the fold, and every entry it writes is a decision cheaper to compute than to store. This lands the MODEL only. No reader migrates and no map is deleted here, because the sweep found something a code-first cutover would have shipped silently: the six slots being replaced DO NOT AGREE on how a scope decides a contested key. `fn_nodes` bare and the scope `item_registry` are first-write-wins under precedence (minimum rank); `service_ops` and `file_module_paths` are unconditional inserts, so the LAST module in order wins. A uniform minimum-rank view would have redispatched a colliding service operation — the failure the `std.resources` / `extdeps.filesystem` `Filesystem` pair already has a receipt for — with nothing refusing. `gunbc.scope_rank_view` carries polarity as a typed per-slot fact so that state is unwritable rather than caught in review, plus the reader census (separating the point lookups the view makes free from the four enumerating readers it does not), the qualified-admission rung, and the acceptance oracles. The admission row is the load-bearing one. "A scope must never resolve a name outside its admitting closure" holds today by construction and INCIDENTALLY — the map is materialized from the scope's own members, so an unadmitted declaration has no key. A corpus-wide index deletes that construction, so the model states how it is re-established: resolution returns only a declarer obtained through the rank map, so an unadmitted answer has no constructor. Also deletes the stale 1155/9573 transcription from the `PreparedScopeIndexes` doc comment. Both figures the lane was handed are readings of one instrument — `floor: N scope construction(s) for M distinct scope(s)` — at two revisions, and the older one was copied into prose where nothing re-derives it, so it rotted in place and then read as a rival present-tense measurement. The comment now names the line instead. The number is not updated; updating it re-commits the defect. Evidence: `test.claim.scope_rank_view_test`, 10 witnesses, all PASS on a remote run, plus two discriminating REDs each of which reds exactly one assertion and nothing else — collapsing both HighestRankWins rows to LowestRankWins reds `the_slots_do_not_share_one_polarity`, and editing the transcribed figure to match the measured one reds `the_two_scope_count_figures_disagree`. Design: docs/plans/scope-rank-view-design.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011umUFhtMDKh27CDmdXHXub
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Aug 31, 2026
…site grep Zero-code prep while step 2 holds for #9825 and #9809 to merge. The brief sizes the reader migration at ~126 item_registry sites. That is the grep count over the token; 112 of them are a DIFFERENT carrier's field of the same name — TypedModule's per-module registry (the authority the corpus index is derived from), v1_compiler_infer's own threaded typecheck registry, field declarations, empty-map initializers, the emitter's qualified overlay, and prose. A cutover scoped to the grep would edit the typecheck layer and the emitter for no reason; one that ignored the difference would edit the wrong authority. The real population is 14 readers plus 4 construction sites, and the sweep found two things the earlier column could not express: ProvenanceTest — four readers do not want the declaration, they want the WINNER'S DECLARING MODULE and decide a builtin dispatch from it (try_witness_evaluation_dispatch, is_v4_bridge_family, is_v2_std_collection_map_grounded_fn, definer_module_for_name). At these sites a polarity or admission error surfaces as a DIFFERENT FUNCTION EXECUTING rather than as a missing name, so they cut over first and the discriminating REDs are exercised against them. UnconsumedConstruction — InterpContext::resolved_graph hands out a whole ResolvedGraph built from the scope map, which no resolver can serve and which would therefore force materialization to survive the cut. It has no call site anywhere in src/. Filed as a row with its proof rather than silently deleted, because 'the blocker is not real' is the claim a reader most wants to check. Also fixes eval_var and eval_data_item_value's classification: both already re-resolve through lookup_fn_from immediately after the registry hit, so they ask the same question twice today and collapse into one resolution. Design doc gains the full non-population table, the per-arm site list and the five-step cut order. Two new witnesses; 12/12 PASS remotely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011umUFhtMDKh27CDmdXHXub
briansrls
pushed a commit
that referenced
this pull request
Aug 31, 2026
…share one polarity (#9825) * Model the scope rank-view before any reader moves, and record that its slots do not share one polarity The seed rebuilds a set of name-keyed maps for every claim scope it enters. `[floor-scope-cost]` already names the remedy beside the number it prices — "a scope that is a view rather than a rebuild costs nothing to enter" — and #9809 removed the module-local half of the derivation. What survives is the fold, and every entry it writes is a decision cheaper to compute than to store. This lands the MODEL only. No reader migrates and no map is deleted here, because the sweep found something a code-first cutover would have shipped silently: the six slots being replaced DO NOT AGREE on how a scope decides a contested key. `fn_nodes` bare and the scope `item_registry` are first-write-wins under precedence (minimum rank); `service_ops` and `file_module_paths` are unconditional inserts, so the LAST module in order wins. A uniform minimum-rank view would have redispatched a colliding service operation — the failure the `std.resources` / `extdeps.filesystem` `Filesystem` pair already has a receipt for — with nothing refusing. `gunbc.scope_rank_view` carries polarity as a typed per-slot fact so that state is unwritable rather than caught in review, plus the reader census (separating the point lookups the view makes free from the four enumerating readers it does not), the qualified-admission rung, and the acceptance oracles. The admission row is the load-bearing one. "A scope must never resolve a name outside its admitting closure" holds today by construction and INCIDENTALLY — the map is materialized from the scope's own members, so an unadmitted declaration has no key. A corpus-wide index deletes that construction, so the model states how it is re-established: resolution returns only a declarer obtained through the rank map, so an unadmitted answer has no constructor. Also deletes the stale 1155/9573 transcription from the `PreparedScopeIndexes` doc comment. Both figures the lane was handed are readings of one instrument — `floor: N scope construction(s) for M distinct scope(s)` — at two revisions, and the older one was copied into prose where nothing re-derives it, so it rotted in place and then read as a rival present-tense measurement. The comment now names the line instead. The number is not updated; updating it re-commits the defect. Evidence: `test.claim.scope_rank_view_test`, 10 witnesses, all PASS on a remote run, plus two discriminating REDs each of which reds exactly one assertion and nothing else — collapsing both HighestRankWins rows to LowestRankWins reds `the_slots_do_not_share_one_polarity`, and editing the transcribed figure to match the measured one reds `the_two_scope_count_figures_disagree`. Design: docs/plans/scope-rank-view-design.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011umUFhtMDKh27CDmdXHXub * Declare the witness substrate-only so the required floor RUNS it rather than declining it A witness with no live_tree_disposition is declined by the floor's fail-closed default: discovered, counted, never run. This one reads only .dag data, so the honest arm is SubstrateInputsOnly and the floor executes it. Re-greened remotely: 10/10 PASS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011umUFhtMDKh27CDmdXHXub * Per-site migration census: the population is 14 readers, not the 126-site grep Zero-code prep while step 2 holds for #9825 and #9809 to merge. The brief sizes the reader migration at ~126 item_registry sites. That is the grep count over the token; 112 of them are a DIFFERENT carrier's field of the same name — TypedModule's per-module registry (the authority the corpus index is derived from), v1_compiler_infer's own threaded typecheck registry, field declarations, empty-map initializers, the emitter's qualified overlay, and prose. A cutover scoped to the grep would edit the typecheck layer and the emitter for no reason; one that ignored the difference would edit the wrong authority. The real population is 14 readers plus 4 construction sites, and the sweep found two things the earlier column could not express: ProvenanceTest — four readers do not want the declaration, they want the WINNER'S DECLARING MODULE and decide a builtin dispatch from it (try_witness_evaluation_dispatch, is_v4_bridge_family, is_v2_std_collection_map_grounded_fn, definer_module_for_name). At these sites a polarity or admission error surfaces as a DIFFERENT FUNCTION EXECUTING rather than as a missing name, so they cut over first and the discriminating REDs are exercised against them. UnconsumedConstruction — InterpContext::resolved_graph hands out a whole ResolvedGraph built from the scope map, which no resolver can serve and which would therefore force materialization to survive the cut. It has no call site anywhere in src/. Filed as a row with its proof rather than silently deleted, because 'the blocker is not real' is the claim a reader most wants to check. Also fixes eval_var and eval_data_item_value's classification: both already re-resolve through lookup_fn_from immediately after the registry hit, so they ask the same question twice today and collapse into one resolution. Design doc gains the full non-population table, the per-arm site list and the five-step cut order. Two new witnesses; 12/12 PASS remotely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011umUFhtMDKh27CDmdXHXub --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Sep 19, 2026
…ount (review 68031) TWO FINDINGS, BOTH CORRECT. 1. NO SeedGrowthJustification ROW LANDED WITH 458 LINES OF NEW HAND RUST. gunbc.seed_growth_admission seed_growth_forward_freeze_policy_note makes unenumerated hand growth in src/v1 a stop-line, and DESIGN section 7 says the same generally: a seed-retained module is a DECLARED ROW with a reason and a migration trigger, "countable, prioritizable -- never a silent escape hatch". The module doc-comment carried the deletion trigger, which is none of those things. The review is right that this is the strongest case for a row rather than the weakest. The module is explicitly a SHADOW of a fold that still stands -- its own header says the carriers land BESIDE it -- so it is admitted debt by construction, with a known owning lane and a known deletion motion. dag/gunbc/scope_rank_view_seed_growth.dag follows the sibling receipt for this same seam (gunbc.claim_scope_fixture_seed_growth): six hand items enumerated; why it is Rust and not .dag (the subject is a seed function, so a .dag re-derivation could not observe the fold at all and would be a second resolver free to drift -- the fork this migration exists to REMOVE); what is not netted (PreparedClaimScope's two fields are ExistingSeedItemModified, moved rather than recomputed); and the trigger, which is step 3's root cut. The `#[cfg(test)] mod equivalence` is inside the receipt's scope rather than exempted as "just tests". It is the carriers' only consumer, which is what makes them consumed rather than dangling, and it dissolves on the SAME trigger: once step 3 deletes the materialized maps there is no legacy arm left to compare against, and a one-armed equivalence law is a decoration that would be cited as coverage. 2. "mean 427 at the measured run" WAS A TRANSCRIBED MEASUREMENT. Same DESIGN section 6 violation as review 68008, in a comment this branch wrote, one commit after fixing the same class in the proposal doc. Replaced by its producers: `module_count` beside the fields, and `[floor-scope-cost]` per run. NOT FIXED, AND NAMED SO IT IS NOT MISTAKEN FOR AN OVERSIGHT: cli_run.rs carries "660 scopes x ~427 modules ... 17.1s" from #9809, the same class and not this branch's line. It sits on the construction site step 3 deletes, so it dissolves with the cut rather than needing a separate correction here. Verified before pushing, since commits auto-push: the new .dag PARSES (compile.frontend and compile.normalize both completed against the real closure); whole-closure typecheck could not be reached because the run was OOM-killed on the remote runner (exit 137), so that half is unverified here and the required floor's parse phase is what covers it. The four equivalence laws still pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits, in the order the second depends on the first.
1 — the split.
[floor-scope-cost]prices a scope in resident bytes andfloor: N scope construction(s)counts them; neither says which of the three constructions insideclaim_scope_forthe time is.ScopeBuildSplittimes the precedence ORDER walk, the item-registry UNION andbuild_scope_indexes_with_module_orderseparately and the fold reports[floor-scope-split].It answered immediately, from this PR's first required floor job (99457075331, run 33382265937):
indexes 59%, order 25%, registry 16% of 75.3s.
2 — the view, cut 1. 660 scopes over a mean of 427 modules re-derived each module's contribution once per scope it appeared in, though the derivation reads nothing but the module.
ModuleScopeFragmentderives it once per prepared subject; the per-scope fold keeps every order-dependent rule (bare-slot precedence, ambiguity counting, the union) unchanged. In-process only, keyed by the subject digest, bounded by the same two-subject populationREFERENCE_CLOSURE_INDEXESis.Why this lane looks like this
The brief named "duplicate in-process closure computation across floor scopes". The loader/entry-closure layer turned out to be already shared and canonical (
load_sources_for_entry_with_pool→entry_closure_sources, one memo consumed by retention arming, the pre-resolve calibration and the per-entry resolve; identity viasubject_digest_for_closure). The measured duplication is one layer down — shared input, duplicated derivation — which is what this PR removes. The tree already named this remedy, in the[floor-scope-cost]comment: "a scope that is a view rather than a rebuild costs nothing to enter."Seen and priced, not missed: the slice-1 entry-closure overlap probe (
measure_selected_entry_closure_overlap, markercli_run_selected_closure_overlap_probe) is inert — its scaffold header names a runner bin that is not in the tree, so its only callers are arithmetic unit tests. It measures the entry layer, which is the already-shared one, so it is deliberately not revived here.Evidence
scope_fragment_memo_equivalence(3 tests, inrust-unit-tests): memoized vs unmemoized scopes over a corpus whose two entries resolve one colliding bare name to different declarations — identical answers at identity grain (fn_nodes by declaration node address, ambiguity set, file-module and import-binding tiers, service ops, item registry, precedence identity, module count); a positive control that the fixture really does exercise order-dependent resolution; and a cross-subject test that two subjects sharing a module name do not share one answer.[floor-scope-split]and[floor-scope-cost]numbers to compare against the ones quoted above.🤖 Generated with Claude Code
https://claude.ai/code/session_01BtVuh6N448SbFMQxCguJx7