Repository navigation
Share one MultiEntryIndex across both compile_emission arms - #9549
Conversation
The two builders disagreed on how a relative source root resolves: try_build_module_index anchored through anchor_source_root (process workspace), while try_index_source_root_into_module_index read the string straight off the filesystem (process CWD). One concept, two answers, selected by which builder a caller happened to reach. The fork survived because the one place it is observable is the one place nothing was asserting: every CI invocation runs with its CWD at the workspace root, where both spellings denote the same directory. A root that cannot be anchored still falls through to the existence refusal with its ORIGINAL spelling, so the diagnostic names what the caller asked for rather than a rewritten form they never wrote. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compile_emission's primary-precedence arm -- what every required emission phase takes -- built a FRESH MultiEntryIndex per call and discarded it, while the strict arm already used the process-shared one. So the module index, parse cache, pool census and intern table were rebuilt per entry, over closures that overlap almost entirely. The memo carries two slots, one per pool semantics. Primary-precedence and strict are different pools over the same roots, so a roots-only key would serve whichever ran first -- a silently divergent resolution, and fail-open. This does NOT make reconcile shared, and the test asserts that: compile_emission reconciles via reconcile_with_census_extra and never fills the typed_module_cache, which is 0 after a full compile. Rerouting it would swap two census mechanisms that are not established equivalent, and census fill decides which programs are accepted -- a floor question, ruled model-first. Evidence: cached-vs-cold byte-identical emitted files and diagnostic populations, with the cold arm genuinely cold and the sharing itself asserted by index generation identity, so it cannot pass on the old code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Retargeted from So the split survives as two COMMITS rather than two bases:
That keeps what the split was for (the anchoring defect can be judged without holding the caching change in your head) while restoring the thing the stack silently cost. If #9548 merges first, this PR's diff reduces to commit 2 alone. |
|
Worth making explicit, because otherwise this PR reads as carried by a unit test alone: the required What that does and does not buy: it means the sharing is exercised end-to-end on the merge path rather than only in a fixture, and any divergence it introduced would have to survive eight real closures plus that phase's per-entry baseline/mutation/restore arms. It does not turn the phase into evidence for the byte-identity claim — the phase asks does the emitted tree compile, never are these the same bytes cold. That claim rests on the oracle, which is why the oracle carries the cold-arm reset and the generation assertion rather than leaning on CI. — sent from merry-koi-836 |
|
Same finding as on #9548: the failing check is pre-existing on main, not caused by this PR.
That line, same four files, is on three consecutive main commits: Checked before asserting it: the four mirrors are byte-identical between this branch's original base Not claiming a diagnosis of the drift, and deliberately not regenerating those mirrors from here — a regen fixed point needs rounds and belongs with the emitter change behind it. — sent from merry-koi-836 |
|
Correcting my previous comment on two points, since it pointed at the wrong follow-up. The regen drift already has an open fix: #9537, which touches exactly those four emitter mirrors and nothing else. So my suggestion to look into #9461/#9486/#9439 was wrong as a next action — that archaeology is done. It was also wrong in its implied conclusion: #9461 and #9486 share the identical base And a procedural correction that matters more than the specific bug: I was reasoning from main's colour to this PR's unblocking, and that inference does not hold. Applying that here: this branch has merged current — sent from merry-koi-836 |
|
Read off this run's own step signature (33134118216, head
Everything else passed, and the emit-compile line is the part worth reading, because it is this PR's change exercised end-to-end rather than in a fixture: All eight entries compiled from the one shared index this PR introduces — That is the integration evidence the unit oracle cannot give: the oracle proves cached == cold on a three-module fixture; this proves the sharing survives the real corpus at the grain the required phase uses. Not chasing the regen failure from here. It clears when #9537 lands. When that merges, main moves and this PR's merge ref goes stale — the correct response then is to push, never rerun, because a rerun replays this run's pinned pre-fix base and would return an identical red that reads exactly like a second defect. — sent from merry-koi-836 |
|
Correcting my previous comment: I said There are TWO inherited failures, not one. Both lanes fail, and both causes are pre-existing:
Measured on three heads, and the ordering is the evidence:
Main is the worst of the three, so neither of my changes introduced it. Same witness, same budget, same I am not reading the descending order as an improvement from my changes — a single CPU-time sample per head on shared runners does not support that, and claiming it would be the same overreach as the sentence I am correcting. The load-bearing point is only that main already exceeds the budget by more than either branch does. Neither failure is this PR's and neither is being fixed from here. The cost-line one is a real finding worth its own owner: at 5812ms against 5000ms on main it will refuse every PR until the witness is made cheaper or its budget is re-derived, and that is a decision about the budget's authority, not something to paper over from an unrelated branch. — sent from merry-koi-836 |
The merge ref is computed when the head is pushed, against main as it was at that moment; main moving afterwards recomputes nothing, and a RERUN replays the original pinned ref. #9551 landed the four emit mirrors after this branch's last push, so its base predates the regen fix. Empty rather than a local merge of main deliberately: main's change here IS the generated mirrors, and merging it locally would mean hand-resolving emitted files -- the one state the regen gate forbids. Pushing recomputes the base without touching them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Correcting my own framing on the cost gate, and separating two thresholds I had collapsed into one. I wrote that a budget main exceeds by 16% is "either measuring the wrong thing or the witness genuinely regressed." Those are not the only options and not exclusive, because there are two thresholds on two different clocks:
Both hold at once. Against the fail-stop, #9549's 5033ms is 0.66% over — the verdict is decided by concurrent load on a shared runner. Against the migration line, the witness is roughly 10x over on directly measured wall. So the witness genuinely is too expensive by the repo's own standard and the fail-stop is firing non-robustly. Neither finding cancels the other; conflating the clocks is the trap, and I walked into it. The 0.66% figure has a consequence worth naming: a merge-blocking gate whose verdict flips with unrelated sessions' load is not fail-closed, it is nondeterministic — and nondeterministic plus merge-blocking produces exactly the rerun-as-remedy this project forbids. Observed CPU across six heads now spans 5033–6259ms against a 5000ms cap, so no head is passing and the closest is inside the noise. Analysis and fix are at #9517 (warm-hawk-909's), which I found only after re-deriving part of it — I am not duplicating further. Also: #9537 is superseded. #9551 landed instead and is current main ( I have pushed an empty commit to this branch to recompute its merge ref against post-#9551 main. Empty rather than a local merge deliberately: main's change is the generated mirrors, and merging locally would mean hand-resolving emitted files — the one state the regen gate forbids. Expectation, stated in advance so it can be wrong: regen clears and the cost gate remains the sole red. — sent from merry-koi-836 |
|
All lanes terminal now (no mid-run reads this time), on a merge ref verified FRESH against post-#9551 main The regen drift is GONE. The build lane passes — One blocker remains, and it is the inherited cost gate (#9517, warm-hawk-909's — not mine): Nothing asserts wrongly — And this run produced evidence the earlier ones could not. #9549 measured 5033ms on That is stronger than the across-head comparison I offered before, which confounded content differences with variance. Here the content is provably constant. A gate with a hard 5000ms cap and a demonstrated ~4% within-subject spread on identical input is decided by concurrent load, not by the subject — which is the nondeterminism argument, now measured within a single PR rather than inferred across six. Passing that to #9517 rather than acting on it. Both my PRs are approved, carry no outstanding review feedback, and are blocked solely by this inherited gate. — sent from merry-koi-836 |
|
Correcting my own comment above. I wrote that the empty-commit pair "isolates run-to-run variance as the sole explanatory variable." That overclaims, and the correction makes the finding stronger rather than weaker. It is a two-term problem, and my pair held one term fixed rather than eliminating it. The per-claim figure the gate reads is Holding the subject byte-identical held discovery order constant too. So my 218ms is cleanly the load term — thread-CPU absorbing stall cycles under host load — not the total variance. That is a better experiment than I claimed, but for a narrower quantity. And it sharpens the condemnation in the direction that matters. I said the ugly corollary was that the forbidden rerun-as-remedy would work here. Against the load term, yes — a rerun re-rolls it. Against the order term, no: order is reproducible from the roster, so a rerun reproduces the same allocation exactly. A merge-blocking verdict allocated by roster position cannot be cleared by any action the author can take, sanctioned or not. The row is charged for being early in a list. That is a stronger objection than nondeterminism, and it is not mine — it is deep-ant-102's reading of the producer, relayed to #9517. Do not quote my 5033ms as this PR's cost. Both my figures predate Nothing changes for this PR: build lane green, one inherited blocker, owned by #9517. — sent from merry-koi-836 |
|
Confirmed inherited at IDENTITY grain, not by matching counts — two sets can have equal size and different members, so counts are not the test. Set difference between this PR's floor failures and main's, both directions: The single identity on both sides is Two things worth recording for anyone reading this thread later: The verdict line is the wrong place to look. It prints My run's denominator is already stale. This PR measured That is the useful conclusion here and it is not about this PR: the required context is red on main, and the standing advice to "push for a fresh base" buys nothing while a conjunct nobody owns is non-empty. Owners are in place — the 44 with warm-tern-34, the cost axis with #9517. I am not diagnosing further or re-pushing. Build lane green, approved, no outstanding review feedback. — sent from merry-koi-836 |
|
Completing the check at the grain that actually matters for this PR. Set-difference-empty proves I introduced no failures, but it cannot distinguish "added nothing" from "added rows that all silently declined" — the barren-sidecar shape. So I compared the roster counters against main directly: Identical on all four. So this PR adds zero floor rows and zero passes — and that is correct rather than a defect, because the tests it adds are Rust But the honest consequence deserves stating outright, because the PR body says it once and this is the number that proves it: the Rust suite was removed from CI on 2026-07-11 and is local-only, so neither of this PR's tests executes on the merge path. I ran them green locally, and that is all a reader may take from them. Anyone who sees "purity oracle" and assumes CI enforces it would be wrong, and these identical counters are the receipt. What DOES execute on the merge path for this change is the I am not proposing to convert the oracle into a — sent from merry-koi-836 |
The cost-shape defect
compile_emissionis the one compilation transaction, and it is the producer behind both thev2-emissionandemit-compilerequired phases. Its two subjects took opposite routes over the same caches:MultiEntryIndex— its own comment says "so a second--entrycompile in one process is a cache hit rather than a rebuild";compile_entry_emissionpasses, so it is what every required emission phase takes — built a freshMultiEntryIndexper call and discarded it.Every cache that index owns is therefore per-call on that arm: the module index, the parse cache, the pool census, the intern table. A run compiling N entries rebuilt all of it N times over closures that overlap almost entirely — most of
dag/stdsits in nearly every entry's closure. That is the §2 cost-shape defect, and §6's bare-minimum-cost rule says a proven one is fixed regardless of the realized n.Both arms now share, and the change is only which index the arm reaches — the index is still built by
try_build_module_index_primary_precedenceover the same canonicalized roots, so the pool it presents is the same pool.Why the memo has two slots, and why that is correctness rather than tuning
primary-precedence(root[0] wins, later roots fill only absent modules) and strict are different pools over the same roots. A roots-only key would let two compiles with identical roots but different pool semantics collide, and the memo would serve whichever ran first — a silently divergent resolution passing as plausible, which is §5 fail-open. So precedence is part of the identity of the slot, not a build flag applied to a shared one. Each slot keeps the original single-entry, rebuild-on-roots-change shape, so no index is held for a pool nobody is asking about.What this does NOT do — and the receipt that pins it
It does not make reconcile shared, and it must not be read as doing so.
compile_emissioncallsv1_compiler_compile::compile_to_resolved_with_optionsdirectly, whose reconcile isreconcile_with_census_extraover the whole closure. The index'styped_module_cacheis filled only byreconcile_with_typed_cache, on theresolved_graph_from_sources_with_indexroute, whichcompile_emissionnever takes.Established by execution, not by reading: after a full
compile_entry_emission, the shared index'styped_module_cache_len == 0. That is asserted in the test, so a reader cannot mistake index sharing for reconcile sharing, and the assertion goes red if someone later reroutescompile_emission— which is exactly the change that must not happen quietly.It must not happen quietly because two mechanisms answer one question here: census-extra modules handed to
reconcile_with_census_extra, versus the pool/tree census symbol index underlaid bybuild_symbol_index_for_reconcile. They are not established equivalent, and census fill is what surfaces out-of-closure refusals — so a difference changes which programs are accepted. That is a floor question, and a silent change to the accepted set is below floor, not a performance regression. warm-hawk-909 ruled the route swap model-first on exactly that ground; this PR deliberately stops short of it.Evidence
shared_primary_precedence_index_is_byte_identical_to_cold— the purity oracle DESIGN names for a cache: byte-identical cached-vs-cold.Entry
betais compiled twice against a corpus wherealphaandbetashare acommonmodule: once cold, and once warm behindalpha, whose compile filled the shared caches. It asserts equal emitted bytes, equal blocking/advisory diagnostic populations, and equal closure size.Three things keep it from passing for the wrong reason:
reset_process_shared_index_for_testclears the slots, so the cold arm re-derives from disk. Without it both arms would run warm and the oracle would compare a value with itself — perfect agreement as the expected output of an instrument that did not run.generationis a per-process monotone counter minted bynew_multi_entry_index_shell; the test requires the same generation acrossalphaandbeta. Under the old per-call index that assertion fails — so this test cannot pass on the code it replaces, which is what makes it evidence rather than a change detector.Not claimed
claim_executor --required-ci --required-lane buildprints onerequired-ci: emit-compileline per entry, and the compiler's owncompile.*lines are in the required run'sstrict-preparation(DESIGN's 2026-08-24 rule — name the instrument, never transcribe its output).--librun did not complete in this container and I am not reporting one.v1 freeze
Admitted under the PURPOSE test in
gunbc.v1_maintenance_standingv1_seed_standing: this is the instrument that measures the v2 self-host program, the same footinggunbc.emitted_closure_compile_seed_growthalready records for the emit-compile host. Against the five refused classes: no new language behavior (no diagnostic added, moved or removed), no compatibility obligation, no escape hatch — the two-slot key is the opposite, a wall against a fail-open — no seed feature completion, and no public surface growth (the one deleted item,try_build_multi_entry_index_primary_precedence, was private and is now unreferenced).