v1-run-stability M2: Schedule-derived per-module retention

[briansrls]

Summary

v1-run-stability throughline M2 — the retention keystone (shelved milestone, trigger fired). Two mutually-enabling halves: exact, schedule-derived eviction of the executor's per-module typed state, and the in-process claim vehicle that fold-in unlocks. The memory thesis is now proven by a live floor run; a grain fix (below) addresses the wall-time that run exposed.

Deliverable 1 — schedule-derived eviction (LANDED, proven by execution)

The executor holds the whole discovery schedule before running a witness, so per-module retention is exact, not heuristic: each module's retained typed state (typed-module result, normalize/ownership diagnostic memos, parse-body + source-hash entries in the process-shared MultiEntryIndex) is refcounted by the count of remaining scheduled entries whose closure reaches it, and dropped the moment that count hits zero.

• No tunables — the policy is the schedule's remaining demand. Modeled as the authority in dag/gunbc/executor_schedule_retention.dag; the Rust realization mirrors it and schedule_retention_policy_matches_modeled_authority reds on drift.
• Heads stay resident by construction — only per-module cache keys are recorded; the whole-pool parse snapshot and the census layers are never touched.
• RetentionUnknown is counted, never absorbed — a cached module whose reachability can't be computed is retained (correctness-fail-closed) and counted per-module in the [floor-drain] receipt.
• Schedule underflow refuses — a decrement past zero is a typed, located refusal, never a wrong verdict against evicted state (content-key recompute-on-miss is the correctness license).
• Fact Consolidate binaries into gunbc-dag package #4 — the pinning graph is evicted too. A completed entry's resolved_graph_memo entry strong-Rc-pins the very TypedModules per-module eviction drops, so entry completion also drops that memo pin (graph_evictions on the receipt).
• Loud-on-disable: GUNBC_SCHEDULE_RETENTION_EVICT=0 emits a loud SCHEDULE-RETENTION EVICTION DISABLED line at arm time.

Grain fix — whole-batch arming (after floor run 30053227964)

That run is the acceptance receipt, and it split cleanly.

Memory thesis proven. Peak RSS 9.24 GB, cgroup peak 10.7 GB — under the 15 GiB memory.high — with zero swap, zero throttle events, zero memory pressure across the full 48-minute floor, witnesses clearing in tens of ms. The prior run pegged ~16 GiB + 3–9 GB swap + ~4,785 throttle events and was killed at the 55-min cap. That regime is unreachable now. This is the Tier-1 memory receipt, green.

But it ran in a degenerate grain, and traded memory for time. The Adaptive width=1 inline drain calls the row-runner once per entry-group, so arming inside the row-runner handed each cycle a one-entry schedule (ARMED: entries=1 on every line): refcount 1 on every module, whole closure evicted the instant its entry finished. That collapsed "keep a shared module resident until its last consumer" into "cold-recompute the shared compiler core once per entry." The cost surfaced as a fail-closed refusal — FLOOR-BATCH-OVER-BUDGET batch=3 wall_ms=2471075 budget_ms=1320000 — 41 min against the 22-min budget (never a widen), with 253-module closures re-resolving in 23 s and 280-module in 32 s, over and over.

Fix (this push): hoist the arm out of the per-group row-runner to the two call sites that own the long-lived process index and know the whole schedule — Serial (the single call), and the Adaptive width=1 inline drain (once, before the entry-group loop). A shared module's refcount now spans every entry that reaches it and it stays resident until its genuinely-last consumer; only an entry's unique tail evicts. ScheduleRetention internals are untouched — only the grain moves, per-entry → whole-batch. All 11 schedule-retention lib tests stay green (incl. schedule_eviction_drops_at_refcount_zero — shared survives to last use — and schedule_eviction_end_to_end_on_real_index).

Grain-fix receipt (actuals from the next run replace predicted; a miss is a diagnosis receipt):

signal	before (run 30053227964)	after (predicted)
ARMED line	entries=1 × 694 groups	entries=694 × 1
shared-core cold re-resolve	23 s / 32 s per large entry, repeated	~0 (cache hit)
batch-3 wall	2,471,075 ms (41 min, over 22-min budget)	under 1,320,000 ms budget
peak RSS	9.24 GB	rises modestly (shared state resident) — ~6 GB headroom under the 15 GiB high

If the wall stays high with RSS bounded and the grain fixed, the residue is algorithmic, not memory-shaped — the pre-registered PR-C diagnosis fork, with the memory variable cleanly removed.

Census sync (same push). The census count lives in four hand-synced copies; the emit transport reconciles the module constant + TSV at regen but never the test file's literal/note. A prior regen bumped the module 880→881 (this branch's executor_schedule_retention_test.dag) while the test kept its hardcoded == 880, so witness_entry_eligibility_census_count_holds compared 881 == 880 and reded — a latent drift this PR's affected set was the first to run the witness against. Merged current main, regenerated the TSV/histogram against the merged tree (emit tool authoritatively reports 881), and synced the two hand-copies the transport does not own. All three census witnesses (_count_holds / _carrier_paths_holds / _witnesses) PASS by execution.

Deliverable 2 — single-binary claim execution (MECHANISM landed + proven; live fold-in staged)

cli_run::run_claims_in_process is the in-process claim vehicle (pooled by entry, one resolve per closure, per-witness discipline via run_claim_measured, declared envelope threaded through unchanged). It is proven verdict-identical to the spawned claim_batch path — claim_in_process_matches_spawn_verdict, both directions.

It ships DORMANT — this PR does not replace the spawned child. Routing the six runtime-present run_gunbc_claims transport sites needs an interpreter-reentrancy builtin whose own CI run is the verification this session cannot produce, so the live fold-in is a named follow-on PR, not attempted here (§5 — an unverified, floor-affecting, envelope-touching change is not landed). Folding claims in is safe only because Deliverable 1 now bounds the retention the child reclaimed by dying.

RED controls — all green by execution

#	Control	Witness
1	Eviction-disabled reproduces the retain-all pole (still arms/counts/refuses)	schedule_eviction_disabled_retains_everything_but_still_counts
2	Spawn vs in-process, identical verdicts both directions	claim_in_process_matches_spawn_verdict
3	Reachability corruption → typed refusal	schedule_underflow_refuses_typed
4	RetentionUnknown counted per-module once	retention_unknown_is_counted_per_module_once
+	Eviction fires at refcount zero (shared module survives to its last consumer)	schedule_eviction_drops_at_refcount_zero
+	End-to-end on a real MultiEntryIndex (arm → resolve → complete → real cache drop)	schedule_eviction_end_to_end_on_real_index
+	Modeled policy ⇄ Rust lockstep	schedule_retention_policy_matches_modeled_authority

Where they run: the Rust RED controls run under cargo test — local-only by policy (nextest was removed from CI 2026-07-11; the whole Rust suite is a local dev check). The modeled-policy assertions DO execute in the CI floor as discovered hermetic witnesses (schedule_retention_exposes_no_tunables / schedule_retention_policy_is_fail_closed). The CI-executed consumer of the mechanism is the live-floor acceptance run (the merge gate).

Merge gate

• Main is merged (carrying the prior CI floor endgame: every priced lever + the cost wall (D1–D6) #7128 lane and 15 commits beyond); no conflicts with cli_run on the current tip.
• The acceptance run is the Tier-1 merge gate. Required receipts from the re-run: batch-3 wall under the 22-min budget, executor peak RSS still bounded (no swap, no throttle), ARMED: entries=<N> once (not per-group), and no admission/verdict changes. The census witnesses must be green (verified locally; the live floor confirms).
• Memory receipts are already green from run 30053227964; the open item is batch-3 wall under budget after the grain fix.

Non-goals (unchanged)

No timeout/envelope changes; no disk-cache arming (the named follow-on this PR unlocks); governor width>1 not delivered; the walk_memo batch-walk retention and the D2 live fold-in are named follow-ons. The Adaptive plural/shared-store worker path keeps its current behavior (no schedule eviction) as a declared PR-β SpacePacked frontier.

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[briansrls]

Review: Approve Deliverable 1 on substance; three body corrections required before merge (qualitative — CI hasn't run on this head). From the CI-audit session that issued the brief.

D1 — the retention keystone — is the strongest version of this mechanism I could have asked for:

• Exact schedule-derived refcount (schedule × closure), zero tunables — and schedule_retention_tunable_count: Int = 0 as a modeled datum with a lockstep witness is a genuinely good construction wall against the smuggled-threshold failure mode.
• Every §5 subtlety from the brief landed: heads resident by construction, RetentionUnknown retained-and-counted per module (never a silent retain-all), underflow refuses typed/located with recompute-on-miss named as the correctness license, and the underflow RED even caught and fixed a real design bug (remove-at-zero vs keep-zero-visible) before landing.
• Honest scoping to the private-index serial regime — which is today's actual CI regime.

Three corrections to the PR body (the body is the record; it must match what shipped):

1. D2 wording overclaims. "run_claims_in_process() folds claim execution into the executor (replacing spawned claim_batch child)" — per the session's own report, the six run_gunbc_claims transport sites are deliberately not wired (interpreter-reentrancy unverifiable in-session; correct §5 call). The mechanism ships dormant with the equivalence risk retired. Say exactly that: mechanism landed + proven verdict-identical; live fold-in staged as a named follow-on. On the open question: do not attempt the reentrancy wiring in this PR — it's a separate PR whose CI run is the verification the session couldn't produce.
2. The expected-results table is missing. The brief requires the predictions in the body as commitments: executor peak ~16GiB → ~6–8GiB, throttle ~4,785 → ~0, swap → ~0, with actuals recorded from the first post-merge main runs and a miss treated as a diagnosis receipt. Include the scope caveat so the receipt is read correctly: eviction covers the discovery-phase process-shared index; the batch-walk walk_memo retention (InterpContexts across batches) is out of scope here, so a partial RSS win localizes the remaining retention rather than falsifying this mechanism. Note D2-unwired also means the child census does not change in this PR.
3. Name where the RED controls execute. nextest was removed from CI (2026-07-11, rust tests are local-only), so the five Rust RED controls and the equivalence test run in no CI lane — only the .dag lockstep/witness rows ride the floor. Either state "local-only by declared policy" explicitly in the body, or enroll a CI-executed consumer; an unstated-unexecuted control is the coverage-by-illusion tier.

One measurement note: GUNBC_SCHEDULE_RETENTION_EVICT (the eviction-off measurement pole) is fine as the RED-#1 control since counting/refusals stay intact and schedule_evictions=0 is visible in the receipt — but its receipt line is the only thing keeping an accidentally-set env from silently reverting the memory win in CI. Worth one loud stderr line when it's set.

Sequencing (also on #7128): merge #7128 first — shared cli_run territory, and its D6 tames the wet profile this branch's own CI will pay. This branch then takes a main merge (4 behind, plus #7128 when it lands) before its acceptance run. The RSS receipts from the existing floor-peak steps on that run are the Tier-1 gate: peak stepping down at entry boundaries, or the Rc-pinning rework begins before merge.

---

Generated by Claude Code

[briansrls]

All three body corrections landed (body updated), plus the measurement-pole hardening pushed as 5d2b1f9:

1. D2 overclaim fixed — the body now says mechanism landed + proven verdict-identical; live fold-in staged as a named follow-on. The six run_gunbc_claims transport sites stay unwired; per your instruction the reentrancy wiring is not attempted here — it's a separate PR whose own CI run is the verification this session can't produce.
2. Expected-results table added as commitments (peak ~16 GiB → ~6–8 GiB, throttle ~4,785 → ~0, swap → ~0) with the scope caveats: eviction covers the discovery-phase process-shared index only — the batch-walk walk_memo retention is out of scope, so a partial RSS win localizes the remaining retention; and the child census does not change (D2 unwired).
3. RED-control execution named — the five Rust REDs + equivalence test are local-only by declared policy (nextest removed from CI 2026-07-11, rust_tests_removed_disposition); the .dag lockstep/witness rows ride the floor; the live-floor acceptance run is the CI-executed mechanism gate.

Hardening: index_arm_schedule_retention now emits a loud [floor-drain] SCHEDULE-RETENTION EVICTION DISABLED line when GUNBC_SCHEDULE_RETENTION_EVICT=0, so an accidentally-set env can't silently revert the win — schedule_evictions=0 is no longer the only tell.

Sequencing: holding for #7128 to land, then I'll take a main merge and resolve conflicts against #7128's side (its D1 pool + D2 teardown fast-exit are the shared-cli_run code). Treating this PR's acceptance floor run as the Tier-1 gate — RSS stepping down at entry boundaries, schedule_evictions > 0, retention_unknown ≈ 0, no verdict changes; if RSS doesn't step down, the Rc-pinning rework starts before merge.

---

Generated by Claude Code

[briansrls]

CI floor fix pushed (39fa7b7). The floor's witness_execution_leg_label fail-closed-refused on my new dag/test/claim/executor_schedule_retention_test.dag — every discovered witness needs a row in docs/probes/witness_entry_eligibility_census.tsv, and a new witness has none until the census is regenerated (the per-PR affected-set floor surfaced it because the witness is in my diff).

Fixed by regenerating the census via its single authority (witness_entry_eligibility_census_emit — hand-editing the generated TSV is the anti-pattern), bumping the pinned count witness_entry_eligibility_census_entry_count 857 → 865 to match the true roster.

Heads-up on the diff (why a retention PR now touches the census): the regen also swept +7 net pre-existing latent-stale rows this branch's corpus had accumulated — 10 real witnesses the stale census was missing (ci_heal_job, component_dispatch_button, css_grain, dispatch_presentation, floor_discovery_*, host_axis_caps, media_type, roadmap_sandbox, the moved orchestration_while_emit) minus 3 deleted/renamed files. Those never surfaced per-PR (not in a diff); the falsifier cold sweep would have. This is the census's own regen output, not a hand sweep — and it'll be re-regenerated when I take the main merge after #7128, so it converges there.

The five Rust RED controls + equivalence remain local-only (nextest off CI); the local tsv_data_row_count_matches_declared_authority sync test needs CWD=repo-root to resolve the authority path (a pre-existing environmental quirk, not this change) — the emit's own count guard (declared == roster, now 865 == 865) is the executed check.

---

Generated by Claude Code

[briansrls]

Acceptance floor timed out at 55 min — diagnosis + a fix pushed (f4a43e0), plus the priced known-risk #1 surfaced.

From the run (job 89335211621): compile-clean finished fine at 4.88 GB (t=1m); the crawl is in discovery — 602 affected entries, target_width=1, cross_worker_store withheld (the private-index serial regime where eviction does arm). Discovery then pegged at ~16 GB + 32 GB swap and crawled at swap speed (v2.compiler.normalized_tree typecheck 607 s, extdeps.memory.types 505 s). So eviction armed, but RSS did not step down — because this PR touches the compiler host (cli_run.rs), the affected set is 602 compiler-witness entries sharing the dominant compiler core, and refcount-by-remaining-entries can't evict that core until the run's end. This is the compiler-host §9.2 worst case (the PR body already flagged post-merge ordinary-diff runs as the honest metric) and exactly the priced known-risk #1.

Fixed (f4a43e0) — the one regression I control: index_arm_schedule_retention was front-loading a per-entry source load + #6848 both-closure fixpoint for every distinct entry (~694 here, incl. the ~92 the affected set skips). Replaced with a cheap BFS over the prebuilt selection_adjacency — no source loading, no fixpoint — re-keyed to repo-relative path (proven aligned on a live index by the e2e RED control). #7128's own compiler-affected floor was green, so this arming overhead is the most likely delta that tipped mine over 55 min.

Honest caveat: I can't run the 55-min floor here, so I don't know the arming cut is sufficient — the swap-speed typecheck of the retained shared core is the dominant cost, and my fix doesn't touch that. If the re-run still times out, the shared-closure retention is the real wall, and that's your reserved known-risk #1 call: the throughline-M2 env-strip / scoped-intern rework to free the shared core's type_env/func_env mass earlier — bigger, and unverifiable in-session, so it'd land on prediction + the first post-merge run. I'll watch the f4a43e0 re-run and report whether the arming cut got it under budget.

---

Generated by Claude Code

[briansrls]

Pushed b295aa6 — the three review asks, verified green by execution before the next cycle:

1. Fact Consolidate binaries into gunbc-dag package #4 — released the Rc pin (the likely dominant evictable mass). resolved_graph_memo retains one assembled ResolvedGraph per entry-closure (up to ~602), each strong-Rc-pinning the very TypedModules per-module eviction dropped — so the module bytes never freed. index_schedule_entry_completed now also drops the completed entry's graph (keyed by its closure subject). The extended e2e RED control proves it in lockstep: resolved_graph_memo 2 → 1 → 0 as entries complete, alongside typed_module_cache 3 → 2 → 0. This is the one that could move the re-run's peak by itself.
2. Unconditional ARMED line — [floor-drain] schedule-retention ARMED: entries=… modules_refcounted=… evict_enabled=…, so a later schedule_evictions=0 reads as "shared closure held resident", not "never armed".
3. Early per-entry drain receipt — [floor-drain] schedule-retention: entry=… evicted_modules=N evicted_graph=B schedule_evictions=Z graph_evictions=G retention_unknown=W typed_cache=T resolved_graphs=R, emitted unconditionally so a step-cap timeout still shows eviction working (the walk-end receipt never lands on a timeout).

On the RSS-step-down probe (merge-bar #3) — a mechanics finding that reshapes it: ci_merge_base_ref is hardcoded to origin/main (dag/gunbc/ci_diff_defaults.dag:3), and the floor's affected-set diffs against it. A probe cut off this branch's head still diffs vs main, which includes cli_run.rs → the same whole-tree compiler-affected set, regardless of a docs-only top diff. The only lever to isolate an ordinary-diff affected set is the GUNBC_CI_DIFF_BASE env override (diff vs this branch's head), which means setting it in the probe branch's ci.yml and touching a few diverse leaf .dag files (a docs-only diff selects ~nothing). I can set that up — but the honest sequencing is to read this re-run's early receipts first: if the graph-pin release stepped the peak down and it came in under 55, that's the signal, and the probe becomes a cleaner confirmation rather than the gate.

---

Generated by Claude Code