CI-humming PR2: ground oomd evidence from the gunbc.oomd_install seam + de-conflate spec from realization-evidence

.github/fleet-runner-deploy.manifest

@@ -1,7 +1,7 @@
 runner-deploy UNSOUND: runner-slot conserving cap not enforced live: the runner-slice MemoryMax drop-in is inert (effective MemoryMax=INFINITY); refuse to derive a governing runner count that assumes a cap the host does not enforce (fail closed until ctrl runner_host_reconcile makes disk==live and the design authority flips it)
 session-slice host=srv1 slice_max_bytes=21474836480 per_session_max_bytes=10737418240 oom_pressure_kill=true oom_pressure_limit_pct=60 cpu_weight=1000
 session-slice host=srv2 slice_max_bytes=21474836480 per_session_max_bytes=10737418240 oom_pressure_kill=true oom_pressure_limit_pct=60 cpu_weight=1000
-session-reservation UNSOUND: REGIME-2 keystone unmet: it over-commits the per-session caps, so it is unsound until per-cgroup memory.max + systemd-oomd eviction is verified live on the hosts (oomd_enforcement = OomdUnverified). Fail closed; do NOT reserve the smaller aggregate without the runtime eviction floor.
+session-reservation UNSOUND: REGIME-2 keystone unmet: it over-commits the per-session caps, so it is unsound until per-cgroup memory.max + systemd-oomd eviction is verified live on the hosts (gunbc_fleet_oomd_evidence = OomdUnverified). Fail closed; do NOT reserve the smaller aggregate without the runtime eviction floor.
 operating-curve (SESSION-side coverage only: operator selects one per-session operating point grounded in measured per-session peaks min 0.28 / median ~2.06 / max 8.27 GiB; sessions_slice = per_session_cap*concurrency. The per-host runner_count is budget-derived from runner_slice_cap and reported on the runner-deploy lines above, independent of this curve):
   per_session_cap_bytes=2211908157 concurrency=16 sessions_slice_bytes=35390530512 peak_coverage=6/13
   per_session_cap_bytes=4294967296 concurrency=16 sessions_slice_bytes=68719476736 peak_coverage=10/13

dsl/gunbc/ci_budget_tree.dag

@@ -24,6 +24,7 @@ import gunbc.ci_runner_placement {
   SessionSliceResolution, SessionSliceResolved, SessionSliceUnresolved,
   resolve_session_slice
 }
+import gunbc.fleet_host_budget { gunbc_fleet_oomd_evidence }
 
 data gunbc_ci_budget_tree_deconfliction_note: String = "this conserving host BudgetNode governs INTER-run co-residence (host_ram -> overhead + sessions_slice + runner_slice; REGIME-1 conservation by construction via byte_size_saturating_sub). It is intentionally NOT the within-run spawn-width budget: within_run_spawn_budget (whole-run RSS x margin) is a separate carrier feeding memory_aware_spawn_width (#5444) for parallelism INSIDE one run, and must stay non-fail-closed so the live floor keeps running when the inter-run operating point is unset. Do not fold the two back together (see gunbc_ci_footprint_granularity_note)."
 
@@ -43,7 +44,7 @@ fn host_allocatable_bytes(offer: ComputeOffer) -> ByteSize {
 }
 
 fn session_pool_bytes_live(offer: ComputeOffer) -> ByteSize {
-  match resolve_session_slice(mode: gunbc_ci_session_reservation_mode) {
+  match resolve_session_slice(mode: gunbc_ci_session_reservation_mode, oomd: gunbc_fleet_oomd_evidence) {
     SessionSliceResolved { bytes: s, per_session_cap: _, concurrency: _ } => s
     SessionSliceUnresolved { reason: _ } => host_allocatable_bytes(offer: offer)
   }

dsl/gunbc/ci_floor_measurement.dag

@@ -92,14 +92,13 @@ type RunnerSliceCapEffectiveness
 
 type SessionReservationMode
   = Regime1WorstCase { point: SessionOperatingPoint }
-  | Regime2AggregateOomd { oomd_eviction_ceiling: ByteSize, session_concurrency: Int, aggregate_reservation: SessionAggregateReservation, oomd_enforcement: OomdEnforcementEvidence }
+  | Regime2AggregateOomd { oomd_eviction_ceiling: ByteSize, session_concurrency: Int, aggregate_reservation: SessionAggregateReservation }
   | SessionReservationModeUnset
 
 data gunbc_ci_session_reservation_mode: SessionReservationMode = Regime2AggregateOomd {
   oomd_eviction_ceiling: byte_size(10737418240),
   session_concurrency: 16,
   aggregate_reservation: SessionAggregateMeasured { bytes: byte_size(21474836480) },
-  oomd_enforcement: OomdUnverified,
 }
 
 data gunbc_ci_regime2_aggregate_dissolution_marker: String = "dissolve-on stage:per-type-baseline-build-pool -- Regime2AggregateOomd.aggregate_reservation is the COARSE single-aggregate stand-in. It DISSOLVES into the grounded decomposition host = overhead + Sigma(container-type baselines: GhaRunnerContainer idle ~0.2-0.3GiB + AgentSessionContainer idle ~0.35-0.5GiB, each a measured ResourceEnvelope) + ONE capped shared_build_pool + headroom. The shared build pool is NOT new infra: it is the EXISTING host-wide ctrl-jobserver (scheduling arm) + ctrl-sccache (caching arm) = the two arms of std.realization RealizedStep already unifying GHA-CI builds and session ctrl-builds; its authority is the ctrl-jobserver instance config (build_pool_budget = TOKENS x MIB_PER_JOB, gunbc owns the SHAPE, ctrl owns the values, cited in extdeps per shapes-vs-instances). Coarse-form scaffold per DESIGN sec 6: honest forward-composing, NOT parallel-representation debt; the single aggregate is exactly Sigma(baselines)+build_pool collapsed."
@@ -108,7 +107,7 @@ data gunbc_ci_per_run_peak_sharpen_followup: String = "LANDED (this PR): the con
 
 data gunbc_ci_session_regime_model: String = "REGIME-1 (Regime1WorstCase mode, shipping, safe-now, no runtime floor required): sessions_slice = per_session_cap C * concurrency N -- reserve as if all N sessions sit at their cap simultaneously. This is what the operating curve renders. It forces the hard tension visible in the curve: on a 128GiB host at 24GiB/run you cannot get high concurrency AND full peak coverage AND 3 runners at once. Both regimes are now SELECTABLE via the SessionReservationMode authority (gunbc_ci_session_reservation_mode): resolve_session_slice dispatches REGIME-1 -> C*N and REGIME-2 -> measured aggregate; the committed mode stays SessionReservationModeUnset (fail-closed) until the operator picks."
 
-data gunbc_ci_session_regime2_followup: String = "REGIME-2 is now the COMMITTED mode (gunbc_ci_session_reservation_mode = Regime2AggregateOomd), staged fail-closed: sessions_slice = measured realistic AGGREGATE concurrent peak (the sessions do not all peak together), with C kept only as the per-session OOM CEILING (oomd_eviction_ceiling). The far-smaller aggregate reservation lets high concurrency (16) AND full coverage AND many runners coexist. Gate (a) aggregate_reservation: CLOSED -- SessionAggregateMeasured { 20GiB } from a live measured concurrent peak. Gate (b) oomd_enforcement: STILL OPEN -- OomdUnverified until per-cgroup memory.max + systemd-oomd eviction is live and verified on the hosts (the managed sessions.slice). That oomd floor is the KEYSTONE: it is both the srv2-reboot fix and the REGIME-2 enabler, and it is also what makes mean-packed runner counts safe. resolve_session_slice therefore still returns Unresolved (live plan UNSOUND) -- a single 1-line flip OomdUnverified -> OomdEnforced (PR2, once the managed slice is live) unlocks the full derivation with NO structural change."
+data gunbc_ci_session_regime2_followup: String = "REGIME-2 is the COMMITTED mode (gunbc_ci_session_reservation_mode = Regime2AggregateOomd), staged fail-closed: sessions_slice = measured realistic AGGREGATE concurrent peak (the sessions do not all peak together), with C kept only as the per-session OOM CEILING (oomd_eviction_ceiling). The far-smaller aggregate reservation lets high concurrency (16) AND full coverage AND many runners coexist. Gate (a) aggregate_reservation: CLOSED -- SessionAggregateMeasured { 20GiB } from a live measured concurrent peak. Gate (b) oomd ENFORCEMENT: the keystone -- both the srv2-reboot fix and the REGIME-2 enabler, and what makes mean-packed runner counts safe. DE-CONFLATED (PR2): the mode is pure operating-point SPEC (ceiling + concurrency + aggregate); oomd enforcement is realization EVIDENCE and is no longer stored on the mode. It is threaded into resolve_session_slice from the SINGLE grounded authority fleet_host_budget.gunbc_fleet_oomd_evidence, which grounds OomdEnforced over a live show-effective read via the gunbc.oomd_install seam (gunbc_oomd_fleet_evidence). Until that read is a real ReadObserved on the managed sessions.slice the authority stays OomdUnverified (fail-closed: resolve_session_slice -> Unresolved, the live plan stays UNSOUND). The earlier '1-line literal flip' framing was unsound: the committed mode field could never be grounded in place (gunbc.oomd_install imports this module, so a cycle forbids the mode reading the seam), so a hand-edit of the field to OomdEnforced would be the DESIGN sec 5 anti-pattern (editing a declaration to green a gate the realization does not back). Grounding the one fleet-side authority and threading it is the verify-not-validate realization: when the live read grounds OomdEnforced BOTH gates (session-slice resolution + host plan) open with no further gunbc edit."
 
 data gunbc_ci_footprint_granularity_note: String = "two distinct memory footprints, intentionally separate carriers (a de-conflation, not a fork): (1) INTER-run whole-run cgroup peak gunbc_ci_job_peaks (~24GiB) decides how many runs/runners CO-RESIDE per host -> the conserving budget tree; (2) WITHIN-run per-shard peak gunbc_ci_floor_per_shard_peak_rss_bytes (~2.9GiB) decides parallelism INSIDE one run against its container cap -> memory_aware_spawn_width (#5444). Different granularity (whole-run vs per-shard) and different decision, so two carriers. Do not consolidate them into one authority."
 

dsl/gunbc/ci_runner_placement.dag

@@ -158,7 +158,7 @@ fn resolve_regime1_slice(c: ByteSize, n: Int) -> SessionSliceResolution {
   }
 }
 
-fn resolve_session_slice(mode: SessionReservationMode) -> SessionSliceResolution {
+fn resolve_session_slice(mode: SessionReservationMode, oomd: OomdEnforcementEvidence) -> SessionSliceResolution {
   match mode {
     SessionReservationModeUnset =>
       SessionSliceUnresolved {
@@ -171,10 +171,10 @@ fn resolve_session_slice(mode: SessionReservationMode) -> SessionSliceResolution
         SessionOperatingPointSet { per_session_cap: c, session_concurrency: n } =>
           resolve_regime1_slice(c: c, n: n)
       }
-    Regime2AggregateOomd { oomd_eviction_ceiling: ceiling, session_concurrency: n, aggregate_reservation: agg, oomd_enforcement: oomd } =>
+    Regime2AggregateOomd { oomd_eviction_ceiling: ceiling, session_concurrency: n, aggregate_reservation: agg } =>
       match oomd {
         OomdUnverified =>
-          SessionSliceUnresolved { reason: "REGIME-2 keystone unmet: it over-commits the per-session caps, so it is unsound until per-cgroup memory.max + systemd-oomd eviction is verified live on the hosts (oomd_enforcement = OomdUnverified). Fail closed; do NOT reserve the smaller aggregate without the runtime eviction floor." }
+          SessionSliceUnresolved { reason: "REGIME-2 keystone unmet: it over-commits the per-session caps, so it is unsound until per-cgroup memory.max + systemd-oomd eviction is verified live on the hosts (gunbc_fleet_oomd_evidence = OomdUnverified). Fail closed; do NOT reserve the smaller aggregate without the runtime eviction floor." }
         OomdEnforced =>
           match agg {
             SessionAggregateUnmeasured =>
@@ -304,7 +304,7 @@ fn runner_deployment_plan_for(
   match enforcement {
     RunnerSlotUnenforced { reason: r } => PlanUnsound { reason: r }
     RunnerSlotEnforced =>
-      match resolve_session_slice(mode: mode) {
+      match resolve_session_slice(mode: mode, oomd: oomd) {
         SessionSliceUnresolved { reason: r } => PlanUnsound { reason: r }
         SessionSliceResolved { bytes: session_slice, per_session_cap: _, concurrency: _ } =>
           match resolve_run_unit_peak(job_peaks: job_peaks) {
@@ -370,7 +370,7 @@ fn samples_covered(cap: ByteSize) -> Int {
 }
 
 fn session_reservation_plan() -> SessionReservationPlan {
-  match resolve_session_slice(mode: gunbc_ci_session_reservation_mode) {
+  match resolve_session_slice(mode: gunbc_ci_session_reservation_mode, oomd: gunbc_fleet_oomd_evidence) {
     SessionSliceUnresolved { reason: r } => SessionReservationUnsound { reason: r }
     SessionSliceResolved { bytes: _, per_session_cap: c, concurrency: n } =>
       SessionReservationDerived {
@@ -401,7 +401,7 @@ type OperatingPointRow {
 }
 
 fn evaluate_operating_point(cap: ByteSize, n: Int) -> OperatingPointRow {
-  let slice = match resolve_session_slice(mode: Regime1WorstCase { point: SessionOperatingPointSet { per_session_cap: cap, session_concurrency: n } }) {
+  let slice = match resolve_session_slice(mode: Regime1WorstCase { point: SessionOperatingPointSet { per_session_cap: cap, session_concurrency: n } }, oomd: gunbc_fleet_oomd_evidence) {
     SessionSliceResolved { bytes: b, per_session_cap: _, concurrency: _ } => b
     SessionSliceUnresolved { reason: _ } => byte_size(0)
   }

dsl/gunbc/fleet_host_budget.dag

@@ -40,6 +40,16 @@ import gunbc.ci_floor_measurement {
   RunnerSliceCapEffectiveness,
   CapEffective,
   CapUnbounded,
+  SessionReservationMode,
+  Regime1WorstCase,
+  Regime2AggregateOomd,
+  SessionReservationModeUnset,
+  gunbc_ci_session_reservation_mode,
+}
+import gunbc.oomd_install {
+  OomdGroundingThresholds,
+  gunbc_oomd_fleet_evidence,
+  gunbc_oomd_sessions_policy,
 }
 
 data fleet_build_realization_marker: String = "build = ONE shared host-wide RealizedStep, NOT per-type. shape = CreateEffect CreateIfAbsent keyed on the content hash (sccache content-addresses the compile -> create-if-absent on content key); materialization = Share (the CACHING arm = ctrl-sccache, host-wide across GHA-CI builds AND session ctrl-builds); placement = LocalFilesystem; cost.space = mib_per_job (the per-build memory envelope = SCHEDULING arm = ctrl-jobserver). The two arms (ctrl-jobserver scheduling + ctrl-sccache caching) already unify GHA and session builds = the two arms of std.realization RealizedStep instantiated. A typed RealizedStep value IS constructed (build_realization below), not only this marker."
@@ -92,7 +102,22 @@ fn resolve_headroom(policy: HeadroomPolicy, floor: ByteSize) -> HeadroomResoluti
 
 data gunbc_fleet_runner_slice_dropin_marker: String = "the runner-slice memory cap ALREADY EXISTS on disk as /etc/systemd/system/system-actions-runner.slice.d/10-resource-caps.conf (MemoryHigh=55307438489 ~51.5GiB = 55% of host, MemoryMax=70391285350 ~65.6GiB = 70% of host; cites the srv2 2026-05-28 reboot incident) but is INERT (effective MemoryMax=INFINITY) = disk!=live DRIFT, which the ctrl reconciler runner_host_reconcile.mjs is meant to fix. gunbc does NOT generate a new cap from scratch: it OWNS the cap VALUE = runner_slice_cap = host - overhead - session_worst_case_slice - headroom (the CONSERVING value that REFINES the 55/70% static-fraction policy; build_pool is NOT subtracted -- the GHA build lives INSIDE this slice, counted once via the per-job peak, see gunbc_fleet_build_pool_counted_once_marker), fed INTO that existing drop-in. The 55/70% fractions are the coarse stand-in this conserving value sharpens."
 
-data gunbc_fleet_oomd_evidence: OomdEnforcementEvidence = OomdUnverified
+data gunbc_fleet_oomd_evidence_seam_marker: String = "the SINGLE grounded oomd-enforcement authority for the fleet capacity model. It binds the gunbc.oomd_install seam (gunbc_oomd_fleet_evidence) over the committed show-effective read, assembling the REGIME-2 thresholds (per-session C = oomd_eviction_ceiling, parent slice = aggregate_reservation, PSI percent = the gunbc_oomd_sessions_policy single authority) from gunbc_ci_session_reservation_mode -- NO literal OomdEnforced. It stays OomdUnverified (fail-closed) until a real ReadObserved on the managed sessions.slice grounds the full conjunction (DESIGN sec 5: verify the realization, not the declaration). Threaded into BOTH oomd gates -- fleet_host_plan_for here AND ci_runner_placement.resolve_session_slice -- so one authority gates the whole derivation; when the live read grounds OomdEnforced the plan goes SOUND with no further gunbc edit. Non-REGIME-2 modes have no aggregate over-commit to protect, so they fail closed to OomdUnverified (the fleet plan needs the eviction backstop regardless)."
+
+fn gunbc_fleet_oomd_evidence_for(mode: SessionReservationMode) -> OomdEnforcementEvidence {
+  match mode {
+    Regime2AggregateOomd { oomd_eviction_ceiling: ceiling, session_concurrency: _, aggregate_reservation: agg } =>
+      gunbc_oomd_fleet_evidence(req: OomdGroundingThresholds {
+        per_session_cap: ceiling,
+        parent_slice_cap: agg,
+        pressure_limit: gunbc_oomd_sessions_policy.managed_oom_memory_pressure_limit,
+      })
+    Regime1WorstCase { point: _ } => OomdUnverified
+    SessionReservationModeUnset => OomdUnverified
+  }
+}
+
+data gunbc_fleet_oomd_evidence: OomdEnforcementEvidence = gunbc_fleet_oomd_evidence_for(mode: gunbc_ci_session_reservation_mode)
 
 data gunbc_fleet_build_pool_counted_once_marker: String = "the sccache build pool is NO LONGER a host-level sibling reservation subtracted before the runner slice (that was a DOUBLE-COUNT: a GHA-CI runner job IS a build -- its rustc lives INSIDE system-actions-runner.slice and is already charged in the per-job whole-tree peak that the runner count divides by; subtracting build_pool again starved the runner slice to ~0). Counted-once fix (bright-stag §2/§3 ruling 2026-06-23): the build memory lives INSIDE the cgroup that builds (runner slice for GHA-CI, session footprint for session ctrl-builds), so it is NOT a separate host line. The shared pool's CONCURRENCY survives as effective_build_tokens (= derive_jobserver_tokens, the jobserver authority) -- the build-bound axis of the min-over-constraints runner count, NOT a memory subtraction. runner_baseline likewise lives INSIDE the runner slice (idle is in the per-job peak), so it is an informational sub-allocation of the slice, not a host sibling. The SESSION ctrl-build burst is the session-reservation-plan's per_session_cap concern (REGIME-1 worst-case), re-attributed there, not deleted."
 

dsl/gunbc/oomd_install.dag

@@ -12,7 +12,7 @@ import std.realization_reconcile {
   Reconciliation,
   Converged,
   NotConverged,
-  reconcile,
+  reconcile_grounded,
 }
 import gunbc.ci_floor_measurement {
   OomdEnforcementEvidence,
@@ -210,13 +210,13 @@ fn ground_oomd_from_read(read: OomdEffectiveRead, req: OomdGroundingThresholds)
 fn reconcile_oomd(read: ShowEffectiveRead<OomdEffectiveRead>, req: OomdGroundingThresholds) -> Reconciliation<OomdApplyEffect, OomdEnforcementEvidence> {
   match read {
     ReadAbsent { reason: why } =>
-      reconcile(
+      reconcile_grounded(
         apply_effect: gunbc_oomd_apply_effect,
         show_read: read,
         grounding: DoesNotGround { reason: why }
       )
     ReadObserved { value: observed } =>
-      reconcile(
+      reconcile_grounded(
         apply_effect: gunbc_oomd_apply_effect,
         show_read: read,
         grounding: ground_oomd_from_read(read: observed, req: req)