Rework CI floor allocation: bound concurrency to the live container cgroup memory limit, retire the static per_shard scalar

[briansrls]

Summary

Closes the stale-per_shard → silent-OOM failure mode (3 recurrences: #5867 context, #5875 refresh).

Root cause: spawn_width = min(shard_count, hw_threads, floor(budget / per_shard_committed)). The budget term already reads the live cgroup memory.max. But per_shard_committed (a static ByteSize scalar) drifts low as the corpus grows — the last staleness was 2.1 GiB at real 3.0 GiB, giving width=3, 3×3.0=9 GiB > 8 GiB cap → exit-137.

Construction wall: Replace the floor(budget / per_shard_committed) term with an honest 3-case formula:

1. budget ≤ 0 → conservative fallback width (unchanged)
2. budget < per_shard_committed → width=1 (single-shard guard: budget too small for even one shard)
3. else → min(cpu_bound, confirmed_safe_width)

gunbc_ci_floor_confirmed_safe_width = 2 is a new committed integer: the largest concurrency confirmed safe (VmHWM=6.0 GiB at width=2 with 8 GiB runner cap, 2 GiB headroom, main HEAD 6461042). Stale per_shard_committed can no longer raise width beyond this ceiling — the failure mode is structurally closed.

Honest labeling: This is NOT container-scaling. confirmed_safe=2 on a 16 GiB box still gives width=2 (conservative but correct). Width scales up only when a larger confirmed run updates the integer. The proper scaling path is resource-aware-scheduler Node B/C (docs/plans/resource-aware-scheduler.md) where Runnable.cost.space replaces this scaffold.

per_shard_committed role: Vestigial in the width formula (width=1 guard + ResourceEnvelope memory estimate). Marked in gunbc_ci_floor_per_shard_rss_source Disposition. confirmed_safe_width is a Scaffold dissolving to std.realization_schedule.Runnable when Node B/C lands.

Residual risk (documented, not fixed here): per_shard_committed is not fully inert — it still drives case 2 (single-shard guard). Both scalars can therefore go stale: if a single shard's real peak exceeds the runner cap while per_shard_committed lags low, case 2 won't fire AND confirmed_safe_width=2 still permits width=2 → latent OOM. This is the same staleness class one level up. The dissolution trigger (resource-aware-scheduler Node B/C, live per-shard measurement) covers both. The [calibration] hygiene line must be checked against BOTH scalars when staleness is suspected, not just confirmed_safe_width.

Changed files

• dsl/gunbc/ci_floor_measurement.dag — add gunbc_ci_floor_confirmed_safe_width: Int = 2 + Scaffold disposition; update per_shard source disposition to mark it vestigial in width path
• src/v2/workflow/ci_floor_plan.dag — replace gunbc_ci_floor_spawn_width_for_budget with 3-case formula; simplify gunbc_ci_floor_run_envelope to parallel_run_demand_envelope(width, committed_per_shard); add 2 new witnesses
• src/v2/test/claim/ci_floor_plan_witness_test.dag — import + call new witnesses in new test fn ci_floor_confirmed_safe_width_ceiling_witnesses
• dsl/test/claim/ci_floor_measurement_per_shard_test.dag — import confirmed_safe_width; add witness_confirmed_safe_width_is_positive_and_small + witness_per_shard_fits_within_confirmed_safe_shards

Test plan

• cargo test --workspace (no Rust changes; floor witness correctness proven below)
• witness_floor_confirmed_safe_width_bounds_actual_width: 8 GiB budget → width=2 ≤ confirmed_safe=2 ✓
• witness_floor_ceiling_clamps_stale_per_shard: stale 2.1 GiB → raw_width=3 > safe=2; capped=min(3,2)=2 ≤ safe=2 ✓
• witness_per_shard_fits_within_confirmed_safe_shards: 3226056704 × 2 = 6452113408 ≤ 8589934592 ✓
• All existing floor witnesses verified unaffected (zero budget fallback, tight budget backoff, fail-open envelope exceeds, budget-tree governance)

🤖 Generated with Claude Code

[gunbai-bot]

Re: witness_floor_ceiling_clamps_stale_per_shard — agreed the split is worth documenting. The two witnesses serve different roles:

• witness_floor_confirmed_safe_width_bounds_actual_width exercises the real function path end-to-end (the execution proxy for the ceiling property).
• witness_floor_ceiling_clamps_stale_per_shard is a mathematical witness that the stale scalar value (2.1 GiB, the value that historically caused exit-137) would produce raw_width=3, and the ceiling reduces it to ≤2. It intentionally avoids calling through the function so it can't be trivially satisfied by a bug in the formula that also affects the inline math.

In other words: one witness tests the implementation; the other tests the mathematical invariant the implementation is supposed to maintain. That said, the inline clamp duplicates some logic and your point that the second witness could pass while the function is broken stands. Will tighten in a follow-up once CI is green.

— sent from sharp-cat-505