Skip to content

CI floor never OOMs: derive spawn_width from measured per-shard memory demand (memory-aware width-fold) + expose a per-run ResourceEnvelope; fail-closed DEMAND->CEIL so the floor under-subscribes rather than OOMs. Consume the merged measure ops (#5470/#5478 concurrent_memory_peak_per_share / measure - #5524

Merged
briansrls merged 2 commits into
mainfrom
session/quiet-bee-906
Jun 22, 2026

Conversation

@briansrls

@briansrls briansrls commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

What

Expose the CI floor's per-run resource DEMAND as a ResourceEnvelope — the carrier cross-run placement consumes — derived from the memory-aware width fold so the floor's projected memory can never exceed the budget it was derived against. This is the item-3 slice of the work item; items 1–2 were already on main (see Grounding below).

DESIGN refs: §2 (reuse the one ResourceEnvelope carrier, no parallel mint), §3 (single authority — same width inputs as gunbc_ci_floor_spawn_width, no re-derived shard_count fork), §5 (fail-closed; derive, don't hand-set; proven by execution).

Changes

dsl/product/compute_fabric.dag (reuses the existing ResourceEnvelope):

src/v2/workflow/ci_floor_plan.dag: gunbc_ci_floor_run_envelope(memory_budget_bytes) reusing the same width inputs as gunbc_ci_floor_spawn_width (breadth, fleet threads, measured per-shard peak, conservative fallback).

Verify by execution (§5)

Discriminating floor-enrolled witnesses (test fn, auto-enrolled):

  • ci_floor_run_demand_envelope_model_holds (dsl) — non-binding fits; a planted ~20 GB shard shrinks the derived width 7→2 and still fits; RED control: a memory-blind (fail-open) width × the same peak (140 GB) fails the same fit predicate; 0-byte budget → conservative width.
  • ci_floor_run_envelope_witnesses (v2) — on the live floor inputs (real breadth + measured peak): a tight ~10 GB budget backs the derived width below shard_count and the envelope fits; RED control over the live shape; 0-byte → conservative.

All green by execution. Perturbation-checked: making gunbc_ci_floor_run_envelope fail-open (memory-blind width) flips ci_floor_run_envelope_witnesses → false (red), so the witness has real teeth.

Grounding (verification notes — already on main, not re-done here)

Cross-run bin-packing / workflow_dispatch run-doubling are operator/infra-owned and out of scope; this produces the demand model they ride on.

Brian Searls and others added 2 commits June 22, 2026 05:54
…ver OOMs)

The memory-aware width-fold (#5444) + the floor/ceil measure ops (#5470/#5478) are
already on main; this is the remaining item-3 slice: expose the floor's per-run resource
DEMAND as a ResourceEnvelope (the carrier cross-run placement consumes), DERIVED from the
same fold so it can never exceed the budget it was derived against.

product.compute_fabric (reuse ResourceEnvelope, no parallel mint):
- projected_concurrent_peak = width × per_shard_peak via measure_scale_fraction_ceil
  (#5478, fail-closed CEIL: demand never under-estimated)
- parallel_run_demand_envelope(width, peak) + _from_budget(...) deriving width via
  memory_aware_spawn_width — budget+measurement in, ⌊0.8·budget⌋-bounded envelope out
- demand_envelope_fits_budget: shared fit predicate (reuses memory_bytes_honor_demand)

src/v2/workflow/ci_floor_plan: gunbc_ci_floor_run_envelope(budget) reusing the SAME width
inputs as gunbc_ci_floor_spawn_width (no re-derived shard_count fork, §3).

Verify by execution (§5): discriminating floor-enrolled witnesses — a planted high-demand
shard / tight budget SHRINKS the derived width and the envelope still fits; the RED control
runs a memory-blind (fail-open) width through the SAME fit predicate and exceeds budget.
Perturbation-checked: a fail-open floor envelope flips the witness RED.

Grounding notes: items 1-2 (derive width, fail-closed DEMAND->CEIL) already merged
(#5444/#5470/#5478); the executor even-width division is gone (#5421, does-not-repro).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review June 22, 2026 05:59
@briansrls
briansrls merged commit baf4873 into main Jun 22, 2026
2 of 3 checks passed
@briansrls
briansrls deleted the session/quiet-bee-906 branch June 22, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant