Repository navigation
CI floor never OOMs: derive spawn_width from measured per-shard memory demand (memory-aware width-fold) + expose a per-run ResourceEnvelope; fail-closed DEMAND->CEIL so the floor under-subscribes rather than OOMs. Consume the merged measure ops (#5470/#5478 concurrent_memory_peak_per_share / measure - #5524
Merged
Merged
Conversation
…ver OOMs) The memory-aware width-fold (#5444) + the floor/ceil measure ops (#5470/#5478) are already on main; this is the remaining item-3 slice: expose the floor's per-run resource DEMAND as a ResourceEnvelope (the carrier cross-run placement consumes), DERIVED from the same fold so it can never exceed the budget it was derived against. product.compute_fabric (reuse ResourceEnvelope, no parallel mint): - projected_concurrent_peak = width × per_shard_peak via measure_scale_fraction_ceil (#5478, fail-closed CEIL: demand never under-estimated) - parallel_run_demand_envelope(width, peak) + _from_budget(...) deriving width via memory_aware_spawn_width — budget+measurement in, ⌊0.8·budget⌋-bounded envelope out - demand_envelope_fits_budget: shared fit predicate (reuses memory_bytes_honor_demand) src/v2/workflow/ci_floor_plan: gunbc_ci_floor_run_envelope(budget) reusing the SAME width inputs as gunbc_ci_floor_spawn_width (no re-derived shard_count fork, §3). Verify by execution (§5): discriminating floor-enrolled witnesses — a planted high-demand shard / tight budget SHRINKS the derived width and the envelope still fits; the RED control runs a memory-blind (fail-open) width through the SAME fit predicate and exceeds budget. Perturbation-checked: a fail-open floor envelope flips the witness RED. Grounding notes: items 1-2 (derive width, fail-closed DEMAND->CEIL) already merged (#5444/#5470/#5478); the executor even-width division is gone (#5421, does-not-repro). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Expose the CI floor's per-run resource DEMAND as a
ResourceEnvelope— the carrier cross-run placement consumes — derived from the memory-aware width fold so the floor's projected memory can never exceed the budget it was derived against. This is the item-3 slice of the work item; items 1–2 were already on main (see Grounding below).DESIGN refs: §2 (reuse the one
ResourceEnvelopecarrier, no parallel mint), §3 (single authority — same width inputs asgunbc_ci_floor_spawn_width, no re-derivedshard_countfork), §5 (fail-closed; derive, don't hand-set; proven by execution).Changes
dsl/product/compute_fabric.dag(reuses the existingResourceEnvelope):projected_concurrent_peak(width, per_shard_peak)=width × peakviameasure_scale_fraction_ceil(std.measure demand-CEIL follow-up (§1 spawn-width; completes the ceil≠floor money-pair after #5470 landed the FLOOR family). Builds DIRECTLY on merged #5470 (measure.dag now has measure_scale_fraction_floor + measure_fit_count_floor). Authority: warm-lark's #5467 (docs/plans/expressibility-frontier. #5478, the fail-closed CEIL — a run's projected memory is never under-estimated)parallel_run_demand_envelope(width, peak)+…_from_budget(…)deriving width viamemory_aware_spawn_width(§1-C: memory-aware CI floor spawn_width — derived from measured VmHWM peak + live budget (not pinned 4) #5444) — budget + measurement in, a ⌊0.8·budget⌋-bounded envelope outdemand_envelope_fits_budget— the shared fit predicate (reusesmemory_bytes_honor_demand); both the green path and the red control run through itsrc/v2/workflow/ci_floor_plan.dag:gunbc_ci_floor_run_envelope(memory_budget_bytes)reusing the same width inputs asgunbc_ci_floor_spawn_width(breadth, fleet threads, measured per-shard peak, conservative fallback).Verify by execution (§5)
Discriminating floor-enrolled witnesses (
test fn, auto-enrolled):ci_floor_run_demand_envelope_model_holds(dsl) — non-binding fits; a planted ~20 GB shard shrinks the derived width 7→2 and still fits; RED control: a memory-blind (fail-open) width × the same peak (140 GB) fails the same fit predicate; 0-byte budget → conservative width.ci_floor_run_envelope_witnesses(v2) — on the live floor inputs (real breadth + measured peak): a tight ~10 GB budget backs the derived width belowshard_countand the envelope fits; RED control over the live shape; 0-byte → conservative.All green by execution. Perturbation-checked: making
gunbc_ci_floor_run_envelopefail-open (memory-blind width) flipsci_floor_run_envelope_witnesses→false(red), so the witness has real teeth.Grounding (verification notes — already on main, not re-done here)
memory_aware_spawn_width, §1-C: memory-aware CI floor spawn_width — derived from measured VmHWM peak + live budget (not pinned 4) #5444) and fail-closed DEMAND→CEIL (concurrent_memory_peak_per_share→measure_scale_fraction_ceil, std.measure demand-CEIL follow-up (§1 spawn-width; completes the ceil≠floor money-pair after #5470 landed the FLOOR family). Builds DIRECTLY on merged #5470 (measure.dag now has measure_scale_fraction_floor + measure_fit_count_floor). Authority: warm-lark's #5467 (docs/plans/expressibility-frontier. #5478) are merged and consumed green-by-execution byclaim_executor(reads live cgroupmemory.max).run_walkgives each runnable the full width;DiscoveryBatchuses it as the corpus shard width,SingleClaimignores it.Cross-run bin-packing /
workflow_dispatchrun-doubling are operator/infra-owned and out of scope; this produces the demand model they ride on.