diff --git a/.githooks/pre-push b/.githooks/pre-push index 20afeec3b95..a7adee22b4a 100755 --- a/.githooks/pre-push +++ b/.githooks/pre-push @@ -75,6 +75,7 @@ fresh_artifact_path() { docs/plans/ci-selection-vs-scheduling.md*) return 0 ;; docs/plans/ci-process-end-to-end.md*) return 0 ;; docs/plans/ci-oom-reclassification.md*) return 0 ;; + docs/plans/bounded-input-cost-envelope-scheduling.md*) return 0 ;; docs/plans/ci-humming.md*) return 0 ;; docs/plans/edge-b-rust-dag-provenance-brief.md*) return 0 ;; docs/plans/resolver-type-name-collision-wall.md*) return 0 ;; diff --git a/ROADMAP.md b/ROADMAP.md index 37bd709da94..a2a77ae6c57 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -58,7 +58,8 @@ State (2026-07-01): required CI is back on the fleet after the Ubicloud detour ( - [x] **resolver pathology A — profile receipt only** — the 518s resolve: reproducible `scripts/profile-cold-resolve.sh`; top-N entry offenders; pair `budget_roster_completeness_test` vs `fold_list_generic_instantiation` (34.7× cold wall); hypothesis: `typecheck_module` in `reconcile_with_typed_cache`. — ✓ signed off: operator - [ ] **resolver pathology B — fix one confirmed pathology** — RED control on the pathological pair from the profile receipt, the fix, and a before/after resolve-count or wall-clock receipt. - [ ] **G5 rust-gate selection** — rust fmt/clippy/run-all is all-or-nothing on `.rs` PRs (the ~530s hot spot); no affected-set, while the `.dag` floor already has one. Matters until v1 is gone. [plan](docs/plans/ci-selection-vs-scheduling.md) -- [ ] **floor runs the right things** — SELECTION (what changed) vs SCHEDULING (by cost); per-PR = run-all sound baseline (#5427) shrunk to the affected set; cost never drives selection. [plan](docs/plans/ci-selection-vs-scheduling.md) +- [ ] **floor runs the right things** — SELECTION (what changed) vs SCHEDULING (by cost); per-PR = run-all sound baseline (#5427) shrunk to the affected set; cost never drives selection. Scheduling authority: [bounded-input / cost-envelope design](docs/plans/bounded-input-cost-envelope-scheduling.md). [plan](docs/plans/ci-selection-vs-scheduling.md) +- [x] **bounded-input / cost-envelope scheduling — operator-signed design** *(2026-07-02 capture)* — unified design record: `InputEnvelope` (data ceiling) → `CostEnvelope` (symbolic Predicted cost at envelope) → scheduling (width/placement); Predicted authority / Measured falsifier; cost → scheduling never selection. P0 InputEnvelope landed; P1–P5 dispatch separately (no scheduler types in this capture). [plan](docs/plans/bounded-input-cost-envelope-scheduling.md) — ✓ signed off: operator - [ ] **resolve-cache default-on** — purity proven (616/616) but opt-in: whole-file JSON IO buffers ~11× the packed graph and OOMed the concurrent floor (#5789 reverted); gated on a streaming IO realization. The warm ~18% lever, downstream of incrementality. - [ ] **G4 dispatch dup** — `workflow_dispatch`+PR fire two same-SHA runs; the concurrency fallback won't collapse them → wasted runners and OOM risk (live instance: #6110's draft-attach + ready-flip fired two runs, the cancelled one reporting as a failing check). Superseded runs also linger `in_progress` for 12–37min — audit cancel-in-progress coverage; leaked runner minutes. [decision record](docs/plans/ci-merge-freshness.md) @@ -86,6 +87,11 @@ Receipts + adjacent levers: [fractal Gantt](docs/plans/ci-floor-fractal-gantt.md - [ ] **slice 2 — fleet_converge emit** — `fleet_converge_emit` → committed `.github/fleet-converge.sh`; the emit half of the re-land [plan](docs/plans/shell-emission-model.md) - [ ] **`apply()` Phases B–F** — B `host_exec`→`apply()` (gated on srv3 OsInstalled) · C Redfish live (partially via #6097) · D converge lane = EmitArtifactThenThinRun handler (first ctrl LOC deleted) · E pull-mode self-converge (autoinstall plants the on-host agent; the push star retires) · F decom. Phase A landed (#5756). [plan](docs/plans/host-effect-orchestration.md) - [ ] **srv3 → OsInstalled** — finish the greenfield install over the solve-driven NBD-proxy actuator (#6097, live BMC); proves the install band end-to-end. [plan](docs/plans/srv3-webui-kvm-virtual-media.md) + - [ ] **resource-namespace-upsert-a — vocabulary only** *(#6134 srv3 bringup follow-up, operator 2026-07-02)* — FileNamespace/ToolNamespace/ProcessNamespace/SecretFileNamespace; shared verdicts Converged|Absent|Stale|Conflict|Inaccessible|UnknownRefused; plans Noop|Create|UpsertAccess|DrainThenStart|Refuse. RED: foreign process on 10809=Conflict not kill; outside-namespace chmod=Refuse; stale seeded ISO ≠ AlreadyPresent. Non-goal: live srv3 mutation. + - [ ] **install-media-generic-layer — extract Ubuntu flavor from process** — split `extdeps/os/install_media.dag` generic shapes (fetch/remaster/serve verdicts) from `ubuntu_*` flavor rows; move `.gunbc-content-hash` sidecar + remaster policy to `gunbc/install_media_*`; mandatory before second OS/distro. Post-#6134. + - [ ] **nbd-serve-held-session-lease** — NbdProxyServeSession intent with port/pid/cmdline/ISO/token fingerprint; classify RunningExpected|RunningStaleExpected|PortOccupiedForeign|TokenMissing|IsoMismatch|Absent; plan Noop|DrainThenStart|Refuse. RED: foreign 10809 no kill; stale matching session drains only. Live consumer: srv3 actuator. + - [ ] **srv3-boot-action-diagnostic — Redfish vs ipmitool fallback state machine** — separate transports for BmcBootAction; boot refused unless serve RunningExpected; NoRebootObserved → optional IpmiResetFallback with separate approval; OutOfBandHostActionReceipt until modeled. + - [ ] **os-install generic naming cleanup** — rename generic behavior tests/modules away from `srv3_*` where testing credential resolution, remaster classification, workflow escalation; keep srv3 only in instance fixtures (`srv3_pre_install_lease_table`, `srv3_seeded_install_media_artifact`). Post-#6134. **Fabric dispatch — RunShape → Allocation → Receipt (GitHub is the queue, not the scheduler):** @@ -101,7 +107,7 @@ Receipts + adjacent levers: [fractal Gantt](docs/plans/ci-floor-fractal-gantt.md - [ ] **the return happened ungated** — #6111 put required CI back on srv1/srv2 by operator call, accepting interim OOM/sccache exposure; the items below are the gates the return *should* have had, now owed as hardening. Ubicloud remains the burst/fallback offer (and the slot-shape template the admission model generalizes). - [ ] **SessionSliceEnforcement + verified-effective caps** — agent-session cgroups have no citable verified-effective cap (the residual over-commit vector on srv1/srv2); the humming apply-rule stands: runner caps must not tighten while sessions are uncapped and oomd is absent. - [ ] **OOM classification consumer** — `ClusteredOutOfMemoryKill` is modeled but absent from the active infra-retry signature roster; build the consumer so an OOM is first-class, never EAGAIN-shaped. [plan](docs/plans/ci-oom-reclassification.md) -- [ ] **resource-aware scheduler** — width derived from per-`Runnable` measured cost, self-calibrating: (A) per-shard peak-RSS plumbing → (B) `CostEstimate.space` on `Runnable` → (C) scheduler reads cost, static side-channel deleted → (D) calibration loop retires the data rows (instrument landed #5431). Construction invariant: more memory → more width → more throughput, no `.dag` edits. [plan](docs/plans/resource-aware-scheduler.md) +- [ ] **resource-aware scheduler (implementation)** — implements the signed [bounded-input / cost-envelope scheduling](docs/plans/bounded-input-cost-envelope-scheduling.md) design: (A) per-shard peak-RSS plumbing → (B) `Runnable.cost` with Predicted `CostEnvelope` → (C) scheduler reads cost, static side-channel deleted → (D) Measured falsifies Predicted, calibration loop retires data rows (#5431 instrument). Construction invariant: more memory → more width → more throughput, no `.dag` edits. [tracker](docs/plans/resource-aware-scheduler.md) - [ ] **resource budget tree** — admission (construction) · conserve-lens (honest residue) · runtime reconcile (fail-closed handler), three verdicts never conflated; subsumes spawn-width/placement/compile-jobs as consumer leaves. Protective only when paired with an enforcement actuator (cgroup `memory.max`, operator-fenced). [carrier](dag/product/budget_tree.dag) [grounding](docs/plans/budget-tree.md) - [ ] **one Placement authority** — jobs (GitHub) · threads (`spawn_width`) · sessions (ctrl capacity) are 3 forks of "put work on a host"; `Placement`/`Materialization` are modeled but inert — CI consuming them is the lane's real deliverable. diff --git a/dag/gunbc/plan_registry_batch_f.dag b/dag/gunbc/plan_registry_batch_f.dag index 898b9f8cd96..fd28e0d0846 100644 --- a/dag/gunbc/plan_registry_batch_f.dag +++ b/dag/gunbc/plan_registry_batch_f.dag @@ -9,6 +9,7 @@ import gunbc.plans.input_envelope_roadmap { input_envelope_roadmap_plan } import gunbc.plans.ci_selection_vs_scheduling { ci_selection_vs_scheduling_plan } import gunbc.plans.ci_process_end_to_end { ci_process_end_to_end_plan } import gunbc.plans.ci_oom_reclassification { ci_oom_reclassification_plan } +import gunbc.plans.bounded_input_cost_envelope_scheduling { bounded_input_cost_envelope_scheduling_plan } data batch_f_plans: List = [ self_applying_lenses_plan, @@ -18,5 +19,6 @@ data batch_f_plans: List = [ input_envelope_roadmap_plan, ci_selection_vs_scheduling_plan, ci_process_end_to_end_plan, - ci_oom_reclassification_plan + ci_oom_reclassification_plan, + bounded_input_cost_envelope_scheduling_plan ] diff --git a/dag/gunbc/plans/bounded_input_cost_envelope_scheduling.dag b/dag/gunbc/plans/bounded_input_cost_envelope_scheduling.dag new file mode 100644 index 00000000000..a4314f57a0b --- /dev/null +++ b/dag/gunbc/plans/bounded_input_cost_envelope_scheduling.dag @@ -0,0 +1,115 @@ +module gunbc.plans.bounded_input_cost_envelope_scheduling + +import gunbc.plan { Plan, DissolutionTrigger, HasTrigger } +import std.markdown { MarkdownBlock, MarkdownTableRow, TableBlock, BlockquoteBlock, AlignNone } +import gunbc.plans.md_helpers { h2, h3, p, li, ul, ol, cell, row, quote } + +fn bounded_input_cost_envelope_scheduling_body() -> List { + [ + BlockquoteBlock { blocks: [ + p(text: "**Status: operator-signed design record (2026-07-02).** Model-before-implement: this doc is the single authority for how bounded input, cost envelopes, and scheduling compose. It declares carrier shapes, the pipeline order, and acceptance tiers — **not** scheduler-type implementations (those dispatch as separate work items). DESIGN refs: §1 (time = cost; safety — cost never drives selection), §2 (one concept every scale — width is a budget fit count), §3 (single authority — do not re-mint InputEnvelope / symbolic_cost / budget tree), §5 (fail-closed — unbounded input and zero predicted cost are the same smell; Predicted authority / Measured falsifier), §6 (this doc dissolves when the carriers state the policy)."), + ] }, + h2(text: "1. The displaced cost (why one design)"), + p(text: "Scheduling work was split across four plan docs — [`input-envelope-roadmap.md`](input-envelope-roadmap.md) (data ceiling), [`resource-aware-scheduler.md`](resource-aware-scheduler.md) (width from cost), [`ci-selection-vs-scheduling.md`](ci-selection-vs-scheduling.md) (selection vs scheduling axis), [`compute-envelope-model.md`](compute-envelope-model.md) (host resource envelope). Each was correct locally; together they lacked one signed answer to: **given a must-run set, how does declared input bound predicted cost, and how does predicted cost bound width — without fusing selection?** This doc is that answer. Companion docs remain implementation trackers; **this doc is the authority they must not contradict**."), + h2(text: "2. Operator-signed invariants (2026-07-02)"), + p(text: "**Phase namespace:** §7 uses **P0–P5** (this design). Companion plans keep their own labels — always qualified (e.g. input-envelope-roadmap **P2** = this design's **P5**)."), + ol(items: [ + li(text: "**Bounded input before cost before schedule.** A floor run whose corpus lacks a declared `InputEnvelope` is fail-open (§5): the scheduler has no principled admission or cost basis. `InputEnvelope = BoundedInput | EnvelopeUnknown` (`gunbc.ci_input_envelope`); `EnvelopeUnknown` is honest bottom — admission refuses (`RefusedUndeclared`), never silent green."), + li(text: "**Predicted cost is authority; Measured is falsifier.** `CostBasis = Predicted` — symbolic cost evaluated at the declared input envelope — is what width, shard balance, and budget consumers read. `CostBasis = Measured` compares against Predicted; a mismatch is a bug in the cost model, not a silent override. Measuring where you can derive is the §5 fail-open smell."), + li(text: "**Cost informs scheduling, never selection** (operator, 2026-06-21 — see [`ci-selection-vs-scheduling.md`](ci-selection-vs-scheduling.md)). Expensive-but-affected work runs per-PR; the answer to \"the affected set is costly\" is parallelize + cache, never \"run it later\"."), + li(text: "**Schedule topology stays central; width/cost realization is peripheral.** `Schedule = List>` is hardware-free partial order (`std.realization_schedule`). Width, shard balance, and cache-layer choice are peripheral realizations folded against deployment facts — measured decisions must not break `schedule_eq` across hosts."), + li(text: "**More memory → more width → more throughput, by construction.** `spawn_width = floor(available_memory / max_runnable_predicted_space)` — derived, never hand-authored in plan data rows. Host memory increase reflected in budget → width rises automatically → no `.dag` ceiling edits."), + ]), + h2(text: "3. The two envelopes"), + h3(text: "3a InputEnvelope — the data-side ceiling"), + p(text: "**Authority:** `gunbc.ci_input_envelope` (`InputSizeAxis` · `InputBound` · `InputEnvelope` · `input_admitted` · `gunbc_ci_corpus_envelope`). **P0 landed (PARTIAL ✓):** types + admission witness (`dag/test/claim/input_envelope_admission_test.dag`). Scaffold ceilings in `gunbc_ci_corpus_envelope` dissolve when **P5** (input-envelope-roadmap **P2**) derives them from corpus discovery (witness roster count, source-node count, corpus-node count) — operator-set operating-point ceilings in P0 are explicitly **not** measurements."), + h3(text: "3b CostEnvelope — symbolic cost at the envelope"), + p(text: "**Definition:** `CostEnvelope = symbolic_cost(graph structure)` evaluated with size variables bound to the `BoundedInput` ceilings (reuse `v2.lens.cost` / `symbolic_cost_of_node` — do **not** mint a parallel cost fold). The envelope binds axes (`WitnessCount`, `SourceNodeCount`, `CorpusNodeCount`) to the symbolic cost's size parameters so predicted space/time is a **function of declared input**, not a static data row."), + p(text: "**On `Runnable`:** `cost: CostAccount` with `basis = Predicted`; `space` (and rolled-up `time` where modeled) come from the CostEnvelope evaluation for that runnable's graph slice. `cost_account_predicted_zero()` is the tell that Predicted authority is not yet wired — it must have no consumers on the floor path when implementation closes."), + h2(text: "4. The pipeline — three stages, never fused"), + TableBlock { + header: row(cells: [ + cell(text: "Stage"), + cell(text: "Question"), + cell(text: "Authority"), + cell(text: "Fail-closed bottom"), + ]), + alignments: [AlignNone, AlignNone, AlignNone, AlignNone], + rows: [ + row(cells: [ + cell(text: "**SELECTION** (§4)"), + cell(text: "does this result depend on what changed?"), + cell(text: "`v2.lens.affected_set` — transitive node closure"), + cell(text: "provenance gap → run-all baseline (#5427)"), + ]), + row(cells: [ + cell(text: "**ADMISSION** (data)"), + cell(text: "is actual input within declared ceiling?"), + cell(text: "`input_admitted(envelope, actual)`"), + cell(text: "`EnvelopeUnknown` → `RefusedUndeclared`"), + ]), + row(cells: [ + cell(text: "**SCHEDULING** (§1/§2)"), + cell(text: "given admitted work, how wide / where / when?"), + cell(text: "CostEnvelope (Predicted) + `gunbc.fleet_container.ResourceEnvelope` / budget tree"), + cell(text: "over-commit → refuse or shrink width, never silent OOM"), + ]), + ], + }, + p(text: "**Load-bearing seam:** selection produces membership; admission gates input; scheduling consumes **Predicted** cost against **available** budget. None of the three may read cost to skip selection, and admission must run before width derivation uses cost (you cannot predict cost honestly on unbounded input)."), + h2(text: "5. Width derivation (construction invariant)"), + p(text: "**Formula:** `spawn_width = min(cpu_bound, floor(memory_budget / max_runnable_predicted_space))` — the RUN-phase lever (`compute-envelope-model.md` §3: discovery shards use the prebuilt binary; width-up is pids-safe on its own). **Budget source:** `extdeps.accounting.budget` / `product.budget_tree` / `std.realization_width` — one capped-resource→claims concept (§3 convergence with complexity `EffortBudget` and `memory_aware_spawn_width`). **Side-channel deletion:** `gunbc_ci_floor_spawn_width_for_budget`, static `gunbc_ci_floor_measured_peak` rows, and a second `eval_spawn_width` pass in `claim_executor` retire when `WidthBoundedRealization` derives width from the plan."), + h2(text: "6. Single authorities — do not re-mint (§3)"), + ul(items: [ + li(text: "**Input envelope** — `gunbc.ci_input_envelope` (not a scheduler-local ceiling type)"), + li(text: "**Symbolic cost** — `v2.lens.cost` (`symbolic_cost_of_node`, `SymbolicCost` lattice)"), + li(text: "**Schedule topology** — `std.realization_schedule` (`Schedule`, `Runnable`, `RealizationPlan`)"), + li(text: "**Width fold** — `std.realization_width` + peripheral `WidthBoundedRealization`"), + li(text: "**Host / run budget** — `product.budget_tree` + `gunbc.fleet_container.ResourceEnvelope` (BUILD vs RUN phases stay distinct — two invariants, not one product; `#5904` moved this type off dissolved `product.compute_fabric`)"), + li(text: "**Measured observation** — `gunbc.fleet_intent.PerformanceReceipt` → roll-up to `CostAccount` with `basis = Measured` (falsifier only; `#5904` live home — not dissolved `compute_fabric.PerformanceReceipt`)"), + ]), + h2(text: "7. Implementation phases (dispatch separately — not this capture)"), + p(text: "Phases are ordered; each closes at a named consumer tier (mirrors §2 fleet acceptance: algebra alone is not done). **This capture lands phases 0–1 of the design record only.**"), + ol(items: [ + li(text: "**P0 — InputEnvelope admission (PARTIAL ✓).** Types + `input_admitted` + CI corpus instance + witness. **Accept:** `input_envelope_admission_test.dag` green; RED on revert of `RefusedUndeclared` / over-envelope paths."), + li(text: "**P1 — CostEnvelope on `Runnable` (Predicted).** Populate `Runnable.cost.space` from symbolic evaluation at `gunbc_ci_corpus_envelope` bindings; delete `cost_account_predicted_zero()` on the floor rollup path. **Accept (T2):** floor plan reads cost from runnables; remove cost side-channel → witness fails."), + li(text: "**P2 — Scheduler consumes Predicted cost.** `WidthBoundedRealization` derives `spawn_width` from plan; delete `gunbc_ci_floor_spawn_width_for_budget` and static peak rows. **Accept (T2):** host memory increase → higher width without data-row edits (discriminating receipt)."), + li(text: "**P3 — Measured falsifier + calibration loop.** `gunbc.fleet_intent.PerformanceReceipt` / per-shard RSS (`resource-aware-scheduler.md` Node A) feeds `CostBasis = Measured`; mismatch vs Predicted → typed diagnostic. Calibration updates symbolic formula inputs, not the Predicted row directly. **Accept (T3):** planted underestimate in Predicted → falsifier RED."), + li(text: "**P4 — Runtime admission gate.** Wire `input_admitted` into the floor path (`ci_floor_plan.dag` Runnable roster + `gunbc.fleet_container.demand_envelope_fits_budget` / `witness_run_demand_envelope_fits_budget`) so runtime refuses before schedule build — **not** dissolved `WorkDemand` (`#5904`). **Accept (T2):** corpus exceeding `gunbc_ci_corpus_envelope` → schedule refuses, not OOM."), + li(text: "**P5 — Ceiling derivation (input-envelope-roadmap P2).** Replace scaffold ceilings in `gunbc_ci_corpus_envelope` with values derived from discovery roster + measured node counts; dissolve `Scaffold` disposition on the envelope data."), + ]), + h2(text: "8. Relationship to companion docs"), + ul(items: [ + li(text: "[`input-envelope-roadmap.md`](input-envelope-roadmap.md) — input-envelope-roadmap P1/P2 detail; maps to this design's P0/P5; defers to §3a here for authority"), + li(text: "[`resource-aware-scheduler.md`](resource-aware-scheduler.md) — Nodes A–D implementation tracker for this design's P2–P3; defers to §5–§7 here"), + li(text: "[`ci-selection-vs-scheduling.md`](ci-selection-vs-scheduling.md) — selection axis framing; §4 pipeline table is the composed view"), + li(text: "[`compute-envelope-model.md`](compute-envelope-model.md) — host envelope framing (live type: `gunbc.fleet_container.ResourceEnvelope`); BUILD-phase fan-out; RUN-phase width is §5 here"), + li(text: "[`realization-measurement-loop.md`](realization-measurement-loop.md) — broader cost→schedule loop coordination; this design is the scheduling-arm authority named by that doc's companion pointer"), + li(text: "[`budget-tree.md`](budget-tree.md) — admission/construction for memory budget; width fold is a consumer leaf"), + ]), + h2(text: "9. Explicit non-goals (this capture)"), + ul(items: [ + li(text: "No new scheduler types in `src/v2/workflow/scheduler.dag` or `std.realization_schedule` beyond what P1–P2 dispatch items specify"), + li(text: "No `claim_executor` / Rust seed edits for measurement plumbing (Node A remains its own PR)"), + li(text: "No fusion of selection and scheduling (no cost-based nightly routing)"), + li(text: "No replacement of GitHub/job placement (`RunShape → Allocation → Receipt` fabric is orthogonal host-level placement)"), + ]), + h2(text: "10. Red controls (design-level — implementation must wire each)"), + ul(items: [ + li(text: "`EnvelopeUnknown` demand → `RefusedUndeclared` (already witnessed in P0)"), + li(text: "Actual axis over declared ceiling → `RefusedOverEnvelope`"), + li(text: "Predicted vs Measured space mismatch → typed falsifier (not silent width change)"), + li(text: "Drop Predicted cost from runnable → width fold fails closed / refuses over-wide spawn"), + li(text: "Affected witness skipped on touched dep → selection soundness RED (independent of this design, but scheduling must not bypass it)"), + ]), + ] +} + +data bounded_input_cost_envelope_scheduling_plan: Plan = Plan { + slug: "bounded-input-cost-envelope-scheduling", + title: "Bounded-input / cost-envelope scheduling — operator-signed design", + body: bounded_input_cost_envelope_scheduling_body(), + dissolution: HasTrigger { + text: "Delete this doc when `gunbc_ci_corpus_envelope` carries derived `BoundedInput` ceilings (P5), every floor `Runnable` carries Predicted `CostEnvelope`-derived cost, `spawn_width` is derived from `WidthBoundedRealization` (no side-channel), `gunbc.fleet_intent.PerformanceReceipt` falsifies Predicted with a discriminating RED, runtime admission consumes `input_admitted` on the live floor path, and the selection/scheduling axis split is witnessed — at which point the carriers state the policy and companion trackers (`resource-aware-scheduler.md`, `input-envelope-roadmap.md`) may dissolve." + } +} diff --git a/dag/gunbc/roadmap_authority.dag b/dag/gunbc/roadmap_authority.dag index f4fc494177d..aa2f127a168 100644 --- a/dag/gunbc/roadmap_authority.dag +++ b/dag/gunbc/roadmap_authority.dag @@ -168,7 +168,8 @@ fn section_2() -> RoadmapSection { sign(rn: owned(rn: authored_wi(id: "1-resolver-pathology-a", done: true, content: "**resolver pathology A — profile receipt only** — the 518s resolve: reproducible `scripts/profile-cold-resolve.sh`; top-N entry offenders; pair `budget_roster_completeness_test` vs `fold_list_generic_instantiation` (34.7× cold wall); hypothesis: `typecheck_module` in `reconcile_with_typed_cache`.", intricacy: IntricacyHigh, volume: VolumeSmall, repo: "gunbc"), owner: "dispatched 2026-07-01 · adhoc-51b0b8e7-0be"), s: signed(by: "operator", works: true, scope_equivalent: true, as_expected: true)), authored_wi(id: "1-resolver-pathology-b", done: false, content: "**resolver pathology B — fix one confirmed pathology** — RED control on the pathological pair from the profile receipt, the fix, and a before/after resolve-count or wall-clock receipt.", intricacy: IntricacyHigh, volume: VolumeMedium, repo: "gunbc"), authored_doc(id: "1-g5-rust-selection", done: false, content: "**G5 rust-gate selection** — rust fmt/clippy/run-all is all-or-nothing on `.rs` PRs (the ~530s hot spot); no affected-set, while the `.dag` floor already has one. Matters until v1 is gone.", path: "docs/plans/ci-selection-vs-scheduling.md"), - authored_doc(id: "1-floor-right-things", done: false, content: "**floor runs the right things** — SELECTION (what changed) vs SCHEDULING (by cost); per-PR = run-all sound baseline (#5427) shrunk to the affected set; cost never drives selection.", path: "docs/plans/ci-selection-vs-scheduling.md"), + authored_doc(id: "1-floor-right-things", done: false, content: "**floor runs the right things** — SELECTION (what changed) vs SCHEDULING (by cost); per-PR = run-all sound baseline (#5427) shrunk to the affected set; cost never drives selection. Scheduling authority: [bounded-input / cost-envelope design](docs/plans/bounded-input-cost-envelope-scheduling.md).", path: "docs/plans/ci-selection-vs-scheduling.md"), + sign(rn: authored_doc(id: "1-bics-design", done: true, content: "**bounded-input / cost-envelope scheduling — operator-signed design** *(2026-07-02 capture)* — unified design record: `InputEnvelope` (data ceiling) → `CostEnvelope` (symbolic Predicted cost at envelope) → scheduling (width/placement); Predicted authority / Measured falsifier; cost → scheduling never selection. P0 InputEnvelope landed; P1–P5 dispatch separately (no scheduler types in this capture).", path: "docs/plans/bounded-input-cost-envelope-scheduling.md"), s: signed(by: "operator", works: true, scope_equivalent: true, as_expected: true)), authored(id: "2-p3", done: false, content: "**resolve-cache default-on** — purity proven (616/616) but opt-in: whole-file JSON IO buffers ~11× the packed graph and OOMed the concurrent floor (#5789 reverted); gated on a streaming IO realization. The warm ~18% lever, downstream of incrementality."), authored_pp(id: "1-g4-dispatch", done: false, content: "**G4 dispatch dup** — `workflow_dispatch`+PR fire two same-SHA runs; the concurrency fallback won't collapse them → wasted runners and OOM risk (live instance: #6110's draft-attach + ready-flip fired two runs, the cancelled one reporting as a failing check). Superseded runs also linger `in_progress` for 12–37min — audit cancel-in-progress coverage; leaked runner minutes.", carriers: [ptr(label: "decision record", path: "docs/plans/ci-merge-freshness.md")]), ], @@ -177,6 +178,7 @@ fn section_2() -> RoadmapSection { RoadmapEdge { child: nid(s: "2-provenance-ingest"), parent: nid(s: "1-affected-set-defork") }, RoadmapEdge { child: nid(s: "1-resolver-pathology-b"), parent: nid(s: "1-resolver-pathology-a") }, RoadmapEdge { child: nid(s: "2-compile-clean-shard-b"), parent: nid(s: "2-compile-clean-shard-a") }, + RoadmapEdge { child: nid(s: "1-sched-resource-aware"), parent: nid(s: "1-bics-design") }, ], ), section_prose(content: "Receipts + adjacent levers: [fractal Gantt](docs/plans/ci-floor-fractal-gantt.md) · [memory chronicle](docs/plans/interpreter-memory-chronicle.md) · [resolver pathology profile](docs/plans/resolver-pathology-profile-receipt.md) · [representation minimization](docs/plans/representation-minimization.md) · [shared-compute memoization](docs/plans/floor-shared-compute-memoization.md) *(M1 ✓ #6008; M2 gated on determinism #5941)* · [OOM reclassification](docs/plans/ci-oom-reclassification.md)."), @@ -260,7 +262,7 @@ fn section_2() -> RoadmapSection { authored(id: "2-fleet-hardening", done: false, content: "**the return happened ungated** — #6111 put required CI back on srv1/srv2 by operator call, accepting interim OOM/sccache exposure; the items below are the gates the return *should* have had, now owed as hardening. Ubicloud remains the burst/fallback offer (and the slot-shape template the admission model generalizes)."), authored(id: "2-session-slice", done: false, content: "**SessionSliceEnforcement + verified-effective caps** — agent-session cgroups have no citable verified-effective cap (the residual over-commit vector on srv1/srv2); the humming apply-rule stands: runner caps must not tighten while sessions are uncapped and oomd is absent."), authored_doc(id: "2-oom-consumer", done: false, content: "**OOM classification consumer** — `ClusteredOutOfMemoryKill` is modeled but absent from the active infra-retry signature roster; build the consumer so an OOM is first-class, never EAGAIN-shaped.", path: "docs/plans/ci-oom-reclassification.md"), - authored(id: "1-sched-resource-aware", done: false, content: "**resource-aware scheduler** — width derived from per-`Runnable` measured cost, self-calibrating: (A) per-shard peak-RSS plumbing → (B) `CostEstimate.space` on `Runnable` → (C) scheduler reads cost, static side-channel deleted → (D) calibration loop retires the data rows (instrument landed #5431). Construction invariant: more memory → more width → more throughput, no `.dag` edits. [plan](docs/plans/resource-aware-scheduler.md)"), + authored(id: "1-sched-resource-aware", done: false, content: "**resource-aware scheduler (implementation)** — implements the signed [bounded-input / cost-envelope scheduling](docs/plans/bounded-input-cost-envelope-scheduling.md) design: (A) per-shard peak-RSS plumbing → (B) `Runnable.cost` with Predicted `CostEnvelope` → (C) scheduler reads cost, static side-channel deleted → (D) Measured falsifies Predicted, calibration loop retires data rows (#5431 instrument). Construction invariant: more memory → more width → more throughput, no `.dag` edits. [tracker](docs/plans/resource-aware-scheduler.md)"), authored_pp(id: "1-budget-tree", done: false, content: "**resource budget tree** — admission (construction) · conserve-lens (honest residue) · runtime reconcile (fail-closed handler), three verdicts never conflated; subsumes spawn-width/placement/compile-jobs as consumer leaves. Protective only when paired with an enforcement actuator (cgroup `memory.max`, operator-fenced).", carriers: [ptr(label: "carrier", path: "dag/product/budget_tree.dag"), ptr(label: "grounding", path: "docs/plans/budget-tree.md")]), authored(id: "1-placement-authority", done: false, content: "**one Placement authority** — jobs (GitHub) · threads (`spawn_width`) · sessions (ctrl capacity) are 3 forks of \"put work on a host\"; `Placement`/`Materialization` are modeled but inert — CI consuming them is the lane's real deliverable."), ], diff --git a/docs/plans/bounded-input-cost-envelope-scheduling.md b/docs/plans/bounded-input-cost-envelope-scheduling.md new file mode 100644 index 00000000000..f3430477872 --- /dev/null +++ b/docs/plans/bounded-input-cost-envelope-scheduling.md @@ -0,0 +1,91 @@ +# Bounded-input / cost-envelope scheduling — operator-signed design + +> **Status: operator-signed design record (2026-07-02).** Model-before-implement: this doc is the single authority for how bounded input, cost envelopes, and scheduling compose. It declares carrier shapes, the pipeline order, and acceptance tiers — **not** scheduler-type implementations (those dispatch as separate work items). DESIGN refs: §1 (time = cost; safety — cost never drives selection), §2 (one concept every scale — width is a budget fit count), §3 (single authority — do not re-mint InputEnvelope / symbolic_cost / budget tree), §5 (fail-closed — unbounded input and zero predicted cost are the same smell; Predicted authority / Measured falsifier), §6 (this doc dissolves when the carriers state the policy). + +## 1. The displaced cost (why one design) + +Scheduling work was split across four plan docs — [`input-envelope-roadmap.md`](input-envelope-roadmap.md) (data ceiling), [`resource-aware-scheduler.md`](resource-aware-scheduler.md) (width from cost), [`ci-selection-vs-scheduling.md`](ci-selection-vs-scheduling.md) (selection vs scheduling axis), [`compute-envelope-model.md`](compute-envelope-model.md) (host resource envelope). Each was correct locally; together they lacked one signed answer to: **given a must-run set, how does declared input bound predicted cost, and how does predicted cost bound width — without fusing selection?** This doc is that answer. Companion docs remain implementation trackers; **this doc is the authority they must not contradict**. + +## 2. Operator-signed invariants (2026-07-02) + +**Phase namespace:** §7 uses **P0–P5** (this design). Companion plans keep their own labels — always qualified (e.g. input-envelope-roadmap **P2** = this design's **P5**). + +1. **Bounded input before cost before schedule.** A floor run whose corpus lacks a declared `InputEnvelope` is fail-open (§5): the scheduler has no principled admission or cost basis. `InputEnvelope = BoundedInput | EnvelopeUnknown` (`gunbc.ci_input_envelope`); `EnvelopeUnknown` is honest bottom — admission refuses (`RefusedUndeclared`), never silent green. +2. **Predicted cost is authority; Measured is falsifier.** `CostBasis = Predicted` — symbolic cost evaluated at the declared input envelope — is what width, shard balance, and budget consumers read. `CostBasis = Measured` compares against Predicted; a mismatch is a bug in the cost model, not a silent override. Measuring where you can derive is the §5 fail-open smell. +3. **Cost informs scheduling, never selection** (operator, 2026-06-21 — see [`ci-selection-vs-scheduling.md`](ci-selection-vs-scheduling.md)). Expensive-but-affected work runs per-PR; the answer to "the affected set is costly" is parallelize + cache, never "run it later". +4. **Schedule topology stays central; width/cost realization is peripheral.** `Schedule = List>` is hardware-free partial order (`std.realization_schedule`). Width, shard balance, and cache-layer choice are peripheral realizations folded against deployment facts — measured decisions must not break `schedule_eq` across hosts. +5. **More memory → more width → more throughput, by construction.** `spawn_width = floor(available_memory / max_runnable_predicted_space)` — derived, never hand-authored in plan data rows. Host memory increase reflected in budget → width rises automatically → no `.dag` ceiling edits. + +## 3. The two envelopes + +### 3a InputEnvelope — the data-side ceiling + +**Authority:** `gunbc.ci_input_envelope` (`InputSizeAxis` · `InputBound` · `InputEnvelope` · `input_admitted` · `gunbc_ci_corpus_envelope`). **P0 landed (PARTIAL ✓):** types + admission witness (`dag/test/claim/input_envelope_admission_test.dag`). Scaffold ceilings in `gunbc_ci_corpus_envelope` dissolve when **P5** (input-envelope-roadmap **P2**) derives them from corpus discovery (witness roster count, source-node count, corpus-node count) — operator-set operating-point ceilings in P0 are explicitly **not** measurements. + +### 3b CostEnvelope — symbolic cost at the envelope + +**Definition:** `CostEnvelope = symbolic_cost(graph structure)` evaluated with size variables bound to the `BoundedInput` ceilings (reuse `v2.lens.cost` / `symbolic_cost_of_node` — do **not** mint a parallel cost fold). The envelope binds axes (`WitnessCount`, `SourceNodeCount`, `CorpusNodeCount`) to the symbolic cost's size parameters so predicted space/time is a **function of declared input**, not a static data row. + +**On `Runnable`:** `cost: CostAccount` with `basis = Predicted`; `space` (and rolled-up `time` where modeled) come from the CostEnvelope evaluation for that runnable's graph slice. `cost_account_predicted_zero()` is the tell that Predicted authority is not yet wired — it must have no consumers on the floor path when implementation closes. + +## 4. The pipeline — three stages, never fused + +| Stage | Question | Authority | Fail-closed bottom | +| --- | --- | --- | --- | +| **SELECTION** (§4) | does this result depend on what changed? | `v2.lens.affected_set` — transitive node closure | provenance gap → run-all baseline (#5427) | +| **ADMISSION** (data) | is actual input within declared ceiling? | `input_admitted(envelope, actual)` | `EnvelopeUnknown` → `RefusedUndeclared` | +| **SCHEDULING** (§1/§2) | given admitted work, how wide / where / when? | CostEnvelope (Predicted) + `gunbc.fleet_container.ResourceEnvelope` / budget tree | over-commit → refuse or shrink width, never silent OOM | + +**Load-bearing seam:** selection produces membership; admission gates input; scheduling consumes **Predicted** cost against **available** budget. None of the three may read cost to skip selection, and admission must run before width derivation uses cost (you cannot predict cost honestly on unbounded input). + +## 5. Width derivation (construction invariant) + +**Formula:** `spawn_width = min(cpu_bound, floor(memory_budget / max_runnable_predicted_space))` — the RUN-phase lever (`compute-envelope-model.md` §3: discovery shards use the prebuilt binary; width-up is pids-safe on its own). **Budget source:** `extdeps.accounting.budget` / `product.budget_tree` / `std.realization_width` — one capped-resource→claims concept (§3 convergence with complexity `EffortBudget` and `memory_aware_spawn_width`). **Side-channel deletion:** `gunbc_ci_floor_spawn_width_for_budget`, static `gunbc_ci_floor_measured_peak` rows, and a second `eval_spawn_width` pass in `claim_executor` retire when `WidthBoundedRealization` derives width from the plan. + +## 6. Single authorities — do not re-mint (§3) + +- **Input envelope** — `gunbc.ci_input_envelope` (not a scheduler-local ceiling type) +- **Symbolic cost** — `v2.lens.cost` (`symbolic_cost_of_node`, `SymbolicCost` lattice) +- **Schedule topology** — `std.realization_schedule` (`Schedule`, `Runnable`, `RealizationPlan`) +- **Width fold** — `std.realization_width` + peripheral `WidthBoundedRealization` +- **Host / run budget** — `product.budget_tree` + `gunbc.fleet_container.ResourceEnvelope` (BUILD vs RUN phases stay distinct — two invariants, not one product; `#5904` moved this type off dissolved `product.compute_fabric`) +- **Measured observation** — `gunbc.fleet_intent.PerformanceReceipt` → roll-up to `CostAccount` with `basis = Measured` (falsifier only; `#5904` live home — not dissolved `compute_fabric.PerformanceReceipt`) + +## 7. Implementation phases (dispatch separately — not this capture) + +Phases are ordered; each closes at a named consumer tier (mirrors §2 fleet acceptance: algebra alone is not done). **This capture lands phases 0–1 of the design record only.** + +1. **P0 — InputEnvelope admission (PARTIAL ✓).** Types + `input_admitted` + CI corpus instance + witness. **Accept:** `input_envelope_admission_test.dag` green; RED on revert of `RefusedUndeclared` / over-envelope paths. +2. **P1 — CostEnvelope on `Runnable` (Predicted).** Populate `Runnable.cost.space` from symbolic evaluation at `gunbc_ci_corpus_envelope` bindings; delete `cost_account_predicted_zero()` on the floor rollup path. **Accept (T2):** floor plan reads cost from runnables; remove cost side-channel → witness fails. +3. **P2 — Scheduler consumes Predicted cost.** `WidthBoundedRealization` derives `spawn_width` from plan; delete `gunbc_ci_floor_spawn_width_for_budget` and static peak rows. **Accept (T2):** host memory increase → higher width without data-row edits (discriminating receipt). +4. **P3 — Measured falsifier + calibration loop.** `gunbc.fleet_intent.PerformanceReceipt` / per-shard RSS (`resource-aware-scheduler.md` Node A) feeds `CostBasis = Measured`; mismatch vs Predicted → typed diagnostic. Calibration updates symbolic formula inputs, not the Predicted row directly. **Accept (T3):** planted underestimate in Predicted → falsifier RED. +5. **P4 — Runtime admission gate.** Wire `input_admitted` into the floor path (`ci_floor_plan.dag` Runnable roster + `gunbc.fleet_container.demand_envelope_fits_budget` / `witness_run_demand_envelope_fits_budget`) so runtime refuses before schedule build — **not** dissolved `WorkDemand` (`#5904`). **Accept (T2):** corpus exceeding `gunbc_ci_corpus_envelope` → schedule refuses, not OOM. +6. **P5 — Ceiling derivation (input-envelope-roadmap P2).** Replace scaffold ceilings in `gunbc_ci_corpus_envelope` with values derived from discovery roster + measured node counts; dissolve `Scaffold` disposition on the envelope data. + +## 8. Relationship to companion docs + +- [`input-envelope-roadmap.md`](input-envelope-roadmap.md) — input-envelope-roadmap P1/P2 detail; maps to this design's P0/P5; defers to §3a here for authority +- [`resource-aware-scheduler.md`](resource-aware-scheduler.md) — Nodes A–D implementation tracker for this design's P2–P3; defers to §5–§7 here +- [`ci-selection-vs-scheduling.md`](ci-selection-vs-scheduling.md) — selection axis framing; §4 pipeline table is the composed view +- [`compute-envelope-model.md`](compute-envelope-model.md) — host envelope framing (live type: `gunbc.fleet_container.ResourceEnvelope`); BUILD-phase fan-out; RUN-phase width is §5 here +- [`realization-measurement-loop.md`](realization-measurement-loop.md) — broader cost→schedule loop coordination; this design is the scheduling-arm authority named by that doc's companion pointer +- [`budget-tree.md`](budget-tree.md) — admission/construction for memory budget; width fold is a consumer leaf + +## 9. Explicit non-goals (this capture) + +- No new scheduler types in `src/v2/workflow/scheduler.dag` or `std.realization_schedule` beyond what P1–P2 dispatch items specify +- No `claim_executor` / Rust seed edits for measurement plumbing (Node A remains its own PR) +- No fusion of selection and scheduling (no cost-based nightly routing) +- No replacement of GitHub/job placement (`RunShape → Allocation → Receipt` fabric is orthogonal host-level placement) + +## 10. Red controls (design-level — implementation must wire each) + +- `EnvelopeUnknown` demand → `RefusedUndeclared` (already witnessed in P0) +- Actual axis over declared ceiling → `RefusedOverEnvelope` +- Predicted vs Measured space mismatch → typed falsifier (not silent width change) +- Drop Predicted cost from runnable → width fold fails closed / refuses over-wide spawn +- Affected witness skipped on touched dep → selection soundness RED (independent of this design, but scheduling must not bypass it) + +## Dissolution trigger (DESIGN §6) + +Delete this doc when `gunbc_ci_corpus_envelope` carries derived `BoundedInput` ceilings (P5), every floor `Runnable` carries Predicted `CostEnvelope`-derived cost, `spawn_width` is derived from `WidthBoundedRealization` (no side-channel), `gunbc.fleet_intent.PerformanceReceipt` falsifies Predicted with a discriminating RED, runtime admission consumes `input_admitted` on the live floor path, and the selection/scheduling axis split is witnessed — at which point the carriers state the policy and companion trackers (`resource-aware-scheduler.md`, `input-envelope-roadmap.md`) may dissolve.