Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 1 addition & 10 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,16 +69,7 @@ here: it's the substrate CI runs on; selling it as an infra piece is downstream.
- [ ] **floor runs reliably & affordably** — memory-aware scheduling (spawn_width is memory-blind → deterministic OOM as the corpus grows; `ResourceEnvelope.memory` modeled but unwired) and kill build flakes (sccache corruption ⇒ false-green: exit-0 with no artifact)
- [ ] **tree-scoped builtin availability (registry partition)** (fail-closed) — the seed `builtin_function_registry` (76 names, `src/v1/04_method.dag`, a marked bridge scaffold) is **global, not scoped to the compiled tree**, so v1-seed intrinsics leak into the dsl-substrate compile and *resolve without a real `.dag` def* (a §5 fail-open, surfaced by `utf8_decode_bytes` in `secret_manager.dag` — the gate is green despite no genuine definition). Fix by construction: the substrate may use only builtins with real `.dag` defs; seed-only names admitted only when entry-root = the v1 seed — this advances the registry's *own* sanctioned dissolve-on ("deleted when builtins are actual `.dag` definitions"). Load-bearing seed + a likely red wave → **measure-first** (leaked-name count: how many of the 76 the substrate relies on), expose-then-triage, escalate before the enforce-flip. *Owned by §1 (quick-ant-298); instance fix (real `utf8_decode_bytes` std fn in `std/encoding` **+** removal of the registry bridge entry & its seed mirror) = **#5452** (verified sound: symbol now resolves via the real `.dag` def, execution is the fail-closed `v1_rt::utf8_decode_bytes` intercept; fleet-gated). Class fix (tree-scoped partition) still open.*
- [ ] repo model (internal repo) on compute fabric
- [ ] **CI on compute fabric** — centralize *all* host management/modeling under the fabric: the host's true `ResourceEnvelope` is the **single authority** and **every** resource/parallelism knob is **derived** from it and **generated** (gunbc owns the unit *shapes*, ctrl owns the srv1/srv2 *instances* — the units are emitted, not hand-typed). Today these knobs are independent hand-set systemd+shell that drift and contradict, so the host oversubscribes itself — this is *one* un-modeled-envelope problem wearing many hats (the EAGAIN clone-reject flake, the spawn_width OOM, the `MemoryMax` drift are all the same root). **The dimensions that must be modeled/calculated, not forgotten** (enumerated *now* precisely because there are too many to hold in our heads):
- [ ] **ground the envelope in measured truth first** — the fleet model is currently *wrong* (says 64c/256GB Altra; measured ground truth is **128c/128GB** on both hosts). Every derived limit inherits this lie until fixed.
- [ ] **cgroup pids cap** (`TasksMax` on `actions-runner@*.service` + the container template) — too low (4096) vs the build's thread fan-out ⇒ `clone()` rejected ⇒ EAGAIN false-failure; host has ~1M-thread headroom unused.
- [ ] **jobserver token count** (`/var/lib/ctrl/jobserver` — the real cross-process parallelism authority, ~120) — must be *derived from* the envelope, not a magic constant the pids cap then contradicts.
- [ ] **floor `spawn_width`** — must be **memory- *and* pids-aware**, not just `min(shard_count, hardware_threads)` (`ResourceEnvelope.memory` modeled but unwired → deterministic OOM as corpus grows).
- [ ] **per-build fan-out** — `CARGO_BUILD_JOBS` × rustc codegen-units (debug default 256) × LLVM threads × sccache; the multiplicand that makes a single "job" burst past the pids cap.
- [ ] **cgroup memory budget** (`MemoryMax`) — currently *drifted* (srv1 52% vs srv2 70%); must be one derived value, not per-host hand edits.
- [ ] **watchdog / wall limits** (`WatchdogLimit.max_wall`), **isolation boundary** (`IsolationBoundary`), and **runner instance count / topology** (how many `actions-runner@srv1-NN` units, on which host) — also envelope-derived, also currently hand-set.

Single carrier with §2 (caching) — this is the **parallelization-by-realization** arm of `std/realization.dag` (Schedule/Placement/Width), the structural twin of caching's Materialization arm; both derived from one `EffectShape`-independence authority. Naming the knobs here is the construction wall (§5): once each is *derived* from the envelope, "jobserver says 120 but pids caps 4096" becomes unwritable.
- [ ] **CI on compute fabric** — centralize host management under the fabric: one host `ResourceEnvelope` is the single authority, every knob **derived** from it and **generated** (gunbc owns unit shapes, ctrl owns srv1/srv2 instances), not the hand-set systemd+shell that drifts and contradicts today (the EAGAIN clone-reject flake, spawn_width OOM, and `MemoryMax` drift are all this one un-modeled-envelope root). **Chore list to model — derive, don't hand-set:** measured host facts (the fleet model is *wrong*: says 64c/256GB, real is 128c/128GB) · cgroup pids cap (`TasksMax`) · jobserver token count · `spawn_width` (memory- & pids-aware) · per-build fan-out (`CARGO_BUILD_JOBS`×codegen-units×LLVM) · `MemoryMax` (drifted srv1≠srv2) · watchdog / isolation / runner topology. This is the **parallelization-by-realization** arm of `std/realization.dag` (the twin of §2 caching).
- [ ] *(downstream / expansion)* compute fabric as a sellable infra piece

## 2. Minimal work — caching by realization (fail-closed)
Expand Down
Loading