diff --git a/docs/design-decomposition-algebra.md b/docs/design-decomposition-algebra.md new file mode 100644 index 00000000000..cc96ed914d6 --- /dev/null +++ b/docs/design-decomposition-algebra.md @@ -0,0 +1,533 @@ +# Decomposition Algebra — ctrl/ migration scoping doc + +**Status**: DRAFT (2026-05-12). Authored per operator directive: migrate ctrl/ processes into .dag substrate; algebra is authoritative, ctrl/ TS becomes projected emission. + +**Authority**: ctrl/ PRs #1192 / #1193 / #1195 / #1197 (decomposition-algebra series, stacked, all OPEN as of 2026-05-12T18:30Z). These four PRs land the algebra in TS; this doc proposes the .dag substrate that makes the algebra source-of-truth. + +**Out of scope**: implementation of emission targets (HTTP REST / SQL migrations / audit-event emission). Those are Phase 3 — substrate-prerequisites that gate the ctrl/ replace. This doc is Phase 0 (readiness audit + integration sketch). + +--- + +## §1. Concept + +The operator wants `ctrl/` (dashboard / planning / replanning) workflows expressed as projections of a typed .dag substrate, with pedantic algebra underneath and a friendly CLI on top (`dashboard-ops open-task` / `replan` / `close` / `archive` / `escalate` / `pause`). The decomposition algebra defined in ctrl PRs #1192-#1197 is the candidate substrate: nodes have a structural type (leaf / composite / bucket / NULL); closure is emergent for composites; replan is more-decomposition; buckets capture algebraic remainder. + +The .dag migration makes that algebra the **single authority**. The TS implementation in `ctrl/` becomes one emission target (REST handlers + SQL schema + CLI surface), but no longer authors the rules. Cost-of-change for a new Mode variant or Operation arm collapses to 1 file (the substrate). Per `feedback_lenses_not_passes.md`: the algebra is a lens over graph state, not a heuristic pass. + +--- + +## §2. The ctrl/ algebra — concrete reference + +### §2.1 Schema (ctrl PR #1192) + +Adds to `nodes` table: +- `mode TEXT CHECK IN ('leaf', 'composite', 'bucket')` (or NULL) — declared shape +- `mode_declared_at`, `mode_declared_by_session_id` — audit fields +- `bucket_drained_at`, `bucket_drained_by_session_id`, `bucket_drain_note` — drain receipt +- `dashboard_migrations` row `decomposition_algebra.cutoff.v1` — cutoff timestamp; pre-cutoff rows grandfathered + +`NODE_MODES` exported as single authority; DB CHECK enforces at boundary. + +### §2.2 Operations (ctrl PR #1193) + +API endpoints: +- `POST /api/nodes/:id/declare { mode, session_id? }` +- `POST /api/nodes/:id/bucket/drain { note, session_id? }` +- `POST /api/internal-work-items` (existing; now auto-flips parent to `composite` on first child via `autoFlipParentModeForChildAddition`) + +Pure helpers in `lib/dag_writes.mjs`: +- `declareNodeMode` — idempotent on same value; refuses redeclare to different mode (caller must replan) +- `drainBucket` — refuses non-bucket nodes + empty notes; mandatory witness string +- `autoFlipParentModeForChildAddition` — "the act of authoring a child IS the declaration of the parent" + +`DecompositionError` with `code` field for HTTP status mapping. + +### §2.3 Closure rule (ctrl PR #1195) + +`canCloseNode(db, nodeId, { cutoff }) → { ok, reason }` — single source of truth. + +Reasons: +- `NODE_NOT_FOUND` +- `ALREADY_CLOSED` (idempotent path) +- `MODE_NOT_DECLARED` (post-cutoff node with mode=NULL) +- `BUCKET_NOT_DRAINED` (mode='bucket' without bucket_drained_at) +- `COMPOSITE_HAS_OPEN_CHILDREN` (mode='composite' with ≥1 open blocks-child) + +`closeNode(db, nodeId, { closedAt, force, cutoff })` wraps the guard; `force: true` is operator override; `node_close_refused` events logged on refusal. + +### §2.4 Replan (ctrl PR #1197) + +`dashboard-ops replan` thin client authors a reconcile work-item under the caller's parent. No new schema; walk-back is operator-driven recursion through normal work-item flow. **Replan IS more decomposition, not a state revert** — the meta-work is itself work, expressed with the same primitives. + +--- + +## §3. Mapping to existing gunbc primitives (grep-verified) + +**Audit scope correction (2026-05-12, post-codex Finding #1)**: prior version scoped audit to `dsl/std/` only and missed the v3 lens substrate. Corrected audit covers `dsl/std/` + `src/v3/std/` + `src/v3/lenses/`. The v3 substrate is the live home for many primitives I previously marked ABSENT. + +| Decomp-algebra concept | Gunbc primitive | Citation | Reuse status | +|---|---|---|---| +| Node (work item) | Recursive type pattern; concrete `Node` is compiler-AST | `dsl/std/node.dag:55-196` | **Concept reuses; carrier needs new domain-specific declaration** (compiler Node is for AST, not work items) | +| Mode (open enum) | Open enum + Practice 4 dissolution framework | `dsl/std/computation.dag:133,192,246` (`SizeBound` / `CallPattern` / `IterationPrimitive`) | ✓ Direct reuse of pattern | +| Operation (closed sum) | Closed enum with payloads | `dsl/std/effects.dag:71-76` (`EffectShape`); `dsl/std/computation.dag:246-249` (`IterationPrimitive`) | ✓ Direct reuse of pattern | +| Closure projection `canCloseNode` | Projection function pattern (Slice 4 substrate) | `dsl/gunbc/ci.dag:29-32` documents shape; `dsl/gunbc/ci_emission.dag` not yet authored (Slice 4 in flight) | ✓ Pattern documented; concrete projection function pattern lands soon | +| Cardinality (single/optional) | `Required \| Optional` | `dsl/std/constructors.dag:74-75` | ✓ Direct reuse | +| Bounded multiplicity (0..N) | List + SetCardinality (size primitive) | `dsl/std/termination.dag:198` (`SetCardinality`) | ◐ Partial — no `Bounded` carrier yet | +| **Witness** (Inhabits \| Violates) | First-class typed witness; the substrate is live in v3 | `src/v3/std/dimensions.dag:35` (`Witness = Inhabits(Carrier) \| Violates { reason, at }`) | ✓ **EXISTS — corrects prior audit miss**. Decomp-algebra Witness can reuse this carrier. | +| OptionalDiagnostic | Optional break channel for `break_diagnostic` | `src/v3/std/dimensions.dag` (`OptionalDiagnostic = NoDiagnostic \| SomeDiagnostic`) | ✓ Reuse | +| DimensionReport | Aggregate result from analysis fold (pass/fail partition) | `src/v3/std/dimensions.dag` (`DimensionReport = DimensionOk { composed, witnesses } \| ...`) | ✓ Reuse (closure-decision shape candidate) | +| Timestamp | Narrowed String (ISO 8601) | `dsl/std/types.dag:299` | ✓ Reuse | +| Clock resource | `Clock` with `now()` | `dsl/std/resources.dag:83-95` | ✓ Reuse | +| Algebraic structures | Magma / Semigroup / Monoid / Group / Ring / Field / FreeMonoid | `dsl/std/algebra.dag:99-320` | ✓ Reuse — closure rules can compose under appropriate structure | +| Graph / edges | Field-based on Node; explicit `GraphEdge` in `CallGraph` | `dsl/std/node.dag` (children: List); `dsl/std/graph.dag:15-30` | ◐ Partial — no labeled edges (parent→child semantic relationship vs dependency vs constraint) | +| Coproduct dissolution receipts | 🟡 MIXED / 🟢 TERMINAL emoji + dissolution-trigger blocks | `dsl/std/computation.dag:126,189,228,244`; `docs/v3-modeling-analysis.md:217-229` (ledger rule); `src/v3/std/coproduct_projection.dag` (substrate dispatch for Practice 4) | ✓ Direct reuse of pattern | +| Effects model | `EffectShape` for idempotency; `IterationPrimitive` for computation | `dsl/std/effects.dag:71-76`; `dsl/std/computation.dag:246-249`; `src/v3/std/effects.dag` (v3 expansion) | ✓ Reuse — algebra operations are EffectShape-class graph mutations | +| **Lens** (compositional analysis carrier) | 🟡 TRANSITIONAL 6-field shape (Director-locked); reuses Witness/OptionalDiagnostic/DimensionReport/Monoid | `src/v3/std/lens.dag` (`Lens { name, read, sequential, branch, iterate, validate }`); `src/v3/lenses/` (~16 worked instances: complexity, cost, parallelism, idempotency, ...) | ✓ **EXISTS — corrects prior audit miss**. Decomposition-algebra closure/state projections fit this carrier shape. | +| Audit log / EventLog | Domain-only at `dsl/gunbc/workflow/types.dag:327-335` (`AuditEntry`); no std/ primitive | — | **✗ GAP — needed for replay-as-truth (see §5 Gap 1)** | +| Task / WorkItem / Workflow | Domain-specific types in `dsl/gunbc/workflow/types.dag` (`ImplementationTask`, `TrackedIssue`, `IssueLifecycleStage`, `StageOutcome`) | `dsl/gunbc/workflow/types.dag:46-194` | **OVERLAP — see §4 (axis-conflation correction post-codex Finding #2)** | + +--- + +## §4. Workflow domain overlap — significant finding + +The audit surfaced an existing workflow type system in `dsl/gunbc/workflow/types.dag` (~340 lines, gunbc-domain): + +| Type | Lines | Decomp-algebra analog | +|---|---|---| +| `ImplementationTask { title, description, file_paths?, dependencies?, done: Bool }` | 106-112 | Leaf-mode Node | +| `IssueLifecycleStage = Idea \| Design \| DesignReview \| Accepted \| Implementing \| CodeReview \| Testing \| Done \| TerminalFailed` | 46-54 | State-machine-residue parallel to Mode | +| `TrackedIssue { id, title, body, stage, url?, author?, labels, created_at?, updated_at? }` | 57-67 | Composite Node with stage tag | +| `StageOutcome { run_key, stage, status, payload?, error?, attempt_count, retry_budget_remaining, ... }` | 183-194 | Operation outcome / execution receipt | +| `AuditEntry { timestamp, actor, action, entity_type, entity_id, before?, after? }` | 327-335 | Single audit-log event | +| `Signal { signal_type, idempotency_key, payload, produced_at, consumed_at? }` | 245-250 | Idempotent operation event | + +These types pre-date the decomposition-algebra work and overlap with it. The key question for this scoping doc: + +**Do the workflow types REPLACE the decomp-algebra carriers, EXTEND them, or DISSOLVE INTO them?** + +**Initial proposal**: DISSOLVE INTO decomp-algebra Mode + closure rules. + +**CORRECTION (post-codex Finding #2 2026-05-12T19:08Z)**: the initial proposal **conflated two orthogonal axes**: + +1. **Decomposition axis** (Mode = Leaf / Composite / Bucket / NULL) — STRUCTURAL; describes how a node closes +2. **Workflow phase axis** (Idea / Design / Implementing / CodeReview / Testing / Done) — TEMPORAL/PROCESS; describes where in the workflow lifecycle a node is + +These are **orthogonal coordinates**. A Composite node can be in "Implementing" OR "CodeReview" phase. A Leaf node can be in "Design" OR "Done" phase. The naive collapse of stages to modes drops information that real consumers use (e.g., review-scheduler decides "trigger review at CodeReview stage transition" — that's not a Mode property). + +**Corrected dissolution shape — preserve both axes as structural coordinates** (per Practice 4 "dimensional" dissolution pattern; M-dimensional space hidden in flat N-variant enum): + +``` +type Node { + mode: Mode // Leaf | Composite | Bucket | NULL (decomposition axis) + phase: Phase // workflow lifecycle axis + ... +} + +type Phase + = Pre // pre-decomposition: idea / design / design-review / accepted + | Active // decomposition committed: implementing + | InReview // composite-with-children-in-witness-collection: code-review / testing + | Closed // canCloseNode passed: done / terminal-failed (with closure-witness) +``` + +`Phase` is itself an open enum subject to Practice 4 — variants will dissolve further as consumers prove out (e.g., `InReview` may split if review-vs-test diverge structurally). + +**ALTERNATIVE — prove every existing stage consumer dissolves**: enumerate every `ctrl/` site that reads `IssueLifecycleStage` and show its question is answerable from `(mode, phase, closure_witnesses)` instead of from raw stage. This is the rigorous dissolution proof; it's deferred to Phase 1.5 work and is the closure predicate for the workflow-types dissolution PR. + +**Until either proof lands**: workflow-types stay extant; decomp-algebra is co-located not replacing. The dissolution claim is **staged with trigger**: trigger = per-consumer enumeration proves no information loss. + +**Pipeline-coordinate facts MUST be preserved as separate axes** (post-codex inline BLOCKING #2 2026-05-12T19:08Z): the grep audit of `dsl/gunbc/workflow/types.dag` surfaces facts USED downstream as pipeline coordinates, that the initial dissolution proposal would have dropped: + +| Fact | Citation | Pipeline role | Preservation strategy | +|---|---|---|---| +| `StageRunKey` | `dsl/gunbc/workflow/types.dag:159` | Unique per-stage-invocation identifier; threads through `StageOutcome:184`, `PipelineArtifact:213`, `MetricRecord:225`, `RetryDue:239`, `TerminalStateReached:243` | KEEP as first-class fact; `RunKey` is a node-decomposition-independent coordinate (work-item can have N runs across its lifecycle). NOT dissolved into Mode/Phase. | +| `ClaimLease` | `dsl/gunbc/workflow/types.dag:166` | Lease-generation + expiry for stage-execution claims | KEEP. Lease-claim semantics are pipeline-coordinate, not decomposition-state. | +| `SignalType` | `dsl/gunbc/workflow/types.dag:235` | Idempotency-keyed signal payload kind | KEEP. Signals are events on the EventLog (Gap 1); SignalType is the event-payload tag, structurally distinct from Mode/Phase. | +| `PipelineArtifact` | `dsl/gunbc/workflow/types.dag:120, 212-214` | Stage-output artifact with type tag (`artifact_id`, `artifact_type`) | KEEP. Artifacts are stage-produced facts; flow forward from run to run. | +| `ArtifactType` | `dsl/gunbc/workflow/types.dag:214, 226` | Artifact taxonomy | KEEP as open enum subject to Practice 4. | +| Metrics (`artifacts_stored: Int` at 318, `MetricRecord` at 225) | `dsl/gunbc/workflow/types.dag:225, 318` | Per-stage telemetry; aggregated upstream | KEEP. Metrics are observability facts, structurally separate. | + +**Per `feedback_projections_must_compose_facts.md`** + **INVARIANTS P2 facts-flow-forward**: facts MUST flow forward through projections. Dissolving stage to mode/phase WITHOUT preserving stage-bound facts violates P2. The corrected dissolution treats `IssueLifecycleStage` as a stage-tag whose VALUE collapses into `(Mode, Phase)` BUT the run-keyed facts (RunKey / ClaimLease / SignalType / Artifacts / Metrics) remain structurally distinct. + +**Other workflow-type mappings** (refined): +- `ImplementationTask.done: Bool` → leaf-Mode + Phase=Closed case (Mode × Phase axes) +- `TrackedIssue.stage` (the stage-value) → collapses into `(currentMode, currentPhase)` projection lens +- `StageOutcome` → KEEP as run-keyed pipeline fact; `StageOutcome.status` → Operation outcome on a specific run; `run_key` threads forward +- `StageRunKey` → KEEP as first-class run-identifier; orthogonal to decomposition +- `ClaimLease` → KEEP as lease-execution coordinate +- `Signal` → audit-log-position model for idempotency; `SignalType` enum tag stays +- `PipelineArtifact` → KEEP; artifacts are stage-output facts forward-flowing +- `AuditEntry` → per-event payload inside `EventLog` primitive (Gap 1 from §5) + +Per `feedback_checkpoint_dissolution_default.md`: at C-checkpoints, dissolution is default — but dissolution requires both structural-coordinate preservation (Mode × Phase axes) AND fact-preservation (run-keyed facts flow forward unaffected). The dissolution proof is bounded: only stage-VALUE collapses; stage-bound facts persist. + +--- + +## §5. Four gaps as extension proposals + +### Gap 1 — EventLog primitive + +**Need**: replay-as-truth model. Current node state is a projection over the audit log; closure is `canCloseNode(replay(log_at(now)))`. + +**Sketch**: +``` +type EventLog { + events: List> + // List has monotonic timestamp invariant per event +} + +type TimestampedEvent { + position: MonotonicSeq // strictly increasing + timestamp: Timestamp + payload: T +} +``` + +**Algebra**: `EventLog` forms a `FreeMonoid>` under append (already in `dsl/std/algebra.dag:390`). Replay is `fold` (already in `IterationPrimitive`). + +**Consequences if landed**: +- `node_close_refused` event in PR #1195 becomes structurally emitted via `EventLog`, not hand-implemented logging call +- Stale-RC parser bug class **dissolves**: review-tally is `latest_per_provider_at_HEAD = fold(log, by_provider, take_latest_for_current_HEAD)`. Old-SHA RC is structurally not-at-HEAD. +- Cutoff timestamp (PR #1192) becomes `log_position` instead of `dashboard_migrations.created_at` + +### Gap 2 — RETRACTED (post-codex inline BLOCKING 2026-05-12T19:08Z) + +**Original proposal**: introduce `Lens { view, update }` as new bidirectional state-projection carrier. + +**Retracted because**: `src/v3/std/lens.dag` already declares a Director-locked `Lens` substrate (6-field shape: name / read / sequential / branch / iterate / validate). Introducing a separate `Lens` creates parallel authority — violates INVARIANTS P2 (single-authority) + MODELING.md M9 (DFS the concept DAG before defining). + +**Corrected approach**: state-projection in decomp-algebra reuses existing substrate. Specifically: +- Mode-as-projection = `fold` over `FreeMonoid>` (FreeMonoid already in `dsl/std/algebra.dag:390`); no Lens carrier needed for one-way projection +- Closure-decision = `canCloseNode` as a typed projection function (pattern per `dsl/gunbc/ci.dag:29-32` `project_github_actions` shape) +- If the v3 Lens 6-field shape (read / sequential / branch / iterate / validate) fits a future ctrl-domain analysis, reuse-as-instantiation; substrate-parameterization is a downstream concern, not this canvas + +**Open question**: if decomposition-algebra surfaces a genuine bidirectional-update use case that doesn't fit v3 Lens (which is fold-only over DAG-with-behaviors), surface to Substrate Mgr for shape audit. Until then, no new lens carrier proposed. + +### Gap 3 — Bounded multiplicity + +**Need**: bucket-mode wants "0..N children with semantic-remainder property." `Cardinality = Required | Optional` is too coarse. + +**Sketch**: +``` +type Bounded { + min: Nat + max: Optional // None = unbounded + current_count: Nat +} +``` + +OR — reuse `SetCardinality` from `dsl/std/termination.dag:198` if it generalizes. + +**Lower priority** than Gaps 1+2 — workable with current List for now. + +### Gap 4 — Attestation carrier (RENAMED from "Witness" to avoid v3 collision) + +**Carrier-name collision audit** (post-codex inline BLOCKING 2026-05-12T19:08Z): v3 substrate already has `Witness = Inhabits(Carrier) | Violates { reason, at }` at `src/v3/std/dimensions.dag:35`. That carrier is per-Behavior cost-basis-inhabitance proof — STRUCTURALLY DISTINCT from decomp-algebra's human-attestation-of-intent (drain note, replan reason, escalate debt-string, operator-override). + +**Per `feedback_self_hosting_md_authority_audit_before_substrate_naming.md`**: same-name carriers across namespaces invite confusion. Rename decomp-algebra's "Witness" to **`Attestation`** to preserve namespace clarity. + +**Need**: typed attestation record for human-authored intent attached to Operation events. + +**Sketch**: +``` +type Attestation { + author: SessionId + text: String + timestamp: Timestamp + evidence: List +} + +type AttestationEvidence = // 🟡 MIXED — will become TERMINAL when locked + StructuralLensReceipt { lens_name: String } // reuses v3 Lens instance result + | IntegrationTest { name: String, passes: Bool } + | ProseAttestation { text: String } +``` + +**Reuse**: where decomp-algebra carriers reference v3 substrate (e.g. an `AttestationEvidence::StructuralLensReceipt` whose lens-instance is a real `Lens` from `src/v3/lenses/`), the existing `Witness` carrier is the inhabitance proof — `Attestation` and v3 `Witness` compose; they don't conflict. + +**Lower priority** — small extension, can land alongside Phase 1 substrate. The `Witness` → `Attestation` rename cascade across §6/§9/§13 in this doc applied below. + +--- + +## §6. Operations as effects + +**Operations** (graph mutations on the typed node-graph): + +``` +type Operation = + Declare { node: NodeId, mode: Mode, attestation: Attestation } + | Decompose { parent: NodeId, children: List, attestation: Attestation } + | Drain { bucket: NodeId, attestation: Attestation } + | Replan { node: NodeId, reason: Attestation } + | Escalate { from: NodeId, to_parent: NodeId, debt: Attestation } + | Pause { node: NodeId, reason: Attestation } + | Reopen { node: NodeId, reopen: ReopenAttestation } // closed → open + | Regress { node: NodeId, retracted_child_ids: List, regression: RegressionAttestation } + | AttestedOverride { rule: ClosureRule, attestation: Attestation } // force: true +``` + +(Renamed: `AttestedOverride` → `AttestedOverride` to match the `Attestation` carrier naming; per Gap 4 carrier-rename.) + +**Reopen + Regress added (post-codex Finding #3 2026-05-12T19:08Z)**: real workflows have closure-reversal cases: +- PR merge reverted post-CI failure +- Gate ratchet retroactively determined wrong +- Subtree decomposition retracted (re-merge-into-parent) + +Without explicit `Reopen` / `Regress` operations, the monotonicity claim was unfounded. Each has explicit witness requirements: + +``` +type ReopenAttestation { + reason: ReopenReason + attestation: Attestation // why reopen is justified +} + +type ReopenReason = + PostMergeRevert { revert_commit: Sha } + | RetroGateInvalidation { gate_id: GateId } + | OperatorIntervention { directive: String } + +type RegressionAttestation { + reason: String + retracted_subtree_signature: String // structural proof of what's being merged back + attestation: Attestation +} +``` + +Each operation is a **pure function** `(EventLog, Operation) → EventLog` (append). Graph state is `state: EventLog → NodeGraph` (projection lens). + +**Closure decisions are NOT monotonic** (post-codex inline BLOCKING #3 2026-05-12T19:08Z): the prior `canCloseNode(state(log + op)) ≥ canCloseNode(state(log))` claim was **false** even for the previously-proposed "forward-stable subset." Specifically: + +- **`Decompose`** adds children → if a parent was leaf-closeable before, decomposing it to a composite-with-open-children makes it `COMPOSITE_HAS_OPEN_CHILDREN` (closure retraction) +- **`Replan`** authors a reconcile work-item under the caller's parent → parent gains a new open child → closure retracted on parent +- **`Declare(node, Bucket)`** without drain → BUCKET_NOT_DRAINED state introduced (closure retraction vs unset) +- **`Reopen`** / **`Regress`** / **`Escalate`** explicitly retract by design + +No subset of Operation is universally monotonic in `canCloseNode`. Dropping the claim. + +**Replacement framing — typed effects per Operation**: each Operation has explicit, type-signature-visible effects on closure-eligibility. Consumers of `canCloseNode` MUST NOT assume monotonicity across event-log composition. Per `feedback_state_space_vs_behavioral_invariants.md`: the type captures the effect explicitly; no behavioral assumption substitutes. + +Effects-per-operation classification: +- `Declare(node, Leaf)` — neutral or improves (sets closure-required-mode; vacuously closeable if no children) +- `Declare(node, Composite)` — neutral if no children; later children retract +- `Declare(node, Bucket)` — retracts (introduces BUCKET_NOT_DRAINED requirement) +- `Decompose(parent, children)` — retracts on parent (introduces COMPOSITE_HAS_OPEN_CHILDREN) +- `Drain(bucket, attestation)` — improves (BUCKET_NOT_DRAINED → resolved) +- `Replan` — retracts on caller's parent (adds reconcile child) +- `Escalate` — retracts on parent (adds upward debt) +- `Pause` — neutral on closure-eligibility (informational lifecycle state); orthogonal to canCloseNode reasons +- `Reopen(node, ...)` — retracts (closed → open) +- `Regress(node, ...)` — retracts on parent + possibly improves on retracted-child-ids +- `AttestedOverride` — bypasses guard with audit-visible attestation (operator-tier escape hatch); per `feedback_parallel_representation_debt.md` it's the structurally-modeled `force: true` from PR #1195 + +**No "lattice claim"** — closure decisions are derived per-state, not constrained by composition order. The operational invariant is: every closure-eligibility transition is witnessed by an explicit Operation; no silent transitions. That's the only invariant claim this doc makes. + +--- + +## §7. Dissolution receipts + +### `Mode = Leaf | Composite | Bucket | NULL` — 🟡 MIXED + +**Dissolution trigger**: when `Mode` becomes derivable from graph state. + +**Derivation candidates**: +- `Leaf` = node has no children AND no decomposition declared +- `Composite` = node has ≥1 child OR explicit composite declaration +- `Bucket` = node has explicit `is_remainder` marker + bound drain rule +- `NULL` = pre-cutoff (grandfathered; dissolves entirely when all grandfathered nodes close) + +Mode dissolution trigger: when `currentMode: EventLog → Mode` projection function lands (a `fold` over `FreeMonoid>` per Gap 2 retraction), Mode collapses from stored field → derived projection. The `mode_declared_at` audit becomes a tag on the fold result (provenance, not state). No new lens carrier needed — reuses existing FreeMonoid + fold substrate. + +### `Operation = Declare | Decompose | ... | AttestedOverride` — 🟢 TERMINAL + +Closed sum over the friendly-CLI vocabulary; payload is structural. No dissolution proposed — each variant is structurally distinct (different effect shape per `EffectShape` mapping). + +### `Attestation` — 🟡 MIXED → 🟢 TERMINAL pending AttestationEvidence enum closure + +When `AttestationEvidence` enum (StructuralLensReceipt / IntegrationTest / ProseAttestation) is locked, `Attestation` becomes terminal. Strength ordering: `StructuralLensReceipt > IntegrationTest > ProseAttestation`. Distinct from v3 `Witness` (`src/v3/std/dimensions.dag:35`) — `Attestation` is human-intent attestation; `Witness` is per-Behavior inhabitance proof. They compose: a `StructuralLensReceipt` may reference a `Lens` instance whose result is a `Witness` chain. + +--- + +## §8. Cost-of-change contract + +**Adding a new Mode variant** (e.g. `Periodic`, `Replicated`) should touch **exactly 1 file**: `dsl/std/process_algebra.dag` (or wherever Phase 1 lands). + +Derived emissions: +- CLI vocabulary (`dashboard-ops open-task --periodic`) — derived from `Mode` enum projection +- REST endpoint validation (`POST /api/nodes/:id/declare { mode: 'periodic' }`) — derived from `Mode` schema projection +- SQL `CHECK IN (...)` constraint — derived from `Mode` enum coordinate dissolution +- Audit-event variant — derived from `Operation` enum payload + +Verification approach: a test that asserts `count(Mode variants) = count(CLI declare flags) = count(SQL CHECK values) = count(REST schema enum values)`. If hand-edit required anywhere, cost-of-change > 1 — failure. + +This is the same shape as `EmissionTarget = YamlStatic | BinaryShim` deciding `project_github_actions` arm dispatch in T-WAD substrate. + +--- + +## §9. Phase 1 substrate-file outline + +`dsl/std/process_algebra.dag` (proposed; new file): + +``` +// Process algebra — workflow planning/replanning as composition of typed +// operations on a node-graph. Source authority for ctrl/ planning state. +// +// Per feedback_lenses_not_passes.md: this substrate is a lens over event-log, +// not a heuristic pass. Current state is a projection of the audit log; closure +// is canCloseNode applied to the projection. + +// === Mode (open enum, Practice 4 MIXED) === +type Mode = Leaf | Composite | Bucket | NULL_TRANSITIONAL + // 🟡 MIXED — dissolution trigger: when `currentMode` projection function lands, + // Mode collapses from stored field → lens projection. NULL dissolves + // entirely when all grandfathered nodes (pre-cutoff) close. + +// === Witness === +// `Attestation` is the decomp-algebra attestation carrier (renamed from +// "Witness" to avoid collision with v3.std.dimensions::Witness). +type Attestation { + author: SessionId + text: String + timestamp: Timestamp + evidence: List +} + +type AttestationEvidence = // 🟡 MIXED — will become TERMINAL when locked + StructuralLensReceipt { lens_name: String } // refs v3 Lens instance result + | IntegrationTest { name: String, passes: Bool } + | ProseAttestation { text: String } + +// === Operation (closed sum, TERMINAL) === +type Operation = // 🟢 TERMINAL + Declare { node: NodeId, mode: Mode, attestation: Attestation } + | Decompose { parent: NodeId, children: List, attestation: Attestation } + | Drain { bucket: NodeId, attestation: Attestation } + | Replan { node: NodeId, reason: Attestation } + | Escalate { from: NodeId, to_parent: NodeId, debt: Attestation } + | Pause { node: NodeId, reason: Attestation } + | AttestedOverride { rule: ClosureRule, attestation: Attestation } + +// === EventLog primitive (Gap 1) === +type EventLog = FreeMonoid> + +type TimestampedEvent { + position: MonotonicSeq + timestamp: Timestamp + payload: T +} + +// === Closure rule (lens projection) === +type CloseDecision = + OK + | NotFound + | AlreadyClosed + | ModeNotDeclared + | BucketNotDrained + | CompositeHasOpenChildren + +projection canCloseNode( + graph: NodeGraph, + node_id: NodeId, + cutoff: Cutoff +) -> CloseDecision + +// === Graph projection over event log === +projection state(log: EventLog) -> NodeGraph + +projection currentMode( + graph: NodeGraph, + node_id: NodeId +) -> Mode // derived, not stored +``` + +This is ~50 lines of substrate. CLI / REST / SQL / audit emissions derive. + +--- + +## §10. First-cut process to migrate + +**Recommendation: review-verdict-parser** (today's pain). + +**Why**: +- 7 parser-class operator-tier surfaces in ~8h validated the cost of the heuristic-pass model (per `feedback_lenses_not_passes.md`) +- Migration is well-bounded: define `ReviewVerdict = Approve | RequestChanges(findings) | Comment` at reviewer-output boundary; review-tally becomes `latest_per_provider_at_HEAD` lens over `EventLog` +- Self-validating: when parser-lag disappears, the substrate works + +**Alternative: the decomposition algebra itself** (the ctrl PRs #1192-#1197). + +- Larger scope (replaces an in-flight TS implementation) +- But more strategic: every other ctrl/ migration consumes the algebra +- Requires Phase 3 emission targets (HTTP REST + SQL) before cut-over + +**Suggested ordering**: review-verdict-parser FIRST (proves substrate emission patterns; small win), decomposition algebra SECOND (replaces the foundational TS-side implementation). + +--- + +## §11. Open questions + +1. **Workflow types dissolution scope** — do we dissolve `IssueLifecycleStage` / `ImplementationTask` / `TrackedIssue` etc. fully into the decomp-algebra, or keep them as gunbc-domain refinements on top? Proposed: dissolve, with the existing types deprecated and removed once consumers migrate. Operator confirm. + +2. **Home placement** — `dsl/std/process_algebra.dag` (universal) vs `dsl/ctrl/process_algebra.dag` (application-specific). Operator confirmed compositional split: universal → `std/`. Confirmed. + +3. **Phase 2 CLI emission target** — Rust binary replaces bash `dashboard-ops`? Or stays as bash shim wrapping a generated client? Proposed: Rust binary (gunbc's strongest emission target), bash shim only for transition period. + +4. **Emission target priority** — HTTP REST first, SQL second, audit-event third? Or different order? Proposed: HTTP REST first (smallest), enables full Phase 4 cut-over without needing SQL emission yet (ctrl/ keeps SQL hand-authored short-term). + +5. **Force-override modeling** — `AttestedOverride` as a structured Operation variant (this doc proposes) vs API flag (ctrl PR #1195 implements). Proposed: structured operation; eliminates the special-case "force: true" boolean flag. + +6. **Phase 1 substrate landing strategy** — single PR for `dsl/std/process_algebra.dag` skeleton (~50 lines), or stacked PRs per type (Mode → Operation → EventLog → projection)? Proposed: single bundled PR (algebra is one substrate; per `feedback_bundle_workstreams_per_pr.md`). + +--- + +## §12. Cross-references + +- ctrl PRs #1192 / #1193 / #1195 / #1197 — decomposition-algebra implementation (TS-side authority until Phase 4 cut-over) +- `dsl/std/algebra.dag:99-320` — algebraic structure precedent (Magma / Semigroup / Monoid / Group / Ring / Field / FreeMonoid) +- `dsl/std/computation.dag:133,192,246` — open-enum + Practice 4 dissolution receipt patterns (`SizeBound` / `CallPattern` / `IterationPrimitive`) +- `dsl/std/effects.dag:71-76` — `EffectShape` closed sum (precedent for `Operation` shape) +- `dsl/std/node.dag:55-196` — compiler-AST Node (recursive-type discipline; carrier reuses pattern, not concrete type) +- `dsl/std/graph.dag:15-30` — `GraphEdge` (precedent for labeled edge types if Gap 7 advances) +- `dsl/gunbc/workflow/types.dag` — pre-existing workflow types (dissolution scope per §4) +- `dsl/gunbc/ci.dag:29-32` — projection-function pattern (`project_github_actions`); concrete implementation in flight via Slice 4 +- `docs/v3-modeling-analysis.md:217-229` — Practice 4 dissolution-receipt ledger rule +- `INVARIANTS.md` P1 (single-authority), P2 (illegal-states-unrepresentable), P5 (Pure Bootstrap) +- `MODELING.md` M9 (DFS the concept DAG before defining) +- Memory `feedback_lenses_not_passes.md` — lenses over physics, not heuristic passes +- Memory `feedback_coproduct_dissolution.md` — 4 dissolution patterns +- Memory `feedback_checkpoint_dissolution_default.md` — at C-checkpoints, dissolution is default +- Memory `feedback_construction_over_ratchets.md` — model first, violations dissolve + +--- + +## §13. Validation case study — PR #2745 misread (2026-05-12) + +The structural algebra proposed in this doc would have caught the PR #2745 misread at PR-close time, not at PM-audit-time. Walked through: + +**Decomposition (implicit, as it stood)**: +- Root: operator intent "T-WAD FULL R3-close" +- → PM scope doc (PR #2744 §0) +- → Criterion 3 "WorkflowRuntime + project_github_actions in gunbc-substrate" +- → WI-2 brief: claim "NEW file `dsl/gunbc/ci_emission.dag` declaring enum + signature" +- → cool-carp-720 worker → PR #2745 + +**The contradiction**: +- WI-2 brief claim: PR #2745 delivers `dsl/gunbc/ci_emission.dag` with `WorkflowRuntime` + `project_github_actions` +- Actual delivery: PR #2745 created `dsl/extdeps/github/ci.dag` (platform substrate) + modifications to `actions.dag` + comment additions to `dsl/gunbc/ci.dag`. The claimed file `dsl/gunbc/ci_emission.dag` was never authored. + +**Walk-back trace (under structural algebra)**: +- Start at PR #2745 leaf node; check parent (WI-2 brief composite node) +- WI-2 brief claim unchanged (meaning intact) +- Composite check: do children's deliveries cover the parent's stated children-set? **NO** — `dsl/gunbc/ci_emission.dag` is listed but missing +- → `COMPOSITE_HAS_OPEN_CHILDREN` would fire at PR-close-event +- → `Replan` operation authored under WI-2 brief; reconcile work-item created + +**What the structural algebra catches that prose-driven workflow missed**: file-list-intersection between brief's declared deliverables and PR's actual file changes is a `StructuralLens` projection over `EventLog`. Mismatch raises `canCloseNode → COMPOSITE_HAS_OPEN_CHILDREN` automatically. Saves the wrong-claim cycle PM hit (which required Director questioning + grep-discovery + correction relay). + +**Mapping to operations**: +- `WalkBackEvent` → sequence of `Replan` operations in the event log +- `StableAncestor` → parent node where `Replan` was authored +- `RootContested` → `Escalate` operation routed to root-asker (operator) +- `force: true` resolution path → `AttestedOverride` operation + +--- + +— Authored by deep-wolf-155 (PM) 2026-05-12 per operator directive for ctrl/ migration scoping. Replaces a prior procedural walk-back draft with this structural-integration scope; validation case study (§13) inlined. diff --git a/docs/r3-remaining-work-dependency-graph.md b/docs/r3-remaining-work-dependency-graph.md new file mode 100644 index 00000000000..b45bd19081f --- /dev/null +++ b/docs/r3-remaining-work-dependency-graph.md @@ -0,0 +1,300 @@ +# R3 remaining-work dependency graph + max-parallelization plan + +**Status**: DRAFT (2026-05-12). Authored per operator directive 2026-05-12T~20:00Z: "define the remainder of R3 now, including all dependencies, so we can max parallelize all the work." + +**Authority**: §1.8 Canonical R3 Closure Ledger at `docs/r3-program-plan.md:225-500` + lane structure at `docs/r3-structure.md` + Cluster F + Cluster M sequencing plans. + +**Companion docs**: +- `docs/audit/r3-cluster-f-sequencing-plan-2026-05-09.md` — Cluster F (gates #81/#82/#83/#95) +- `docs/audit/r3-cluster-m-sequencing-plan-2026-05-09.md` — Cluster M (gates #84/#85/#86/#87 — **critical-path**) + +--- + +## §1. State snapshot + +**Total**: 103 enumerated gates → **102 R3-load-bearing** (only #11 canvas-deferred per Director 2026-05-09 (a)-disposition). + +| Status | Count | % | +|---|---|---| +| 🟢 CLOSED (PASSING) | ~32 | 31% | +| 🟡 IN-FLIGHT (CONSUMER_LANDED) | ~40 | 39% | +| 🔴 OPEN (DECLARED, work-not-started OR no-PR) | ~31 | 30% | + +**R3 close formula** (from `r3-program-plan.md`): +> R3 close = ALL 102 R3-load-bearing §1.8 gates GREEN + `r3_debt_paydown_zero_remaining` GREEN. + +`r3_debt_paydown_zero_remaining` (#75) is already PASSING — debt-receipt CI discipline live since 2026-05-10. + +--- + +## §2. Critical-path clusters + +### **Cluster M — Tests-As-Data-Completeness** (the load-bearing one) + +**Gates**: #84 (bulk-port) + #85 (quantifier substrate) + #86 (generator substrate) + #87 (cementing discipline) + +**Why critical-path**: `#84 every_rust_test_ports_to_dag_or_generated` dissolves **~80-90 of 101 SG-0 hand-Rust test entries** in a single gate closure. Without #84 closing, the "zero hand-authored" thesis for PB-0 R3 close is violated. + +**Phase structure**: +- **Phase 1** (parallel-dispatchable NOW): #85 quantifier substrate + #86 generator substrate. Locked-design-resolved, no canvas needed. Substrate Mgr owns. +- **Phase 2** (cascade post-Phase 1): #87 cementing discipline pattern. Verification Mgr owns. Migrates hand-Rust cementing tests → .dag TestClaim + frozen-oracle witness. +- **Phase 3** (cascade post-Phase 2): #84 bulk-port. Coordinator + 6 per-class workers (cementing / reflected-Dag / generic DimensionReport / boundary / R1C residuals / L4/L7/L5 skeleton). **~80-90 entries dissolve**. + +**Timeline estimate** (per `docs/audit/r3-cluster-m-sequencing-plan-2026-05-09.md`): +- Phase 1: 1-2 weeks +- Phase 2: 1-2 weeks +- Phase 3: 2-4 weeks parallel per-class +- **Total: 4-8 weeks** for full Cluster M closure + +### **Cluster F — Lens Behavioral Parity** (lens-producer retirement) + +**Gates**: #81 (parallelism) + #82 (effect_enum) + #83 (register status) + #95 (opt-in iteration demo) + +**Why critical**: Director carve-promoted these IN-R3 per 2026-05-09 ratification (`#issuecomment-4412330468`). No R4 carve-out; all 4 in-R3-load-bearing. + +**Phase structure** (per `docs/audit/r3-cluster-f-sequencing-plan-2026-05-09.md`): +- **F-α** (parallel-dispatchable NOW): #81 parallelism walker port from `src/v3/compiler/src/workflow_parallelism.rs` → `.dag`. Substrate-ready. Substrate Mgr owns. M-sized. +- **F-β.1** (canvas-tier, parallel with F-α): #82 effect_enum migration-shape canvas. Substrate Mgr authors; Director ratifies. NO predicate-block on F-α. +- **F-β.2** (cascade post-F-β.1): #82 implementation per ratified shape. 4 sub-phases (4a resource-threading, 4b metadata, 4c lens body, 4d full retirement). +- **F-γ.1** (cascade post-F-α + T-LAS #91): #95 opt-in parallelism demo via `apply_lens(parallelism, fn, Enforce)`. +- **F-γ.2** (cascade post-all-4-lenses COMPLETE): #83 register status (all 4 lenses zero-proxy zero-stub). + +**Timeline**: F-α 1-2w, F-β.1 <1w, F-β.2 1-2w post-canvas, F-γ cascade → **3-4 weeks total**. + +### **T-Workflow-As-Data (T-WAD FULL R3)** — NEW 2026-05-12 + +**Gates**: #98-#103 (6 gates added today per PR #2744 §1, Director ratification msg_5cbdad24) + +**Status today**: +- #99 `workflow_runtime_open_enum_landed` — CLOSED (PR #2745 merged earlier today, BUT see audit caveat; substrate landed elsewhere) +- #100 `project_github_actions_landed` — CLOSED-ish (substrate in flight via PR #2774) +- #101 `test_cost_dimension_landed` — CLOSED (PR #2761 merged) +- #102 `slow_test_exemptions_dissolved` — CLOSED (PR #2764 merged) +- #98 `ci_yml_hand_authority_dissolved` — IN-FLIGHT/OPEN +- #103 `ci_uses_affected_set_selection` — OPEN (Slice 7; cool-crab-565 canvas at PR #2766) + +**Sequencing**: Slice 4 (#100 substrate / Slice 5 (#98 actual ci.yml swap) / Slice 7 (#103 affected-set) / Slice 8 (substrate completion). Slice 4 in flight (PR #2774); rest cascade. + +--- + +## §3. Remaining work by lane (post-audit 2026-05-12) + +| # | Lane | Mgr | Total | 🟢 | 🟡 | 🔴 | Critical? | +|---|---|---|---|---|---|---|---| +| 1 | T-Tier3-Dissolution | PB Mgr | 4 | 2 | 1 | 1 | No | +| 2 | T-LensProducer-Retirement | PB Mgr | 4 | 1 | 1 | 2 | **PB-Runtime gated** | +| 3 | T-V-L4-L7-Direct | Verification | 6 | 3 | 1 | 2 | No | +| 4 | T-V-L5-Corpus | Verification | 1 | 0 | 0 | 1 | Depends on L4 | +| 5 | T-FixedPoint | PB Mgr | 1 | 0 | 1 | 0 | No (R1+R3 interp) | +| 6 | T-Numeric-Construction | Substrate | 8 | 2 | 0 | 6 | No | +| 7 | T-Omni-Shape-B | Substrate | 4 | 4 | 0 | 0 | ✓ DONE | +| 8 | T-Anthropic-Wire | Substrate | 3 | 1 | 1 | 1 | No | +| 9 | T-Bridge-Retirement | Distributed | 6 | 3 | 2 | 1 | No | +| 10 | **T-CostLens-Composition** | Substrate | 4 | 0 | 1 | 3 | Cascade impact | +| 11 | T-V2-Retirement | PB Mgr | 3 | 2 | 0 | 1 | No | +| 12 | T-Free-Consequences-Demonstration | Verification | 10 | 10 | 0 | 0 | ✓ DONE | +| 13 | T-E-P-Producer-Broadening | Substrate | 7 | 0 | 4 | 3 | No | +| 14 | **T-Lens-Behavioral-Parity** | Cross | 5 | 1 | 0 | 4 | **Cluster F** | +| 15 | **T-Tests-As-Data-Completeness** | Verification | 4 | 2 | 1 | 1 | **Cluster M ★** | +| 16 | **T-Lens-Application-Surface** | Cross | 7 | 3 | 1 | 3 | Cascade for F-γ.1 | +| 17 | **T-Workflow-As-Data** | Substrate | 8 | 4 | 3 | 5 | T-WAD FULL R3 | +| 18 | T-Lens-Self-Application | Verification | 3 | 0 | 0 | 3 | Cascade post-T-WAD + T-LAS | +| - | R3 Debt-Paydown (standing) | DP Mgr | 1 | 1 | 0 | 0 | ✓ DONE | + +**Two lanes 100% done**: T-Omni-Shape-B, T-Free-Consequences-Demonstration. Standing program gate also done. + +**Star lane**: Cluster M (T-Tests-As-Data-Completeness) — critical-path. + +--- + +## §4. Dependency graph — parallel-dispatchable NOW + +### Independent dispatch (no prerequisite blocking; can fire today) + +These gates have NO unsatisfied prerequisites. Each is an independent worker spawn: + +| Gate | Lane | Mgr | Scope | Size | +|---|---|---|---|---| +| **#81** parallelism_lens_behaviorally_complete | T-LBP / Cluster F | Substrate | walker port `workflow_parallelism.rs` → `.dag` | M | +| **#82** F-β.1 canvas | T-LBP / Cluster F | Substrate | migration-shape ratification (canvas-tier; no predicate block on F-α) | S | +| **#85** forall_exists_quantifier_substrate_landed | Tests-As-Data / Cluster M | Substrate | Quantifier/{ForAll, Exists}, QuantifiedTestClaim carriers; PR #2734 in flight | S-M | +| **#86** program_generator_carrier_landed | Tests-As-Data / Cluster M | Substrate | ProgramGenerator + ProgramShape carriers; PR #2752 DRAFT (close out) | S-M | +| **#87** cementing_discipline (Phase 2 brief authoring) | Tests-As-Data / Cluster M | Verification | cementing-discipline pattern: hand-Rust cementing test → .dag TestClaim + frozen-oracle | Brief NOW | +| **#84** bulk-port coordinator (framework brief) | Tests-As-Data / Cluster M | Verification | coordinator framework + 6 per-class brief stubs | Brief NOW | +| **#98** ci_yml_hand_authority_dissolved | T-WAD | Substrate | actual ci.yml swap with WAD-emitted artifact (Slice 5) | M (post-Slice 4) | +| **#103** ci_uses_affected_set_selection | T-WAD | Substrate / Verification | Slice 7 affected-set integration; canvas at PR #2766 | M | +| **#2** tier3_computation_mirror_dissolved | Tier3 | PB Mgr | scope-matched to #1/#4 pattern (already landed) | S | +| **#15** l5_cross_target_consistency | V-L5-Corpus | Verification | corpus-driven; depends on L4 status | M | +| **#36** bridge_retirement_ledger_zero | Bridge-Retirement | Substrate | unified ledger zero | S | +| **#40** symbolic_cost_expr_equals_executable | CostLens | Substrate | ε path ratified 2026-05-07; Verification Mgr ratchet authoring | S-M | +| **#70** cost_lens_demonstration | CostLens | Substrate | Rust-side composition fixture + ≥2 algebra-instances | M | +| **#73** lens_behavioral_parity_demonstration | T-LBP | Substrate/Verification | all 4 lenses vs frozen v2-oracle snapshot | M | + +**Total parallel-dispatchable NOW: ~14 gates** (some are brief authoring, some are immediate worker spawn). + +### Cascade-gated (waiting on specific predecessors) + +| Gate | Blocked by | When unblocked | +|---|---|---| +| #82 F-β.2 implementation | F-β.1 canvas Director-ratified | post-canvas-ratification | +| #83 register status | #81 + #82 + #79 + #80 all COMPLETE | end-chain Cluster F | +| #95 opt-in iteration demo | #81 + T-LAS #91 | mid-chain Cluster F | +| #84 Phase 3 bulk-port workers | #87 Phase 2 pattern landing | mid-Cluster M | +| #91 T-LAS enforce_violation_routing | T-LBP cascade COMPLETE | post-cluster-F | +| #56-#59 T-Lens-Self-Application | T-WAD + T-LAS cascade | post-cluster + slice-cascade | +| #15 l5_cross_target_consistency | L4 (#9) corpus completion | L4 mature | + +--- + +## §5. Dispatch wave plan (max parallelization) + +### **Wave 1 — IMMEDIATE (today / Day 1)** + +**Substrate Mgr (warm-wolf-698)** can spawn UP TO 16 workers (current max_active_children). Currently 1 worker (proud-dove-838). Spawn 5-7 more: +- Worker S1: **#81 parallelism walker port** (M-sized, substrate-ready) +- Worker S2: **#85 + #86 substrate carriers** (bundled per `feedback_bundle_workstreams_per_pr.md`; locked-design-ready) +- Worker S3: **#82 F-β.1 canvas authoring** (Director ratification target) +- Worker S4: **#36 bridge_retirement_ledger_zero** (small, isolated) +- Worker S5: **#73 lens_behavioral_parity_demonstration** (4-lens vs frozen oracle) +- Worker S6: **#103 Slice 7 implementation** (consumes PR #2766 canvas) +- Worker S7: **#98 Slice 5 (post-Slice 4 / PR #2774 land)** — pre-stage brief; spawn worker on Slice 4 merge + +**Verification Mgr (clever-tern-670)** can spawn up to 8. Currently 3 (sharp-deer-47, calm-newt-227, tidy-gull-504). Spawn 3 more: +- Worker V1: **#87 Phase 2 cementing-discipline pattern brief** +- Worker V2: **#84 Phase 3 coordinator framework brief** (6 per-class stubs) +- Worker V3: **#40 symbolic_cost_expr_equals + #70 cost_lens_demonstration** (bundled) + +**Debt-Paydown Mgr (zesty-boar-261)** can spawn up to 8. Currently 0. Spawn 2-3: +- Worker D1: **Slow-test residual sweep** (post-cost-dim-landing follow-on) +- Worker D2: **SG-0 trajectory follow-on** per `docs/audit/r3-sg0-trajectory-tracker.md` + +**PB Mgr** (currently no active session — needs successor spawn or Director re-allocation): +- Worker PB1: **#2 tier3_computation_mirror_dissolved** (scope-matches #1/#4) +- Worker PB2: **#16 pb_self_compile_fixed_point** (R3 interpretation strengthening) + +**Wave 1 worker count: ~13-15 parallel workers**. Mgr capacity not exhausted; can scale further if needed. + +### **Wave 2 — Day 7-10 (cascade-gated)** + +Triggered by Wave 1 landings: +- **#82 F-β.2 implementation** (4 sub-phase workers post-canvas-ratification) +- **#84 Phase 3 bulk-port** (6 per-class workers post-#87 pattern landing — ~80-90 SG-0 dissolutions begin) +- **#83 register status** (post-all-4-lenses-COMPLETE) +- **#95 opt-in iteration demo** (post-#81 + post-T-LAS #91) + +### **Wave 3 — Day 14-21 (final cascade)** + +- T-Lens-Application-Surface Slice B (#91 enforce_violation_routing) +- T-Lens-Self-Application (#56-#59) +- Final lane closures + #36 ledger zero verification + +--- + +## §6. Worker spawn capacity check + +Current state (per `dashboard-ops graph deep-wolf-155`): + +| Mgr | max_active_children | Currently active | Spawn budget | +|---|---|---|---| +| warm-wolf-698 (Substrate) | 16 | 1 (proud-dove-838) | +15 | +| zesty-boar-261 (Debt-Paydown) | 8 | 0 | +8 | +| clever-tern-670 (Verification) | 8 | 3 (sharp-deer-47 + 2 under sharp-deer-47) | +5 | + +**Total R3 spawn budget: +28 workers**. We've been operating at ~5 active R3 workers; could scale 5-6x. + +**Pre-condition for high-parallelism dispatch**: pre-authored briefs per `feedback_pre_authored_brief_queue.md`. Director surfaced 3 lane queues earlier — these need to be brief-authored ahead of spawn. + +--- + +## §7. Throughput levers (ranked by leverage) + +1. **Land your review-parser fix** (in-flight per your earlier message). Eliminates per-PR PM bypass overhead + rebase-cascades. Single biggest unblock. + +2. **Pre-author Wave-1 briefs in bulk** (Director-surfaced queues + this doc's catalog). Mgrs author 5+ briefs each, then spawn workers in batch. Avoids the "Mgr idle → author brief on-demand → spawn → repeat" serialization. + +3. **Spawn to Mgr capacity** (up to +28 R3 workers). Right now we're at ~5; we can run 5-6x more parallel work if briefs are queued. + +4. **Cluster M Phase 1 is dispatch-ready NOW** — #85 + #86 substrate carriers are locked-design-resolved (no canvas needed). Substrate Mgr can dispatch immediately. **Phase 1 → Phase 2 → Phase 3 cascade IS the critical-path.** + +5. **F-β.1 canvas immediate authoring** — #82 migration-shape decisions need Director ratification. Substrate Mgr authors canvas, Director ratifies, Phase F-β.2 unblocks. **Earliest possible canvas dispatch = earliest possible cascade trigger.** + +6. **PB Mgr successor spawn** — currently no active PB Mgr session. T-LensProducer-Retirement + Tier3 + V2 + FixedPoint lanes need owner. Operator or Director call. + +7. **Class-authorization batch merges** (Director-ratified per msg_836c373b) — eliminates PM-per-PR-auth bottleneck for parser-lag PRs. + +--- + +## §8. Concrete first-day action list + +**Operator (Brian) — Day 0**: +1. Review + ratify this dispatch plan +2. Confirm/spawn PB Mgr successor (no active session) +3. Optionally fire reviewer-schedule for any PRs that need it (or trust auto-cycle) + +**PM (deep-wolf-155) — Day 1 immediately**: +1. Relay Wave-1 dispatch plan to zesty-bear-812 (Director) for ratification + Mgr-tier dispatch +2. Director ratifies and authors next-queue briefs OR delegates to Mgr-tier brief-queue depth +3. Mgrs bulk-author briefs (5+ each) per Wave-1 catalog +4. Mgrs spawn workers up to Wave-1 count (~13-15 workers) + +**Cluster M sequencing (critical path)**: +- Day 1: Substrate Mgr spawns S2 (#85 + #86 bundled brief) +- Day 1-3: Workers ship Phase 1 substrate; PR cycles +- Day 4-7: Verification Mgr lands Phase 2 pattern (#87 first migration) +- Day 7+: Verification Mgr spawns 6 per-class Phase 3 bulk-port workers; ~80-90 SG-0 entries dissolve over 2-4 weeks + +**Cluster F sequencing (parallel critical path)**: +- Day 1: Substrate Mgr spawns S1 (#81 walker port) + S3 (#82 F-β.1 canvas) +- Day 3-5: Director ratifies F-β.1 canvas +- Day 5-12: F-β.2 implementation 4 sub-phase workers +- Day 12+: F-γ cascade (#83 + #95) post-completion + +--- + +## §9. Open questions for operator + +**Q-A**: PB Mgr successor — currently no active session. Spawn under Director (zesty-bear-812) for T-LP-Retirement + Tier3 + V2 + FixedPoint lanes? +- Proposed: spawn via `dashboard-ops work-items create` under Director + +**Q-B**: Worker-spawn limits — Mgr capacity is +28 R3 budget. Should we cap aggressive parallelism somewhere (token cost, review-pipeline capacity, etc.)? +- Proposed: spawn up to ~15 Wave-1; observe; scale if reviewer schedule absorbs + +**Q-C**: Cluster M Phase 3 bulk-port — 6 per-class workers; should each be its own PR or bundled by 2-3 (per `feedback_bundle_workstreams_per_pr.md`)? +- Proposed: per-class PR (each is its own dissolution domain); merge order can batch + +**Q-D**: T-WAD Slice 7 dispatch — PR #2766 (cool-crab-565 canvas) is in clever-tern-670's lane. Slice 7 implementation worker would also go under Verification Mgr? +- Proposed: yes; Slice 7 implementation = Verification Mgr lane; coordinates with T-WAD Substrate via PM relay + +**Q-E**: Review-parser fix ETA — knowing approximately when bypass-overhead ends helps cadence planning. Any indication? +- Open + +**Q-F**: SG-0 trajectory target — what's the SG-0 count we want to hit before considering R3 close-ready? Current is ~101 per `r3-sg0-trajectory-tracker.md`? +- Proposed: 0 (per PB-0 thesis); Cluster M Phase 3 dissolves ~80-90; final ~10-20 close via individual gate landings + +--- + +## §10. Timeline to R3 close (honest projection) + +**Per `docs/audit/r3-cluster-f-sequencing-plan-2026-05-09.md` + `docs/audit/r3-cluster-m-sequencing-plan-2026-05-09.md`**: + +- **Day 1-3**: Wave-1 brief authoring + worker spawn-up +- **Week 1**: Phase 1 carriers land (#85 + #86); F-α + F-β.1 canvas in flight +- **Week 2**: Phase 2 cementing-discipline pattern lands (#87); F-β.2 implementation begins; T-WAD Slice 4 lands +- **Week 3-4**: Phase 3 bulk-port begins (per-class workers); F-γ cascade; T-WAD Slice 5/7 cascade +- **Week 5-6**: ~80-90 SG-0 entries dissolved via #84; Cluster F all-4-lenses COMPLETE +- **Week 6-8**: Final cascade — T-LAS Slice B + T-Lens-Self-Application + remaining lane closures + ledger zero + +**Total**: 6-8 weeks honest timeline from full Wave-1 dispatch to R3-close-ready, contingent on: +- Worker spawn-up reaches +13-15 parallel (currently 5) +- Brief-queue-depth maintained (pre-authored) +- Review-parser fix lands mid-window (reduces coordination tax) +- No major substrate-cascade re-evaluations + +**Critical-path nodes**: +- M1(2.8) compiler-surface lowering gap (PR #2774 — proud-dove-838) — substantive engineering; unblocks Slice 5 +- Cluster M Phase 2 cementing-discipline pattern landing — unblocks 80-90 entries +- F-β.1 canvas Director-ratification — unblocks F-β.2 → F-γ cascade + +--- + +— Authored by deep-wolf-155 (PM/CEO) 2026-05-12 per operator directive for R3 remaining-work definition + max-parallelization plan. Consumes §1.8 ledger + Cluster F + Cluster M sequencing plans; PM-tier proposal for Director ratification + Mgr-tier dispatch. diff --git a/docs/r4-c-compiler-and-llvm-in-dag-program-plan.md b/docs/r4-c-compiler-and-llvm-in-dag-program-plan.md new file mode 100644 index 00000000000..6f70769af08 --- /dev/null +++ b/docs/r4-c-compiler-and-llvm-in-dag-program-plan.md @@ -0,0 +1,293 @@ +# C Compiler + LLVM in .dag — execution promotion plan + +**Status**: DRAFT (2026-05-12). Authored per operator directive 2026-05-12T~19:30Z: parallel program to ctrl-migration; promote existing research-viability artifacts to executing production program. + +**Authority**: existing research plan at `~/ctrl/research/market/viability/demos/c-compiler-in-dag/PLAN.md` (Brian-approved 2026-05-04 via `gunb-ai/ctrl#339` with locks A1-A10). This doc proposes the **execution shape** that consumes the research plan; the workstream content stays authoritative there. + +**Companion docs**: +- `docs/design-decomposition-algebra.md` — decomposition algebra (PR #2775) +- `docs/r4-ctrl-dag-migration-project-plan.md` — sibling parallel program (PR #2775) + +--- + +## §1. Mission + scope decision + +**Mission**: model C compiler + LLVM IR + (optionally) XLS IR in `.dag` substrate so the .dag model is **single authority** for compiler behavior. Bidirectional extdeps for C/C++ (ingestion + emission). Demonstrates the gunbc thesis (omni-emission; substrate-as-integration-surface-coherence) on a production compiler infrastructure surface. + +**Scope decision required** — two interpretations of "C compiler + LLVM in .dag": + +**Interpretation (a) — "LLVM IR + C frontend modeled in .dag"** (existing research scope): +- C subset frontend (Kaleidoscope-Ch8-equivalent per lock A4: numeric expressions + control flow + functions + locals + integer/double types) ingested → typed `.dag` +- LLVM IR (v20.1.7 per lock A6) modeled as `.dag` substrate +- LLVM IR emitted (`emit_module: Module -> String`) — W4 spike done +- LLVM IR ingested (`parse_module: String -> Module`) — W2d, ~6-10 worker-weeks +- Bidirectional symmetry: same `.dag` substrate compiles to native binaries (LLVM) OR to Verilog (XLS, per lock A2) +- This is the existing PLAN.md scope. Multi-month effort but bounded. + +**Interpretation (b) — "LLVM entirely in .dag"** (much larger scope): +- All of (a), plus: +- LLVM optimizer passes (Mem2Reg, InstCombine, GVN, LICM, ...) re-implemented as `.dag` lenses + transforms +- Target-specific lowering (x86, ARM, etc.) modeled in `.dag` +- Register allocation, instruction selection, scheduling as `.dag` +- Replaces LLVM-the-C++-library entirely +- Multi-year effort; ~10M LOC equivalent + +**Proposed**: start with (a). It's already scoped + has W4 spike landed. (b) is a Phase 6+ extension *after* (a) ships. Operator confirm. + +--- + +## §2. Existing research scaffolding audit + +Inventoried from `~/ctrl/research/market/viability/demos/c-compiler-in-dag/`: + +### Authoritative planning doc +- `PLAN.md` — Brian-approved Round-2 plan, 13 workstreams (W0-W11), locks A1-A10. **This stays the authoritative workstream content**; the doc you're reading is the execution shape that consumes it. + +### W0 — shared abstract IR primitives (DONE on paper) +`research/.../extdeps/common/ir/`: +- `bitvector.dag` — fixed-width bitvectors +- `integer.dag` — integer types (load-bearing pair with bitvector) +- `array.dag` — array primitive +- `product.dag` — product type +- `ssa_value.dag` — SSA value primitive + +**Promotion**: `dsl/extdeps/common/ir/` in gunbc per PLAN.md §"Bidirectional-extdeps reframe." + +### W4 — LLVM IR text emitter spike (DONE) +`research/.../spike-llvm-ir/`: +- `ir_substrate.dag` + `ir_types.dag` — Module / Function / BasicBlock / Instruction (Ret) / Value (IntegerConstant | LocalRef) / IrType (I32 | Void) declared +- `ir_emit.dag` — `emit_module: Module -> String` pure emitter +- `trivial_program.dag` — constructs `define i32 @main() { ret i32 42 }` +- `expected_output.ll` — hand-written reference +- `VALIDATION.md` — pass/fail criteria + +**Tier 1 surface deliberately omitted** (per spike scope): named basic-block emission + CFG successors / arithmetic / memory / branch / call / phi / conversion / comparison / pointers / aggregates / metadata / debug / attributes / datalayout / target triple / linkage / calling conventions. These are W2 production work. + +### W7 — XLS/Verilog IR spike (DONE) +`research/.../spike-verilog/`: +- Parallel structure to W4 spike: `ir_substrate.dag` / `ir_emit.dag` / `ir_types.dag` / `trivial_program.dag` / `expected_verilog.v` / `expected_xls_ir.txt` + +### Phase-2 expression evaluator (DONE) +`research/.../lens-counterfactuals/phase-2-expression-evaluator/`: +- `evaluator_types.dag` — typed expression AST +- `evaluator_eval.dag` — pure evaluator +- `evaluator_effects.dag` — effect modeling +- `evaluator_lens_slot.dag` — lens slot for invariant application +- `evaluator_fixture.dag` — test fixtures + +### Lens counterfactuals (3 real-world bug case studies) +`research/.../lens-counterfactuals/path-1/`: +- `llvm-tbaa-metadata-instcombine-101164` — LLVM TBAA metadata bug; gunbc-decl.dag + lens-application.dag +- `pytorch-scatter-add-determinism-50469` — PyTorch determinism bug +- `xls-useless-io-recv-tuple-4008` — XLS bug + +Each shows: "this bug would be impossible by construction if the IR were modeled in .dag." + +### Existing gunbc compiler lens substrate (CONSUMED) +`src/v3/lenses/` — 16 working lens instances ready to consume: +- complexity / cost / cost_target_realization / dag_shape +- effect_enumeration / emission_provenance / idempotency +- infer_helpers / lens_composition_associative_witness / lower_helpers +- named_function_count / parallelism / provenance +- structural_resolution / unused_parameters / variant_payload + +These are EXACTLY the compiler-tier lens instances a production optimizer needs. + +--- + +## §3. Promotion plan: research → production substrate + +### What promotes immediately (Phase A — small, NOW) + +| Research artifact | Promote to | Effort | Why now | +|---|---|---|---| +| W0 `extdeps/common/ir/*.dag` (5 files) | `dsl/extdeps/common/ir/` in gunbc | 1-2 PRs, ~3 days | Already authored; review + merge | +| W4 spike (`ir_*.dag`) | `dsl/extdeps/llvm/v20/ir/` in gunbc | 1-2 PRs, ~3-5 days | Spike validated; promote to versioned extdeps | +| W7 spike (XLS/Verilog) | `dsl/extdeps/hardware/xls_ir/ir/` in gunbc | 1-2 PRs, ~3-5 days | Spike validated; same promotion shape | +| Phase-2 expression evaluator | `dsl/std/expression_evaluator/` OR `src/v3/expression_evaluator/` | 1-2 PRs, ~2-3 days | Already typed; minor refactor | + +**~6-9 PRs over 1-2 weeks** to land all current research into production substrate. + +### What requires real engineering (Phase B — multi-week) + +| Workstream | Effort | Description | +|---|---|---| +| W2a — LLVM IR Tier 1 surface | 4-8 worker-weeks | Add named basic-block emission, CFG, arithmetic, memory, branch, call, phi, conversion, comparison, pointers — the rest of LLVM IR beyond the Ret/i32 spike | +| W2d — LLVM IR ingestion (`parse.dag`) | 6-10 worker-weeks | Parser for LLVM IR text → typed `.dag` (bidirectional symmetry) | +| W3a — C ingestion (Ch8-equivalent) | 6-10 worker-weeks | C subset lexer/parser/AST in `.dag` | +| W3b — C emission | 4-6 worker-weeks | `.dag` → C source for backwards compatibility | +| W11 — C++ ingestion (xlscc-subset) | 8-12 worker-weeks | C++ ingestion pinned to xlscc commit `bab31d0817` (per lock A5) | +| W4-extended — XLS IR Tier 1 surface | 4-6 worker-weeks | Same as W2a for XLS IR | +| W8 — Mem2Reg as lens (pressure test) | 3-5 worker-weeks | First real optimization pass as `apply_lens` invariant | +| W10 — `apply_lens` substrate (v0 demo per A8) | 2-4 worker-weeks | Lens-binding pattern; v0 = lens calls inside Transform body | +| W1 / W5 / W6 / W9 — substrate-prereq work | varies | per PLAN.md | + +**~6-12 months** for full Phase B with 4-6 parallel workers. + +### What requires Phase 3 emission targets (Phase C — gated) + +Mostly the C compiler EXECUTION path: +- LLVM linking / object-file generation — needs `dsl/extdeps/llvm/v20/codegen.dag` or shell-out to llc +- Native binary execution — needs runtime linkage emission +- C++ template instantiation — much harder substrate + +--- + +## §4. Program tree (new Director under PM/CEO) + +``` +operator (Brian) +└── deep-wolf-155 (CEO/PM, root) + ├── zesty-bear-812 (gunbc R3-close Director, idle) + ├── clever-ant-97 (Ctrl-Migration Director, idle — just spawned) + └── C-Compiler+LLVM Director ← NEW (proposed; awaiting spawn confirmation) + ├── Substrate Mgr (Phase A — promote research to production substrate) + │ └── ~6-9 PRs landing W0/W4/W7/phase-2-evaluator into gunbc + ├── Frontend Mgr (Phase B — W3a/W11 — C/C++ ingestion) + │ └── 2-4 parallel workers on C + C++ ingestion + ├── IR Mgr (Phase B — W2/W4-extended/W2d — LLVM IR + XLS IR Tier 1 + ingestion) + │ └── 2-3 parallel workers on LLVM IR surface + ├── Lens-Application Mgr (Phase B — W8/W10 — Mem2Reg + apply_lens substrate) + │ └── 1-2 workers consuming src/v3/lenses/ for production optimizer + └── Pressure-Test Mgr (Phase B — lens-counterfactual validation) + └── 1-2 workers extending the 3 case studies + integration tests +``` + +**Why 5 Mgrs (vs ctrl-migration's 4)**: the workstream parallelism is higher because Frontend / IR / Lens-Application / Pressure-Test are genuinely independent lanes once Phase A substrate lands. Substrate Mgr is the critical-path bottleneck Phase A; rest spawn after. + +**Coordination with ctrl-migration + gunbc R3-close**: all three programs are PM children. PM owns inter-program coordination. + +--- + +## §5. Phase sequencing + +``` +Phase 0 (DONE — research already in ~/ctrl/research/) + ├── PLAN.md authored, Brian-approved with A1-A10 locks + ├── W0/W4/W7/phase-2-evaluator/lens-counterfactuals authored + └── 13 workstreams scoped + ↓ +Phase A (~1-2 weeks; promote research → production substrate) + ├── Substrate Mgr lands W0 + W4 + W7 + phase-2-evaluator into gunbc + └── ~6-9 PRs, doc-shape + minor refactor + ↓ +Phase B (~6-12 months parallel; the real engineering) + ├── Frontend Mgr: W3a (C) + W11 (C++) ingestion + ├── IR Mgr: W2a + W4-extended Tier 1 surfaces + W2d (LLVM IR ingestion) + ├── Lens-Application Mgr: W8 (Mem2Reg-as-lens) + W10 (apply_lens v0) + └── Pressure-Test Mgr: extend lens-counterfactuals; validation harnesses + ↓ +Phase C (multi-month; emission gates) + ├── `dsl/extdeps/llvm/v20/codegen.dag` — native binary emission + ├── `dsl/extdeps/hardware/xls_ir/codegen.dag` — Verilog emission via XLS + └── Runtime / linker / object-file emission targets + ↓ +Phase D (open-ended; "LLVM entirely" expansion if pursued) + └── Optimizer passes, target lowering, regalloc, instruction selection as .dag +``` + +--- + +## §6. First-week concrete actions + +**Operator (Brian) — Day 0**: +1. Review + ratify this execution plan + scope decision (a) "C frontend + LLVM IR" vs (b) "LLVM entirely" +2. Confirm spawn of C-Compiler+LLVM Director under PM (parallel to clever-ant-97) +3. Optionally: ratify research → production promotion authority (existing PLAN.md A1-A10 locks carry forward to production-tier work) + +**PM (deep-wolf-155) — Day 1**: +1. Spawn C-Compiler+LLVM Director via `dashboard-ops work-items create` +2. Dispatch charter (this doc + PLAN.md + research artifacts) + +**C-Compiler+LLVM Director — Day 1**: +1. Ratify scope per operator directive +2. Spawn Substrate Mgr first (Phase A critical-path) +3. Defer Frontend / IR / Lens-Application / Pressure-Test Mgrs to Day-N when Phase A nears landing (~Day 7-10) + +**Substrate Mgr — Day 2-7**: +1. Author 5 promotion briefs: + - Brief A1: W0 promotion (`dsl/extdeps/common/ir/`) — 5 files + - Brief A2: W4 promotion (`dsl/extdeps/llvm/v20/ir/`) + - Brief A3: W7 promotion (`dsl/extdeps/hardware/xls_ir/ir/`) + - Brief A4: phase-2-evaluator promotion + - Brief A5: PLAN.md → gunbc-side doc move (or symlink) per `feedback_dissolution_authority_not_file_presence.md` +2. Spawn 3-5 workers in parallel (each brief is small; ~3-5 day worker time) + +**Phase A wraps ~Day 10** — Phase B Mgrs spawn: +- Frontend Mgr (W3a + W11) +- IR Mgr (W2a + W4-extended + W2d) +- Lens-Application Mgr (W8 + W10) +- Pressure-Test Mgr (lens-counterfactual extension) + +--- + +## §7. Cross-program coordination + +Three sibling programs under PM: + +| Program | Director | Critical interface | +|---|---|---| +| gunbc R3-close | zesty-bear-812 | Substrate primitives in `dsl/std/`; lenses in `src/v3/lenses/` | +| Ctrl-migration | clever-ant-97 | `dsl/ctrl/*.dag` for ctrl/ subsystems | +| C-Compiler+LLVM | TBD (proposed spawn) | `dsl/extdeps/{common,llvm,hardware}/` | + +**Shared concerns**: +- **`dsl/std/` substrate** — all three programs may need substrate-shape changes. Conflicts route through PM. +- **`src/v3/lenses/`** — C-Compiler+LLVM Lens-Application Mgr consumes lens instances; coordination with zesty-bear-812 R3 Substrate Mgr on lens-instance shape. +- **Practice 4 dissolution discipline** — applies uniformly across all three programs. +- **`Lens` substrate** at `src/v3/std/lens.dag` — Director-locked 6-field shape; any extension request from C-Compiler+LLVM program routes to Substrate Mgr via PM. + +**Director-tier autonomy**: each Director owns their program scope independently; PM bridges substrate-shape conflicts. + +--- + +## §8. What this plan does NOT do + +- Implement any compiler code (Phase A is doc-shape promotion only) +- Override the existing PLAN.md A1-A10 locks (those stay authoritative for workstream content) +- Commit to Interpretation (b) "LLVM entirely" — that's Phase D, operator-decision +- Set hard timelines on Phase B (worker-weeks per workstream are estimates from PLAN.md; actual cadence depends on staffing) + +--- + +## §9. Open questions for operator + +**Q-A: Scope decision** — Interpretation (a) "C frontend + LLVM IR substrate" or (b) "LLVM entirely"? +- Proposed: (a) now, (b) as Phase D extension if pursued. + +**Q-B: Director spawn** — should PM spawn the new Director now (as 3rd child under PM, parallel to clever-ant-97), or hold pending ratification? +- Proposed: spawn now per CEO authority; Director can ratify charter when alive. + +**Q-C: Research-to-production authority** — do A1-A10 locks in `~/ctrl/research/.../PLAN.md` carry forward to the production-tier program automatically, or require re-ratification? +- Proposed: carry forward (locks were Brian-approved); document any deltas as new locks. + +**Q-D: PLAN.md home** — keep authoritative content at `~/ctrl/research/.../PLAN.md` with gunbc-side doc pointing to it, OR migrate PLAN.md to `~/gunbc/docs/r4-c-compiler-llvm-research-plan.md`? +- Proposed: migrate the authoritative content to gunbc once Phase A lands (production substrate lives in gunbc; the plan should live with the substrate). + +**Q-E: XLS / hardware target priority** — keep Tier 4 (XLS) ship-at-launch per A2, or drop given the urgency of "replace ctrl/ ASAP" framing for the sibling program? +- Proposed: keep XLS per A2; the bidirectional-thesis claim depends on it; XLS work doesn't compete with ctrl-migration for the same Mgr time. + +**Q-F: Lens consumption from `src/v3/lenses/`** — should the C-Compiler+LLVM program treat the 16 existing lenses as authoritative-and-reused, or audit each for compiler-domain fit and propose extensions? +- Proposed: audit each (Lens-Application Mgr's first task); propose extensions via Substrate Mgr if needed. + +--- + +## §10. Cross-references + +- `~/ctrl/research/market/viability/demos/c-compiler-in-dag/PLAN.md` — Round-2 authoritative plan +- `~/ctrl/research/market/viability/demos/c-compiler-in-dag/README.md` — research README +- `~/ctrl/research/market/viability/demos/c-compiler-in-dag/spike-llvm-ir/` — W4 spike +- `~/ctrl/research/market/viability/demos/c-compiler-in-dag/spike-verilog/` — W7 spike +- `~/ctrl/research/market/viability/demos/c-compiler-in-dag/extdeps/common/ir/` — W0 primitives +- `~/ctrl/research/market/viability/demos/c-compiler-in-dag/lens-counterfactuals/` — 3 real-world bug case studies +- `dsl/std/algebra.dag`, `dsl/std/logic.dag`, `dsl/std/bit.dag` — gunbc primitives W0 builds on +- `src/v3/std/lens.dag` — Director-locked Lens substrate (production lens carrier) +- `src/v3/lenses/` — 16 production lens instances ready to consume +- `~/gunbc/THESIS.md` — omni-emission / bidirectional-extdeps thesis +- `~/gunbc/docs/thesis/what-else-falls-out.md` §"Frontend/backend agnosticism" — bidirectional-extdeps thesis claim +- `feedback_lenses_not_passes.md` — lenses are analyses (read-only); passes are Transforms +- gunb-ai/ctrl#339 — Brian's approval of A1-A10 locks + +--- + +— Authored by deep-wolf-155 (PM/CEO) 2026-05-12 per operator directive for parallel C compiler + LLVM program planning. Promotion of existing research-viability artifacts to executing production program; existing PLAN.md A1-A10 locks carry forward. diff --git a/docs/r4-ctrl-dag-migration-project-plan.md b/docs/r4-ctrl-dag-migration-project-plan.md new file mode 100644 index 00000000000..cf3c6788b47 --- /dev/null +++ b/docs/r4-ctrl-dag-migration-project-plan.md @@ -0,0 +1,448 @@ +# ctrl/ → .dag migration — project plan + +**Status**: DRAFT (2026-05-12). Authored per operator directive 2026-05-12T18:30Z: "make a project plan for this... separate tree (beside the director) that models all dependencies out... migrate as much of ctrl/ onto dag as possible ASAP." + +**Authority**: this is a Mgr-tier proposal for a NEW parallel program tree (parallel to gunbc R3-close Director `zesty-bear-812`). Operator ratifies + spawns Director session for the program; PM (`deep-wolf-155`) authored as project-tier proposal. + +**Companion doc**: `docs/design-decomposition-algebra.md` — algebra scope; consume as substrate authority. + +--- + +## §1. Mission + +**Replace the existing dashboard with .dag code ASAP.** Per operator directive 2026-05-12T~19:05Z: this is not a "future authority" project; it's a direct replacement. ctrl/ TS dies when .dag emission proves out. Apply the gunbc thesis (lens-not-pass; cost-of-change = 1 file; coproduct dissolution at C-checkpoints; M9 DFS-the-concept-DAG-before-defining) to a real product surface. + +**Compositional-modeling discipline**: the intent layer is THIN — "replace dashboard with .dag." The substrate work underneath is where the rigor goes. Per `MODELING.md` M9: every new type DFS-traces back to existing `dsl/std/` primitives; no parallel hierarchies; no opaque-string bridges; no heuristic passes. Per `feedback_lenses_not_passes.md`: lenses over physics, no heuristic enforcement. + +**Why now**: today's 7 parser-class operator-tier-bypass surfaces in 8h (3 distinct fingerprints) validate the heuristic-pass-on-unstructured-text cost. The ctrl/ decomposition-algebra series (PRs #1192-#1197) is in flight and the right substrate to consume. Existing .dag precedents (`workflows/review.dag`, `research/.../inbox_delivery_slice.dag`) prove the approach. + +**ASAP execution**: Phase 1 algebra substrate + Phase 1.5 subsystem modeling + Phase 3 emission targets all run **in parallel** as critical-path. No "deferred until Phase N" — emission targets land as fast as substrate, so per-subsystem cut-over (Phase 4) can fire as soon as the trio converges for that subsystem. Per `feedback_holistic_over_patches.md`: systematic fix, not patches. + +--- + +## §2. Audit summary — ctrl/ subsystems + +Inventoried from `gh api repos/gunb-ai/ctrl/contents/` (2026-05-12). Major subsystems: + +### Already partially .dag-modeled +| Subsystem | Existing .dag | Status | +|---|---|---| +| PR review workflow | `workflows/review.dag` | Partial — pure helpers; awaits `uses` + extdep chain compile | +| Branch review config | `workflows/branch_review.dag` | Partial | +| Review config data | `workflows/review_config.dag` | Partial | +| DAG modeling proposal | `scripts/session-dashboard/DAG_MODELING_PROPOSAL.md` | Design-only doc | + +### TS-only subsystems with documented designs (.md only) +| Subsystem | Documented at | Migration priority | +|---|---|---| +| Decomposition algebra | ctrl PRs #1192/#1193/#1195/#1197 | **HIGH** — substrate for everything else | +| PR-review verdict parser | (across api-reviewer + chatgpt-reviewer) | **HIGH** — today's pain | +| Session lifecycle | `scripts/session-dashboard/SESSION_LIFECYCLE.md` | HIGH — spawn/archive emergence | +| Inbox | `scripts/session-dashboard/INBOX_DESIGN.md` | MEDIUM | +| Dashboard messaging | `scripts/session-dashboard/CONTROL_PLANE_MESSAGE_AUDIT.md` + `control_plane_messages.mjs` | MEDIUM | +| Container lifecycle | `scripts/session-dashboard/CONTAINER_LIFECYCLE.md` + `container_runtime.mjs` | MEDIUM | +| Reviews API | `scripts/session-dashboard/REVIEWS_DESIGN.md` | HIGH (consumes verdict parser) | +| Review posting unification | `scripts/session-dashboard/REVIEW_POSTING_UNIFICATION_DESIGN.md` | MEDIUM | +| Scheduler resilience | `scripts/session-dashboard/SCHEDULER_RESILIENCE_DESIGN.md` | LOW | +| Sessions on remote node | `scripts/session-dashboard/SESSIONS_ON_REMOTE_NODE_DESIGN.md` | LOW | +| Provider path fallback | `scripts/session-dashboard/PROVIDER_PATH_FALLBACK_DESIGN.md` | LOW | +| Settings / API keys | `scripts/session-dashboard/SETTINGS_API_KEYS_DESIGN.md` | LOW | +| Test discipline | `scripts/session-dashboard/TEST_DISCIPLINE_DESIGN.md` | LOW | +| Chat reliability/architecture | 3 .md files | LOW | +| Analyses pipeline | `analyses_api.mjs` / `analyses_sync.mjs` / `analyses_table.mjs` | LOW | +| CI integration | `ci.mjs` | LOW | +| DAG API (work-items HTTP) | `dag_api.mjs` | **HIGH** — sibling of decomposition algebra | + +### Existing top-level docs (constitutional) +`AGENTS.md` / `AUDIT.md` / `CODING.md` / `INVARIANTS.md` / `REVIEW_CONTROL_PLANE.md` / `SCOPE_PORTABILITY.md` / `TESTING.md` — these are project-level authority docs that the .dag substrate must respect. + +--- + +## §3. What can migrate NOW — comprehensive subsystem catalog + +**Strategy**: model **service contracts** (types + typed function signatures + pure helpers), NOT just types. Per `feedback_lenses_not_passes.md` + the existing demo `research/market/viability/demos/agent-ctrl-session-dashboard/inbox_delivery_slice.dag` (which proves: `service InboxDeliverySlice { fn ... }` shape is workable today even without emission). + +**Authority claim correction (post-codex inline BLOCKING #4 2026-05-12T19:08Z)**: per INVARIANTS P2, **declarations alone are staging, not landed authority**. A merged `.dag` model is 🟡 STAGED, NOT 🟢 AUTHORITY. Authority requires a generated consumer or emission target that exercises the substrate — until then the .dag file is "proposed shape pending realization." This is consistent with the existing `feedback_no_textual_enforcement_bridges.md` discipline (textual claims of authority don't substitute for structural enforcement). + +**What "staged" means concretely for a Phase 1.5 PR**: +- The `.dag` file lands as a structurally-checked, compiler-validated artifact (verifiable structure) +- It is NOT yet substrate authority for runtime behavior — the TS implementation in `ctrl/` remains authoritative until cut-over +- The dissolution trigger that retires the staged state and fires authority: **emission target lands + parity test passes + cut-over PR deletes the TS file** (the trio from §6 convergence) + +The phased framing (Phase 1.5 + Phase 3 + Phase 4 in parallel) explicitly times the staged→authority transition per subsystem; no subsystem becomes authoritative until its full trio converges. + +**Total audit** (2026-05-12 via gh API on `gunb-ai/ctrl`): +- 237 .mjs files in `scripts/session-dashboard/` organized into ~16 subsystems +- ~20 .mjs files in `scripts/chatgpt-reviewer/` (browser-based review automation) +- 3 .mjs files in `scripts/api-reviewer/` (CLI-based review backends) +- 29 design docs in `scripts/session-dashboard/*.md` (constitutional + per-subsystem) +- 3 existing partial .dag files in `workflows/` (review, branch_review, review_config) +- 1 existing demo .dag at `research/market/viability/demos/agent-ctrl-session-dashboard/inbox_delivery_slice.dag` — **promote to substrate** (≈90% already there) + +**Staging discipline (post-codex Finding #4 2026-05-12T19:08Z)**: every "doable NOW" row below is **staged with an explicit dissolution trigger**, NOT authoritative-on-arrival. A landed `.dag` model is NOT yet substrate authority — the file is one half of the contract; the other half is a consumer that exercises it. Per Practice 4 dissolution-receipt discipline: every staged carrier has a named trigger that retires the staged state. For all Phase 1.5 modeling PRs the trigger is **"per-subsystem realization receipt + consumer parity"**: emission-target projection ✓ + parity-test ✓ + cut-over PR deletes TS file ✓. Until that trigger fires, the `.dag` file is 🟡 STAGED, not 🟢 AUTHORITY. + +### Catalog (in priority order) + +| # | Subsystem | Source files | TS LOC | Model now? | Notes | +|---|---|---|---|---|---| +| 1 | **Review verdict** | `findings_extract.mjs`, `findings_parser.mjs`, `findings_store.mjs`, `findings_triage.mjs`, `review_subprocess_runner.mjs`, `pr_feedback_format.mjs`, `api-reviewer/*` (3 files) | ~2000 | ✓ NOW | Today's pain; in flight per operator (Q-D) | +| 2 | **Decomposition algebra (work-item)** | `dag_api.mjs`, `lib/dag_writes.mjs`, `lib/dag_schema.mjs` (per ctrl PRs #1192-#1197) | ~800 | ✓ NOW + Phase 1 | Algebra substrate from companion §9 | +| 3 | **Inbox delivery** | `inbox_policies.mjs`, `inbox_schema.mjs` + `INBOX_DESIGN.md` | ~600 | ✓ NOW | **DEMO EXISTS** at research/.../inbox_delivery_slice.dag — promote | +| 4 | **Control plane messages** | `control_plane_messages.mjs`, `control_plane_inject.mjs`, `send_eligibility.mjs` + 3 `CONTROL_PLANE_*.md` | ~1500 | ✓ NOW | Dashboard-message routing + sender-marker discipline | +| 5 | **Session lifecycle** | `sessions_schema.mjs`, `watcher.mjs`, `container_runtime.mjs`, `runtime_tmux.mjs`, `runtime_helpers.mjs` + `SESSION_LIFECYCLE.md`, `CONTAINER_LIFECYCLE.md` | ~2500 | ✓ NOW | Spawn/idle/archive emergence | +| 6 | **Review pipeline (extended)** | extend existing `workflows/review.dag` + `review_scheduler.mjs`, `reviews_schema.mjs` + `REVIEWS_DESIGN.md`, `REVIEW_POSTING_UNIFICATION_DESIGN.md` | ~1800 | ✓ NOW | Promote existing .dag; consumes #1 | +| 7 | **Pools (capacity / billing / dispatch)** | `pools_api.mjs`, `pools_billing.mjs`, `pools_dispatch.mjs`, `pools_schedule.mjs`, `pools_schema.mjs`, `pools_validate.mjs`, `pools_writes.mjs` | ~2000 | ✓ NOW | Big subsystem; 7-file group | +| 8 | **PR digests** | `pr_attached_urls.mjs`, `pr_ci_digest.mjs`, `pr_conflict_digest.mjs`, `pr_merge_ready_digest.mjs`, `pr_rest_fallback.mjs` | ~1200 | ✓ NOW | Pure-function-heavy; easy candidates | +| 9 | **Scheduler** | `scheduler.mjs`, `review_scheduler.mjs` + `SCHEDULER_RESILIENCE_DESIGN.md` | ~1000 | ✓ NOW | Decision contract pure; trigger execution gated | +| 10 | **Work-advancement prompts** | `work_advancement_prompts.mjs` | ~400 | ✓ NOW | Template construction = pure functions | +| 11 | **Analyses pipeline** | `analyses_api.mjs`, `analyses_sync.mjs`, `analyses_sync_targets.mjs`, `analyses_table.mjs` | ~1500 | ✓ NOW | Sync + table queries | +| 12 | **CI integration** | `ci.mjs` | ~500 | ✓ NOW | Poll + gate decisions pure | +| 13 | **chatgpt-reviewer (browser automation)** | `scripts/chatgpt-reviewer/*` ~20 files | ~3000 | ◐ PARTIAL NOW | Browser automation = side-effectful; model CONTRACT today, execution deferred | +| 14 | **api-reviewer (CLI backends)** | `scripts/api-reviewer/*` 3 files | ~600 | ✓ NOW | Backend selection + invocation contract | +| 15 | **Server / HTTP routes** | `server.mjs` | ~2000 | ◐ PARTIAL NOW | Route TABLE doable today; handler bodies gated on Phase 3 HTTP emission | +| 16 | **Utility helpers** | `disk_pressure.mjs`, `effort_picker.mjs`, `parse_int_env.mjs`, `transcript_excerpt.mjs` | ~400 | ✓ NOW (fold) | Small; fold into consuming subsystems rather than standalone | + +**Total TS LOC under audit**: ~21,800 lines across 16 subsystems. Realistic to model all 16 in 3-4 weeks of parallel worker dispatch (Phase 1.5). + +### Independent NOW (no Phase 1 dependency) + +Items **3, 5, 8, 10, 11, 12, 14, 16** can dispatch immediately as Phase 1.5 work without waiting for Phase 1 algebra substrate. ~8 parallel workers feasible. + +Items **2, 4, 6, 7, 9, 13, 15** consume the algebra substrate or have non-trivial cross-dependencies; dispatch after Phase 1 lands or Phase 1.5 first-wave validates the pattern. + +Item **1** (review verdict) is already in flight per operator directive. + +### Subsystems that GATE on emission targets (Phase 3+) + +| Concern | Why gated | +|---|---| +| HTTP route handlers | Need `dsl/extdeps/http/server.dag` for actual server replacement | +| SQL schema + migrations | Need `dsl/extdeps/sql/migration.dag` for migration emission | +| Audit event persistence | Need `dsl/extdeps/audit/event.dag` for event-log emission | +| Browser DOM automation | Need `dsl/extdeps/browser/dom.dag` for Puppeteer-class emission | +| Cron / launchd / hooks | OS-level scheduling — need shell/cron emission | +| Provider authentication | Auth + secret handling — separate Phase 3 concern | + +Each is a distinct R4-class extdeps workstream. None are blocking for Phase 1.5 modeling. + +--- + +## §4. Phases NOT doable NOW (gated) + +**Phase 2 — Friendly CLI projection** (Rust binary): +- Requires Phase 1 substrate landed +- Adds `dashboard-ops` as Rust binary projection of `dsl/ctrl/cli_surface.dag` +- Replaces bash dashboard-ops over time + +**Phase 3 — Emission targets** (gates Phase 4): +- `dsl/extdeps/http/server.dag` — HTTP REST handler carriers +- `dsl/extdeps/sql/migration.dag` — SQL schema emission +- `dsl/extdeps/audit/event.dag` — audit-event emission + +Each is real R4-class substrate work, parallel to T-WAD's `dsl/extdeps/github/actions.dag`. + +**Phase 4 — Ctrl/ cut-over**: +- Stop hand-maintaining `ctrl/lib/dag_writes.mjs`, `ctrl/scripts/session-dashboard/*.mjs` +- Generated emission replaces TS implementation +- Byte-identity / behavior-parity verification + +**Phase 5 — Generalize**: +- Once first ctrl/ subsystem cuts over cleanly, others follow with no new emission-target work +- Each new subsystem migration = add types + projections in `dsl/ctrl/*.dag` + +--- + +## §5. Program tree (Ctrl-Migration Director under PM/CEO) + +**UPDATED per operator directive 2026-05-12T~19:20Z**: deep-wolf-155 (PM) operates at CEO-tier above the gunbc R3-close Director `zesty-bear-812`. Ctrl-Migration Director is **a child under deep-wolf-155**, not parallel-root. Spawn fired via `dashboard-ops work-items create` at `node://adhoc-dc298bc7-9f7` 2026-05-12T19:20:39Z. + +``` +operator (Brian) +└── deep-wolf-155 (CEO/PM, root session) + ├── zesty-bear-812 (gunbc R3-close Director, operationally under PM) + │ └── R3-close subtree (3 R3 Mgrs + workers) + └── Ctrl-Migration Director (NEW child, just spawned at adhoc-dc298bc7-9f7) + ├── Substrate Mgr (Phase 1 — critical path) + │ └── algebra substrate (`dsl/std/process_algebra.dag` + Attestation) + ├── Subsystem-Modeling Mgr (Phase 1.5 — critical path, 8-14 workers across waves) + │ └── 16 subsystem .dag modeling PRs per §3 catalog + ├── Emission-Targets Mgr (Phase 3 — critical path, PARALLEL not deferred) + │ └── HTTP extdeps → SQL extdeps → audit-event extdeps + └── Verification Mgr (Phase 4 — convergence + cut-over) + └── parity tests; per-subsystem cut-over PRs; TS deletion +``` + +**4 Mgrs spawn on Day 1**, not 3 — Emission-Targets Mgr is NOT deferred. Phase 3 work starts in parallel with Phase 1 + 1.5. + +**Auto-spawn mechanics**: per the dashboard ladder, `dashboard-ops work-items create` creates a work-item bound to the caller; the dashboard auto-spawns a child session within ~30s with the title as charter. The Director then runs its own `dashboard-ops work-items create` for each Mgr to populate the next tier. + +**Decisions per ladder**: +- PM/CEO (deep-wolf-155): inter-program coordination between zesty-bear-812 (R3-close) and the new Ctrl-Migration Director; ratifies cross-program signals; surfaces structural conflicts to operator +- Director: scopes phases, ratifies cross-Mgr signals, surfaces to PM, enforces compositional-modeling discipline (M9 DFS before any new carrier) +- Substrate Mgr: own Phase 1 — single algebra substrate file + Practice 4 receipts; lands `dsl/std/process_algebra.dag` +- Subsystem-Modeling Mgr: own Phase 1.5 — dispatches 8 parallel workers Wave 1 (Day 2-5), 6 more Wave 2 (Day 6-10); each lands one subsystem `.dag` PR with service-contract shape +- Emission-Targets Mgr: own Phase 3 — HTTP/SQL/audit-event extdeps; **starts Day 3** authoring HTTP extdeps brief (parallel with Substrate Mgr Phase 1) +- Verification Mgr: own Phase 4 — byte-identity / behavior-parity tests; spawns cut-over PRs as each subsystem's trio converges (algebra ✓ + subsystem-modeled ✓ + emission-target-landed ✓) + +**Coordination between programs**: PM (deep-wolf-155) bridges zesty-bear-812 (gunbc R3-close) and Ctrl-Migration Director. Cross-program substrate-shape conflicts route through PM. Each Director owns their program's scope independently; PM owns the inter-program interface. + +**Compositional-modeling enforcement**: Director's standing brief discipline = every new carrier requires (1) M9 DFS trace to existing `dsl/std/` primitive showing why-not-reuse, (2) Practice 4 dissolution receipt for any open enum, (3) cost-of-change-1 verification (adding next variant touches 1 file), (4) cross-reference to source TS file being replaced. Per `feedback_grep_substrate_before_naming_ratification.md`: grep dsl/std/ + docs/audit/ before ratifying new carrier names. + +--- + +## §6. Phase sequencing — parallel critical paths (revised per "replace ASAP") + +**Old framing** (rejected): Phase 1 → Phase 3 → Phase 4 sequential. Treats emission as "deferred." + +**New framing** (per operator 2026-05-12): Phase 1 + Phase 1.5 + Phase 3 **in parallel**, all critical-path. Per-subsystem cut-over (Phase 4) fires as soon as the three converge for that subsystem. + +``` +Phase 0 (DONE) + ├── docs/design-decomposition-algebra.md (PR #2775) + ├── docs/r4-ctrl-dag-migration-project-plan.md (PR #2775) + └── ctrl audit complete (16 subsystems / ~21,800 TS LOC) + │ + ┌───────────────────┼───────────────────┐ + ↓ ↓ ↓ + Phase 1 Phase 1.5 Phase 3 + (algebra) (subsystem (emission + modeling) targets) + │ │ │ + │ Day 2-5: │ Day 2-5: │ Day 3-7: + │ Substrate Mgr │ Subsystem-Mgr │ Emission-Mgr + │ lands │ dispatches 8 │ HTTP extdeps + │ process_algebra │ parallel workers │ brief authored + │ │ (independent │ + │ │ items) │ + ↓ ↓ ↓ + Phase 1 lands Wave 1 PRs land HTTP extdeps land + │ │ + ↓ ↓ + Day 6-10: Day 8-14: + Subsystem-Mgr Emission-Mgr + Wave 2 (algebra- SQL + audit + consumers) extdeps + │ │ + ─────┴───────────────────┴───── + │ + ↓ + Phase 4: per-subsystem cut-over + ↓ (as each subsystem's trio converges) + Phase 5: generalize / full replacement +``` + +**Critical-path** (all three lanes simultaneously): +1. **Algebra substrate** → enables: work_item / messaging / scheduler subsystem modeling +2. **Subsystem modeling** → enables: contracts that emission projects against +3. **Emission targets** → enables: actual TS replacement at runtime + +**Convergence trigger for cut-over** (per subsystem): subsystem's `.dag` contract authored ✓ AND relevant emission target landed (HTTP for routes / SQL for schema / etc.) ✓ AND parity test passes ✓ → cut-over PR lands; TS file deleted. + +**No "future authority" phase**: Q-G resolved per operator directive — substrate becomes authority THE MOMENT emission proves out for that subsystem. No co-authority window. + +**Parallel ≠ independent — Verification Mgr is the throttle** (per claude #10327 exploratory observation 2026-05-12T19:22Z): "Phase 1 + 1.5 + 3 in parallel" does NOT mean Phase 1.5 PRs can outrun their matching Phase 3 emission target indefinitely. Each Phase 1.5 PR that lands before its matching Phase 3 emission target **explicitly accepts staged-debt** — the .dag file is 🟡 STAGED with no convergence trigger in sight. The Verification Mgr **throttles** by tracking the staged-debt backlog per subsystem; when staged-debt exceeds a budget (proposed: 3 subsystems with no matching emission target landed), pause Phase 1.5 dispatch and prioritize the emission target Mgr's work. + +**Wave-1-trio checkpoint** (per claude #10327 exploratory observation): before fanning Wave 2 dispatch (Day 6-10), require **at least one full trio convergence** on a Wave-1 subsystem (algebra ✓ + Phase 1.5 modeling PR ✓ + Phase 3 emission target ✓). This validates the convergence pattern works END-TO-END on a real subsystem before spinning up the other 13 in parallel. If Wave-1 trio fails to converge by Day 10, surface to PM for re-scope. Risk this mitigates: 16 staged `.dag` files with no dissolution receipts firing. + +--- + +## §7. First-wave dispatch shape + +Once operator spawns the Ctrl-Migration Director: + +**Day 1**: +- Director ratifies project plan + companion scope doc +- Director spawns 3 Mgr sessions (Substrate / Subsystem-Modeling / Verification) + - Emission-Targets Mgr deferred to Day-N when Phase 1 nears landing +- Substrate Mgr authors brief for `dsl/std/process_algebra.dag` Phase 1 substrate + +**Day 2-5** (parallel) — first wave of Phase 1.5 (independent items, no Phase 1 dependency): +- Substrate Mgr's worker drafts Phase 1 substrate PR +- Subsystem-Modeling Mgr authors 8 first-wave briefs from §3 catalog rows: + - Worker A: `dsl/ctrl/inbox.dag` (catalog #3 — **promote existing demo**) + - Worker B: `dsl/ctrl/session_lifecycle.dag` (catalog #5) + - Worker C: `dsl/ctrl/pr_digests.dag` (catalog #8 — pure-function-heavy, easy) + - Worker D: `dsl/ctrl/work_prompts.dag` (catalog #10 — small) + - Worker E: `dsl/ctrl/analyses.dag` (catalog #11) + - Worker F: `dsl/ctrl/ci.dag` (catalog #12) + - Worker G: `dsl/ctrl/api_reviewer.dag` (catalog #14) + - Worker H: utility-helper consolidation (catalog #16 — fold into consumers) +- 8 workers dispatched in parallel; each lands one PR (bundled by subsystem per §11 Q-H) + +**Day 6-10** — second wave (consumes Phase 1 algebra substrate): +- Phase 1 algebra substrate lands (`dsl/std/process_algebra.dag`) +- Dispatch next 6 subsystem briefs: + - Worker I: `dsl/ctrl/work_item.dag` (catalog #2 — algebra-consumer) + - Worker J: `dsl/ctrl/messaging.dag` (catalog #4 — control plane) + - Worker K: `dsl/ctrl/review_pipeline.dag` (catalog #6 — extends `workflows/review.dag`) + - Worker L: `dsl/ctrl/pools.dag` (catalog #7 — big 7-file group) + - Worker M: `dsl/ctrl/scheduler.dag` (catalog #9) + - Worker N: `dsl/ctrl/chatgpt_reviewer.dag` (catalog #13 — partial; contract today, browser-execution deferred) +- Review verdict (catalog #1) parallel-tracked under operator's existing text-parsing fix work; merges in as a co-authority once both stabilize. + +**Day 11-14**: +- First + second wave PRs cycle reviews + land +- Verification Mgr framework brief (parity-test scaffolding for Phase 4) +- Emission-Targets Mgr spawned; Phase 3 design briefs author for HTTP/SQL/audit extdeps + +**Day 15+**: +- Phase 3 emission target PRs land sequentially (HTTP first per §4 ordering) +- Phase 4 cut-over begins per-subsystem (review verdict first as proven minimum) + +--- + +## §8. Per-subsystem brief shape (template) + +Each Phase 1.5 worker brief should declare: + +``` +# Worker brief — .dag modeling + +**Authority**: Ctrl-Migration project plan §3 item N; operator directive 2026-05-12. +**Parent**: Subsystem-Modeling Mgr lane; project Director. +**Closure predicate**: `dsl/ctrl/.dag` authored + 2-distinct-provider APPROVE. + +## Output +`dsl/ctrl/.dag` (or `~/ctrl/workflows/.dag` per §11 Q-A) + +## Scope +- Type-only modeling; no emission target yet +- Define carriers + closed/open enums + projections +- **Practice 4 dissolution receipts for every enum/sum with ≥2 variants** (including closed sums; per codex inline BLOCKING #5 — modeling-discipline requires 🟢/🟡/🔴 classification for every N≥2 coproduct, not just open enums). See acceptance gate #2 below for receipt-format details. +- Cite the corresponding ctrl/ TS file(s) as "current authority; this PR proposes STAGED substrate, becomes authority on Phase 4 cut-over trio convergence" + +## Reference materials +- `docs/r4-ctrl-dag-migration-project-plan.md` §3 item N (this brief's parent) +- `docs/design-decomposition-algebra.md` (algebra substrate) +- Source TS files in ctrl/ +- Existing design .md in scripts/session-dashboard/.md if available + +## Acceptance gates +1. Carriers + enums declared +2. **Practice 4 receipts on every enum/sum with ≥2 variants** (NOT just open enums — closed sums need dissolution analysis too, per codex Finding #5 2026-05-12T19:08Z). Each receipt names: (a) classification (🟢 TERMINAL / 🟡 STAGED / 🔴 NEEDS-DISSOLUTION), (b) dissolution pattern if non-terminal (fact-placement / variant-is-data / algebraic-form / dimensional), (c) trigger that fires dissolution. +3. Cross-references to current ctrl/ TS authority files (the consumers being modeled) +4. **Consumer receipt named**: identify the specific consumer (TS file or future emission target) whose parity/cut-over fires the "STAGED → AUTHORITY" trigger for this subsystem +5. Cost-of-change check: adding a new variant/operation touches 1 file +6. Doc-only — no emission code + +## STOP / PING criteria +- STOP if substrate-shape question requires re-evaluation of algebra carriers — surface to Substrate Mgr +- STOP if subsystem semantics conflict with companion algebra — surface to project Director +- STOP if a closed sum/enum encountered with no clear dissolution pattern (TERMINAL classification unjustified) — surface to project Director for ratification +- PING project Director on PR-open +``` + +--- + +## §9. Risk register + +| Risk | Probability | Impact | Mitigation | +|---|---|---|---| +| Phase 1 substrate shape questioned during Phase 1.5 modeling | MEDIUM | HIGH (rework) | Land Phase 1 before bulk Phase 1.5 dispatch; only items not consuming algebra start before | +| Cost-of-change verification fails on first new variant | MEDIUM | MEDIUM (signals modeling gap) | Write the cost-of-change test in Phase 1; treat as ratchet | +| Workflow-types dissolution (companion §4) requires deprecation of in-flight gunbc work | LOW | MEDIUM | Audit `dsl/gunbc/workflow/types.dag` consumers before Phase 1 starts | +| ctrl/ TS keeps evolving during migration (drift) | HIGH | MEDIUM | Verification Mgr maintains parity tests; flag drift as substantive review finding | +| Emission targets (HTTP/SQL/audit) require new gunbc extdeps work that's deeper than estimated | MEDIUM | HIGH (Phase 3 slips) | Audit `dsl/extdeps/` complexity baseline early; consider deferring SQL emission if too deep | +| Director-tier coordination conflict with zesty-bear-812 (R3-close) | LOW | LOW (orthogonal programs) | Both Directors report to operator; cross-tier coordination via operator | +| Practice 4 dissolution discipline drift in subsystem modeling | MEDIUM | MEDIUM | Every brief requires explicit Practice 4 receipt declaration | +| Review-verdict-parser migration validates substrate but doesn't prove broader emission | MEDIUM | LOW (intentional — small first) | Sequence: review-verdict first (proves approach), then decomp-algebra (proves emission stack) | + +--- + +## §10. First-week concrete actions + +**Operator (Brian) — Day 0**: +1. Review + ratify this project plan (PR #2775 merge) +2. Confirm remaining §11 Qs (B Director shape, C workflow-types dissolution, E ctrl PRs disposition, F cross-Director coord, H per-subsystem PR cadence) +3. **Spawn Ctrl-Migration Director session via dashboard** — one-time operator action (e.g. `dashboard-ops work-items create "Ctrl-Migration Director"` or analogous spawn-root-session command) + +**Ctrl-Migration Director — Day 1**: +1. Ratify scope per operator directive (compositional-modeling discipline; M9 DFS enforcement) +2. Spawn **4 Mgrs simultaneously**: Substrate / Subsystem-Modeling / Emission-Targets / Verification +3. Dispatch Substrate Mgr to author Phase 1 brief AND Emission-Targets Mgr to author HTTP extdeps brief — both critical-path Day 2+ + +**Substrate Mgr — Day 2-5**: +1. Author Phase 1 brief: `dsl/std/process_algebra.dag` substrate (companion doc §9 skeleton) +2. Spawn worker; PR cycle; coordinate with Subsystem-Modeling Mgr on algebra-consumer briefs + +**Subsystem-Modeling Mgr — Day 2-5 (parallel)**: +1. Author **8 first-wave briefs** for independent items (catalog #3, 5, 8, 10, 11, 12, 14, 16) +2. Spawn 8 workers in parallel +3. PR cycle + +**Emission-Targets Mgr — Day 3-7 (parallel — NOT deferred)**: +1. Author HTTP extdeps brief: `dsl/extdeps/http/server.dag` carriers (route + handler + middleware shapes) +2. Spawn worker; PR cycle +3. Author SQL extdeps brief (Day 6+); audit-event extdeps brief (Day 8+) + +**Verification Mgr — Day 5+ (parallel)**: +1. Author parity-test framework brief +2. Spawn worker to scaffold byte-identity test infrastructure +3. Author first per-subsystem cut-over brief when first subsystem's trio (algebra + modeling + emission) converges (~Day 10-14) + +**WAVE-1-TRIO CHECKPOINT (~Day 7-10)** — REQUIRED before Wave 2 dispatch: +- Verification Mgr validates one Wave-1 subsystem's full trio convergence: algebra ✓ + Phase 1.5 modeling PR merged ✓ + Phase 3 emission target landed ✓ + parity test passes ✓ +- If trio converges: Wave 2 fans out (Day 6-10 timeline holds) +- If trio does NOT converge by Day 10: PAUSE Wave 2 dispatch; surface to PM for re-scope. Risk mitigated: 16 staged `.dag` files with no dissolution receipts firing (per claude #10327 exploratory observation 2026-05-12T19:22Z) + +**Wave-2 dispatch (~Day 6-10, gated on Wave-1-trio checkpoint)**: +- Subsystem-Modeling Mgr: 6 second-wave briefs (catalog #2, 4, 6, 7, 9, 13) for algebra-consuming items +- Emission-Targets Mgr: SQL + audit-event briefs in parallel +- Verification Mgr: parity test infrastructure landing + ongoing staged-debt-budget throttle + +**Staged-debt budget** (Verification Mgr enforces): if at any time 3+ subsystems have merged 🟡 STAGED .dag files with no matching Phase 3 emission target landed, Subsystem-Modeling Mgr PAUSES new Phase 1.5 dispatch until Emission-Targets Mgr catches up. Prevents runaway staged-debt accumulation. + +**Convergence cut-over (~Day 14+)**: +- First subsystem cut-over PR lands (TS file deleted; emission projection becomes authority) +- Per `feedback_parity_script_over_comment_reframe.md`: parity script with fail-close, not process discipline +- Subsequent cut-overs cascade as each subsystem's trio converges + +--- + +## §11. Open questions for operator + +**Q-A — RESOLVED (operator 2026-05-12T~18:55Z)**: `.dag` file placement is **gunbc-side**. `dsl/ctrl/*.dag` lives in gunbc as application/tool substrate (parallel to existing `dsl/gunbc/*` application-level types). Universal primitives go to `dsl/std/`. The 3 existing partial `workflows/*.dag` files in ctrl repo eventually migrate to gunbc-side too. + +**Q-D — RESOLVED (operator 2026-05-12T~18:55Z)**: review-verdict-parser migration is **already in flight** per operator (text-parsing fix in progress). Phase 1.5 dispatch order is no longer the question; the operator directive is "audit ALL session-dashboard work + migrate real functionality ASAP." See §3 comprehensive catalog (16 subsystems). + +--- + +### Remaining open questions + +**Q-B: Director-tier session shape** — single Director or operator-acting-as-director? +- Single Director (proposed): one session owns the program; ratifies + delegates +- Operator-acting (alternative): operator directly spawns Mgrs; saves one tier +- Proposed: single Director given scope (16 subsystems = > 1 week program) + +**Q-C: Workflow-types dissolution scope** (also in companion doc §4) — dissolve `dsl/gunbc/workflow/types.dag` into the decomp-algebra, or extend? +- Proposed: dissolve; existing types become structural projections over decomp-algebra +- Operator confirm — has substantial downstream impact on gunbc R3-close work + +**Q-E: Existing ctrl PRs #1192-#1197 disposition** — let them land in TS (treat as the "current authority" the migration will eventually replace), or hold pending substrate landing? +- Proposed: let them land in TS; they are the current authority; the migration eventually projects-from-substrate to replace them +- Operator confirm — has implications for ctrl team velocity + +**Q-F: Cross-Director coordination** — how do `zesty-bear-812` (gunbc R3-close) and the new Ctrl-Migration Director coordinate when their work touches the same substrate? +- Proposed: cross-tier coordination via operator-relay (operator routes signals between Directors); substrate-shape conflicts get operator ratification +- Operator confirm — sets the inter-program protocol + +**Q-G — RESOLVED (operator 2026-05-12T~19:05Z)**: substrate becomes authority **immediately when emission proves out per subsystem**. No "future authority" / "co-authority" phases. Per `feedback_parallel_representation_debt.md`: when canonical source exists, consume it rather than scaffold. Cut-over PR deletes the TS file in the same PR that lands the emission projection. + +**Q-H (new): Per-subsystem PR cadence** — bundle related subsystems (e.g., all 4 `findings_*` files → one PR) or one PR per .mjs file? +- Proposed: bundle by subsystem (catalog row) per `feedback_bundle_workstreams_per_pr.md`; ~16 PRs total for Phase 1.5 + +--- + +## §12. Cross-references + +- `docs/design-decomposition-algebra.md` — companion scope doc; algebra substrate authority +- ctrl PRs `gunb-ai/ctrl#1192`, `gunb-ai/ctrl#1193`, `gunb-ai/ctrl#1195`, `gunb-ai/ctrl#1197` — current TS-side algebra authority +- `gunb-ai/ctrl/workflows/review.dag` — existing partial .dag in ctrl (PR-review workflow) +- `gunb-ai/ctrl/scripts/session-dashboard/DAG_MODELING_PROPOSAL.md` — existing design doc +- `dsl/gunbc/workflow/types.dag` — existing gunbc workflow types (dissolution scope per companion §4) +- `INVARIANTS.md` + `MODELING.md` — governing constraints +- Memory `feedback_lenses_not_passes.md` — substrate authority over heuristic pass + +--- + +— Authored by deep-wolf-155 (PM) 2026-05-12 per operator directive for ctrl-migration project planning. Project Mgr-tier proposal; operator ratifies + spawns Director session for execution.