diff --git a/ROADMAP.md b/ROADMAP.md index 18cc6726031..89acf40cf6e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -311,6 +311,7 @@ Per-row tagging is a scheduled follow-up sweep — **trigger:** post-merge of th ### Post-merge debt (2026-04-21 receipt-closure wave) - **Class 5 Gap 1 — `Bool` inhabits `BooleanAlgebra` grounding not wired**: surfaced during #610's closing. Current `OperatorKind::Logical(_)` dispatch in the three emitters is hardcoded (`if let OperatorKind::Logical(_)` branches bypass the algebra-field resolution that arithmetic operators use). The substrate already models `BooleanAlgebra` with `meet/join/complement` per `dsl/std/algebra.dag:230-236`, and comments there state *"Bool inhabits BooleanAlgebra"* — but the `inhabits` edge isn't structurally landed. Dissolution: wire `Bool inhabits BooleanAlgebra` + extend `resolve_operator_arrow` for `OperatorKind::Logical(_)` + delete the hardcoded emitter branches. Owner: 1e-2b lane (Path A). +- **Class 5 Gap 3 — `data` body shape boundary (substrate capability gap; precise shape requires live-state audit)**: `ValueBody` in `src/v3/compiler/src/dag.rs:259-287` carries three variants — `Unparsed(SourceSpan)`, `Structural { fields: Vec<(String, FieldValue)> }`, `Scalar(LiteralBits)`. **Field-level** shapes inside `Structural` are FULL per `FieldValue` at `src/v3/compiler/src/dag.rs:328-353` (`Literal`, `Reference`, `Record`, `List`, `Variant { constructor, payload }`) — nested records, list literals, declaration references, `Var` references, sum-variant literals all lower today via `lower_structural_field_value` at `src/v3/compiler/src/lower.rs:2616+`. **Remaining gap is at the top-level `ValueBody` boundary**: `data foo: List = [...]`, `data foo: SumType = SomeVariant`, and `data foo: T = other_decl_ref` cannot parse as top-level bodies today — only scalar literals (→ `ValueBody::Scalar`) and record-structural bodies (→ `ValueBody::Structural`) are accepted; anything else falls to `ValueBody::Unparsed`. The comment at `dag.rs:269-287` is explicit: growing `Scalar` to swallow `Reference` / `List` / `Variant` is rejected; extension is via new `ValueBody` variants. **Authority caveat:** `src/v3/DOWNSTREAM_REQUIREMENTS.md:239` describes the *pre-PR-B-unwind* shape (where `FieldValue` was `LiteralBits`-only and "port-carried field values" were the gap); the code has since extended `FieldValue` with the four additional variants, so the DOWNSTREAM entry's "remaining gap" framing is itself stale against the live code. Read `dag.rs` / `lower.rs` for the current boundary. **Consumer impact surfaced 2026-04-24** by PR #693 escalation (sub-child `sharp-bear-829` under Surface Manager): `sub_charclass_in_std_unicode` phase-2 (`tokenize.dag` retype to `Char` / `List` / `CharClass`) failed during execution; the specific shape of the failure (top-level `ValueBody` boundary vs. something narrower) needs concrete reproduction from the escalating sub-child before the dissolution direction is finalized — phase-2 "blocked on Class 5 Gap 3" is a provisional classification pending that reproduction. Dissolution direction: extend `ValueBody` with the additional top-level variants needed by downstream consumers, informed by concrete reproduction. Alternative design call: shift authority shape for config declarations from `data` bodies to extractor-over-unparsed-source. Owner: unassigned; substrate-capability scope — routes through a self-hosting / substrate-capability lane, not through the consuming lanes. **Audit pattern — recursive application.** The 2026-04-23 Character-level row (this file, `:354`) claimed "no substrate capability gap" and was factually wrong; retracted 2026-04-24 by the update to that row in this PR (see the `:354` Status block). The 2026-04-24 initial authoring of *this* row then described the remaining gap incorrectly (claimed field-level shapes were blocked when they were actually supported, citing the stale DOWNSTREAM entry as authority); corrected later the same day after gpt-5.4's independent audit caught the drift. Both incidents instantiate the rule: **claims about substrate capability must be verified against live `ValueBody` / `FieldValue` / `lower_*` code paths before the claim lands — reading a summary authority (ROADMAP row, DOWNSTREAM entry) is necessary but not sufficient, because authority text itself drifts.** - **`emit_rust_module` gap: SurfaceLiteral → LiteralBits rename**: substrate declares `LitInt/LitBool/LitString` in `LiteralBits`; Rust enum is `Int/Bool/String`; no rename facility in `emit_rust_module` causes connective-producing slices (per #612's staging choice notes) to fail on this. Dissolution: add per-variant name mapping in spec or rename-facility in emit. Owner: unassigned; blocks deeper SG-3g slices. - **`emit_rust_module` gap: `render_variant_constructor` fails on external tuple variants**: `src/v3/compiler/src/emit/rust_target.rs:4296` unconditionally emits `Variant { _0: value }` which is wrong for external tuple variants. Dissolution: spec-driven variant-constructor template per source kind. Owner: unassigned; blocks deeper SG-3g slices. - **File-preference rank is a ratified-parallel-authority scaffold**: `Dag::declaration_name_preference_rank` (`src/v3/compiler/src/dag.rs:1985-2015`) and its mirror in `collect_symbols` (`src/v3/compiler/src/lower.rs:1258-1278`) resolve same-named declarations by preferring `src/v3/` over `dsl/`. This is a P2 Boundary Discipline violation — a stable lookup rule that *normalizes* duplicate authority instead of dissolving it. Held as a scaffold rather than deleted because the duplicates carry v3-only substrate content (`v3.std.substrate` imports, `ElementRef` / `PortId` references, partitioned `EffectShape`, diagnostic-assertion surface) that cannot yet live under `dsl/std/` — v2 CI recursively ingests `dsl/` and cannot parse v3 grammar. Dissolution trigger: when every module currently duplicated between `dsl/std/` and `src/v3/std/` (or `src/v3/spec/`) has converged to a single canonical home, delete the rank function + mirror policy and remove the duplicate-authority preference rule entirely. After convergence, `declaration_by_name` must resolve against the single surviving authority or fail closed on multiple matches; no rank-based bridge remains. Convergence checklist: (1) `module std.effects` — `dsl/std/effects.dag` ↔ `src/v3/std/effects.dag` (v3 copy imports `v3.std.substrate`, partitions `EffectShape`, uses `ElementRef`); (2) `module std.verification` — `dsl/std/verification.dag` ↔ `src/v3/std/verification.dag` (surfaces are disjoint today: v2 `AssertKind/TestClaim/TestCase` vs v3 `DiagnosticKind / DiagnosticReference / PortStateExpectation`; convergence requires a design call on which surface wins); (3) embedded `http_path` mirror inside `src/v3/std/effects.dag:118-260` (resolves via either staging `http_path.dag` in v3 or rewiring imports to consume `std.http_path` directly once v3-substrate staging lands). Owner: unblocked by v3.std.substrate staging lane + v3-grammar CI convergence. @@ -350,7 +351,7 @@ Tracked debt not dispatched with the 2026-04-21 lane wave (Lane 1e P3.0, SG-2c-4 - **`extdeps/browser.dag` typed-carrier service-boundary collapse**: `dsl/extdeps/browser.dag:21-32` declares typed carriers (`BrowserConfig`, opaque `BrowserContext` / `Page` / `Element` handles, imports `Url`), but service ops use raw `String`: `Launch.input.headless: String = "false"`, `output.context_id: String`; `Goto.input.url: String`, `wait_until: String`, `output.final_url: String`; selector/query/evaluate outputs all remain `String` (`:42-47,56-60,75-103`). Same class as the LLM service flattening and the GitHub auth model bypass (both tracked separately). P1 Modeling Faithfulness / M8 "dispatch structural, not string extraction". **Dissolution trigger**: when the transport layer (REST/shell/file) can consume typed carriers end-to-end rather than stringly — this requires transport-level capability support that currently forces the String fallback. The Lane H brief (PR #658) pilots the pattern on browser.dag specifically; if that lands cleanly, LLM + GitHub dissolutions can consume the same pattern. Until then, the transport-capability gap is the upstream blocker. Surfaced by 2026-04-22 exploratory analysis. Owner: unassigned (pilot in-flight on Lane H #658); M per service family, contingent on transport capability. -- **Character-level under-consumption in tokenize + syntax authorities (consumption gap, not substrate gap)**: `src/v3/compiler/tokenize.dag` and `dsl/extdeps/languages/dag/syntax.dag` both slice the ASCII/Unicode codepoint space in two parallel non-canonical forms. **Reserved individual codepoints as opaque strings**: `StringEscapeSpec.suffix`, `output_codepoint: Int`, `LocalPunctSpec.pattern`, `string_literal_delimiter`, `line_comment_prefix`, `OperatorSpec.symbol`, `dag_keyword_set` keys — all encode specific codepoints as byte strings. **Reserved codepoint classes as hidden Rust predicates**: `is_ascii_whitespace` / `is_ascii_digit` / `is_ascii_alphabetic` / `is_ascii_alphanumeric` plus a bare `byte == b'_'` `push_str`'d into `regen_tokenize.rs:700,727,750,752`. Not mentioned in `.dag` at all. **The character-level concepts already exist in `dsl/std/`** and are imported cross-tree today (e.g., `src/v3/compiler/regen.dag:3` imports `std.types`): `std.types::Char = Int` (Unicode scalar, U+0000–U+10FFFF), `std.string_type::String = FreeMonoid`, `std.unicode` (`DisplayWidth`, `UnicodeBlock`, block/width classification), `std.encoding` (`ASCII | UTF8 | Latin1 | Text | Binary | Unknown` lattice with `ASCII <: UTF8 <: Text` — literally the ASCII→Unicode causal chain), `std.bit::Byte`. **Framing**: this is a consumption gap — the modeling is done, the tokenizer and syntax authorities just aren't using it. Only one substrate delta is needed. **Dissolution (follow-up lane)**: (1) add `CharClass = Whitespace | Digit | IdentStart | IdentContinue` (or superset) to `std.unicode` as a sibling to `DisplayWidth`, plus classification predicates as data/functions; (2) retype the opaque-string fields in `tokenize.dag` (`suffix: Char`, `output_codepoint: Char`, `pattern: List`, `string_literal_delimiter: Char`, `line_comment_prefix: List`) and the parallel fields in `syntax.dag` (`OperatorSpec.symbol: List`, keyword-set keys as `List`); (3) rewire `regen_tokenize` to read the class-predicate list structurally rather than hardcoding host-stdlib method names. Scaffold note lives in the `tokenize.dag` header. Owner: unassigned; migration-gate: **ready-to-dispatch** (no substrate capability gap). If the `tokenize.dag → std/tokenize.dag` consolidation migration (from the compiler–std consolidation program above) is dispatched first, the rewrite piggybacks on it so the structurally-labeled borrow lands in its final home. +- **Character-level under-consumption in tokenize + syntax authorities (mixed: consumption gap for steps 1+3, substrate gap for step 2 — see Status block below)**: `src/v3/compiler/tokenize.dag` and `dsl/extdeps/languages/dag/syntax.dag` both slice the ASCII/Unicode codepoint space in two parallel non-canonical forms. **Reserved individual codepoints as opaque strings**: `StringEscapeSpec.suffix`, `output_codepoint: Int`, `LocalPunctSpec.pattern`, `string_literal_delimiter`, `line_comment_prefix`, `OperatorSpec.symbol`, `dag_keyword_set` keys — all encode specific codepoints as byte strings. **Reserved codepoint classes as hidden Rust predicates**: `is_ascii_whitespace` / `is_ascii_digit` / `is_ascii_alphabetic` / `is_ascii_alphanumeric` plus a bare `byte == b'_'` `push_str`'d into `regen_tokenize.rs:700,727,750,752`. Not mentioned in `.dag` at all. **The character-level concepts already exist in `dsl/std/`** and are imported cross-tree today (e.g., `src/v3/compiler/regen.dag:3` imports `std.types`): `std.types::Char = Int` (Unicode scalar, U+0000–U+10FFFF), `std.string_type::String = FreeMonoid`, `std.unicode` (`DisplayWidth`, `UnicodeBlock`, block/width classification), `std.encoding` (`ASCII | UTF8 | Latin1 | Text | Binary | Unknown` lattice with `ASCII <: UTF8 <: Text` — literally the ASCII→Unicode causal chain), `std.bit::Byte`. **Framing**: this is a consumption gap — the modeling is done, the tokenizer and syntax authorities just aren't using it. Only one substrate delta is needed. **Dissolution (follow-up lane)**: (1) add `CharClass = Whitespace | Digit | IdentStart | IdentContinue` (or superset) to `std.unicode` as a sibling to `DisplayWidth`, plus classification predicates as data/functions; (2) retype the opaque-string fields in `tokenize.dag` (`suffix: Char`, `output_codepoint: Char`, `pattern: List`, `string_literal_delimiter: Char`, `line_comment_prefix: List`) and the parallel fields in `syntax.dag` (`OperatorSpec.symbol: List`, keyword-set keys as `List`); (3) rewire `regen_tokenize` to read the class-predicate list structurally rather than hardcoding host-stdlib method names. Scaffold note lives in the `tokenize.dag` header. **Status (2026-04-24, post-PR #693):** phase-1 landed — step (1) done (`CharClass` + `char_in_class` in `dsl/std/unicode.dag`, ASCII-aligned); step (3) partially done via a `tokenize_char_class.rs` Rust mirror (gate-locked 0..=127 against Rust ASCII helpers per `sub_charclass_in_std_unicode_gate`), so generated tokenizer carries no `is_ascii_*` calls. Phase-2 (step 2 — the `.dag`-native retype to `Char` / `List` / `CharClass` variants inside `data` bodies) failed at execution time and the sub-child classified the blocker as Class 5 Gap 3; the row above now correctly describes that gap as a top-level `ValueBody` boundary (top-level non-scalar, non-record bodies — field-level shapes inside `Structural` already work). **The specific shape of the CharClass phase-2 failure — whether it hits the top-level `ValueBody` boundary or a narrower limitation — needs concrete reproduction from PR #693's sub-child before the blocker classification is finalized.** See the Class 5 Gap 3 row above for the substrate-capability dissolution direction and authority pointers. **The 2026-04-23 authoring of this row claimed "no substrate capability gap"; retracted — PR #693's execution found the gap.** See the `char_in_class` interpreter-parity follow-up row below for the bridge-finish step (retiring the host-stdlib-only ratchet on `tokenize_char_class.rs`). Owner: phase-1 landed (PR #693); phase-2 unassigned, blocked on Class 5 Gap 3. If the `tokenize.dag → std/tokenize.dag` consolidation migration (from the compiler–std consolidation program above) is dispatched before Class 5 Gap 3 closes, the phase-1 Rust mirror piggybacks on it; the full structural `.dag`-native path lands only when Class 5 Gap 3 closes. - **`char_in_class` interpreter parity (tokenizer bridge finish, PR #693):** `dsl/std/unicode.dag::char_in_class` is canonical; `src/v3/compiler/src/tokenize_char_class.rs` stays hand-synced until M1(2.8) class-5 gap #3 allows structural scanner rows. Today unit tests ratchet against `u8::is_ascii_*` only. **Next step:** once `char_in_class` is evaluable from the v3 compiler test harness, assert code-point parity on `0..=127` against `byte_matches` and retire the host-stdlib-only ratchet / duplicate sync prose. Same migration gate as deleting `tokenize_char_class.rs` when `regen_tokenize` reads classes from lowered `tokenize.dag`. ### Post-merge debt (2026-04-23 thesis-doc surface — PR #672)