Skip to content
94 changes: 74 additions & 20 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Previously eliminated:
| Gate | Command | Status |
|------|---------|--------|
| Lint | `cargo clippy --workspace -- -D warnings` | GREEN |
| Tests | `cargo test -p v2-compiler-tests` | GREEN (294 pass, 0 fail, 41 ignored) |
| Tests | `cargo test -p v2-compiler-tests` | GREEN (316 pass, 0 fail, 44 ignored) |
| Full DSL | `full_dsl_compiles -- --ignored` | GREEN (93 dsl + 29 v2) |
| Diagnostic ratchet | `strict_compile_diagnostic_count -- --ignored` | 314 (all complexity violations) |
| L1 ratchet | `scripts/l1-ratchet.sh --check` | 37 (target: 0) |
Expand Down Expand Up @@ -116,7 +116,7 @@ Previously eliminated:
- **Fixed-point:** `regen pass N == regen pass N+1`. Two-pass bootstrap
required when the emitter changes its own output.
- **External dependencies:** cargo, rustc (opaque transforms, not modeled).
- **Hand-maintained files (9):** Copied back during regen, not overwritten.
- **Hand-maintained files (2):** Copied back during regen, not overwritten.
These are source, not derived. See Bootstrap Status for list.

**Merge workflow problem:** Stage0 `.rs` files are derived, but git
Expand Down Expand Up @@ -269,14 +269,14 @@ correct-but-suboptimal code and blocks new backends.
fuses Node → String in one pass, consuming coercion data directly.
Clone semantics unified in SharingStrategy (CloneTemplates removed).
TLC-1 complete: ExprVar → ExprCall normalization at inference time
(PR #329). TLC-3 done: all three backends dispatch per collection type
via `IndexingSemantics` templates. TLC-4 partial:
`emit_let_binding_annotated` infrastructure exists but disabled for
bootstrap convergence (.dag type inference produces `()` for empty
collection element types). TLC-2 partial: `wraps_result` on
`RuntimeFunction` registry but consumers still read parallel maps.
Higher-order method templates (filter/any/all/flat_map) data-driven
via `HigherOrderMethodSpec`.
(PR #329). TLC-2 complete: `SimpleMethodSpec` co-locates template +
wrapping flag, both maps derive from single authority. TLC-3 done: all
three backends dispatch per collection type via `IndexingSemantics`
templates. TLC-4 partial: `emit_let_binding_annotated` infrastructure
exists but disabled for bootstrap convergence (.dag type inference
produces `()` for empty collection element types). Higher-order method
templates (filter/any/all/flat_map) data-driven via
`HigherOrderMethodSpec`.

One root cause, two symptoms:
1. Facts that exist upstream are lost before emit — emission compensates
Expand Down Expand Up @@ -331,12 +331,14 @@ backends.
ExprVar → ExprCall for zero-arg function references (PR #329). All three
backends emit `name()` via existing ExprCall handling. `is_zero_arg_callable_ref`
dissolved (L1 33→32). Imported zero-arg function refs untested.
- [~] **TLC-2: Runtime bridge signature derivation.** Runtime helper
- [x] **TLC-2: Runtime bridge signature derivation.** Runtime helper
return types and wrapping conventions must derive from the same
type/coercion authority as emission. `wraps_result` field added to
`RuntimeFunction` registry (PR #331); `rt_wraps_result()` derives
from registry. Remaining: `rust_method_wraps_result` is a parallel
map for method templates — should unify into single spec authority.
from registry. `SimpleMethodSpec` type co-locates template + wrapping
flag for method templates (same pattern as `HigherOrderMethodSpec`).
Both `rust_method_templates()` and `rust_method_wraps_result()` now
derive from `rust_simple_method_specs` — single authority, no parallel maps.
- [x] **TLC-3: Indexing / character access semantics.** `IndexingSemantics`
in LanguageSpec — per-collection-type templates (list/map/string index/slice).
All three backends dispatch per collection type via shared
Expand All @@ -361,10 +363,52 @@ Make emission fully data-driven. Adding a backend = adding data.
with `method_name`, `inline_template`, `fn_ref_template`, `wraps_in_sharing`.
Shared `emit_rust_higher_order_method` replaces 4 hardcoded emitters
(filter/any/all/flat_map). Data-driven dispatch via method name lookup.
- [~] Transport/config: `TransportKind` enum + `classify_transport()` centralize dispatch; `ServiceFieldSet` + `compute_service_fields()` centralize field queries. Remaining per-backend sites are inherent rendering differences.
- [x] Transport/config: `TransportKind` enum + `classify_transport()` centralize dispatch; `ServiceFieldSet` + `compute_service_fields()` centralize field queries. Remaining per-backend rendering is inherent language differences (HTTP clients, shell runners, file I/O) — addressed by the 3→1 homomorphism.
- [ ] LanguageSpec completion — all target-language facts data-driven (see LS lane below)
- [x] TypeRendering dissolved — `render_node_type` consumes coercion data directly (PR #331)
- [ ] 3 backends → 1 parameterized homomorphism (~2,500 lines eliminated)
- [~] 3 backends → 1 parameterized homomorphism (PR #338).
**Phase 1-3 complete** (-206 .dag lines, -481 stage0 lines):
shared expr wrappers (`emit_expr_var_shared`, `emit_expr_field_access_shared`,
`extract_string_interp_parts`), shared typed handlers (`emit_typed_cast_shared`,
`emit_typed_index_shared`, `emit_typed_slice_shared`), ExprData accessors
(`expr_field_access_summary`, `expr_method_call_semantics`), dead code
deletion (`emit_*_typed_bin_op/cast/index/slice`).
**Phases 4-6 deferred** — blocked on depth/indent asymmetry (see below).
**Review feedback** (recorded, not yet addressed):
(a) Unify `SimpleMethodSpec` + `HigherOrderMethodSpec` into shared
`MethodTemplateSpec` to reduce schema drift across method families.
(b) Method names could be a structural enum (M4 direction) rather than
strings for clearer algebraic dispatch across language extdeps.

### Depth/indent asymmetry (cross-cutting blocker)

Go renders sub-expressions at `depth: 0` and manually prepends
`make_indent(level: depth)` at the wrapper level. Python threads
`depth` through all recursive calls. This fundamental difference
pervades every handler and prevents clean parameterization of:
- TCO handlers (~10 functions per backend, ~130 lines potential)
- Method call dispatch (~4 functions per backend, ~80 lines potential)
- Block statement emitters (scope-threading + depth)
- `emit_typed_let`, `emit_typed_for_each`

**Resolution options:**
1. **Align Go to Python's strategy.** Go's `wrap_result` callback in
`emit_shared_expr` already handles indentation for simple expressions.
If Go switched to `depth: depth` recursion and removed per-wrapper
`prefix` wrapping, the two backends would become structurally
identical. Risk: changes Go's emitted output formatting (whitespace
only, not semantics).
2. **Add `DepthMode` to shared layer.** Shared handlers take a depth
resolver: `sub_depth: fn(Int) -> Int` where Go passes `d => 0` and
Python passes `d => d`. More complex but preserves current behavior.
3. **Accept current state.** -206 lines is the practical limit without
resolving this asymmetry. The remaining ~210 lines of savings may
not justify the design cost.

**LanguageSpec extensions needed** (once depth is resolved):
- `async_call_prefix: String` — Python: `"await "`, Go/Rust: `""`
- `TcoSyntax` — loop_open, loop_close, break_keyword, continue_str,
stmt_terminator (data-driven TCO formatting)

### Coercion infrastructure (reference)

Expand All @@ -385,10 +429,10 @@ Remaining parallel authorities:
than Copy (excludes Unit; PR #333)
- [x] Dead code: deleted `try_target_primitive_type`, `target_primitive_type`,
`target_container_template`, `is_value_type`, `emit_primitive_type` + unused imports
- [~] Runtime bridge wrapping: `wraps_result` field on `RuntimeFunction` registry
(map_keys, map_values, append data-driven; method templates split/enumerate/chars/skip/take
use `rust_method_wraps_result` data map — consumer still reads parallel maps
instead of a single spec type)
- [x] Runtime bridge wrapping: `wraps_result` field on `RuntimeFunction` registry
(map_keys, map_values, append) and `SimpleMethodSpec.wraps_result` for method
templates (split/enumerate/chars/skip/take). Both derived maps come from their
respective single authorities — no parallel maps.

### Acceptance

Expand Down Expand Up @@ -1081,7 +1125,7 @@ distinction as structural data.
- [ ] **BP-1b: Bootstrap cycle.** Model the self-compile cycle: stage0
as bootstrap seed, the fixed-point condition (pass N == pass N+1),
and the two-pass rule (required when emitter changes its own output).
Hand-maintained files (9) enumerated as source artifacts distinct
Hand-maintained files (2) enumerated as source artifacts distinct
from generated artifacts.
- [ ] **BP-1c: `.gitattributes` generation.** Files the pipeline model
marks as "generated" get `-merge` in `.gitattributes`. The
Expand Down Expand Up @@ -1116,6 +1160,16 @@ distinction as structural data.
- **Cost algebra extensions**: Pow, Sqrt, Exp; amortized analysis via
potential functions; space as peer dimension. See
`docs/cost-algebra.md`.
- **Style emission**: Emission formatting (indentation, braces,
statement terminators) should be data-driven from two authorities:
(1) Language spec — significant whitespace (Python), block delimiters,
statement terminators. These are correctness requirements.
(2) Style spec — indent unit, brace placement, readability conventions.
These are readability requirements (correctness during bootstrap,
optional post-bootstrap via external formatters like gofmt/rustfmt).
`BlockSyntax` on `LanguageSpec` is the first step. Future work:
TCO syntax templates, async call prefix, identifier escaping
conventions, post-emission formatting pass.
- **Post-Rust path**: .dag → native code (LLVM/Cranelift) directly,
no Rust intermediate. Optional — Rust/Go as intermediates may
suffice indefinitely.
44 changes: 32 additions & 12 deletions dsl/extdeps/languages/rust/emit.dag
Original file line number Diff line number Diff line change
Expand Up @@ -48,20 +48,40 @@ data rust_container_templates: Map<String, String> = {
// Named-placeholder templates for simple algebra method calls.
// Templates use {recv} (receiver) and {arg} (first argument).
// Applied via apply_named_template in shared emit.
data rust_method_templates: Map<String, String> = {
"count": "(\{recv\}.len() as i64)",
"join": "\{recv\}.join(&\{arg\})",
"split": "\{recv\}.split(&\{arg\}).map(|s| s.to_string()).collect::<Vec<_>>()",
"last": "\{recv\}.last().cloned()",
"first": "\{recv\}.first().cloned()",
"enumerate": "\{recv\}.iter().cloned().enumerate().map(|(i, v)| (i as i64, v)).collect::<Vec<_>>()",
"chars": "\{recv\}.chars().map(|c| c as i64).collect::<Vec<_>>()",
"skip": "\{recv\}.iter().cloned().skip(\{arg\} as usize).collect::<Vec<_>>()",
"take": "\{recv\}.iter().cloned().take(\{arg\} as usize).collect::<Vec<_>>()"
//
// Single authority: SimpleMethodSpec carries both the template and
// wraps_result flag. Derived maps feed LanguageSpec (template only)
// and Rust emitter (wrapping). Same pattern as HigherOrderMethodSpec.

type SimpleMethodSpec {
method_name: String
template: String
wraps_result: Bool
}

data rust_simple_method_specs: List<SimpleMethodSpec> = [
{ method_name: "count", template: "(\{recv\}.len() as i64)", wraps_result: false },
{ method_name: "join", template: "\{recv\}.join(&\{arg\})", wraps_result: false },
{ method_name: "split", template: "\{recv\}.split(&\{arg\}).map(|s| s.to_string()).collect::<Vec<_>>()", wraps_result: true },
{ method_name: "last", template: "\{recv\}.last().cloned()", wraps_result: false },
{ method_name: "first", template: "\{recv\}.first().cloned()", wraps_result: false },
{ method_name: "enumerate", template: "\{recv\}.iter().cloned().enumerate().map(|(i, v)| (i as i64, v)).collect::<Vec<_>>()", wraps_result: true },
{ method_name: "chars", template: "\{recv\}.chars().map(|c| c as i64).collect::<Vec<_>>()", wraps_result: true },
{ method_name: "skip", template: "\{recv\}.iter().cloned().skip(\{arg\} as usize).collect::<Vec<_>>()", wraps_result: true },
{ method_name: "take", template: "\{recv\}.iter().cloned().take(\{arg\} as usize).collect::<Vec<_>>()", wraps_result: true }
]

// Derived: template map for LanguageSpec consumption (backend-neutral).
fn rust_method_templates() -> Map<String, String> {
rust_simple_method_specs
|> fold(init: empty_map(), f: (acc, spec) => map_insert(acc, spec.method_name, spec.template))
}

data rust_method_wraps_result: Map<String, Bool> = {
"split": true, "enumerate": true, "chars": true, "skip": true, "take": true
// Derived: wraps_result map for Rust Rc-wrapping decisions.
fn rust_method_wraps_result() -> Map<String, Bool> {
rust_simple_method_specs
|> filter(s => s.wraps_result)
|> fold(init: empty_map(), f: (acc, spec) => map_insert(acc, spec.method_name, true))
}

// -- Reserved words ------------------------------------------------------
Expand Down
6 changes: 6 additions & 0 deletions src/v2/00_core.dag
Original file line number Diff line number Diff line change
Expand Up @@ -982,6 +982,9 @@ fn expr_var_name_at(texpr: Node, source_index: NewlineIndex?) -> String { author
fn field_access_base(texpr: Node) -> Node { expr_child_at(texpr: texpr, index: 0, role: "field access base") }
fn field_access_field(texpr: Node) -> String { texpr.name }
fn field_access_field_at(texpr: Node, source_index: NewlineIndex?) -> String { authored_name_at(source_index: source_index, node: texpr) }
fn expr_field_access_summary(texpr: Node) -> FieldSummary? {
match texpr.expr_data { ExprFieldAccess { summary: s } => s _ => none }
}

// -- ExprCall (D3: func dissolved into Node.name) --
fn expr_call_func(texpr: Node) -> String { texpr.name }
Expand All @@ -992,6 +995,9 @@ fn method_receiver(texpr: Node) -> Node { expr_child_at(texpr: texpr, index: 0,
fn method_arg_nodes(texpr: Node) -> List<Node> { texpr.children |> skip(1) }
fn expr_method_name(texpr: Node) -> String { texpr.name }
fn expr_method_name_at(texpr: Node, source_index: NewlineIndex?) -> String { authored_name_at(source_index: source_index, node: texpr) }
fn expr_method_call_semantics(texpr: Node) -> MethodSemantics? {
match texpr.expr_data { ExprMethodCall { method_semantics: ms } => ms _ => none }
}

// -- ExprLambda (D3: params dissolved into child Nodes after body) --
fn lambda_body(texpr: Node) -> Node { expr_child_at(texpr: texpr, index: 0, role: "lambda body") }
Expand Down
85 changes: 83 additions & 2 deletions src/v2/05_emit.dag
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ import v2.std.core {
}

import v2.compiler.infer_env { TypeEnv, TypeBinding, RecursiveVariantFieldWitness }
import v2.compiler.infer_types { rt_type, emit_map_has, node_is_collection, node_is_keyed_collection, node_is_element_collection }
import v2.compiler.infer_types { rt_type, emit_map_has, node_is_collection, node_is_keyed_collection, node_is_element_collection, normalize_access_type_node }
import std.types { is_container_type, container_to_algebra_name }
import v2.compiler.coercion { coerce_primitive_type, coerce_container_template, target_optional_template, target_callable }
import v2.compiler.infer_sigs { ResolvedFuncSig, ResolvedFuncEnv }
Expand All @@ -81,7 +81,7 @@ import v2.compiler.languages {
ImportRule,
TypeUsageTrigger, TraitImplTrigger, DeriveMacroTrigger,
ContainerUsageTrigger, AsyncUsageTrigger,
language_spec_for_target,
language_spec_for_target, is_string_like,
test_conventions_for_target,
target_keyword,
wrap_shared_type
Expand Down Expand Up @@ -1664,6 +1664,87 @@ fn emit_null_coalesce(l_str: String, r_str: String, target: RenderTarget) -> Str
}


// =========================================================================
// Shared expression handler helpers (CG-3: 3→1 homomorphism)
//
// These replace duplicated per-backend wrapper functions. Since
// emit_shared_expr already dispatches on ExprData, wrapper functions
// that re-match are redundant. Backends inline child extraction into
// closures; these helpers share non-trivial logic.
// =========================================================================

// Shared ExprVar emission — target-parameterized, no backend-specific logic.
fn emit_expr_var_shared(expr: Node, target: RenderTarget, source_index: NewlineIndex?) -> String {
let n = expr_var_name_at(texpr: expr, source_index: source_index)
if n == "none" { emit_keyword(key: "null", target: target) }
else if n == "true" || n == "false" { emit_keyword(key: n, target: target) }
else { emit_ident(name: n, target: target) }
}

// Shared ExprFieldAccess service-receiver check. Delegates to emit_field
// for the backend-specific field access rendering.
fn emit_expr_field_access_shared(expr: Node, target: RenderTarget, emit_field: fn(Node) -> String) -> String {
if is_typed_service_call_receiver(receiver: expr) {
match extract_typed_service_name(receiver: expr) {
Some { value: svc_name } => service_var_name(service_name: svc_name)
None => emit_field(expr)
}
} else {
emit_field(expr)
}
}

// Extract StringPart list from ExprStringInterp children.
fn extract_string_interp_parts(expr: Node) -> List<StringPart> {
expr.children |> map(child => match child.expr_data {
ExprLiteral { value: LitStr { value: text } } => Text { value: text }
_ => Interpolation { expr: arg_value(n: child) }
})
}

// Shared ExprCast — target-parameterized, recurse for sub-expression.
fn emit_typed_cast_shared(expr: Node, cast_target_node: Node, target: RenderTarget, recurse: fn(Node) -> String) -> String {
let expr_str = recurse(expr)
let ty_str = emit_node_type(n: cast_target_node, target: target)
concat(ty_str, "(", expr_str, ")")
}

// Shared ExprIndex — dispatches on string/map/list via LanguageSpec.indexing.
fn emit_typed_index_shared(base: Node, index: Node, target: RenderTarget, recurse: fn(Node) -> String) -> String {
let spec = language_spec(target: target)
let base_str = recurse(base)
let index_str = recurse(index)
let base_node = normalize_access_type_node(n: rt_type(n: base))
let is_map = node_is_keyed_collection(n: base_node)
if is_string_like(target: target, name: base_node.name) {
apply_type_template2(template: spec.indexing.string_index, arg0: base_str, arg1: index_str)
} else if is_map {
apply_type_template2(template: spec.indexing.map_index, arg0: base_str, arg1: index_str)
} else {
apply_type_template2(template: spec.indexing.list_index, arg0: base_str, arg1: index_str)
}
}

// Shared ExprSlice — dispatches on string/list via LanguageSpec.indexing.
fn emit_typed_slice_shared(base: Node, start: Node, end: Node, target: RenderTarget, recurse: fn(Node) -> String) -> String {
let spec = language_spec(target: target)
let base_str = recurse(base)
let start_str = recurse(start)
let end_str = recurse(end)
let base_node = normalize_access_type_node(n: rt_type(n: base))
if is_string_like(target: target, name: base_node.name) {
match spec.indexing.string_slice {
Some { value: tmpl } => apply_type_template3(template: tmpl, arg0: base_str, arg1: start_str, arg2: end_str)
None => emit_error_expr(message: "unsupported string slice for target", target: target)
}
} else {
match spec.indexing.list_slice {
Some { value: tmpl } => apply_type_template3(template: tmpl, arg0: base_str, arg1: start_str, arg2: end_str)
None => emit_error_expr(message: "unsupported slice for target", target: target)
}
}
}

// =========================================================================
// emit_shared_expr — full shared ExprData dispatcher (P4.2 step 5)
//
Expand Down
Loading
Loading