PERF

ROADMAP.md

@@ -56,13 +56,22 @@ when it self-compiles (fixed point convergence). **Blocks all other
 lanes from editing stage0 Rust directly.**
 
 Note: "zero manual patches" means zero patches to *generated* files.
-Nine hand-maintained files are still copied during regeneration
-(main.rs, v2_rt.rs, compiler_tests.rs, extdeps_languages_dag_syntax.rs,
-v2_coercion.rs, v2_compiler_infer_method.rs, std_types.rs,
-v2_compiler_tokenize.rs, extdeps_languages_rust_emit.rs).
-The last five are thread-local cache shims for performance;
-eliminating these is tracked as future work under M5-full
-(language plugin extraction).
+Four hand-maintained files are still copied during regeneration
+(main.rs, v2_rt.rs, compiler_tests.rs, v2_coercion.rs).
+Goal is zero — all stage0 content should be 100% generated.
+Remaining files and what blocks elimination:
+
+| File | Role | Blocker |
+|------|------|---------|
+| `main.rs` | CLI entrypoint | Need .dag entrypoint construct |
+| `v2_rt.rs` | Runtime shims (Rc ops, string ops) | Need .dag runtime codegen or inline emit |
+| `compiler_tests.rs` | Test harness | Need .dag test generation |
+| `v2_coercion.rs` | Coercion registries | No .dag counterpart yet |
+
+Previously eliminated (PR #316): v2_compiler_infer_method.rs, std_types.rs,
+extdeps_languages_rust_emit.rs, v2_compiler_tokenize.rs,
+extdeps_languages_dag_syntax.rs — via thread_local! caching in emitter
+and `chars_to_string` runtime function for O(1) tokenizer substring.
 
 ## CI Gates
 
@@ -85,6 +94,33 @@ eliminating these is tracked as future work under M5-full
 | Emitted Rust errors | 0 | 0 | GREEN |
 | DSL complexity ratchet | 2 | 0 | stack_size + fold_stack (deferred to CX lane) |
 
+## Bootstrap Dependency Chain
+
+```
+.dag source ──(v2-compiler)──▶ stage0 .rs ──(cargo/rustc)──▶ v2-compiler binary
+     ▲                                                              │
+     └──────────────────────────────────────────────────────────────┘
+```
+
+- **Source of truth:** `.dag` files. Stage0 `.rs` is a derived artifact.
+- **Cycle-breaker:** Committed stage0 allows fresh clones to bootstrap.
+- **Fixed-point:** `regen pass N == regen pass N+1`. Two-pass bootstrap
+  required when the emitter changes its own output.
+- **External dependencies:** cargo, rustc (opaque transforms, not modeled).
+- **Hand-maintained files (9):** Copied back during regen, not overwritten.
+  These are source, not derived. See Bootstrap Status for list.
+
+**Merge workflow problem:** Stage0 `.rs` files are derived, but git
+treats them as text and produces line-level merge conflicts. These
+conflicts carry zero information — the only correct resolution is
+regen. Fix: `.gitattributes` marks generated stage0 files as `-merge`
+(no line-level merge), CI freshness gate ensures committed stage0
+matches `.dag` source. Merge workflow becomes: resolve `.dag` conflicts
+→ accept either side of stage0 → regen → commit.
+
+This dependency chain is currently implicit (shell scripts + convention).
+Modeling it as `.dag` types is tracked under BP-1 (Layer 3).
+
 ## Bootstrap Health Rules
 
 Clean-repo workflow:
@@ -97,6 +133,7 @@ Clean-repo workflow:
 Stabilization rules:
 - No manual `src/v2/stage0/` edits once regeneration is green.
 - CI gate: `check-stage0-freshness.sh` (regenerate → diff → empty).
+- Stage0 generated `.rs` files use `-merge` in `.gitattributes` (no line-level merge).
 - CX gate disabled in both stage0 and `compile.dag` — emission not blocked by complexity violations. Re-enable when CX violations reach 0.
 
 ## Reviewer Root Cause Analysis (2026-04-03)
@@ -587,6 +624,76 @@ Missing std/ concepts: `std/discrimination.dag` (pattern matching),
   `MatchPattern`, `InferredNode` wrapper — all become graph structure
   or metadata on Node.
 
+## BP-1: Build Pipeline Model
+
+The full compilation pipeline — `.dag` source → intermediate
+representation (Rust/Python/Go) → executable artifact — is a
+dependency DAG that should be modeled in `.dag`. Each stage has an
+artifact type, a transform tool, and its own dependencies. This is
+the same problem the compiler solves for user code: resolve a
+dependency graph, detect cycles, determine evaluation order.
+
+```
+.dag source ──(v2-compiler)──▶ .rs/.py/.go ──(cargo/python/go)──▶ binary/runtime
+     ▲                              │
+     └──────── bootstrap cycle ─────┘  (Rust only: stage0 self-compile)
+```
+
+### Existing infrastructure
+
+Pieces already modeled:
+
+| Concept | Where | What it captures |
+|---------|-------|-----------------|
+| Intermediate targets | `src/v2/artifact.dag` `RenderTarget` | Rust, Python, Go, Dag |
+| Artifact taxonomy | `src/v2/artifact.dag` `ArtifactKind` | ServiceBinary, Library, Frontend, GeneratedSupport |
+| Per-target scaffold | `src/v2/languages.dag` `ProjectScaffold` | manifest file, source dir, extension |
+| Cargo as tool | `dsl/extdeps/cargo.dag` | packages, targets, profiles, Build/Test/Clippy ops |
+| Workspace graph | `dsl/extdeps/gunbc.dag` | `CrateRole`, `WorkspacePackage`, 13 packages |
+| Build phases | `dsl/tools/build.dag` | multi-stage cargo pipeline with result aggregation |
+| Codegen execution | `dsl/tools/codegen.dag` | stamp-file-gated conditional codegen |
+| Output paths | `dsl/config/codegen_paths.dag` | output dirs, stamp files, path templates |
+| Pipeline runs | `dsl/gunbc/workflow/types.dag` | `PipelineRun`, `PipelineArtifact`, stage outcomes |
+
+What's missing: the **derivation chain** connecting these — which
+artifact is produced from which source by which tool, the bootstrap
+cycle and its fixed-point condition, and the generated-vs-source
+distinction as structural data.
+
+### Work items
+
+- [ ] **BP-1a: Derivation model.** Types for pipeline stages, derivation
+  edges (source artifact → tool → output artifact), and external tool
+  dependencies. Compose existing `RenderTarget`, `ProjectScaffold`,
+  cargo service, and `ArtifactKind` into an explicit derivation DAG.
+  Location: `dsl/std/pipeline.dag` or extend `dsl/extdeps/gunbc.dag`.
+- [ ] **BP-1b: Bootstrap cycle.** Model the self-compile cycle: stage0
+  as bootstrap seed, the fixed-point condition (pass N == pass N+1),
+  and the two-pass rule (required when emitter changes its own output).
+  Hand-maintained files (9) enumerated as source artifacts distinct
+  from generated artifacts.
+- [ ] **BP-1c: `.gitattributes` generation.** Files the pipeline model
+  marks as "generated" get `-merge` in `.gitattributes`. The
+  `.gitattributes` file is a derived artifact from the pipeline model,
+  not hand-maintained config. Analogous to how `dsl/tools/bootstrap.dag`
+  generates `.gitignore` from `dsl/config/gitignore.dag`.
+- [ ] **BP-1d: Merge workflow.** Document and enforce: resolve `.dag`
+  conflicts → accept either side of generated stage0 → regen → commit.
+  CI freshness gate (`check-stage0-freshness.sh`) is the safety net.
+
+### Acceptance
+
+1. The derivation chain from `.dag` source through intermediate
+   representation to executable is explicit `.dag` data, not
+   shell-script convention.
+2. `.gitattributes` is generated from the pipeline model. Generated
+   files have `-merge`; hand-maintained files do not.
+3. Every file in `src/v2/stage0/src/` is classified as either
+   generated or hand-maintained by the model.
+4. The bootstrap cycle, fixed-point condition, and two-pass rule are
+   structural facts in the model.
+5. Stage0 merge conflicts no longer require manual resolution.
+
 ## Exploratory Directions
 
 - **Bounded iteration**: three primitives — fold (structural descent

dsl/extdeps/languages/rust/emit.dag

@@ -161,7 +161,8 @@ data rt_function_registry: List<RuntimeFunction> = [
   { name: "map_contains_key",   bridge_name: "map_contains_key", passes_by_ref: true },
   { name: "map_has",            bridge_name: "map_has",          passes_by_ref: true },
   { name: "reverse",            bridge_name: "reverse",          passes_by_ref: false },
-  { name: "replace",            bridge_name: "replace",          passes_by_ref: false }
+  { name: "replace",            bridge_name: "replace",          passes_by_ref: false },
+  { name: "chars_to_string",    bridge_name: "chars_to_string",  passes_by_ref: true }
 ]
 
 // Helper functions: O(1) lookup via indexed maps built from the registry.
@@ -185,7 +186,8 @@ data rt_functions: Map<String, Bool> = {
   "filesystem_read": true, "list_push": true, "map_get": true,
   "map_keys": true, "map_values": true,
   "parse_int": true, "map_contains_key": true, "map_has": true,
-  "reverse": true, "replace": true
+  "reverse": true, "replace": true,
+  "chars_to_string": true
 }
 
 // Derived from rt_function_registry — single authority for passes_by_ref.

scripts/regenerate-stage0.sh

@@ -62,31 +62,11 @@ $STAGE0_CMD compile \
 #   main.rs: FF-9 import resolution, CLI entrypoint
 #   v2_rt.rs: runtime shims (Rc::make_mut, string ops, etc.)
 #   compiler_tests.rs: test harness
-#   extdeps_languages_dag_syntax.rs: thread_local cache for dag_syntax_spec
-#     and dag_keyword_set (called per-token in parser/tokenizer hot loops;
-#     without cache each call reconstructs full data structures. Remove
-#     when emitter supports data-def memoization.)
-#   v2_compiler_infer_method.rs: thread_local cache for builtin_function_registry
-#     (called per-expression during type inference; without cache each call
-#     reconstructs 27-entry HashMap with O(n^2) Rc clone overhead.)
-#   std_types.rs: thread_local cache for kernel_type_set and container_type_set
-#     (called per-node in infer/resolve/emit; without cache each call
-#     reconstructs 8 and 5 HashMap entries respectively.)
-#   v2_compiler_tokenize.rs: thread_local cache for single_punct
-#     (called per-token in scan_token; without cache reconstructs 11 entries.)
-#   extdeps_languages_rust_emit.rs: thread_local cache for rt_functions,
-#     rt_ref_map_functions, rt_bridge_function_names, rust_type_map,
-#     rust_keywords, rust_container_templates (called per-expression/type
-#     in emit; rt_ref_map_functions also eliminates JSON deser + O(n^2) fold.)
+#   v2_coercion.rs: lazy_static coercion registries (no .dag counterpart yet)
 cp "$STAGE0_DIR/src/v2_rt.rs" "$OUTPUT_DIR/src/v2_rt.rs" 2>/dev/null || true
 cp "$STAGE0_DIR/src/compiler_tests.rs" "$OUTPUT_DIR/src/compiler_tests.rs" 2>/dev/null || true
 cp "$STAGE0_DIR/src/main.rs" "$OUTPUT_DIR/src/main.rs" 2>/dev/null || true
-cp "$STAGE0_DIR/src/extdeps_languages_dag_syntax.rs" "$OUTPUT_DIR/src/extdeps_languages_dag_syntax.rs" 2>/dev/null || true
 cp "$STAGE0_DIR/src/v2_coercion.rs" "$OUTPUT_DIR/src/v2_coercion.rs" 2>/dev/null || true
-cp "$STAGE0_DIR/src/v2_compiler_infer_method.rs" "$OUTPUT_DIR/src/v2_compiler_infer_method.rs" 2>/dev/null || true
-cp "$STAGE0_DIR/src/std_types.rs" "$OUTPUT_DIR/src/std_types.rs" 2>/dev/null || true
-cp "$STAGE0_DIR/src/v2_compiler_tokenize.rs" "$OUTPUT_DIR/src/v2_compiler_tokenize.rs" 2>/dev/null || true
-cp "$STAGE0_DIR/src/extdeps_languages_rust_emit.rs" "$OUTPUT_DIR/src/extdeps_languages_rust_emit.rs" 2>/dev/null || true
 
 # --output-dir mode: leave output in place, caller handles it
 if [ -n "$OUTPUT_ONLY" ]; then

src/v2/01_tokenize.dag

@@ -125,7 +125,7 @@ fn source_len(source: SourceRef) -> Int {
 }
 
 fn source_substring(source: SourceRef, start: Int, end: Int) -> String {
-  substring(s: source.text, start: start, end: end)
+  chars_to_string(chars: source.source_chars, start: start, end: end)
 }
 
 fn source_scan_while(source: SourceRef, start: Int, pred: fn(String) -> Bool) -> Int {
@@ -163,44 +163,46 @@ fn tokenize(source: String, file: String) -> List<Token> {
 
 // === Main loop ===
 
-fn tokenize_loop(source: SourceRef, tokens: List<Token>, pos: TokPos, fuel: Int) -> TokenizerState {
-  let s = skip_spaces_and_comments(source: source, pos: pos, fuel: fuel)
-  if s.pos >= source_len(source: source) {
-    return TokenizerState { pos: s.pos, tokens: tokens, interp_depth: s.interp_depth }
-  }
-  let ch = source_char(source: source, pos: s.pos)
+// Dispatch newlines, interpolation-close, and normal tokens to a ScanResult.
+// Factored out so tokenize_loop has exactly one tail call (TCO-friendly).
+fn scan_next_token(source: SourceRef, pos: TokPos) -> ScanResult {
+  let ch = source_char(source: source, pos: pos.pos)
 
   // Newlines are significant (statement separation)
   if ch == "\n" {
-    let tok = make_token(text: "\n", span: make_file_span(file: source.file, start: s.pos, end: s.pos + 1), shape: ShNewline)
-    return tokenize_loop(source: source, tokens: list_push(tokens, tok), pos: TokPos {
-      pos: s.pos + 1, interp_depth: s.interp_depth
-    }, fuel: fuel - 1)
+    return ScanResult {
+      pos: pos.pos + 1,
+      token: make_token(text: "\n", span: make_file_span(file: source.file, start: pos.pos, end: pos.pos + 1), shape: ShNewline),
+      interp_depth: pos.interp_depth
+    }
   }
 
   // When inside string interpolation and we hit }, check if this
   // closes the interpolation expression (top of depth stack == 0).
-  if ch == "}" && count(s.interp_depth) > 0 {
-    let top = last(s.interp_depth).value
+  if ch == "}" && count(pos.interp_depth) > 0 {
+    let top = last(pos.interp_depth).value
     if top == 0 {
-      // Pop the interpolation depth entry, advance past }, resume string scanning
-      let popped = drop_last(stack: s.interp_depth)
-      let cont_pos = TokPos {
-        pos: s.pos + 1, interp_depth: popped
-      }
-      let result = scan_str_cont(source: source, pos: cont_pos, span_start: s.pos)
-      return tokenize_loop(source: source, tokens: list_push(tokens, result.token), pos: TokPos { pos: result.pos, interp_depth: result.interp_depth }, fuel: fuel - 1)
+      let popped = drop_last(stack: pos.interp_depth)
+      let cont_pos = TokPos { pos: pos.pos + 1, interp_depth: popped }
+      return scan_str_cont(source: source, pos: cont_pos, span_start: pos.pos)
     } else {
-      // Nested brace inside interpolation expression -- decrement depth
-      let new_depth = replace_last(stack: s.interp_depth, value: top - 1)
-      let tok = make_token(text: "}", span: make_file_span(file: source.file, start: s.pos, end: s.pos + 1), shape: ShRBrace)
-      return tokenize_loop(source: source, tokens: list_push(tokens, tok), pos: TokPos {
-        pos: s.pos + 1, interp_depth: new_depth
-      }, fuel: fuel - 1)
+      return ScanResult {
+        pos: pos.pos + 1,
+        token: make_token(text: "}", span: make_file_span(file: source.file, start: pos.pos, end: pos.pos + 1), shape: ShRBrace),
+        interp_depth: replace_last(stack: pos.interp_depth, value: top - 1)
+      }
     }
   }
 
-  let result = scan_token(source: source, pos: s, ch: ch)
+  scan_token(source: source, pos: pos, ch: ch)
+}
+
+fn tokenize_loop(source: SourceRef, tokens: List<Token>, pos: TokPos, fuel: Int) -> TokenizerState {
+  let s = skip_spaces_and_comments(source: source, pos: pos, fuel: fuel)
+  if s.pos >= source_len(source: source) {
+    return TokenizerState { pos: s.pos, tokens: tokens, interp_depth: s.interp_depth }
+  }
+  let result = scan_next_token(source: source, pos: s)
   tokenize_loop(source: source, tokens: list_push(tokens, result.token), pos: TokPos { pos: result.pos, interp_depth: result.interp_depth }, fuel: fuel - 1)
 }
 

src/v2/04_emit_info.dag

@@ -101,6 +101,12 @@ type EmitGraphInfo {
   // bindings use move semantics instead of .clone(). Set for fold
   // accumulators after Rc::try_unwrap.
   owned_bindings: Map<String, Bool>
+  // Per-function fold accumulator unwrap eligibility: func_name → (acc_type_name → true).
+  // Built from structural proofs in ownership.dag. The emitter combines this
+  // with rc_types to decide Rc::try_unwrap rendering.
+  fold_eligible_index: Map<String, Map<String, Bool>>
+  // Current function's eligible fold accumulator types. Set by emit_typed_item.
+  fold_eligible: Map<String, Bool>
 }
 
 // TypeRendering: structural descriptor for type rendering.
@@ -142,7 +148,9 @@ fn empty_emit_graph_info() -> EmitGraphInfo {
     value_contexts: empty_map(),
     ownership_index: empty_map(),
     movable: empty_map(),
-    owned_bindings: empty_map()
+    owned_bindings: empty_map(),
+    fold_eligible_index: empty_map(),
+    fold_eligible: empty_map()
   }
 }
 

src/v2/04_infer.dag

@@ -3005,7 +3005,9 @@ fn build_emit_graph_info(modules: List<TypedModule>) -> EmitGraphInfo {
     value_contexts: value_ctxs,
     ownership_index: empty_map(),
     movable: empty_map(),
-    owned_bindings: empty_map()
+    owned_bindings: empty_map(),
+    fold_eligible_index: empty_map(),
+    fold_eligible: empty_map()
   }
 }
 

src/v2/04_method.dag

@@ -85,6 +85,7 @@ fn builtin_function_registry() -> Map<String, Node> {
   let m = map_insert(m, "char_at", string_type)
   let m = map_insert(m, "substring", string_type)
   let m = map_insert(m, "from_code_point", string_type)
+  let m = map_insert(m, "chars_to_string", string_type)
   let m = map_insert(m, "to_string", string_type)
   let m = map_insert(m, "concat", string_type)
   // Map-returning functions