v2 stage0 independence — investigation (analysis-only, no implementation)

[briansrls]

v2 stage0 independence — investigation

Author: sunny-otter-371 (worker)
Brief: operator-direct 2026-05-19 via sunny-wolf-435 — "how do we NOT edit stage0 anymore"
Scope: investigation only; PR body is the deliverable

---

TL;DR

1. There is no working v2-side regeneration today. scripts/regenerate-stage0.sh and scripts/check-stage0-freshness.sh both exit 1 with a "retired" message (left over from T-V2-Retirement deleting src/v2/; v2 was restored in Restore src/v2 — comparison artifact + frozen-buildable #3148 but neither script was). The script-driven .dag → .rs propagation path that BOOTSTRAP.md documents does not currently exist.
2. The CI gate that should catch stage0 staleness is silently passing on every PR. .github/workflows/ci.yml:189 invokes cargo test … bootstrap_fixed_point -- --ignored --exact. With --exact, libtest filters against full test paths; the test's path is bootstrap::bootstrap_fixed_point, which does not match the bare filter — so the v2 job runs zero tests and exits clean. Independently flagged by swift-swift-37; corroborated here by inspection.
3. Even if the gate ran, it would panic on a missing v3 bin. All run_self_compile paths in src/v2/tests/src/bootstrap.rs:58-66 shell out to cargo run -p v3-compiler --bin emit_method_template_projection. That bin does not exist in src/v3/compiler/src/bin/ (deleted in 39ba75728; the v2 restore in Restore src/v2 — comparison artifact + frozen-buildable #3148 did not restore the bin). Affects bootstrap_fixed_point, ci_fixed_point, ci_freshness, bootstrap_stage0_to_stage1, strict_compile_diagnostic_count, and performance_ratchet.
4. Consequence in the field: PR v2 complexity.dag rework — well-modeled lens applying v4 modeling discipline (CALIBRATION investigation, brief at /tmp/v2_complexity_lens_brief.md) #3398 (deep-seal-431) had to hand-mirror ~277 lines of src/v2/stage0/src/v2_compiler_complexity.rs against the .dag change because there was no automatic regen path. This is exactly the gap the operator's framing is asking us to close.
5. The fix is already prototyped — in v3. src/v3/compiler/src/bin/regen_bootstrap.rs is the pattern (feature-gated regen bin, supports --verify for CI, writes snapshots on intentional regen). v2 needs the same shape, owned by v2 itself.

Recommended path: (D) "Replicate v3's regen_bootstrap pattern inside v2." Restore the regen mechanism by building a regen_stage0 v2-compiler bin (or thin wrapper) with --verify, wire CI to it, fix the gate filter, and unblock or stub the emit_method_template_projection dependency. Keep stage0 .rs files committed for fresh-checkout buildability. Rejects (A)/(B) binary-caching as inappropriate for this stage; (C) would regress.

The operator owns the call; this is feed-not-rule.

---

1. Current state — concrete mechanism

1.1 What's in the repo

• Stage0 is a workspace crate. Cargo.toml:10 lists src/v2/stage0; src/v2/stage0/Cargo.toml is a hand-maintained 14-line manifest. Building anything in the workspace triggers cargo to compile stage0 directly from the 65 .rs files in src/v2/stage0/src/ (94,927 lines total).
• Stage0 .rs files are committed. They all carry a // Generated by v2 compiler -- do not edit. header. Largest single file: v2_compiler_emit_rust.rs (16,015 lines). v2_compiler_complexity.rs is 9,545 lines.
• Four files in src/v2/stage0/src/ are hand-maintained (excluded from regen diff by diff_excluding_hand_maintained in bootstrap.rs:174-200): v2_interpreter.rs, cli_run.rs, rest_transport_facts.rs, and Cargo.toml. Everything else is supposed to be a regen artifact.

1.2 What the build does

• cargo build -p v2-compiler --release reads only the committed .rs files. No regen happens automatically. This is the path CI takes; this is the path local devs take.
• The compiler binary, once built, can self-emit: target/release/v2-compiler compile --source-root src/v2 --source-root dsl --output-dir <out> reads .dag + emits fresh .rs. But producing the binary in the first place requires the committed .rs files to be correct.

1.3 What's supposed to keep .dag and .rs in sync

• BOOTSTRAP.md documents ./scripts/regenerate-stage0.sh as the regen tool.
• The script is retired. scripts/regenerate-stage0.sh reads (verbatim, from the file):

  echo "ERROR: scripts/regenerate-stage0.sh is retired — the v2 stage0 tree no longer exists."
  exit 1
  

  Pointer is to docs/design-pure-bootstrap-zero.md and v3 regen bins, but those are v3-specific.
• scripts/check-stage0-freshness.sh is identically retired.
• BOOTSTRAP.md is therefore documenting a mechanism that no longer works. The doc-to-reality gap was set when v2 was deleted (39ba75728, T-V2-Retirement) and never closed when v2 was restored (fdaedb127, Restore src/v2 — comparison artifact + frozen-buildable #3148, 2026-05-15).

1.4 What CI actually runs for v2

.github/workflows/ci.yml:159-189 defines the v2 job, gated by needs.affected.outputs.v2 == 'true' (skipped unless src/v2/** is touched in the PR). When it runs, the only v2-specific check is:

cargo test -p v2-compiler-tests --release bootstrap_fixed_point -- --ignored --exact

Two independent reasons this gate produces zero signal today:

• Filter mismatch. --exact requires the filter to equal the full test path. The test is defined at src/v2/tests/src/bootstrap.rs:569 in module bootstrap (file-as-module), so its full path is bootstrap::bootstrap_fixed_point. The filter bootstrap_fixed_point does not match. libtest reports 0 filtered-in, exits 0, CI green.
• Missing dependency even if it ran. run_self_compile (bootstrap.rs:47-53) calls write_method_template_projection_generated_root (bootstrap.rs:55-75), which shells out to cargo run -p v3-compiler --bin emit_method_template_projection. That bin does not exist (ls of src/v3/compiler/src/bin/: gunbc_ci.rs, r1c_e_emit_gates.rs, regen_bootstrap.rs, regen_parse.rs, regen_parse_tables.rs, regen_tokenize.rs, regen_v3.rs, self_host_fixed_point.rs — no emit_method_template_projection). The deleted bin was a v3 artifact removed in T-V2-Retirement and never restored.

The "right" CI tests (ci_freshness and ci_fixed_point, defined bootstrap.rs:987-1014) are not invoked by ci.yml at all, and would hit the same missing-bin panic in their LazyLock initialization if they were.

1.5 Observed consequence

• PR v2 complexity.dag rework — well-modeled lens applying v4 modeling discipline (CALIBRATION investigation, brief at /tmp/v2_complexity_lens_brief.md) #3398 (deep-seal-431, v2 complexity.dag rework): src/v2/complexity.dag +250/-11, src/v2/stage0/src/v2_compiler_complexity.rs +277/-1. The worker had to hand-mirror the algorithmic change into the generated file because there is no automatic regen path. The hand-mirror is not verifiable against a fresh compile; it is a manually-asserted match.
• Since the 2026-05-15 restoration (fdaedb127) there are zero regen commits to src/v2/stage0/src/*.rs. Either the .dag has not changed in ways that require regen (lucky), or changes have silently diverged from stage0.

1.6 What v3 does (the existing reference)

v3 has the same conceptual problem (Rust seed compiling .dag source for the next pipeline) and has built the answer:

• src/v3/compiler/src/bin/regen_bootstrap.rs — bin that regenerates src/v3/compiler/src/bootstrap_*_generated.rs snapshots from .dag source.
• Cargo-feature-gated: Cargo.toml:14 bootstrap-regen-fresh = []; bin lists required-features = ["bootstrap-regen-fresh"] (Cargo.toml:60). The fresh-parse subgraph is kept out of the default library build graph so the bootstrap chicken-egg stays one-way.
• Supports --verify: CI runs cargo run -p v3-compiler --features bootstrap-regen-fresh --bin regen_bootstrap -- --verify (ci.yml:139) which fails the build if committed snapshots disagree with a fresh compile. Local dev runs the same bin without --verify to write updates.
• Registry: REGEN_OUTPUTS const in src/v3/compiler/build.rs:657, exposed at runtime as v3_compiler::generated_files::GENERATED_FILES, both the regen and the verify path enumerate from one source.

This pattern is the obvious template for v2.

1.7 What v4 does

v4 has no Rust pipeline yet (per src/v4/STRUCTURE.md + brief context). v4's bootstrap chain currently uses v2's compiled binary as stage-minus-one (ci.yml:248-252 runs target/release/v2-compiler compile --source-root src/v4 --target dag as a bootstrap-viability gate). v4's eventual zero-floor design (docs/design-pure-bootstrap-zero.md) is data-driven all the way down, but is not yet operational. v4 is not a near-term answer for v2.

---

2. Candidate paths

(A) Binary-cached bootstrap — vendor or remote-cache the v2-compiler binary; stop committing .rs

Mechanism change. Drop src/v2/stage0/src/*.rs from the repo. CI fetches a tagged v2-compiler binary from artifact storage on every build; locally, cargo build would need a way to fetch + invoke the binary before its own build graph runs. Workspace member src/v2/stage0 becomes either (a) a synthetic crate populated by build.rs from the fetched binary's output, or (b) deleted entirely with v2-compiler distributed solely as a prebuilt binary.

What breaks.

• Fresh-checkout builds require network access to artifact storage. Today cargo build -p v2-compiler works offline; (A) doesn't.
• Cargo does not have first-class "fetch this binary before compilation" semantics; would need a build.rs shim that downloads + caches. Workspace ordering with cross-crate dependencies makes this fragile.
• v4 bootstrap viability (ci.yml:248-252) requires a real working v2-compiler binary — if v2's source ceases to be locally-buildable, the v4 gate has to fetch too.
• Tag/binary mismatch: any local .dag edit needs the binary updated before stage0 can rebuild; round-trip latency is higher than today's local-build.

Rollback. Reverting requires recovering the deleted .rs files (from git history of a prior commit) and re-adding them to source control. Tractable but disruptive.

Effort tier. XL. Touches Cargo workspace shape, CI fetch infrastructure, fresh-checkout developer flow, and the v4 viability gate. Not the right move for "stage0 churn is annoying."

Verdict. Reject for v2 today. The problem is not stage0's existence in repo — it's that there's no working regen. Removing stage0 entirely solves the wrong problem and creates several new ones.

(B) Self-host via prior binary — CI fetches a tagged binary and regenerates locally per-build, .rs still in repo

Mechanism change. Same fetch mechanism as (A), but .rs files stay committed; CI regenerates them every run and diffs against the committed copy as the freshness gate. Local dev can either fetch the binary (online) or build from committed .rs (offline, current behavior).

What breaks.

• Same artifact-storage dependency for CI as (A).
• Adds a regen-every-build CI cost (build the binary OR fetch + run a full self-compile every PR, even on PRs that don't touch v2).
• Does not address the "stage0 diff is much larger than the .dag diff" PR-UX complaint at all — the regenerated .rs still gets committed.

Rollback. Drop the fetch step, restore the build-from-source path.

Effort tier. L. Less disruptive than (A) but with similar dependency-shape additions and no observable PR-UX improvement.

Verdict. Reject. Adds infrastructure without solving the documented operator complaint.

(C) Pure declaration mode — .dag becomes spec, .rs becomes hand-maintained

Mechanism change. Drop the self-host claim; the .dag files in src/v2/ become documentation/spec, and the .rs files in src/v2/stage0/src/ become the real, hand-maintained compiler.

Verdict. Strict regression from v2's documented "1-residual hand-Rust" achievement; contradicts THESIS.md and the no-Rust-hand-edits standing rule the parallel calibration workers are operating under. Listed only for completeness. Reject.

(D) Replicate v3's regen_bootstrap pattern inside v2 (RECOMMENDED)

Mechanism change. Build a regen_stage0 bin owned by v2-compiler that mirrors v3's regen_bootstrap:

1. A regen bin in src/v2/stage0/src/bin/regen_stage0.rs (or a sibling crate src/v2/regen/) that:
   • Builds the current v2-compiler in-process (or shells out to cargo build -p v2-compiler --release).
   • Self-compiles src/v2 + dsl to a temp dir.
   • With no flags: copies the temp output over src/v2/stage0/src/, runs cargo fmt, exits 0.
   • With --verify: diffs the temp output against src/v2/stage0/src/ (excluding the four hand-maintained files), exits non-zero on drift with a unified diff in stderr.
2. A regen output registry — explicit list of which .rs files under src/v2/stage0/src/ are regen-owned vs hand-maintained. Single source of truth, consumed by both the regen step and the --verify diff. Mirrors v3's REGEN_OUTPUTS const.
3. CI wiring in ci.yml's v2 job: replace bootstrap_fixed_point -- --ignored --exact with cargo run -p v2-compiler --bin regen_stage0 -- --verify. Failure means committed stage0 disagrees with what v2-compiler would produce from .dag.
4. Fix the emit_method_template_projection dependency. Either restore the v3 bin, stub it inside v2 so v2's tests don't reach into v3, or remove the dependency entirely (the harness shouldn't need a v3 bin to run a v2 self-compile). This is a separate piece of work but is a prerequisite for any test in bootstrap.rs that calls run_self_compile to even start.
5. Update BOOTSTRAP.md to reflect the actual mechanism (this PR or a follow-up).
6. Keep .rs files committed for fresh-checkout buildability — the regen mechanism is what guarantees they stay correct, not what replaces them.

What breaks.

• The PR that lands the regen bin necessarily also lands a fresh regen of stage0 (the current state will likely show drift). That commit is large (potentially many of the 94,927 lines reformatted/reordered/corrected) but is one-time.
• Workers landing .dag changes must run cargo run -p v2-compiler --bin regen_stage0 before pushing. CI's --verify will catch misses, so the consequence of forgetting is a CI failure, not silent drift.
• The emit_method_template_projection work is a prerequisite blocker for any test that runs the harness end-to-end — but the regen bin itself does not need that harness; it can run an in-process or subprocess self-compile directly against src/v2 + dsl, with the same source-root contract v2's main compile subcommand already uses.

Rollback. Remove the regen bin + CI step + revert BOOTSTRAP.md. Repository state is unchanged; only the workflow file and the new bin file would need to be reverted.

Effort tier. M. New bin is ~50-150 lines (most logic already exists in v2-compiler's compile subcommand and bootstrap.rs's diff helpers). Wiring CI is one-line. The prerequisite emit_method_template_projection cleanup is independently scoped (S/M depending on whether we delete the dependency or restore the bin). One-time stage0 regen commit is mechanical.

Verdict. Matches the operator's framing. Closes the immediate "stage0 churn is manual" complaint (regen is automatic; the diff still exists but is mechanically produced and verifiable). Replaces a documented-but-broken mechanism with a documented-and-working one. Uses a pattern that already exists in v3.

(E) Generate-on-build via build.rs (mentioned, not recommended)

A more aggressive variant of (D): use build.rs to run regen during every cargo build -p v2-compiler. Avoids committed stage0 drift entirely — but creates a circular dependency (the binary that regenerates the crate is the crate being built), forcing a multi-pass build with the same chicken-egg regen_bootstrap.rs is feature-gated to avoid. The complexity gain over (D)'s explicit invocation is not worth the build-graph cost.

Verdict. Mention only. (D) is cleaner.

---

3. Preconditions to verify before committing to (D)

• The --exact filter mismatch on bootstrap_fixed_point is the actual cause of zero filtered-in tests — claim is from swift-swift-37 + this analysis's inspection; running cargo test -p v2-compiler-tests --release bootstrap_fixed_point -- --ignored --exact 2>&1 | tail and observing "0 filtered out" would close it. Did not run here (test path is bootstrap::bootstrap_fixed_point per the file's directory layout, but unverified by execution).
• The emit_method_template_projection v3 bin truly is unused outside the broken harness — verified by ls; needs grep across src/v3/ to confirm no other consumer depends on it before deletion vs restoration is decided.
• v2's existing compile subcommand entry point can be invoked in-process by a sibling bin in the same crate — src/v2/stage0/src/main.rs uses clap and exits via std::process::exit. A regen bin would either invoke v2 as a subprocess (simpler, no API risk) or call the library functions directly (v2_compiler::v2_compiler_compile::compile_sources). Subprocess invocation is the conservative starting point and matches how bootstrap.rs already does it.
• The four hand-maintained files (v2_interpreter.rs, cli_run.rs, rest_transport_facts.rs, Cargo.toml) remain hand-maintained or get a dissolution plan. The brief is "stop editing stage0 anymore" — hand-maintained files are a remaining edit surface. Out of scope for this investigation; flag as follow-up.
• The one-time regen commit's blast radius is acceptable to the operator. If the current committed .rs files have drifted from what .dag would produce, landing (D) means a large initial diff. Operator may want that staged or scoped.

---

4. Open questions for the operator

1. Restore vs replace emit_method_template_projection. Is this v3 bin worth restoring, or should v2's test harness be cut free of it entirely? Restoring is mechanical (find the deleted file in git history, re-add); cutting v2 free is cleaner long-term but requires identifying what the bin produced and whether v2's tests still need it.
2. What does the operator want done about the silently-passing CI gate in the interim? Even before (D) lands, the broken --exact filter + missing bin should be either fixed in a small standalone PR (fix the filter, fix or remove the v3 dependency) or explicitly marked as a known gap with a tracking work item.
3. Hand-maintained files in stage0. The four files excluded from the regen diff are themselves still hand-edited. Are they in scope for "stop editing stage0 anymore"? They are not generated; they are stage0's small irreducible Rust seed. Pure-bootstrap-zero (v4's framing) would eventually drive them to zero, but that's a v4 program.
4. Where does BOOTSTRAP.md go. It currently describes a mechanism that doesn't exist. Either it gets corrected as part of (D), or it gets retired with a pointer to whatever v3-style doc the operator wants v2 to share.
5. Calibration outcome. This investigation suggests (D) is unambiguously the right shape. If the operator agrees, this can be handed to an implementation worker; if not, the framing assumptions in §1 and §2 are the right place to push back.

---

5. Honesty bar / what I did not verify

• Did not execute cargo test -p v2-compiler-tests bootstrap_fixed_point -- --ignored --exact to empirically confirm the libtest filter behavior. The claim is structural and aligns with libtest documentation, but the live observation is from swift-swift-37, not from this session.
• Did not run any cargo build or self-compile during this investigation (workspace was clean; investigation-only mandate).
• Did not exhaustively grep src/v3/ for consumers of emit_method_template_projection — only confirmed the bin file's absence.
• All line-count and file-presence claims are from wc -l, ls, and direct Read of the cited files in this session.
• The recommendation in §0 / §2.4 (D) is feed-not-rule; the operator owns the call.

---

6. Sibling-worker findings folded in

From the brief:

• deep-seal-431, PR v2 complexity.dag rework — well-modeled lens applying v4 modeling discipline (CALIBRATION investigation, brief at /tmp/v2_complexity_lens_brief.md) #3398 — hand-mirrored 277 lines of stage0 .rs against a .dag change; cited as the immediate field consequence (§1.5).
• swift-swift-37 + cool-bee-832 — silent CI pass via --exact filter mismatch + missing v3 bin; folded into §1.4.
• crisp-dove-396 — v2 substrate primitive gap (no zip). Noted for completeness; does not directly affect the regen mechanism but is a related "v2's substrate is incomplete for the modeling discipline we want to apply" data point. Out of scope for stage0 independence.

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: e54c12f7 · Trigger: schedule
• Thinking: 90s wall

---

✅ The provided PR diff is empty, so there are no changed file lines or PR-introduced invariant violations to block.

[briansrls]

Closing per operator wrap-up directive 2026-05-20.