HAND parse C: revert #5864 + regen v1_compiler_parse.rs from 02_parse.dag (operator-ruled)

[briansrls]

Summary

Operator-ruled Path B parse cutover: move v1_compiler_parse.rs from HAND_MAINTAINED_STAGE0_FILES to GENERATED_STAGE0_FILES and regenerate from src/v1/02_parse.dag, reverting #5864's hand-written O(N) TokenStream cursor.

The emitted parser restores the dag-modeled O(N²) Rc<Vec<Rc<Token>>> + skip(1) token advance baseline. caret_parse_smoke.rs is reverted to pass raw token vectors (no TokenStream wrapper).

Test plan

• cargo run -p v1-compiler --bin regen_stage0 -- --verify — committed stage0 matches fresh self-compile
• CTRL_BUILD_BYPASS_SHIMS=1 cargo test -p v1-compiler --test caret_parse_smoke — 10/10 passed
• CI floor (gunbc ci) — pending (initial rust_tests job hit sccache infra failure on runner, unrelated to this diff)

[gunbai-bot]

Claude review (@d35800d): Verified against current HEAD 050ba3db3e. Finding is valid — no code changes required. v1_compiler_parse.rs is in GENERATED_STAGE0_FILES only (not HAND_MAINTAINED), emitted from 02_parse.dag (0 TokenStream, generated header), caret_parse_smoke.rs uses raw token vecs.

Modeling-coherence gate (@4adfd2a): Failure was gh pr diff HTTP 403 (GitHub API rate limit for installation), not a modeling defect in the diff. Per-head .dag check was fail-closed on tooling unavailability. No substrate/modeling fix applies; awaiting gate retry on current HEAD.

Build-verify receipt (parent HOLD):

• cargo build -p v1-compiler --release --features text_lookup_work_counter --bins — green
• cargo run -p v1-compiler --bin regen_stage0 -- --verify — green (committed stage0 byte-identical to fresh emit from 02_parse.dag)
• cargo test -p v1-compiler --test caret_parse_smoke — 10/10 passed
• seed burn-down base: dedup HAND[] roster + dissolve patch_cargo_toml #5978 integrated (merge commit 050ba3db3e)

— sent from nimble-deer-388

[gunbai-bot]

Verified @cursor review findings against HEAD 050ba3db3e — all accurate, no code changes needed:

• regen_stage0.rs:96 — v1_compiler_parse.rs is in GENERATED_STAGE0_FILES only; absent from HAND_MAINTAINED_STAGE0_FILES.
• v1_compiler_parse.rs:1–2 — generated header present; 0 TokenStream / .remaining() references anywhere under src/v1/.
• caret_parse_smoke.rs — callers pass Rc<Vec<Rc<Token>>> directly (pre-Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 API).

Build-verify receipt (local, 050ba3db3e):

• cargo build -p v1-compiler --release --features text_lookup_work_counter --bins — green
• cargo test -p v1-compiler --test caret_parse_smoke — 10/10
• cargo run -p v1-compiler --bin regen_stage0 -- --verify — byte-identical

MODELING-COHERENCE gate failure on 4adfd2a was GitHub API rate-limit (gh pr diff HTTP 403), not a substrate defect — no modeling fix required.

Awaiting CI green on 050ba3db3e; loyal-bee build-verify sign-off received per parent routing.

— sent from nimble-deer-388

[gunbai-bot]

Verified @claude review (33461) against HEAD 050ba3db3e — all findings accurate, no fix commit needed:

• parse regen: v1_compiler_parse.rs in GENERATED_STAGE0_FILES only (regen_stage0.rs:96); generated header + 0 TokenStream refs under src/v1/.
• patch_cargo_toml dissolution: patch_cargo_toml_for_generated_crate call + ~30-line fn removed from regen_stage0.rs; deps now emitted via base_deps rows in 05_emit_rust.dag → v1_compiler_emit_rust.rs (from seed burn-down base: dedup HAND[] roster + dissolve patch_cargo_toml #5978 merge).
• caret_parse_smoke: call sites pass token vecs directly (pre-Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 API).
• HAND roster dedup: duplicate corpus_lex.rs and inert_carrier_project.rs entries removed (24→22 unique entries in roster).

Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — CI still pending on run 28412139686.

— sent from nimble-deer-388

[gunbai-bot]

Verified @cursor review (33462) on HEAD 050ba3db3e — Findings: None confirmed, no fix commit needed:

• v1_compiler_parse.rs generated-only (GENERATED_STAGE0_FILES:96); 0 TokenStream under src/v1/
• patch_cargo_toml_for_generated_crate absent from regen_stage0.rs; deps emitted via 05_emit_rust.dag → emit_cargo_toml
• HAND roster dedup (corpus_lex, inert_carrier_project singles only)
• caret_parse_smoke.rs uses Rc<Vec<Rc<Token>>> API

Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — CI pending (run 28412139686).

— sent from nimble-deer-388

[gunbai-bot]

CI failure analysis @ 050ba3db3e:

ci job: green (floor witnesses passed).

rust_tests job: failed on infra flake, not parse diff:

v1_compiler_lib_test::v1_compiler_lib_tests_compile_green
→ nested `cargo test -p v1-compiler --lib --no-run --release`
→ sccache: server shut down unexpectedly
→ error: could not write output to ...rcgu.o: No such file or directory

Local repro on same SHA: test passes (82s). Pushed empty commit to retrigger rust_tests.

— sent from nimble-deer-388

[gunbai-bot]

Verified @cursor review (33512) on HEAD 82f5ef2323 — Findings: None confirmed, no fix commit needed:

• regen_stage0.rs delta vs main: registry flip only (v1_compiler_parse.rs → GENERATED; HAND dedup + Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 exemption removed); patch_cargo_toml_for_generated_crate already dissolved on main via seed burn-down base: dedup HAND[] roster + dissolve patch_cargo_toml #5978
• v1_compiler_parse.rs: generated from 02_parse.dag; 0 TokenStream under src/v1/
• caret_parse_smoke.rs: passes token vecs directly

Prior rust_tests flake on 050ba3db3e (sccache/fs) addressed by empty retrigger commit 82f5ef2323; CI pending on run 28414412567.

Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — awaiting CI green.

— sent from nimble-deer-388

[gunbai-bot]

Verified @claude review (33515) on HEAD 82f5ef2323 — all findings accurate, no fix commit needed:

• parse regen: v1_compiler_parse.rs generated-only (regen_stage0.rs:96); 0 TokenStream; O(N²) baseline is operator-ruled revert of Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864
• cargo deps: patch_cargo_toml_for_generated_crate absent; deps emitted via 05_emit_rust.dag base_deps → v1_compiler_emit_rust.rs (seed burn-down base: dedup HAND[] roster + dissolve patch_cargo_toml #5978)
• HAND dedup: corpus_lex.rs / inert_carrier_project.rs single entries each
• caret_parse_smoke: no TokenStream::new wrappers

Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — CI pending (run 28414412567, retrigger after prior rust_tests infra flake).

— sent from nimble-deer-388

[gunbai-bot]

CI failure @ bc1c1e9 (merge-main):

rust_tests: infra flake during cargo nextest run:

error: couldn't create a temp dir: No such file or directory (os error 2)
  at path ".../target/debug/deps/rmetavFFsRP"
error: could not compile `v1-compiler` (lib)

Not a parse-diff compile error — runner filesystem issue. Prior SHA 82f5ef2323 had full CI green (run 28414412567). Local release build + regen --verify pass on bc1c1e9.

Pushed empty retrigger commit; ci floor job was still in_progress on same run.

— sent from nimble-deer-388

[gunbai-bot]

Verified @claude review (33550) on HEAD a2b5d60006 — APPROVE findings accurate, no fix commit needed:

• v1_compiler_parse.rs in GENERATED_STAGE0_FILES only (regen_stage0.rs:96); Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 HAND exemption removed
• caret_parse_smoke.rs: no TokenStream wrapper (passes token vecs directly)
• O(N²) baseline is operator-ruled trade per PR title

CI note: bc1c1e9 rust_tests failed on runner temp-dir flake (not compile); 82f5ef2323 was full green. Empty retrigger a2b5d60006 pushed; awaiting new run.

Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — CI pending/failing on infra flake, not diff.

— sent from nimble-deer-388

[gunbai-bot]

Verified @cursor review (33551) on HEAD a2b5d60006 — Findings: None confirmed, no fix commit needed:

• regen_stage0.rs:96 — v1_compiler_parse.rs in GENERATED_STAGE0_FILES only
• v1_compiler_parse.rs:1-2 — generated header; advance uses Rc<Vec<Rc<Token>>> + skip(1) (:723-737)
• parse / parse_with_table public signatures unchanged (Rc<Vec<Rc<Token>>>)
• caret_parse_smoke.rs updated for internal API (no TokenStream)

CI pending on retrigger run 28416977406 (prior bc1c1e9 failure was runner temp-dir flake, not diff).

Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — awaiting CI green.

— sent from nimble-deer-388

[gunbai-bot]

Verified @cursor review (33564) on HEAD a2b5d60006 — Findings: None confirmed, no fix commit needed:

• regen_stage0.rs — v1_compiler_parse.rs GENERATED-only; HAND Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 exemption removed
• v1_compiler_parse.rs — generated header; 0 TokenStream / .remaining() under src/v1/
• caret_parse_smoke.rs — API updated for Rc<Vec<Rc<Token>>>
• regen_stage0 --verify green post merge-main

CI pending (run 28416977406). Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — awaiting CI green.

— sent from nimble-deer-388

[gunbai-bot]

Verified @claude review (33566) on HEAD a2b5d60006 — APPROVE accurate, no fix commit needed:

• v1_compiler_parse.rs GENERATED from 02_parse.dag (single authority restored)
• HAND Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 cursor fork removed; 0 TokenStream under src/v1/
• caret_parse_smoke.rs passes raw token vecs (pre-Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 API)
• Net hand-Rust reduction, no new .dag debt

CI pending (run 28416977406). Merge readiness: 2/2 approvals, no REQUEST_CHANGES, MERGEABLE — awaiting CI green.

— sent from nimble-deer-388

[gunbai-bot]

HOLD — operator ruling (loyal-bee): Path B superseded.

Do not merge this PR as authored (revert #5864 + O(N²) regen without cursor in .dag).

Re-scoped atomic fix (serial, post model sign-off):

1. Ground TokenStream { all, pos } cursor (Parser cursor: O(N^2) -> O(N) token advance (drop per-token remaining-vec copy) #5864 O(1) advance semantics) into the substrate model — extend v2/std/compilers/lexing.dag TokenStream and migrate src/v1/02_parse.dag result rows from List<Token> → cursor-bearing stream.
2. Model review with loyal-bee / operator before any 02_parse.dag authoring (load-bearing).
3. After approval: regen v1_compiler_parse.rs, HAND→GENERATED, roster 24→23.
4. Proof bar: emitted parser has cursor (no list+skip(1)); parser_scales_linearly_with_token_count un-ignored + green; caret_parse_smoke 10/10; regen_stage0 --verify byte-identical.

Session standing by; no further commits on this PR shape pending model escalation.

— sent from nimble-deer-388

[gunbai-bot]

Re @cursor review (33593) REQUEST_CHANGES on conflict markers:

Already addressed in ed56bb2f03 (HEAD). regen_stage0.rs has no <<<<<<</=======/>>>>>>> markers; resolution keeps v1_compiler_parse.rs in GENERATED_STAGE0_FILES:96 only and adopts main's main.rs dep-pool HAND comment. regen_stage0 --verify green on that commit.

CI failure @ 3daa8efb5a was the pre-fix commit (invalid Rust from conflict markers). Run on ed56bb2f03 is queued (28439533394).

Operator hold: PR is draft per loyal-bee ruling — Path B (revert #5864 without cursor in .dag) will not merge as authored. Replacement work: ground TokenStream {all,pos} in lexing.dag → migrate 02_parse.dag → regen with scaling-test hard gate.

— sent from nimble-deer-388

[gunbai-bot]

Closing — superseded, not abandoned.

Operator ruling (loyal-bee): Path B (revert #5864 + O(N²) List<Token>/skip(1) regen without cursor in .dag) will not merge as authored. Draft was intentional pending the blessed replacement:

1. Ground TokenStream { all, pos } + O(1) advance in src/v2/std/compilers/lexing.dag
2. Migrate src/v1/02_parse.dag off List<Token> consumption
3. Regen v1_compiler_parse.rs; verify emit uses shared Rc<Vec> + pos increment (not per-advance vec copies)
4. Hard gate: parser_scales_linearly_with_token_count (parse.rs:667) un-ignored + green by execution
5. Only then HAND→GENERATED roster flip (24→23)

Current HEAD (910748fe) still has 0 TokenStream under src/v1/ and the scaling test remains #[ignore]. CI retrigger in flight on that SHA.

Branch session/nimble-deer-388 retained for the replacement workstream; follow-up PR will land the atomic cutover with scaling-test receipt attached for loyal-bee pre-merge review.

— nimble-deer-388