[codex] retire pipeline include_str side channel ratchet

[briansrls]

Summary

• Remove the remaining include_str!("../../pipeline.dag") parse fixture from the L1.5 fixed-point integration test.
• Replace it with a structural bootstrap assertion that the pipeline authority declarations are loaded from the bootstrapped DAG.
• Broaden the bridge ratchet to scan both compiler src/**/*.rs and tests/**/*.rs for non-comment include_str! references to pipeline.dag.

Why

bridge_include_str_side_channels_retired is supposed to close the pipeline authority slice without a source-text side channel. The production path was already structural, but the integration test still had an active compile-time embed and explicitly allowed that exception.

P5 Receipt

• Hand-Rust census impact: no new src/v3/ production paths or substrate authority paths added.
• Test-surface impact: one existing Rust integration test is tightened to retire the pipeline.dag include_str! side channel; the new helper is a ratchet that fails closed over src/**/*.rs and tests/**/*.rs.
• Authority direction: pipeline ordering remains sourced through bootstrapped Dag structure, not duplicated through Rust string embedding.

Validation

• cargo fmt --check
• cargo test -p v3-compiler --test integration l1_5_fixed_point_test -- --nocapture

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 948d6584 · Trigger: schedule
• Thinking: 152s wall

---

Non-blocking — Strengths

• src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs The test-only change retires the direct pipeline.dag include_str! side channel and widens the ratchet to compiler tests while keeping pipeline authority structural through Dag::new() provenance.

ROADMAP — Verified

• T-PB-B / pb_rust_tests_outside_residual_zero: The PR reduces a Rust-authored test-side authority bridge rather than adding a new one, consistent with the Pure Bootstrap tests-as-data lane.

✅ No blocking concerns found.

[briansrls]

Verified the exploratory parser note against the current include_str_pipeline_dag_offenders loop. The ratchet is intentionally scoped to ordinary include_str!(...) Rust source shapes and has a focused self-test for split string literals; handling fully general Rust lexical structure would be broader than this bridge-retirement guard. No code change needed for this optional observation. — sent from neat-deer-474

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 17dd2d4a · Trigger: schedule
• Thinking: 183s wall

---

Non-blocking — Strengths

• src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs The changed test replaces the direct pipeline.dag include_str side channel with Dag::new-backed provenance checks and a widened ratchet, which preserves pipeline authority structurally.

ROADMAP — Verified

• T-PB-B / pb_rust_tests_outside_residual_zero: This retires a Rust test-side authority bridge rather than adding a new hand-Rust test path, consistent with the tests-as-data zero-residual lane.

✅ No blocking concerns; the change is implementation-local and aligns with P5 progress-as-dissolution.

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 7f50c5d1 · Trigger: schedule
• Thinking: 226s wall

---

Non-blocking — Improvements (fix in-PR if easy, else defer to roadmap)

• src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs Line 57 matches only contiguous include_str!, while Rust accepts whitespace/comments before !; a token-aware match would make this B4 bridge-retirement ratchet harder to bypass if deferred.

✅ No blocking concerns in the current diff.

[briansrls]

Addressed the codex REQUEST_CHANGES in commit 5c21923 by replacing the raw substring ratchet with code-position masking for strings, raw strings, char literals, line comments, and nested block comments. I also extended the scanner to token-match include_str with whitespace/comment trivia before ! and added regressions for inert text plus split/trivia macro spellings. Validation: cargo fmt --check; cargo test -p v3-compiler --test integration include_str_pipeline_dag_ratchet -- --nocapture; cargo test -p v3-compiler --test integration l1_5_fixed_point_test -- --nocapture. — sent from neat-deer-474

[briansrls]

For the optional scanner-scope note: agreed this is intentionally a lightweight Rust-source scanner, not a complete lexer. The intended coverage is now pinned by the focused regressions: it ignores strings/raw strings/chars/line comments/nested block comments, catches split string literals, and token-matches include_str with trivia before !. I’m leaving that as test-pinned scope rather than adding another code-only comment and resetting CI again. — sent from neat-deer-474

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 59f4bc49 · Trigger: schedule
• Thinking: 190s wall

---

BLOCKING (1)

Root Cause

• src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs hand-rolled Rust literal masking conflates lifetimes with char literals → use Rust tokenization or add lifetime-aware handling with a regression case before relying on this as the bridge-retirement receipt.

ROADMAP — Incomplete

• bridge_include_str_side_channels_retired: The PR moves the right authority toward structural loading, but the ratchet is not yet fail-closed for normal Rust syntax.

⚠️ One scanner false-negative needs fixing before this can serve as the retirement ratchet.

[briansrls]

Addressed the codex delimiter REQUEST_CHANGES in commit 0ac4ccc by extending the ratchet scanner to accept all valid Rust macro delimiters for include_str: (), [], and {}. The split-literal regression now covers paren, bracket, and brace spellings, including concat!("../../", "pipeline", ".dag"). Validation: cargo fmt --check; cargo test -p v3-compiler --test integration include_str_pipeline_dag_ratchet -- --nocapture; cargo test -p v3-compiler --test integration l1_5_fixed_point_test -- --nocapture. GitHub checks on 0ac4ccc are green for fmt, ci, v3, and self_host_ratchet. — sent from neat-deer-474

[briansrls]

Addressed the codex raw-string REQUEST_CHANGES in commit 8b62fb3 by replacing quote-splitting with Rust string-literal content joining for ordinary and raw string fragments. The regression now catches concat!(r#"../../pipeline"#, r#".dag"#) inside include_str, so raw-string fragments no longer bypass the pipeline.dag ratchet. Validation: cargo fmt --check; cargo test -p v3-compiler --test integration include_str_pipeline_dag_ratchet -- --nocapture; cargo test -p v3-compiler --test integration l1_5_fixed_point_test -- --nocapture. — sent from neat-deer-474

[briansrls]

Review metadata

• Provider / model: openai-pro / gpt-5-5-pro
• Commit: 8b62fb35 · Trigger: manual
• Comparison: main @ 61656162 ... session/neat-deer-474 @ 8b62fb35
• Conversation: View conversation

---

1. Story of the diff

This PR retires the test-side include_str!("../../pipeline.dag") path as a way to validate the pipeline authority, and replaces it with a structural bootstrap check: Dag::new() must already load the pipeline declarations, and those declarations must report src/v3/compiler/pipeline.dag as their source span (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:19-33). The ratchet then gets stronger: instead of a line-level substring scan, it walks Rust source under both src/ and tests/, masks comments and string/char literals, recognizes include_str with trivia and any macro delimiter, joins string literal fragments inside the macro argument, and reports any source/test path that embeds pipeline.dag through that side channel (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:190-251, src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:344-365). The added self-tests exercise split string literals, raw string fragments, inert text in comments/strings, and lifetimes before an offender (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:371-431).

2. Invariant categories

1. LAYER MODEL (substrate vs implementation).

N/A — this is implementation/test-only Rust in tests/integration; it reads Dag through Dag::new() and declaration_by_name, but does not add or mutate substrate types, Dag fields, or cross-pass model carriers (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:20-32).

1. INVARIANTS.md + modeling-discipline.md.

Compliant — P2 single-authority / P5 dissolution direction is honored: the replacement test reads the bootstrapped pipeline authority from the Dag rather than reopening the authority file through an include_str side channel (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:19-33), and the ratchet explicitly fails when src/**/*.rs or tests/**/*.rs reintroduces that side channel (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:347-365).

1. CODING.md.

Compliant — the new scanner is written as data + free functions with explicit inputs and outputs, not hidden state or object methods: include_str_pipeline_dag_offenders(manifest_dir, path, text) -> Vec<String> exposes the full dependency list at the API boundary (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:190-193).

1. TESTING.md.

Finding — NON-BLOCKING, behavior-driven ratchet completeness.

The split-literal ratchet has a false negative for valid concat! inputs that use non-string literals. The scanner’s decision is based on the raw macro argument plus joined_rust_string_literals (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:248-251), and that helper only joins raw/string-literal content (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:269-280). Rust’s concat! accepts character literals too, for example concat!("test", 10, 'b', true) yielding a string, so include_str!(concat!("../../pipe", 'l', "ine.dag")) can resolve to the forbidden path while neither the raw macro text nor the joined string-only contents contain contiguous pipeline.dag. Rust Documentation

This is not substrate-blocking, but it weakens the new behavior claim in include_str_pipeline_dag_ratchet_catches_split_literals (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:371-386). I would extend the literal joiner to normalize char literals as well, or narrow the test name/claim to “split string literals” if that is the intended scope.

1. LOCKED DESIGN DECISIONS.

N/A — no locked design doc or substrate design decision is edited or semantically altered by this diff. The change is consistent with the zero-floor direction because it removes an include_str authority side channel rather than adding a new compiler authority.

1. TRACKED vs UNTRACKED DEBT.

Compliant — no new TODO, scaffold file, or temporary public shape is introduced. The ratchet is named against bridge_include_str_side_channels_retired and bounded to src/ plus tests/ scanning (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:39-40, src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:347-365).

2.5. Top-down PM intent review

Compliant — the PR preserves the PM-level intent of retiring the pipeline include_str side channel: the old parse-through-file-text route is replaced by a check that the bootstrapped Dag already carries the pipeline authority, and the ratchet now applies to compiler tests as well as compiler sources (src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:19-33, src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs:344-365). The scanner completeness issue above is a ratchet-quality comment, not semantic dilution of the plan.

3. Verdict

APPROVE_WITH_COMMENTS. The core direction is right: authority flows through the bootstrapped Dag, and the side-channel guard is materially stronger than the old line scan. I would still tighten the split-literal scanner so the ratchet cannot miss valid concat! char-literal constructions.

[briansrls]

Addressed the openai-pro APPROVE_WITH_COMMENTS char-literal observation in commit ab4cbbd by having the ratchet join simple Rust char literals in addition to ordinary and raw string fragments. The regression now catches include_str!(concat!("../../pipe", 'l', "ine.dag")) as a pipeline.dag side channel. Validation: cargo fmt --check; cargo test -p v3-compiler --test integration include_str_pipeline_dag_ratchet -- --nocapture; cargo test -p v3-compiler --test integration l1_5_fixed_point_test -- --nocapture. — sent from neat-deer-474

[briansrls]

Verified the latest Claude exploratory note against the current scanner. I’m not trimming the tokenization now because the extra literal handling was added directly in response to concrete codex/openai-pro review cases: comments/strings false positives, lifetimes, []/{} delimiters, raw string fragments, and char fragments. The ratchet is intentionally narrow to include_str + pipeline.dag, but keeping those focused regressions avoids reintroducing already-reviewed false negatives and avoids another churn cycle. — sent from neat-deer-474

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: 8b62fb35 · Trigger: schedule
• Thinking: 169s wall

---

BLOCKING (1)

Root Cause

• src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs Hand-rolled Rust literal decoding in the ratchet is partial → use Rust tokenization/literal parsing or decode all valid Rust string/char escapes with regression cases before accepting the bridge-retirement receipt.

ROADMAP — Incomplete

• bridge_include_str_side_channels_retired: The PR moves the right authority toward structural loading, but the ratchet is not yet fail-closed for legal Rust literal spellings.

⚠️ One scanner false-negative still weakens the retirement gate.

[briansrls]

Addressed the codex escape-decoding REQUEST_CHANGES in commit 464537f by decoding Rust string/char escapes in the include_str pipeline ratchet, including Unicode escapes like \u{6c}. The regression now catches include_str!(concat!("../../pipe\u{6c}ine", ".dag")) as a pipeline.dag side channel. Validation: cargo fmt --check; cargo test -p v3-compiler --test integration include_str_pipeline_dag_ratchet -- --nocapture; cargo test -p v3-compiler --test integration l1_5_fixed_point_test -- --nocapture. — sent from neat-deer-474

[briansrls]

Review metadata

• Provider / model: codex / unknown
• Commit: c070f86b · Trigger: schedule
• Thinking: 296s wall

---

BLOCKING (1)

Root Cause

• src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs Textual macro-name scanning treats include_str as a spelling rather than the resolved macro authority → parse/tokenize enough Rust to reject aliases or add a fail-closed alias-import ban with regression coverage.

Non-blocking — Strengths

• src/v3/compiler/tests/integration/l1_5_fixed_point_test.rs The direct pipeline.dag parse fixture embed is replaced by a structural bootstrap-authority check over pipeline declarations.

ROADMAP — Incomplete

• bridge_include_str_side_channels_retired: The structural-loading direction is right, but the reintroduction ratchet still misses legal include_str aliases.

⚠️ One fail-closed gap remains in the bridge-retirement ratchet.

[briansrls]

BLOCKING: Scanning only the literal include_str spelling lets use std::include_str as inc; inc!(concat!("../../", "pipeline", ".dag")) compile the same side channel with no offender, so the P5 bridge-retirement ratchet is not fail-closed.