feat(ir): inline_single_trip_scan_bodies graph transform (Inc-1b PR-1, unwired)

[justinchuby]

Inc-1b PR-1 — inline_single_trip_scan_bodies graph transform

First bounded step of the Inc-1b lane (cohaagen-27b-inc1b-design.md §1). Adds a new IR graph transform in onnx-runtime-ir and nothing else.

What the transform does

inline_single_trip_scan_bodies(&Graph) -> Graph lowers each single-trip-eligible Scan node to one straight-line iteration of its body spliced into the parent graph:

• Eligibility (structural, safe-by-construction): default-domain Scan with a registered body subgraph, carrying recurrent state (num_state >= 1, i.e. a LinearAttention body threading state_pairs), whose every scan input's scan axis is not a static extent > 1 (Static(1) or dynamic — the decode case), with no nested control flow, and whose lexical captures all resolve by name in parent scope. Anything else is left untouched.
• Scan input → body: a Squeeze drops the size-1 scan axis and feeds the body scan-input formal.
• Body splice: every body node is cloned into the parent with fresh, remapped ValueIds (remap: body ValueId → new parent ValueId). Body state formals bind to the Scan's parent state-input values; the body scan-input formal binds to the squeezed slice; lexical captures resolve by name to values already in parent scope (same resolution ChildExecutor::new does via capture_names); body-local initializers are promoted to parent initializers.
• Body outputs → Scan outputs: the first num_state body outputs become the parent present-state values (written directly by their producer; an Identity is used only for a rare pass-through); scan outputs get an Unsqueeze re-adding the size-1 axis to match the Scan's declared scan-output shape.
• Delete the Scan node and drop its body subgraph.

The transform is a pure Graph -> Graph function — the input graph is never mutated.

Emphatically UNWIRED / zero behavior change / dead until PR-2

This pass is not called from any executor, decoder, loader, or run path. It is dead code exercised only by the unit tests. The prefill / multi-trip / dense paths are completely unaffected. No changes to control_flow.rs, run.rs, state.rs, or decode routing.

• PR-2 (later) will build a decode_inline_exec from the transformed graph and route decode to it behind a default-off flag (eager, transform-correctness proof).
• PR-3 (later) lets the inlined body capture — that is the highest-blast-radius step and still needs Justin's greenlight + capture-team (Mary / fix(session): expose bound inputs to control flow #443–feat(native-decode): rank-3 mrope native positions unblock qwen3.5 hybrid native-CUDA (#384, #67) #543) sign-off. This PR does not and is safe to review/merge on its own.

Crate-boundary note (why inference isn't called inside the pass)

onnx-runtime-ir is the dependency-free base contract and cannot depend on onnx-runtime-shape-inference (which depends on it) — not even as a dev-dependency, since that pulls in a second, incompatible copy of the IR types. So the transform performs the structural merge only (mirroring the graph-building half of ChildExecutor::compile); a caller (PR-2) re-runs InferenceRegistry::infer_graph (Permissive) to re-resolve interior shapes, exactly as ChildExecutor::compile does today. The "inference re-converges" check is therefore placed in the shape-inference crate's integration tests, on the side of the edge that can see both.

Tests (CPU, fast, non-ignored)

onnx-runtime-ir — src/scan_inline/tests.rs (5 tests, shared tiny-graph builder):

• lowers_single_trip_scan_to_straight_line_body — proves the Scan is removed; body ops (MatMul + 3 Adds) are present; one Squeeze + one Unsqueeze appear; the parent present-state is produced by an inlined Add; the scan output is produced by Unsqueeze; the lexical capture w resolves by name to the pre-existing parent weight the inlined MatMul reads; the body-local bias initializer is promoted; the graph is structurally valid; and the rewired boundary/interior values carry the expected static shapes.
• leaves_multi_trip_scan_untouched — recurrent Scan with a static scan-axis extent of 3 is a no-op (must not statically collapse a genuine multi-trip Scan).
• leaves_non_recurrent_scan_untouched — a pure element-wise map Scan (num_state == 0) is a no-op.
• leaves_dense_scanless_graph_untouched — a dense Scan-free graph is returned structurally unchanged (non-vacuous both directions).
• remap_produces_fresh_non_colliding_interior_ids — with body/parent id ranges deliberately overlapping, asserts every node-referenced id is live, every interior value id is freshly allocated above the pre-transform high-water mark (no leaked body id, no alias of a pre-existing parent value), and validate() passes (no id collisions / consistent edges).

onnx-runtime-shape-inference — tests/scan_inline_inference.rs (1 test):

• permissive_inference_reconverges_over_inlined_scan_body — inlines the eligible hybrid, blanks every produced value's shape, then asserts Permissive whole-graph inference re-derives the interior + boundary shapes (present state [2,4], scan output [1,2,4]) from the graph inputs/initializers through the inlined nodes.

Verification

• cargo test -p onnx-runtime-ir — 67 passed (incl. the 5 new).
• cargo test -p onnx-runtime-shape-inference --test scan_inline_inference — 1 passed.
• cargo fmt --all --check — clean.
• cargo clippy -p onnx-runtime-ir --all-targets -- -D warnings — exit 0; cargo clippy -p onnx-runtime-shape-inference --tests -- -D warnings — exit 0.
• cargo check for onnx-runtime-ir, -shape-inference, -optimizer, -loader (its dependents) — clean. (A workspace-wide cargo check fails only on the unrelated onnx-runtime-cpuinfo cmake vendor submodule, which is not populated in a fresh git worktree — environmental, not from this change; PR-1 does not touch ep-cuda or any GPU code.)

Scope held

Size M, LOW blast radius (new pass, dead until PR-2), exactly as the design's staging table predicts. No execution/capture surfaces were touched to compile or test it.

Per Inc-1b design: cohaagen-27b-inc1b-design.md. Please review before merge — do not enable auto-merge (independent reviewer: Harry).

[codecov]

Codecov Report

❌ Patch coverage is 85.15284% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.50%. Comparing base (cbcaff3) to head (727b27a).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/onnx-runtime-ir/src/scan_inline.rs	85.15%	16 Missing and 18 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #580      +/-   ##
==========================================
+ Coverage   81.36%   81.50%   +0.13%     
==========================================
  Files         316      317       +1     
  Lines      123952   124181     +229     
  Branches   123952   124181     +229     
==========================================
+ Hits       100859   101216     +357     
+ Misses      19016    18869     -147     
- Partials     4077     4096      +19     

Flag	Coverage Δ
cli-ort-linux	86.69% <ø> (ø)
cli-ort-windows	83.73% <ø> (ø)
mlas	77.91% <ø> (-0.82%)	⬇️
offline	81.34% <85.15%> (+0.15%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
crates/onnx-runtime-ir/src/scan_inline.rs	85.15% <85.15%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[justinchuby]

VERDICT: APPROVE

Independent review by Harry (author-lockout in effect; I re-ran everything myself in a detached worktree at 727b27af, did not trust the author's self-report).

Blast radius — LOW, confirmed. The branch commit vs its fork point (cbcaff3a) and gh pr view --json files both show EXACTLY 4 files (+1039): scan_inline.rs (new), scan_inline/tests.rs (new), lib.rs (+2 wiring), scan_inline_inference.rs (new integration test). No control_flow.rs / run.rs / state.rs / decode routing / ep-cuda / loader touched.

UNWIRED — confirmed (critical). Workspace grep for inline_single_trip_scan_bodies returns ONLY the definition, the pub use re-export, and test files. No executor/decoder/loader/run path calls it. "Zero behavior change / dead until PR-2" holds.

Build + tests — confirmed. cargo test -p onnx-runtime-ir → 67 passed. cargo test -p onnx-runtime-shape-inference --test scan_inline_inference → 1 passed.

Tests non-vacuous — mutation-verified:

• Positive: emit_unsqueeze → Identity (drop scan-axis re-add) ⇒ positive test FAILS at tests.rs:236 (Unsqueeze count), integration FAILS at :165 (scan_out [2,4] vs [1,2,4]).
• Remap: reuse body ValueId instead of create_value ⇒ remap test FAILS at tests.rs:422 (leaked id not live).
• State-binding: bind state formal to scan input ⇒ integration test FAILS at :164 (present_state shape). (5 structural unit tests alone did NOT catch this shape-compatible-adjacent case — non-blocking, see below.)
  All mutations reverted; source verified identical to HEAD before this verdict.

Semantics — independently reasoned sound. Squeeze drops exactly the normalized scan axis; Unsqueeze re-adds the size-1 axis at the declared output axis; state formals→parent state inputs and first num_state body outputs→present-state (correct ONNX ordering); captures resolve by name in parent scope (index built before initializer promotion, no shadowing); every interior value gets a fresh parent id, validate() passes (no leak/alias/edge inconsistency). Eligibility guard is conservative: static extent>1, num_state==0, nested control flow, and unresolved captures each ⇒ no-op; a genuine multi-trip Scan cannot be statically collapsed (extent-3 test confirms).

fmt/clippy — clean. cargo fmt --all --check exit 0; clippy -D warnings on onnx-runtime-ir --all-targets and onnx-runtime-shape-inference --tests both exit 0.

cpuinfo caveat — confirmed environmental. git submodule status shows the vendor/cpuinfo submodule uninitialized (leading -); the workspace-wide cmake failure is the unpopulated submodule in a fresh worktree, not this diff (which touches zero cpuinfo files).

Non-blocking recommendations for PR-2:

1. Add the design §4 byte-identical decode-inline == child-session parity test (per-token outputs AND identical final state) — the real guard for state-continuity/ordering that PR-1's structural tests can't fully cover.
2. Implement the runtime scan-axis extent == 1 assertion + fallback to the Scan exec for the dynamic/symbolic-axis case the guard intentionally allows through.
3. Bind the decode-inline exec to the identical persistent state buffers at the prefill→decode hand-off (feat(native-decode): auto-derive io kv/state_pairs from graph ports (#384) #573 io.state_pairs, design §3).
4. Verify present-output ordering matches io.state_pairs and add the inlined-interior shape-signature check when warming.

PR-3 (capture) correctly remains gated on Justin greenlight + capture-team (Mary / #443–#543) sign-off and is not in this PR. Not enabling auto-merge; coordinator handles merge.

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	sampling_latency/top_k_per_token	52.50 µs	472.09 µs	+799.2%
🔴	logit_processing/seven_processor_chain_per_step	233.87 µs	1.17 ms	+400.2%
🔴	sampling_latency/top_p_per_token	363.20 µs	995.13 µs	+174.0%
🔴	sampling_latency/min_p_per_token	207.30 µs	337.24 µs	+62.7%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.28 ms	2.36 ms	+3.5%
✅	tokenization/encode_tokens_per_second	383.75 µs	391.41 µs	+2.0%
✅	tokenization/decode_tokens_per_second	6.22 ms	6.24 ms	+0.3%
✅	matmul/large_generic_f16_threads=8/32x1024x1024	85.34 µs	85.45 µs	+0.1%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.13 ms	2.13 ms	+0.0%
✅	qwen3_sampling_processors/top_k_partial_selection	139.08 µs	139.08 µs	+0.0%
✅	qwen3_sampling_processors/top_k_top_p_fast	246.05 µs	246.05 µs	+0.0%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	5.71 ms	5.71 ms	+0.0%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	108.10 µs	108.10 µs	+0.0%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.51 ms	3.51 ms	+0.0%
✅	sampling_latency/greedy_per_token	3.23 µs	3.23 µs	-0.2%
✅	grammar_masking/llguidance_compute_mask/32	75.15 µs	74.85 µs	-0.4%
✅	matmul/medium_generic_f16_threads=1/32x512x512	30.51 µs	30.31 µs	-0.7%
✅	kv_cache/alloc_dealloc_pages	36.74 µs	36.49 µs	-0.7%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.40 ms	9.34 ms	-0.7%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	81.14 µs	79.97 µs	-1.4%
✅	matmul/medium_generic_f16_threads=8/32x512x512	30.38 µs	29.86 µs	-1.7%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	535.02 µs	520.17 µs	-2.8%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.02 ms	1.95 ms	-3.3%
✅	matmul/large_generic_f32_threads=8/32x1024x1024	4.08 ms	3.94 ms	-3.4%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.41 ms	1.34 ms	-5.1%
✅	matmul/small_generic_f16_threads=1/1x256x256	34.31 µs	30.73 µs	-10.4%
✅	matmul/small_generic_bf16_threads=1/1x256x256	35.47 µs	31.70 µs	-10.6%
✅	add/large_f16_threads=1-internal/4194304	46.00 ms	40.99 ms	-10.9%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	415.38 µs	368.81 µs	-11.2%
✅	add/large_bf16_threads=1-internal/4194304	48.39 ms	42.30 ms	-12.6%
✅	gather/medium_f16_threads=1-internal/32768	2.76 µs	2.40 µs	-12.8%
✅	add/small_f16_threads=1-internal/1024	14.72 µs	12.72 µs	-13.6%
✅	reduce_mean/medium_f32_threads=1-internal/65536	283.77 µs	244.07 µs	-14.0%
✅	matmul/medium_generic_f32_threads=8/32x512x512	1.07 ms	911.36 µs	-14.5%
🟢	gather/large_bf16_threads=1-internal/131072	13.65 µs	11.49 µs	-15.8%
🟢	add/medium_f16_threads=1-internal/262144	3.08 ms	2.56 ms	-16.9%
🟢	add/small_bf16_threads=1-internal/1024	15.36 µs	12.76 µs	-17.0%
🟢	add/medium_bf16_threads=1-internal/262144	3.13 ms	2.60 ms	-17.0%
🟢	matmul/small_generic_f32_threads=1/1x256x256	45.51 µs	37.00 µs	-18.7%
🟢	reduce_mean/large_f32_threads=1-internal/262144	1.21 ms	979.36 µs	-19.2%
🟢	add/large_f32_threads=1-internal/4194304	51.46 ms	40.78 ms	-20.8%
🟢	matmul/small_generic_f16_threads=8/1x256x256	37.95 µs	29.98 µs	-21.0%
🟢	add/medium_f32_threads=1-internal/262144	3.18 ms	2.51 ms	-21.2%
🟢	matmul/small_generic_bf16_threads=8/1x256x256	40.79 µs	31.37 µs	-23.1%
🟢	gather/large_f16_threads=1-internal/131072	14.68 µs	11.28 µs	-23.2%
🟢	reduce_mean/small_f32_threads=1-internal/4096	19.84 µs	15.13 µs	-23.8%
🟢	add/small_f32_threads=1-internal/1024	253.8 ns	192.4 ns	-24.2%
🟢	gather/medium_f32_threads=1-internal/32768	5.33 µs	3.79 µs	-28.9%
🟢	gather/medium_bf16_threads=1-internal/32768	3.56 µs	2.40 µs	-32.6%
🟢	matmul/small_generic_f32_threads=8/1x256x256	51.01 µs	34.39 µs	-32.6%
🟢	gather/small_f32_threads=1-internal/4096	998.1 ns	672.3 ns	-32.6%
🟢	gather/large_f32_threads=1-internal/131072	50.48 µs	30.89 µs	-38.8%
🟢	gather/small_f16_threads=1-internal/4096	790.0 ns	470.2 ns	-40.5%
🟢	gather/small_bf16_threads=1-internal/4096	850.0 ns	468.7 ns	-44.9%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 3.21 4.08 7.12 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)