fix(cli): load native pipelines without ORT preflight

[justinchuby]

Summary

• inspect multimodal pipeline graph I/O without constructing ORT sessions
• lazily create the ORT environment only when a component actually uses ORT
• resolve image and audio bindings through the backend-neutral GraphIo interface
• accept Hugging Face conditional namespace keyword expressions in MiniJinja without modifying model-authored templates
• add regressions for native-only pipeline inspection and the Jinja compatibility form

Validation

• cargo test -p onnx-genai-ort --test pipeline_loader native_only_component_loads_without_ort_session --no-default-features
• cargo test -p onnx-genai-server multimodal
• cargo test -p onnx-genai-ort chat_template --no-default-features
• cargo clippy -p onnx-genai-ort -p onnx-genai-server --tests -- -D warnings
• built onnx-genai-cli --release --features native-cuda
• loaded and generated one token with the real 30B int4 VLM using its unmodified chat template and --backend native --device cuda

[codecov]

Codecov Report

❌ Patch coverage is 59.12162% with 121 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.95%. Comparing base (fdeaf77) to head (d1f7cd5).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/onnx-genai-cli/src/output.rs	62.68%	68 Missing and 7 partials ⚠️
crates/onnx-genai-cli/src/lib.rs	46.93%	24 Missing and 2 partials ⚠️
crates/onnx-genai-cli/src/interactive.rs	47.05%	3 Missing and 6 partials ⚠️
...nnx-runtime-session/src/executor/dynamic_shapes.rs	71.42%	2 Missing and 4 partials ⚠️
crates/onnx-genai-cli/src/generate.rs	40.00%	1 Missing and 2 partials ⚠️
crates/onnx-genai-cli/src/transcribe.rs	0.00%	0 Missing and 1 partial ⚠️
...ates/onnx-runtime-session/src/executor/geometry.rs	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             main    #1141     +/-   ##
=========================================
  Coverage   79.95%   79.95%             
=========================================
  Files         368      370      +2     
  Lines      160826   162449   +1623     
  Branches   160826   162449   +1623     
=========================================
+ Hits       128582   129882   +1300     
- Misses      27524    27813    +289     
- Partials     4720     4754     +34     

Flag	Coverage Δ
cli-ort-linux	82.52% <58.24%> (-1.28%)	⬇️
cli-ort-windows	82.05% <58.24%> (-1.26%)	⬇️
mlas	85.67% <ø> (-0.07%)	⬇️
offline	79.74% <69.56%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...es/onnx-runtime-ep-cpu/src/kernels/matmul_nbits.rs	75.88% <ø> (-0.03%)	⬇️
crates/onnx-genai-cli/src/transcribe.rs	78.73% <0.00%> (-0.18%)	⬇️
...ates/onnx-runtime-session/src/executor/geometry.rs	90.95% <50.00%> (-0.23%)	⬇️
crates/onnx-genai-cli/src/generate.rs	90.45% <40.00%> (-0.58%)	⬇️
...nnx-runtime-session/src/executor/dynamic_shapes.rs	36.78% <71.42%> (+1.54%)	⬆️
crates/onnx-genai-cli/src/interactive.rs	75.59% <47.05%> (-0.74%)	⬇️
crates/onnx-genai-cli/src/lib.rs	92.43% <46.93%> (-1.97%)	⬇️
crates/onnx-genai-cli/src/output.rs	76.92% <62.68%> (-6.92%)	⬇️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

⚠️ Benchmark Change Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
⚠️	qwen3_sampling_processors/top_k_top_p_fast	725.12 µs	936.52 µs	+29.2%
⚠️	add/small_bf16_threads=1-internal/1024	478.3 ns	569.2 ns	+19.0%
✅	add/medium_bf16_threads=1-internal/262144	115.21 µs	129.31 µs	+12.2%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.57 ms	10.61 ms	+10.9%
✅	gather/small_bf16_threads=1-internal/4096	487.9 ns	539.5 ns	+10.6%
✅	add/small_f16_threads=1-internal/1024	549.8 ns	605.8 ns	+10.2%
✅	gather/small_f32_threads=1-internal/4096	755.6 ns	830.9 ns	+10.0%
✅	add/medium_f32_threads=1-internal/262144	29.86 µs	32.12 µs	+7.5%
✅	grammar_masking/llguidance_compute_mask/32	99.52 µs	105.88 µs	+6.4%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	550.72 µs	584.73 µs	+6.2%
✅	sampling_latency/top_p_per_token	442.56 µs	469.71 µs	+6.1%
✅	add/small_f32_threads=1-internal/1024	239.8 ns	253.0 ns	+5.5%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.11 ms	1.17 ms	+4.9%
✅	matmul/small_generic_bf16_threads=8/1x256x256	39.90 µs	41.47 µs	+3.9%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.37 ms	2.41 ms	+1.7%
✅	kv_cache/alloc_dealloc_pages	46.27 µs	46.65 µs	+0.8%
✅	reduce_mean/medium_f32_threads=1-internal/65536	297.37 µs	297.63 µs	+0.1%
✅	add/medium_f16_threads=1-internal/262144	132.40 µs	131.81 µs	-0.4%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	672.38 µs	668.72 µs	-0.5%
✅	logit_processing/seven_processor_chain_per_step	377.93 µs	370.61 µs	-1.9%
✅	gather/small_f16_threads=1-internal/4096	592.4 ns	576.4 ns	-2.7%
✅	sampling_latency/top_k_per_token	63.15 µs	61.35 µs	-2.8%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.27 ms	2.19 ms	-3.5%
✅	matmul/small_generic_f16_threads=8/1x256x256	37.75 µs	36.31 µs	-3.8%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	90.94 µs	87.43 µs	-3.9%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	193.65 µs	185.19 µs	-4.4%
✅	matmul/small_generic_f32_threads=1/1x256x256	47.02 µs	44.39 µs	-5.6%
✅	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	695.83 µs	652.84 µs	-6.2%
✅	gather/medium_f16_threads=1-internal/32768	2.64 µs	2.47 µs	-6.5%
✅	matmul/medium_generic_f16_threads=1/32x512x512	35.96 µs	33.37 µs	-7.2%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	6.73 ms	6.23 ms	-7.5%
✅	matmul/medium_generic_f16_threads=8/32x512x512	44.59 µs	41.15 µs	-7.7%
✅	tokenization/encode_tokens_per_second	486.16 µs	446.20 µs	-8.2%
✅	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	837.07 µs	765.04 µs	-8.6%
✅	qwen3_sampling_processors/top_k_partial_selection	167.66 µs	152.68 µs	-8.9%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	4.23 ms	3.84 ms	-9.0%
✅	matmul/small_generic_bf16_threads=1/1x256x256	40.81 µs	36.82 µs	-9.8%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	595.82 µs	534.37 µs	-10.3%
✅	gather/medium_f32_threads=1-internal/32768	4.56 µs	4.09 µs	-10.3%
✅	gather/medium_bf16_threads=1-internal/32768	2.84 µs	2.53 µs	-11.0%
✅	matmul/large_generic_f16_threads=8/32x1024x1024	110.98 µs	98.59 µs	-11.2%
🟢	gather/large_f16_threads=1-internal/131072	16.51 µs	13.64 µs	-17.4%
🟢	gather/large_f32_threads=1-internal/131072	44.62 µs	36.53 µs	-18.1%
🟢	matmul/small_generic_f16_threads=1/1x256x256	42.68 µs	34.83 µs	-18.4%
🟢	matmul/large_generic_bf16_threads=8/32x1024x1024	1.90 ms	1.55 ms	-18.5%
🟢	reduce_mean/small_f32_threads=1-internal/4096	23.29 µs	18.94 µs	-18.7%
🟢	qwen3_sampling_processors/top_k_full_sort_baseline	2.62 ms	2.12 ms	-19.0%
🟢	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	103.20 µs	82.16 µs	-20.4%
🟢	sampling_latency/greedy_per_token	4.61 µs	3.66 µs	-20.6%
🟢	tokenization/decode_tokens_per_second	10.50 ms	8.28 ms	-21.1%
🟢	add/large_f32_threads=1-internal/4194304	991.68 µs	771.38 µs	-22.2%
🟢	sampling_latency/min_p_per_token	282.28 µs	217.04 µs	-23.1%
🟢	matmul/medium_generic_f32_threads=8/32x512x512	1.70 ms	1.29 ms	-24.2%
🟢	matmul/small_generic_f32_threads=8/1x256x256	55.61 µs	40.16 µs	-27.8%
🟢	add/large_bf16_threads=1-internal/4194304	3.10 ms	2.13 ms	-31.2%
🟢	matmul/large_generic_f32_threads=8/32x1024x1024	6.87 ms	4.70 ms	-31.6%
🟢	add/large_f16_threads=1-internal/4194304	3.11 ms	2.06 ms	-33.6%
🟢	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	128.19 µs	72.37 µs	-43.5%
🟢	gather/large_bf16_threads=1-internal/131072	26.14 µs	12.57 µs	-51.9%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 10.23 5.41 5.89 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

[justinchuby]

Superseded by focused PRs:

• fix(pipeline): inspect native graphs without ORT sessions #1156 native pipeline graph inspection
• fix(chat): accept conditional namespace kwargs #1157 chat-template namespace compatibility
• fix(native): improve backend diagnostics and context handling #1158 native backend diagnostics/context plumbing
• fix(cuda): run Muse vision encoder natively #1159 CUDA vision encoder support
• fix(cli): preserve native device in interactive sessions #1160 interactive native-device propagation
• fix(cli): render structured model responses #1161 structured response rendering and multi-EOS
• fix(pipeline): expose model sampling defaults #1162 pipeline sampling defaults
• fix(cuda): preserve attention mask across KV growth #1163 CUDA KV-growth attention-mask correctness
• fix(cli): defer diagnostics while streaming output #1164 deferred INFO logs during streaming

Closing this mixed PR so each concern can be reviewed and merged independently.