[justinchuby]

Symptom

Sending a prompt that references an image without attaching one answered with the raw engine error:

missing required pipeline input 'encoder.pixel_values'

instead of the advice the REPL is supposed to give ("this model declares image input; attach one with /image <path>"). Two repl_e2e tests were failing on main because of it.

Cause

The front end recognized the condition by matching "input not found: " — the wording a single decoder graph raises. A pipeline raises its own wording from routing.rs, so once a multimodal package started failing through the pipeline router the match stopped firing.

The unit test guarding this hand-wrote the same string it tested against, so the test and the matcher agreed with each other while neither agreed with the engine. That is why the drift was invisible.

Fix

Name the message once, next to where the pipeline writes it (MISSING_REQUIRED_INPUT), and export the recognizer (is_missing_required_input) from the engine. A front end now asks the engine whether an error is a missing input rather than guessing at its prose, and the message and its recognizer cannot drift apart because they share the constant.

Tests

The test moves to the engine, next to the constant, and additionally pins the case that must not be advised as a forgotten attachment: an optional input that a presence key declared present is a package-authoring fault, not a missing file.

cargo test -p onnx-genai-cli -p onnx-genai-engine — the two previously failing repl_e2e tests pass (47/47). clippy and fmt clean.

Note: decode_position_and_state_e2e::pipeline_load_rejects_kv_page_pool_before_fixed_state_when_host_budget_cannot_fit_either also fails on main, from an unrelated change to how the KV budget resolves under a tiny host limit. It is not touched here.

[github-actions]

✅ Benchmarks — No Regression

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.12 ms	2.26 ms	+6.3%
✅	matmul/small_generic_f16_threads=8/1x256x256	39.29 µs	41.05 µs	+4.5%
✅	matmul/large_generic_f16_threads=8/32x1024x1024	103.16 µs	107.67 µs	+4.4%
✅	sampling_latency/min_p_per_token	220.82 µs	230.39 µs	+4.3%
✅	matmul/small_generic_f16_threads=1/1x256x256	35.77 µs	37.16 µs	+3.9%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	538.13 µs	553.85 µs	+2.9%
✅	matmul/large_generic_f32_threads=8/32x1024x1024	6.68 ms	6.87 ms	+2.9%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	674.58 µs	693.09 µs	+2.7%
✅	gather/medium_f32_threads=1-internal/32768	4.41 µs	4.51 µs	+2.2%
✅	kv_cache/alloc_dealloc_pages	41.45 µs	42.22 µs	+1.9%
✅	matmul/medium_generic_f32_threads=8/32x512x512	1.73 ms	1.76 ms	+1.7%
✅	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	1.23 ms	1.24 ms	+1.3%
✅	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	586.48 µs	590.40 µs	+0.7%
✅	reduce_mean/medium_f32_threads=1-internal/65536	268.89 µs	270.64 µs	+0.6%
✅	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	101.34 µs	101.72 µs	+0.4%
✅	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	128.39 µs	128.81 µs	+0.3%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.38 ms	2.39 ms	+0.3%
✅	gather/large_bf16_threads=1-internal/131072	15.49 µs	15.45 µs	-0.3%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	6.03 ms	5.95 ms	-1.3%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	130.50 µs	128.51 µs	-1.5%
✅	matmul/small_generic_bf16_threads=8/1x256x256	39.93 µs	39.18 µs	-1.9%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.74 ms	3.66 ms	-2.1%
✅	matmul/small_generic_f32_threads=8/1x256x256	52.52 µs	51.11 µs	-2.7%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.12 ms	1.09 ms	-2.8%
✅	matmul/medium_generic_f16_threads=1/32x512x512	36.24 µs	35.21 µs	-2.8%
✅	matmul/medium_generic_f16_threads=8/32x512x512	44.85 µs	43.49 µs	-3.0%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	10.01 ms	9.66 ms	-3.6%
✅	tokenization/decode_tokens_per_second	6.92 ms	6.67 ms	-3.6%
✅	qwen3_sampling_processors/top_k_top_p_fast	691.71 µs	664.47 µs	-3.9%
✅	logit_processing/seven_processor_chain_per_step	339.17 µs	325.16 µs	-4.1%
✅	grammar_masking/llguidance_compute_mask/32	80.78 µs	77.41 µs	-4.2%
✅	gather/small_bf16_threads=1-internal/4096	520.2 ns	497.8 ns	-4.3%
✅	sampling_latency/top_p_per_token	432.31 µs	412.93 µs	-4.5%
✅	qwen3_sampling_processors/top_k_partial_selection	157.27 µs	150.00 µs	-4.6%
✅	gather/large_f32_threads=1-internal/131072	40.98 µs	38.55 µs	-5.9%
✅	gather/small_f16_threads=1-internal/4096	547.7 ns	513.5 ns	-6.3%
✅	sampling_latency/greedy_per_token	3.56 µs	3.33 µs	-6.5%
✅	matmul/small_generic_bf16_threads=1/1x256x256	38.71 µs	36.13 µs	-6.7%
✅	matmul/small_generic_f32_threads=1/1x256x256	46.38 µs	43.09 µs	-7.1%
✅	tokenization/encode_tokens_per_second	428.79 µs	397.50 µs	-7.3%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.47 ms	2.26 ms	-8.5%
✅	reduce_mean/small_f32_threads=1-internal/4096	17.14 µs	15.61 µs	-8.9%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	2.49 ms	2.24 ms	-10.2%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	626.81 µs	555.20 µs	-11.4%
✅	gather/large_f16_threads=1-internal/131072	17.69 µs	15.27 µs	-13.7%
✅	add/small_bf16_threads=1-internal/1024	561.6 ns	478.2 ns	-14.8%
🟢	add/small_f16_threads=1-internal/1024	597.2 ns	492.2 ns	-17.6%
🟢	add/large_bf16_threads=1-internal/4194304	2.17 ms	1.78 ms	-18.2%
🟢	sampling_latency/top_k_per_token	67.58 µs	55.11 µs	-18.5%
🟢	matmul/large_generic_f16_threads=1/32x1024x1024	108.87 µs	87.88 µs	-19.3%
🟢	add/medium_bf16_threads=1-internal/262144	147.73 µs	117.72 µs	-20.3%
🟢	add/small_f32_threads=1-internal/1024	272.4 ns	211.2 ns	-22.5%
🟢	gather/medium_bf16_threads=1-internal/32768	3.21 µs	2.45 µs	-23.6%
🟢	gather/small_f32_threads=1-internal/4096	926.4 ns	700.2 ns	-24.4%
🟢	add/medium_f32_threads=1-internal/262144	35.72 µs	26.55 µs	-25.7%
🟢	add/large_f32_threads=1-internal/4194304	999.62 µs	738.78 µs	-26.1%
🟢	gather/medium_f16_threads=1-internal/32768	3.33 µs	2.43 µs	-27.0%
🟢	add/large_f16_threads=1-internal/4194304	2.34 ms	1.70 ms	-27.2%
🟢	add/medium_f16_threads=1-internal/262144	177.27 µs	115.16 µs	-35.0%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 3.88 3.73 6.09 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.99%. Comparing base (06c62e0) to head (f9ddb06).
⚠️ Report is 103 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1413       +/-   ##
===========================================
+ Coverage   80.88%   82.99%    +2.10%     
===========================================
  Files         364       12      -352     
  Lines      160729     5638   -155091     
  Branches   160729     5638   -155091     
===========================================
- Hits       130005     4679   -125326     
+ Misses      26069      761    -25308     
+ Partials     4655      198     -4457     

Flag	Coverage Δ
cli-ort-linux	82.94% <ø> (?)
cli-ort-windows	82.46% <ø> (?)
mlas	?
offline	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
crates/onnx-genai-cli/src/interactive.rs	76.61% <ø> (ø)

... and 375 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.