[justinchuby]

Scribe state-only batch: merged 2 inbox drops (#938 roper-decode-remaining-levers, #936 holden-zerocopy-linux-default) into decisions.md, archived older bodies to decisions-archive/2026-08.md (49,264 B → 48,169 B, under the 50 KB gate), appended histories.

State-only; no code.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.81%. Comparing base (2a16f9a) to head (0628e36).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #940      +/-   ##
==========================================
- Coverage   78.41%   77.81%   -0.61%     
==========================================
  Files         354      354              
  Lines      148153   148153              
  Branches   148153   148153              
==========================================
- Hits       116176   115282     -894     
- Misses      27479    28377     +898     
+ Partials     4498     4494       -4     

Flag	Coverage Δ
cli-ort-linux	83.42% <ø> (ø)
cli-ort-windows	82.92% <ø> (ø)
mlas	80.73% <ø> (-0.19%)	⬇️
offline	77.55% <ø> (-0.64%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	45.40 µs	106.62 µs	+134.9%
🔴	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	378.88 µs	620.26 µs	+63.7%
🔴	matmul/small_generic_f32_threads=8/1x256x256	35.96 µs	58.48 µs	+62.7%
🔴	matmul/large_generic_f32_threads=8/32x1024x1024	3.81 ms	5.34 ms	+40.1%
🔴	gather/large_f32_threads=1-internal/131072	27.37 µs	38.26 µs	+39.8%
🔴	matmul/small_generic_f16_threads=8/1x256x256	31.24 µs	43.18 µs	+38.2%
🔴	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	576.59 µs	795.71 µs	+38.0%
🔴	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	44.09 µs	60.35 µs	+36.9%
🔴	gather/small_bf16_threads=1-internal/4096	468.9 ns	638.7 ns	+36.2%
🔴	matmul/small_generic_f16_threads=1/1x256x256	29.96 µs	39.62 µs	+32.2%
🔴	matmul/medium_generic_f32_threads=8/32x512x512	931.93 µs	1.23 ms	+32.2%
⚠️	gather/small_f16_threads=1-internal/4096	476.1 ns	584.6 ns	+22.8%
⚠️	matmul/small_generic_bf16_threads=8/1x256x256	31.01 µs	38.05 µs	+22.7%
⚠️	sampling_latency/top_p_per_token	386.16 µs	470.90 µs	+21.9%
⚠️	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	75.95 µs	89.70 µs	+18.1%
⚠️	tokenization/encode_tokens_per_second	371.12 µs	427.71 µs	+15.2%
⚠️	sampling_latency/min_p_per_token	235.07 µs	270.81 µs	+15.2%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.35 ms	1.55 ms	+14.8%
✅	gather/medium_f16_threads=1-internal/32768	2.37 µs	2.72 µs	+14.5%
✅	add/large_f32_threads=1-internal/4194304	40.72 ms	46.35 ms	+13.8%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.42 ms	3.86 ms	+12.7%
✅	matmul/small_generic_bf16_threads=1/1x256x256	31.23 µs	34.79 µs	+11.4%
✅	tokenization/decode_tokens_per_second	6.50 ms	7.23 ms	+11.3%
✅	gather/small_f32_threads=1-internal/4096	664.1 ns	734.8 ns	+10.6%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	79.17 µs	87.30 µs	+10.3%
✅	sampling_latency/top_k_per_token	50.74 µs	55.54 µs	+9.5%
✅	matmul/small_generic_f32_threads=1/1x256x256	37.04 µs	40.42 µs	+9.1%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	486.83 µs	531.16 µs	+9.1%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.28 ms	2.48 ms	+8.9%
✅	gather/large_f16_threads=1-internal/131072	13.21 µs	14.26 µs	+7.9%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	5.59 ms	5.98 ms	+6.9%
✅	qwen3_sampling_processors/top_k_partial_selection	140.35 µs	149.69 µs	+6.7%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	525.41 µs	554.72 µs	+5.6%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.18 ms	2.26 ms	+3.8%
✅	add/small_bf16_threads=1-internal/1024	13.82 µs	14.32 µs	+3.7%
✅	kv_cache/alloc_dealloc_pages	40.07 µs	41.20 µs	+2.8%
✅	add/large_f16_threads=1-internal/4194304	42.81 ms	43.78 ms	+2.3%
✅	logit_processing/seven_processor_chain_per_step	323.92 µs	329.96 µs	+1.9%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	1.98 ms	2.00 ms	+1.0%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.29 ms	9.36 ms	+0.7%
✅	gather/large_bf16_threads=1-internal/131072	12.75 µs	12.80 µs	+0.4%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	514.07 µs	515.27 µs	+0.2%
✅	add/medium_f32_threads=1-internal/262144	2.67 ms	2.67 ms	-0.2%
✅	qwen3_sampling_processors/top_k_top_p_fast	655.25 µs	651.48 µs	-0.6%
✅	grammar_masking/llguidance_compute_mask/32	78.35 µs	77.90 µs	-0.6%
✅	add/medium_bf16_threads=1-internal/262144	2.78 ms	2.74 ms	-1.4%
✅	matmul/medium_generic_f16_threads=8/32x512x512	43.25 µs	42.17 µs	-2.5%
✅	gather/medium_f32_threads=1-internal/32768	4.38 µs	4.27 µs	-2.5%
✅	add/small_f16_threads=1-internal/1024	13.82 µs	13.41 µs	-3.0%
✅	gather/medium_bf16_threads=1-internal/32768	2.48 µs	2.41 µs	-3.1%
✅	reduce_mean/small_f32_threads=1-internal/4096	17.05 µs	16.35 µs	-4.1%
✅	matmul/large_generic_f16_threads=8/32x1024x1024	89.71 µs	85.36 µs	-4.9%
✅	matmul/medium_generic_f16_threads=1/32x512x512	32.54 µs	30.88 µs	-5.1%
✅	sampling_latency/greedy_per_token	3.31 µs	3.14 µs	-5.3%
✅	add/medium_f16_threads=1-internal/262144	2.92 ms	2.76 ms	-5.6%
✅	add/large_bf16_threads=1-internal/4194304	45.13 ms	41.36 ms	-8.4%
🟢	add/small_f32_threads=1-internal/1024	238.1 ns	202.2 ns	-15.1%
🟢	reduce_mean/medium_f32_threads=1-internal/65536	319.57 µs	266.44 µs	-16.6%
🟢	reduce_mean/large_f32_threads=1-internal/262144	1.19 ms	951.52 µs	-20.3%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 3.92 3.42 5.56 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)