[justinchuby]

Merge Deckard's speculative-capture feasibility decision drop into decisions.md (CONDITIONAL-GO gated behind Marlin; break-even B* = 8.5 today → ~1.2–2.0 post-Marlin). Adds orchestration + session logs. State-only; follows #957.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.98%. Comparing base (2f0b62b) to head (2ee89a1).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #958      +/-   ##
==========================================
- Coverage   79.00%   78.98%   -0.03%     
==========================================
  Files         356      354       -2     
  Lines      150096   147926    -2170     
  Branches   150096   147926    -2170     
==========================================
- Hits       118590   116839    -1751     
+ Misses      26855    26447     -408     
+ Partials     4651     4640      -11     

Flag	Coverage Δ
cli-ort-linux	83.52% <ø> (ø)
cli-ort-windows	83.11% <ø> (ø)
mlas	?
offline	78.81% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	gather/large_f32_threads=1-internal/131072	33.35 µs	48.70 µs	+46.0%
🔴	qwen3_sampling_processors/top_k_partial_selection	136.07 µs	189.30 µs	+39.1%
🔴	gather/medium_f16_threads=1-internal/32768	2.94 µs	3.90 µs	+32.9%
⚠️	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.50 ms	4.30 ms	+23.0%
⚠️	qwen3_sampling_processors/top_p_fast_after_top_k	500.97 µs	615.32 µs	+22.8%
⚠️	matmul/medium_generic_f32_threads=8/32x512x512	1.02 ms	1.25 ms	+22.4%
⚠️	gather/medium_bf16_threads=1-internal/32768	2.60 µs	3.14 µs	+20.7%
⚠️	grammar_masking/llguidance_compute_mask/32	73.34 µs	88.51 µs	+20.7%
✅	sampling_latency/greedy_per_token	2.97 µs	3.40 µs	+14.4%
✅	logit_processing/seven_processor_chain_per_step	321.43 µs	363.78 µs	+13.2%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	5.46 ms	6.17 ms	+13.2%
✅	tokenization/decode_tokens_per_second	6.16 ms	6.94 ms	+12.7%
✅	kv_cache/alloc_dealloc_pages	37.61 µs	41.42 µs	+10.1%
✅	matmul/medium_generic_f16_threads=8/32x512x512	32.00 µs	34.51 µs	+7.8%
✅	matmul/small_generic_bf16_threads=8/1x256x256	32.61 µs	34.53 µs	+5.9%
✅	gather/medium_f32_threads=1-internal/32768	4.27 µs	4.42 µs	+3.5%
✅	tokenization/encode_tokens_per_second	364.41 µs	369.15 µs	+1.3%
✅	matmul/small_generic_bf16_threads=1/1x256x256	34.37 µs	34.80 µs	+1.3%
✅	sampling_latency/top_k_per_token	48.11 µs	48.60 µs	+1.0%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.41 ms	2.35 ms	-2.4%
✅	qwen3_sampling_processors/top_k_top_p_fast	686.54 µs	664.50 µs	-3.2%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.08 ms	2.01 ms	-3.6%
✅	gather/small_f32_threads=1-internal/4096	795.8 ns	766.0 ns	-3.7%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.04 ms	998.42 µs	-4.4%
✅	gather/small_bf16_threads=1-internal/4096	550.4 ns	518.9 ns	-5.7%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	83.66 µs	77.36 µs	-7.5%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	80.75 µs	74.18 µs	-8.1%
✅	matmul/small_generic_f16_threads=1/1x256x256	33.84 µs	30.40 µs	-10.2%
✅	sampling_latency/min_p_per_token	225.76 µs	202.40 µs	-10.3%
✅	reduce_mean/small_f32_threads=1-internal/4096	17.12 µs	15.33 µs	-10.5%
✅	gather/small_f16_threads=1-internal/4096	548.9 ns	490.3 ns	-10.7%
✅	reduce_mean/medium_f32_threads=1-internal/65536	253.04 µs	225.74 µs	-10.8%
✅	matmul/medium_generic_f16_threads=1/32x512x512	33.50 µs	29.84 µs	-10.9%
✅	matmul/small_generic_f16_threads=8/1x256x256	34.79 µs	30.89 µs	-11.2%
✅	add/large_bf16_threads=1-internal/4194304	47.25 ms	41.65 ms	-11.9%
✅	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	53.34 µs	46.55 µs	-12.7%
✅	sampling_latency/top_p_per_token	412.35 µs	357.72 µs	-13.2%
🟢	add/medium_f16_threads=1-internal/262144	2.84 ms	2.41 ms	-15.1%
🟢	matmul/medium_generic_bf16_threads=1/32x512x512	592.19 µs	499.38 µs	-15.7%
🟢	add/medium_bf16_threads=1-internal/262144	3.04 ms	2.56 ms	-15.9%
🟢	add/large_f32_threads=1-internal/4194304	45.02 ms	37.70 ms	-16.3%
🟢	gather/large_f16_threads=1-internal/131072	17.10 µs	14.29 µs	-16.4%
🟢	matmul/large_generic_f32_threads=1/32x1024x1024	10.34 ms	8.61 ms	-16.7%
🟢	add/large_f16_threads=1-internal/4194304	47.05 ms	38.87 ms	-17.4%
🟢	gather/large_bf16_threads=1-internal/131072	18.75 µs	15.45 µs	-17.6%
🟢	matmul/large_generic_f32_threads=8/32x1024x1024	4.52 ms	3.69 ms	-18.5%
🟢	add/medium_f32_threads=1-internal/262144	3.26 ms	2.61 ms	-19.9%
🟢	add/small_f16_threads=1-internal/1024	15.26 µs	12.19 µs	-20.2%
🟢	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	58.29 µs	44.90 µs	-23.0%
🟢	matmul/large_generic_bf16_threads=1/32x1024x1024	2.47 ms	1.89 ms	-23.6%
🟢	add/small_f32_threads=1-internal/1024	244.5 ns	186.4 ns	-23.8%
🟢	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	487.62 µs	363.65 µs	-25.4%
🟢	matmul/large_generic_f16_threads=8/32x1024x1024	111.24 µs	80.90 µs	-27.3%
🟢	add/small_bf16_threads=1-internal/1024	17.15 µs	12.07 µs	-29.7%
🟢	matmul/medium_generic_bf16_threads=8/32x512x512	505.43 µs	353.12 µs	-30.1%
🟢	matmul/small_generic_f32_threads=1/1x256x256	62.55 µs	36.40 µs	-41.8%
🟢	matmul/small_generic_f32_threads=8/1x256x256	91.91 µs	49.27 µs	-46.4%
🟢	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	1.32 ms	594.19 µs	-54.8%
🟢	matmul/large_generic_bf16_threads=8/32x1024x1024	3.47 ms	1.23 ms	-64.5%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 4.49 3.65 6.81 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)