[justinchuby]

A trap I walked into while validating #1214, worth writing down because the failure mode is invisible.

Restoring a falsified file with Copy-Item/Move-Item preserves the original LastWriteTime, so the restored file looks older than the artifact cargo built from the broken version. cargo's freshness check is mtime-based, sees nothing newer, and reuses the stale binary.

The direction it bit me in was benign -- I saw two red tests against source I could read as correct. The dangerous mirror image is breaking the code, restoring, seeing green, and concluding the test is worthless. A falsification loop is exactly when you are toggling one line and trusting the pass/fail signal completely, which makes it exactly when a stale build does the most damage.

Fix is one line: (Get-Item <path>).LastWriteTime = Get-Date after any restore.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.49%. Comparing base (508920d) to head (c1f3f40).
⚠️ Report is 33 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1216      +/-   ##
==========================================
+ Coverage   79.88%   80.49%   +0.60%     
==========================================
  Files         359      359              
  Lines      157592   157898     +306     
  Branches   157592   157898     +306     
==========================================
+ Hits       125897   127098    +1201     
+ Misses      27131    26184     -947     
- Partials     4564     4616      +52     

Flag	Coverage Δ
mlas	85.33% <ø> (-0.07%)	⬇️
offline	80.39% <ø> (+0.61%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 19 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

⚠️ Benchmark Change Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
⚠️	gather/large_bf16_threads=1-internal/131072	15.61 µs	20.13 µs	+28.9%
⚠️	gather/small_f32_threads=1-internal/4096	680.7 ns	844.1 ns	+24.0%
⚠️	matmul/small_generic_bf16_threads=1/1x256x256	34.50 µs	42.01 µs	+21.8%
⚠️	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	6.50 ms	7.86 ms	+21.0%
⚠️	qwen3_sampling_processors/top_k_partial_selection	160.17 µs	190.52 µs	+19.0%
⚠️	reduce_mean/large_f32_threads=1-internal/262144	989.83 µs	1.17 ms	+18.6%
⚠️	matmul/large_generic_f16_threads=8/32x1024x1024	107.40 µs	125.61 µs	+17.0%
⚠️	grammar_masking/llguidance_compute_mask/32	86.92 µs	101.64 µs	+16.9%
⚠️	matmul/large_generic_f32_threads=1/32x1024x1024	12.64 ms	14.62 ms	+15.7%
✅	add/large_bf16_threads=1-internal/4194304	1.76 ms	2.02 ms	+14.8%
✅	logit_processing/seven_processor_chain_per_step	343.04 µs	391.88 µs	+14.2%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	578.98 µs	656.99 µs	+13.5%
✅	reduce_mean/small_f32_threads=1-internal/4096	15.51 µs	17.52 µs	+13.0%
✅	tokenization/encode_tokens_per_second	394.89 µs	444.13 µs	+12.5%
✅	add/large_f32_threads=1-internal/4194304	900.80 µs	1.01 ms	+12.2%
✅	qwen3_sampling_processors/top_k_top_p_fast	728.57 µs	812.33 µs	+11.5%
✅	matmul/small_generic_f16_threads=1/1x256x256	36.81 µs	40.98 µs	+11.3%
✅	kv_cache/alloc_dealloc_pages	46.16 µs	51.15 µs	+10.8%
✅	gather/small_f16_threads=1-internal/4096	488.0 ns	529.8 ns	+8.6%
✅	matmul/small_generic_f32_threads=1/1x256x256	42.39 µs	45.40 µs	+7.1%
✅	matmul/small_generic_f16_threads=8/1x256x256	41.48 µs	44.00 µs	+6.1%
✅	reduce_mean/medium_f32_threads=1-internal/65536	254.47 µs	269.84 µs	+6.0%
✅	matmul/small_generic_f32_threads=8/1x256x256	54.17 µs	57.31 µs	+5.8%
✅	matmul/large_generic_f32_threads=8/32x1024x1024	6.65 ms	7.00 ms	+5.4%
✅	matmul/medium_generic_f32_threads=8/32x512x512	1.57 ms	1.66 ms	+5.2%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	187.46 µs	196.77 µs	+5.0%
✅	tokenization/decode_tokens_per_second	7.32 ms	7.66 ms	+4.6%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	96.47 µs	100.16 µs	+3.8%
✅	add/small_f16_threads=1-internal/1024	600.0 ns	621.9 ns	+3.7%
✅	matmul/medium_generic_f32_threads=1/32x512x512	3.15 ms	3.25 ms	+3.3%
✅	sampling_latency/min_p_per_token	256.17 µs	259.16 µs	+1.2%
✅	gather/large_f32_threads=1-internal/131072	62.51 µs	62.73 µs	+0.3%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	4.23 ms	4.22 ms	-0.1%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.27 ms	2.26 ms	-0.4%
✅	add/large_f16_threads=1-internal/4194304	2.01 ms	1.99 ms	-0.8%
✅	add/medium_f32_threads=1-internal/262144	31.22 µs	30.85 µs	-1.2%
✅	add/medium_bf16_threads=1-internal/262144	133.79 µs	130.84 µs	-2.2%
✅	matmul/medium_generic_bf16_threads=8/32x512x512	676.11 µs	647.16 µs	-4.3%
✅	add/small_f32_threads=1-internal/1024	254.0 ns	242.1 ns	-4.7%
✅	gather/medium_f16_threads=1-internal/32768	3.22 µs	3.06 µs	-5.0%
✅	sampling_latency/greedy_per_token	3.83 µs	3.63 µs	-5.2%
✅	matmul/medium_generic_f16_threads=1/32x512x512	43.29 µs	40.49 µs	-6.5%
✅	gather/medium_bf16_threads=1-internal/32768	3.65 µs	3.38 µs	-7.4%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	695.21 µs	641.71 µs	-7.7%
✅	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	106.14 µs	97.56 µs	-8.1%
✅	matmul/medium_generic_f16_threads=8/32x512x512	51.83 µs	47.52 µs	-8.3%
✅	add/small_bf16_threads=1-internal/1024	610.9 ns	548.4 ns	-10.2%
✅	gather/small_bf16_threads=1-internal/4096	588.7 ns	527.3 ns	-10.4%
✅	sampling_latency/top_p_per_token	497.98 µs	442.63 µs	-11.1%
✅	gather/large_f16_threads=1-internal/131072	26.49 µs	23.36 µs	-11.8%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	2.15 ms	1.87 ms	-12.9%
✅	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	616.09 µs	536.46 µs	-12.9%
✅	matmul/small_generic_bf16_threads=8/1x256x256	49.17 µs	42.01 µs	-14.6%
✅	gather/medium_f32_threads=1-internal/32768	6.16 µs	5.25 µs	-14.7%
🟢	add/medium_f16_threads=1-internal/262144	169.71 µs	141.05 µs	-16.9%
🟢	sampling_latency/top_k_per_token	78.48 µs	65.07 µs	-17.1%
🟢	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	1.22 ms	866.80 µs	-29.0%
🟢	matmul/large_generic_bf16_threads=1/32x1024x1024	3.08 ms	2.18 ms	-29.1%
🟢	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	124.53 µs	81.33 µs	-34.7%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 8.09 5.48 7.16 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)