[justinchuby]

onnx-genai-ort::cuda_rt listed only CUDA 12 cudart names, while the CUDA EP's own loader (onnx-runtime-ep-cuda::dynamic_library::candidates) already listed CUDA 13. Two lists for one job, drifted apart.

Why it looked like something else. On a CUDA 13 host every candidate in this list fails, and the error reports the last one tried: could not load CUDA runtime (cudart): libcudart.so: LoadLibraryExW failed -- a Linux name failing on Windows. That reads as this machine has no CUDA, not this list is stale. It is why native CUDA appeared unavailable on a box with a working RTX 4060, driver 591.55, CUDA 13.1, and a complete CUDA 13 runtime present.

Verified on that host.

• Before, with only CUDA 13 on PATH: load fails at cudart.
• After, same PATH: gets past cudart and commits real device memory --- cuda_ep: VMM arena closing: committed 2097152 B of 68719476736 B reserved.
• Control: adding a CUDA 12 runtime directory to PATH produced that same progression before the fix, which isolates the cause to the missing name rather than to anything else in the environment.

Guard, falsified. every_cuda_major_the_ep_loads_is_also_resolvable_here pins every major the EP accepts. Dropping cudart64_13.dll reddens it: missing cudart64_13.dll; the CUDA EP loads this major but cuda_rt would not.

Follow-up, not in this PR. The real fix is one table, not two agreeing tables. Both crates already depend on onnx-genai-cuda-version-guard, which is the natural home for it. Filed separately; this PR unblocks CUDA 13 hosts now.

Note that a second, unrelated defect sits immediately behind this one and is now reachable: native CUDA decode fails at node 14 (op '::Attention') reached execution without prepared SessionPersistent workspace. Filed separately.

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	matmul/small_generic_f32_threads=8/1x256x256	35.39 µs	86.45 µs	+144.3%
🔴	matmul/large_generic_f32_threads=8/32x1024x1024	3.75 ms	6.75 ms	+79.7%
🔴	matmul/medium_generic_f16_threads=8/32x512x512	29.41 µs	44.43 µs	+51.1%
🔴	gather/large_bf16_threads=1-internal/131072	11.09 µs	16.34 µs	+47.3%
🔴	matmul/small_generic_bf16_threads=1/1x256x256	30.15 µs	40.02 µs	+32.7%
🔴	matmul/small_generic_f16_threads=1/1x256x256	30.25 µs	40.07 µs	+32.5%
🔴	matmul/medium_generic_bf16_threads=8/32x512x512	368.52 µs	481.60 µs	+30.7%
🔴	matmul/small_generic_f32_threads=1/1x256x256	36.88 µs	48.01 µs	+30.2%
🔴	matmul/medium_generic_f16_threads=1/32x512x512	29.00 µs	37.72 µs	+30.1%
⚠️	gather/large_f32_threads=1-internal/131072	28.61 µs	36.93 µs	+29.1%
⚠️	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	91.68 µs	108.28 µs	+18.1%
⚠️	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	125.37 µs	145.32 µs	+15.9%
⚠️	matmul/medium_generic_bf16_threads=1/32x512x512	508.85 µs	585.93 µs	+15.1%
✅	matmul/small_generic_bf16_threads=8/1x256x256	30.93 µs	34.73 µs	+12.3%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.22 ms	2.48 ms	+11.8%
✅	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	1.14 ms	1.27 ms	+11.6%
✅	sampling_latency/top_p_per_token	362.84 µs	399.54 µs	+10.1%
✅	gather/large_f16_threads=1-internal/131072	11.39 µs	12.50 µs	+9.7%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	78.44 µs	85.94 µs	+9.6%
✅	tokenization/encode_tokens_per_second	377.82 µs	412.17 µs	+9.1%
✅	matmul/small_generic_f16_threads=8/1x256x256	29.79 µs	32.39 µs	+8.7%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.00 ms	9.77 ms	+8.6%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.07 ms	2.21 ms	+6.8%
✅	tokenization/decode_tokens_per_second	6.02 ms	6.42 ms	+6.6%
✅	sampling_latency/greedy_per_token	3.11 µs	3.31 µs	+6.4%
✅	sampling_latency/top_k_per_token	49.97 µs	52.49 µs	+5.0%
✅	gather/medium_f32_threads=1-internal/32768	3.62 µs	3.79 µs	+4.8%
✅	matmul/large_generic_f16_threads=8/32x1024x1024	82.51 µs	86.28 µs	+4.6%
✅	gather/medium_bf16_threads=1-internal/32768	2.38 µs	2.43 µs	+2.1%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	1.88 ms	1.91 ms	+1.7%
✅	gather/medium_f16_threads=1-internal/32768	2.41 µs	2.45 µs	+1.6%
✅	add/large_bf16_threads=1-internal/4194304	1.56 ms	1.57 ms	+0.6%
✅	add/small_bf16_threads=1-internal/1024	429.5 ns	430.8 ns	+0.3%
✅	add/medium_bf16_threads=1-internal/262144	98.55 µs	98.23 µs	-0.3%
✅	reduce_mean/small_f32_threads=1-internal/4096	14.37 µs	14.30 µs	-0.5%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	177.09 µs	175.69 µs	-0.8%
✅	qwen3_sampling_processors/top_k_top_p_fast	635.90 µs	629.95 µs	-0.9%
✅	grammar_masking/llguidance_compute_mask/32	73.69 µs	72.69 µs	-1.4%
✅	kv_cache/alloc_dealloc_pages	37.87 µs	37.12 µs	-2.0%
✅	gather/small_bf16_threads=1-internal/4096	465.3 ns	453.4 ns	-2.6%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.47 ms	3.36 ms	-3.1%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	501.67 µs	484.52 µs	-3.4%
✅	logit_processing/seven_processor_chain_per_step	310.14 µs	297.91 µs	-3.9%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	5.50 ms	5.27 ms	-4.3%
✅	reduce_mean/medium_f32_threads=1-internal/65536	244.78 µs	234.33 µs	-4.3%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.00 ms	957.92 µs	-4.5%
✅	gather/small_f32_threads=1-internal/4096	678.6 ns	644.5 ns	-5.0%
✅	sampling_latency/min_p_per_token	205.07 µs	192.61 µs	-6.1%
✅	qwen3_sampling_processors/top_k_partial_selection	140.43 µs	131.05 µs	-6.7%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.57 ms	1.46 ms	-7.3%
✅	matmul/medium_generic_f32_threads=8/32x512x512	1.08 ms	1.00 ms	-7.4%
✅	add/large_f16_threads=1-internal/4194304	1.74 ms	1.57 ms	-9.8%
✅	add/small_f16_threads=1-internal/1024	473.6 ns	426.5 ns	-9.9%
✅	gather/small_f16_threads=1-internal/4096	523.4 ns	455.2 ns	-13.0%
✅	add/medium_f32_threads=1-internal/262144	27.41 µs	23.51 µs	-14.2%
✅	add/large_f32_threads=1-internal/4194304	639.49 µs	546.63 µs	-14.5%
🟢	add/medium_f16_threads=1-internal/262144	120.77 µs	101.83 µs	-15.7%
🟢	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	645.55 µs	516.30 µs	-20.0%
🟢	add/small_f32_threads=1-internal/1024	257.3 ns	185.9 ns	-27.8%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 2.25 3.37 5.33 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.57%. Comparing base (73c3df0) to head (59d05d6).
⚠️ Report is 35 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1178      +/-   ##
==========================================
+ Coverage   80.47%   80.57%   +0.10%     
==========================================
  Files         368      357      -11     
  Lines      161309   154551    -6758     
  Branches   161309   154551    -6758     
==========================================
- Hits       129807   124530    -5277     
+ Misses      26763    25470    -1293     
+ Partials     4739     4551     -188     

Flag	Coverage Δ
cli-ort-windows	?
mlas	?
offline	80.57% <ø> (+0.31%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 36 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.