[justinchuby]

Fixes the nxrt-ep-cuda wheel job (failed in the first publish run) and makes the NVIDIA runtime wheels required dependencies.

Build fix

The CUDA job used nvidia/cuda:13.0.0-devel-ubi9, which is not a manylinux image — it lacks cibuildwheel's /opt/python/* CPython toolchains:

'/opt/python/cp310-cp310/bin/python' executable doesn't exist in image 'nvidia/cuda:13.0.0-devel-ubi9'

Switched the CUDA job to the same PyPA manylinux image as the CPU job (quay.io/pypa/manylinux_2_28_x86_64). No CUDA toolkit is installed: onnx-runtime-ep-cuda binds CUDA via cudarc's dynamic-loading, so cargo build --features cuda needs no nvcc/headers/GPU — CUDA libs are dlopen'd at runtime. Verified inside the real image: cargo build --features cuda OK, readelf -d shows the .so links zero CUDA libs, auditwheel repair → py3-none-manylinux_2_28_x86_64, pip install + import nxrt_ep_cuda OK.

Required CUDA 13 dependencies

The four NVIDIA runtime wheels are required deps (not an extra), so pip install nxrt-ep-cuda pulls them automatically:

nvidia-cuda-runtime>=13,<14
nvidia-cublas>=13,<14
nvidia-cuda-nvrtc>=13,<14
nvidia-cuda-cupti>=13,<14

Uses the unsuffixed names — verified on PyPI these are the real CUDA 13 wheels (nvidia-cuda-runtime 13.3.29 etc.); the -cu13-suffixed names are 0.0.1 placeholder stubs. <14 locks the CUDA 13 major.

CPU job/package untouched. cpu/cuda jobs remain fully independent.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	53.91 µs	75.75 µs	+40.5%
⚠️	matmul/medium_generic_f16_threads=1/32x512x512	40.12 µs	47.62 µs	+18.7%
⚠️	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	433.63 µs	511.28 µs	+17.9%
⚠️	matmul/medium_generic_bf16_threads=8/32x512x512	455.27 µs	531.24 µs	+16.7%
✅	grammar_masking/llguidance_compute_mask/32	82.08 µs	90.30 µs	+10.0%
✅	logit_processing/seven_processor_chain_per_step	330.00 µs	342.60 µs	+3.8%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.80 ms	1.86 ms	+3.1%
✅	matmul/medium_generic_f16_threads=8/32x512x512	39.41 µs	39.97 µs	+1.4%
✅	kv_cache/alloc_dealloc_pages	40.68 µs	40.62 µs	-0.2%
✅	gather/small_bf16_threads=1-internal/4096	464.3 ns	459.0 ns	-1.1%
✅	matmul/small_generic_bf16_threads=8/1x256x256	37.71 µs	36.87 µs	-2.2%
✅	add/small_bf16_threads=1-internal/1024	13.52 µs	12.92 µs	-4.4%
✅	matmul/small_generic_f16_threads=1/1x256x256	29.74 µs	28.39 µs	-4.5%
✅	matmul/small_generic_bf16_threads=1/1x256x256	31.40 µs	29.78 µs	-5.2%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.33 ms	2.21 ms	-5.4%
✅	reduce_mean/small_f32_threads=1-internal/4096	16.70 µs	15.74 µs	-5.8%
✅	matmul/small_generic_f32_threads=8/1x256x256	34.55 µs	32.01 µs	-7.3%
✅	gather/medium_bf16_threads=1-internal/32768	2.38 µs	2.20 µs	-7.3%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.83 ms	2.60 ms	-8.0%
✅	qwen3_sampling_processors/top_k_top_p_fast	757.29 µs	694.94 µs	-8.2%
✅	add/medium_f32_threads=1-internal/262144	2.85 ms	2.60 ms	-8.8%
✅	matmul/small_generic_f32_threads=1/1x256x256	38.18 µs	34.63 µs	-9.3%
✅	gather/small_f16_threads=1-internal/4096	487.4 ns	441.8 ns	-9.4%
✅	matmul/medium_generic_f32_threads=8/32x512x512	1.94 ms	1.75 ms	-10.1%
✅	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	828.82 µs	737.25 µs	-11.0%
✅	gather/small_f32_threads=1-internal/4096	703.7 ns	625.8 ns	-11.1%
✅	add/medium_bf16_threads=1-internal/262144	3.12 ms	2.77 ms	-11.3%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	623.10 µs	540.81 µs	-13.2%
✅	matmul/medium_generic_bf16_threads=1/32x512x512	684.57 µs	591.19 µs	-13.6%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	89.79 µs	77.15 µs	-14.1%
✅	gather/medium_f16_threads=1-internal/32768	2.62 µs	2.24 µs	-14.5%
✅	matmul/small_generic_f16_threads=8/1x256x256	33.57 µs	28.64 µs	-14.7%
🟢	gather/medium_f32_threads=1-internal/32768	4.24 µs	3.53 µs	-16.7%
🟢	matmul/large_generic_f32_threads=1/32x1024x1024	11.93 ms	9.88 ms	-17.2%
🟢	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	4.49 ms	3.68 ms	-18.0%
🟢	gather/large_f16_threads=1-internal/131072	13.38 µs	10.97 µs	-18.1%
🟢	sampling_latency/top_p_per_token	470.70 µs	381.89 µs	-18.9%
🟢	gather/large_bf16_threads=1-internal/131072	13.63 µs	10.89 µs	-20.1%
🟢	sampling_latency/top_k_per_token	65.21 µs	50.53 µs	-22.5%
🟢	reduce_mean/large_f32_threads=1-internal/262144	1.20 ms	919.18 µs	-23.5%
🟢	add/large_f16_threads=1-internal/4194304	54.46 ms	41.64 ms	-23.5%
🟢	add/small_f16_threads=1-internal/1024	17.23 µs	13.11 µs	-23.9%
🟢	sampling_latency/min_p_per_token	283.68 µs	211.45 µs	-25.5%
🟢	gather/large_f32_threads=1-internal/131072	33.35 µs	24.80 µs	-25.6%
🟢	reduce_mean/medium_f32_threads=1-internal/65536	308.19 µs	228.53 µs	-25.8%
🟢	add/large_f32_threads=1-internal/4194304	56.13 ms	41.46 ms	-26.1%
🟢	qwen3_sampling_processors/top_k_partial_selection	184.83 µs	136.23 µs	-26.3%
🟢	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	8.09 ms	5.94 ms	-26.7%
🟢	matmul/large_generic_f16_threads=8/32x1024x1024	126.87 µs	90.63 µs	-28.6%
🟢	add/medium_f16_threads=1-internal/262144	3.43 ms	2.44 ms	-29.0%
🟢	qwen3_sampling_processors/top_k_full_sort_baseline	2.87 ms	2.03 ms	-29.3%
🟢	sampling_latency/greedy_per_token	4.58 µs	3.14 µs	-31.4%
🟢	add/large_bf16_threads=1-internal/4194304	62.04 ms	41.79 ms	-32.6%
🟢	matmul/large_generic_f16_threads=1/32x1024x1024	124.71 µs	78.36 µs	-37.2%
🟢	add/small_f32_threads=1-internal/1024	361.7 ns	200.8 ns	-44.5%
🟢	tokenization/encode_tokens_per_second	670.29 µs	371.04 µs	-44.6%
🟢	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	83.05 µs	42.16 µs	-49.2%
🟢	tokenization/decode_tokens_per_second	12.29 ms	6.21 ms	-49.5%
🟢	matmul/large_generic_f32_threads=8/32x1024x1024	7.48 ms	3.69 ms	-50.6%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 5.13 4.13 5.30 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)