perf(cpu-ep): name the prefill/MLAS Rayon workers

[justinchuby]

The problem

Rayon does not name the workers build_global creates, and an unnamed thread's comm defaults to the process name. So the pool that bound_process_to_decode_budget builds appears in ps, top and /proc/<pid>/task/*/comm as N extra copies of the host binary.

This is not a cosmetic gap. It was the largest unattributed block of threads in any budgeted process, and it is what kept "the process has roughly twice the budget in threads and we don't know whose they are" open as a question across several phases of the CPU scheduler campaign.

A flat thread census does not merely fail to explain these threads — it misattributes them to the caller, which is worse, because the census looks complete.

What the naming immediately revealed

Census on gemm_nbits_qwen3_0p6b_qkv_t8, ONNX_GENAI_CPU_DECODE_THREADS=16, decode-only:

== thread census (48 threads) ==
                        name   count        cpu_ms
             onnx-genai-deco      16          20.0
             bench_decode_ga       1          40.0
              nxgn-prefill-0       1           0.0
              nxgn-prefill-1       1           0.0
              ... (16 total)              all 0.0

Before this change the sixteen nxgn-prefill-* rows read as bench_decode_ga — indistinguishable from the benchmark's own main thread.

All sixteen report 0.0 ms of CPU. The pool is built eagerly at EP init, is never used by a decode-only workload, and holds N threads for the process lifetime.

That also quantifies the budget's real cost, which is larger than it looks:

configuration	threads	composition
default (no budget)	22	1 main + 6 decode + 15 task-runtime
budget 16	48	1 + 16 prefill Rayon + 16 decode + 15 task-runtime
budget 32	80	1 + 32 prefill Rayon + 32 decode + 15 task-runtime

Setting an explicit budget of N does not size one pool to N — it sizes the decode pool to N and builds a second, separate N-wide pool. Half of that was previously anonymous.

A useful negative result falls out of the same census: the ORT session contributes exactly zero threads. Whatever else is going on, our EP is not double-provisioning against ORT.

Why the names are short

Linux stores comm in 15 bytes. This crate's existing onnx-genai--prefixed convention (decode_numa.rs, decode_spmd.rs) already exceeds that and collapses to onnx-genai-deco / onnx-genai-spmd, losing the worker index — visible in the census above, where all 16 decode workers fold into one row. nxgn-prefill-15 is exactly 15 bytes and survives intact, matching the short form the task runtime already uses (nxrt-task-N).

The test asserts the property — fits in comm, keeps its prefix, keeps its index, stays distinct — rather than the literal string, since the string is not the thing that has to hold.

Scope

Thread names only. No behaviour change, no sizing change, no scheduling change. 1448/1448 onnx-runtime-ep-cpu lib tests pass; fmt and clippy clean.

Making the pool lazy is the real fix and is deliberately not attempted here. It would require every global-Rayon entry point in the default build to route through a guard before first use, and missing one silently un-bounds the user's explicit budget — a correctness regression traded for a thread-count saving. Named as a measured cost with a known shape (§41) so the decision is on the record rather than rediscovered.

Includes the one-line rustfmt repair of onnx-runtime-ep-cuda/src/runtime.rs that main is currently failing on (same as #1393/#1395 — whichever lands first makes the others a no-op). Without it this branch cannot be green.

Follows #1395, which built the census that made this diagnosable.

🤖 Generated with GitHub Copilot CLI

[github-actions]

🔴 Benchmark Regression Detected

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
🔴	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	48.70 µs	138.80 µs	+185.0%
🔴	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	60.10 µs	141.92 µs	+136.1%
🔴	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	571.09 µs	1.25 ms	+119.7%
🔴	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	44.48 µs	97.29 µs	+118.7%
🔴	matmul/medium_generic_bf16_threads=8/32x512x512	391.95 µs	759.61 µs	+93.8%
🔴	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	387.58 µs	600.20 µs	+54.9%
🔴	matmul/medium_generic_f16_threads=8/32x512x512	35.24 µs	51.14 µs	+45.1%
🔴	grammar_masking/llguidance_compute_mask/32	72.10 µs	101.86 µs	+41.3%
🔴	kv_cache/alloc_dealloc_pages	37.85 µs	51.92 µs	+37.2%
🔴	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	5.37 ms	7.19 ms	+33.9%
🔴	matmul/small_generic_f16_threads=8/1x256x256	30.88 µs	40.49 µs	+31.1%
⚠️	qwen3_sampling_processors/top_k_top_p_fast	626.63 µs	811.09 µs	+29.4%
⚠️	logit_processing/seven_processor_chain_per_step	313.34 µs	394.80 µs	+26.0%
⚠️	matmul/large_generic_f16_threads=8/32x1024x1024	84.90 µs	106.31 µs	+25.2%
⚠️	matmul/large_generic_f32_threads=8/32x1024x1024	3.68 ms	4.56 ms	+23.9%
⚠️	tokenization/decode_tokens_per_second	6.06 ms	7.47 ms	+23.2%
⚠️	matmul/medium_generic_bf16_threads=1/32x512x512	540.98 µs	664.18 µs	+22.8%
⚠️	matmul/large_generic_bf16_threads=1/32x1024x1024	2.06 ms	2.46 ms	+19.5%
⚠️	tokenization/encode_tokens_per_second	367.22 µs	428.38 µs	+16.7%
⚠️	qwen3_sampling_processors/top_k_partial_selection	149.62 µs	173.69 µs	+16.1%
✅	sampling_latency/top_p_per_token	379.18 µs	434.56 µs	+14.6%
✅	matmul/large_generic_f16_threads=1/32x1024x1024	78.96 µs	89.30 µs	+13.1%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	519.70 µs	577.98 µs	+11.2%
✅	sampling_latency/min_p_per_token	204.86 µs	223.39 µs	+9.0%
✅	matmul/small_generic_bf16_threads=1/1x256x256	30.40 µs	33.07 µs	+8.8%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.15 ms	2.32 ms	+8.3%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.64 ms	3.87 ms	+6.5%
✅	matmul/small_generic_f16_threads=1/1x256x256	31.24 µs	33.12 µs	+6.0%
✅	add/medium_f32_threads=1-internal/262144	25.48 µs	26.87 µs	+5.5%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.44 ms	9.85 ms	+4.4%
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	1.86 ms	1.94 ms	+4.3%
✅	sampling_latency/greedy_per_token	3.29 µs	3.37 µs	+2.4%
✅	sampling_latency/top_k_per_token	53.17 µs	54.20 µs	+1.9%
✅	matmul/small_generic_f32_threads=1/1x256x256	35.57 µs	35.78 µs	+0.6%
✅	matmul/small_generic_f32_threads=8/1x256x256	33.15 µs	33.12 µs	-0.1%
✅	gather/medium_f32_threads=1-internal/32768	3.63 µs	3.60 µs	-0.9%
✅	matmul/medium_generic_f16_threads=1/32x512x512	31.44 µs	31.17 µs	-0.9%
✅	gather/small_f16_threads=1-internal/4096	489.0 ns	484.2 ns	-1.0%
✅	gather/small_f32_threads=1-internal/4096	649.6 ns	623.0 ns	-4.1%
✅	gather/large_bf16_threads=1-internal/131072	11.49 µs	10.94 µs	-4.8%
✅	gather/small_bf16_threads=1-internal/4096	574.9 ns	541.5 ns	-5.8%
✅	gather/medium_f16_threads=1-internal/32768	2.39 µs	2.25 µs	-6.0%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.01 ms	938.75 µs	-6.8%
✅	gather/medium_bf16_threads=1-internal/32768	2.52 µs	2.29 µs	-9.1%
✅	add/small_bf16_threads=1-internal/1024	463.1 ns	414.1 ns	-10.6%
✅	add/medium_f16_threads=1-internal/262144	115.26 µs	101.34 µs	-12.1%
✅	matmul/medium_generic_f32_threads=8/32x512x512	1.07 ms	933.01 µs	-12.4%
✅	reduce_mean/medium_f32_threads=1-internal/65536	260.58 µs	226.68 µs	-13.0%
✅	reduce_mean/small_f32_threads=1-internal/4096	16.33 µs	14.16 µs	-13.3%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.50 ms	2.16 ms	-13.8%
✅	add/large_bf16_threads=1-internal/4194304	1.80 ms	1.55 ms	-13.8%
🟢	add/small_f16_threads=1-internal/1024	516.2 ns	431.9 ns	-16.3%
🟢	gather/large_f16_threads=1-internal/131072	13.55 µs	11.19 µs	-17.4%
🟢	add/small_f32_threads=1-internal/1024	250.3 ns	189.1 ns	-24.4%
🟢	matmul/small_generic_bf16_threads=8/1x256x256	40.06 µs	29.68 µs	-25.9%
🟢	add/medium_bf16_threads=1-internal/262144	144.55 µs	104.67 µs	-27.6%
🟢	gather/large_f32_threads=1-internal/131072	38.32 µs	26.24 µs	-31.5%
🟢	add/large_f16_threads=1-internal/4194304	2.72 ms	1.56 ms	-42.4%
🟢	add/large_f32_threads=1-internal/4194304	1.09 ms	606.27 µs	-44.6%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 5.59 4.15 5.42 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

[codecov]

Codecov Report

❌ Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.14%. Comparing base (4a9f4ec) to head (e3072ca).
⚠️ Report is 72 commits behind head on main.

Files with missing lines	Patch %	Lines
...es/onnx-runtime-ep-cpu/src/kernels/matmul_nbits.rs	87.50%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1398       +/-   ##
===========================================
- Coverage   82.10%   80.14%    -1.97%     
===========================================
  Files          12      376      +364     
  Lines        5471   164303   +158832     
  Branches     5471   164303   +158832     
===========================================
+ Hits         4492   131675   +127183     
- Misses        780    27801    +27021     
- Partials      199     4827     +4628     

Flag	Coverage Δ
cli-ort-linux	82.60% <ø> (?)
cli-ort-windows	82.10% <ø> (ø)
offline	80.05% <87.50%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...es/onnx-runtime-ep-cpu/src/kernels/matmul_nbits.rs	77.05% <87.50%> (ø)

... and 364 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.