perf(ep): tell kernels which of their inputs ORT holds constant

[justinchuby]

The bug

Kernel::set_constant_inputs is how a kernel learns that an input is a weight ORT owns for the life of the session. Kernels use it to decide whether a prepack may be built once and kept, and MatMulNBits goes further: mlas_sqnbit_owns_fp32_compute(can_prepack, …) gates the MLAS SQNBit path itself on the same flag.

It was called from exactly one place in the workspace — onnx-runtime-session's executor. The ORT plugin EP never called it. So on the only path a real ORT model takes, every kernel saw every input as a runtime tensor: MatMulNBits rebuilt its packed weights on every Run and ran the slower non-SQNBit kernel, and QLinearMatMul re-packed B per call. A decoder paid a load-time cost per token.

Why the existing test could not be reused

node_inputs_all_routable already distinguishes weights, as "no producer and not a graph input". That is right for the whole model at GetCapability time and wrong at Compile time: ORT hands a fused node's subgraph over with the initializers it kept inside listed as graph inputs of that subgraph, because they are inputs of the fused node. Instrumenting the compile path on a MatMulNBits model prints

[const] op=MatMulNBits flags=[false, false, false] is_graph_input=[true, true, true] has_producer=[false, false, false]

Graph_GetInitializers still tells the two apart, so OutboundGraphReader now records initializer names (no tensor data: the existing read_initializers_int64 deliberately copies only small int64 tensors, and a 1 GB B must not be copied to answer a yes/no question), and constant_input_flags keys against that set.

Measurement

Interleaved A/B against plain ORT through the ORT session API — the same generated single-node model loaded twice, once with this EP registered and session.disable_cpu_ep_fallback=1, once with no EP appended at all. 3 warmups, then 41 interleaved iterations each side, p50/p90 of Run only (input OrtValues built once). Assignment is asserted before any timing, so a ratio is never reported for a node ORT actually ran.

Host: AMD EPYC 9V74, 32 vCPU / 16 cores, AVX2+FMA+F16C (no AVX-512/VNNI), ORT 1.27.0, release build with --features onnx-runtime-ep-cpu/mlas, K=N=2048. The box is shared, so p90s are contended; the ours ms columns are the load-bearing evidence and the ORT column moves between runs.

case	ours ms before	ours ms after	speedup	ours/ORT before	ours/ORT after
MatMulNBits int4 M=1	1.144	0.425	2.7x	15.0	2.34
MatMulNBits int4 M=128	108.4	7.79	13.9x	91.6	6.36
MatMulNBits int4 f16-act M=1	3.659	0.416	8.8x	16.0	1.80
MatMulNBits int8 M=1	8.226	4.845	1.7x	3.85	2.53
MatMulNBits int8 M=256	47.98	12.04	4.0x	5.03	1.05
QLinearMatMul u8 M=1	2.234	0.079	28x	43.6	1.84
QLinearMatMul u8 M=128	13.32	11.11	1.2x	2.15	3.78
QLinearMatMul i8 M=1	1.968	0.091	21.6x	0.52	0.055
MatMul f32 M=1 (control)	0.147	0.178	—	1.72	1.49
MatMul f32 M=128 (control)	9.675	8.698	—	1.36	1.43
MatMul f16 M=1 (control)	1.368	2.358	—	1.08	0.37
MatMul f16 M=128 (control)	21.04	22.64	—	7.04	2.84

The four dense cases are the control: they declare both operands as graph inputs, so they have no initializer, no flag changes for them, and they move only with host noise. Every case that moved has a constant weight.

Cold session-creation time is reported by the same harness (cold_ours_ms) and does not regress: the prepack moved from per-Run to first-Run, not to CreateSession.

Still losing after this change

Reporting all of it, per the standing rule:

• MatMulNBits int4 M=128 at 6.36x and M=1 at 2.34x.
• QLinearMatMul u8 M=128 at 3.78x (u8 M=1 is now 1.84x).
• MatMul f16 M=128 at 2.84x — no initializer involved, so untouched here.
• MatMul f32 at 1.43-1.49x — likewise.

These are kernel gaps, not wiring gaps, and they stay open on my task list.

Separate finding, not fixed here: the plugin cdylib is built without the mlas feature by default, and docs/architecture/CROSS_PLATFORM.md documents that feature as x86-64-Linux-only. Rebuilding the same benchmark against the default cdylib gives, after this fix, MatMulNBits int4 M=128 at 80x and QLinearMatMul u8 M=128 at 56x ours/ORT: the pure-Rust paths are the ones a portable build actually ships, and they are far behind. That is a distinct piece of work and needs its own PR.

Tests

• constant_weights_are_reported_to_kernels_as_constant (new, plugin_ort_e2e) — reads the counter this PR exports from the very cdylib ORT loaded and asserts symmetric int4 reports 2 constant inputs, asymmetric int8 reports 3, and the all-graph-input dense case reports 0. A wiring that marks everything constant fails the third assertion; caching a prepack of an activation is wrong, not merely slow. Falsified by reverting constant_input_flags to the producer/is_graph_input form: reported 0 constant inputs, expected 2.
• initializers_are_constant_even_when_the_subgraph_calls_them_inputs and nothing_is_constant_without_an_initializer_list (new, onnx-runtime-ep-plugin unit) — hardware-independent, pin the exact shape ORT presents and the absent-optional-input case.
• no_matmul_family_node_escapes_to_the_ort_cpu_ep (extended) — every case now runs a second time in the same session with the activation rotated by one element, and is compared against ORT again. A weight cache still matches ORT; anything activation-derived that outlived the call does not. ORT is the oracle for whether a case can detect staleness at all (a saturating u8 output cannot), and the suite asserts at least 5 of the 10 cases are activation-sensitive so the check cannot go vacuous.
• plugin_path_ab_vs_plain_ort (new, #[ignore] + NXRT_MM_BENCH=1) — the harness that produced the table, committed so the numbers are reproducible rather than asserted.
• plugin_export_abi — the two new exports are added to both the required-symbol list and the unexpected-symbol filter.

Verification

• cargo test -p onnx-runtime-ep-cpu-plugin — 51 + 9 + 6 + 1 passed, 1 ignored (the benchmark)
• cargo test -p onnx-runtime-ep-plugin --lib — 231 passed (was 229)
• cargo fmt --all -- --check clean (given style: reformat the int4 tail-block slice #1104 left unformatted #1109, which repairs main)
• cargo clippy --all-targets clean with and without --features onnx-runtime-ep-cpu/mlas

---

Post-review fix (commit 2): overridable initializers are not constant

Independent review found a correctness hole in commit 1, and it was right.

Graph_GetInitializers is documented in ORT's own header as including "constant and non-constant initializers". From ONNX IR version 4, an initializer whose name also appears in the graph's input list is only a default: the caller may hand a different tensor in on any Run. ORT says so at load time:

[W:onnxruntime:, graph.cc:1419 Graph] Initializer B appears in graph inputs and will not be
treated as constant value/weight.

Keying the flags on that name list alone therefore marked such a value constant, and MatMulNBits would have cached a session-lifetime prepack of the default weight and returned its answer for every later Run — a wrong result, not a slow one.

Fix: each name from Graph_GetInitializers is now filtered through ValueInfo_IsConstantInitializer (ORT ≥1.23, reads no tensor data). Fail-closed: if the entry point is missing or the call errors, the initializer is treated as non-constant. A false negative costs a repeated prepack; a false positive costs correctness.

New regression test an_overridable_initializer_is_not_treated_as_a_constant_weight builds exactly that model — B and scales declared both as initializers and as graph inputs — and asserts two independent things:

1. the EP reports 0 constant inputs (the classification), and
2. running the session twice with two different B payloads matches plain ORT run for run, with the test first proving via ORT that the two payloads do produce different outputs (the consequence — a stale prepack would return run one's answer twice).

Falsified by deleting the is_constant filter and rebuilding the cdylib:

assertion `left == right` failed: an initializer that is also a graph input may be replaced
on any Run, but this EP reported 2 of them as constant weights

The two doc comments that asserted "every initializer is constant" are corrected, and the unit test is renamed to constant_initializers_are_flagged_even_when_the_subgraph_calls_them_inputs.

The measured win survives the stricter gate

Re-measured after the fix on the same host, same build (--features onnx-runtime-ep-cpu/mlas, release, K=N=2048, 3 warmups + 41 interleaved iterations, p50 ms):

case	ours before PR	ours after commit 1	ours after commit 2
nbits4_m1	1.144	0.425	0.400
nbits4_m128	108.4	7.79	8.80
nbits4_f16_m1	3.659	0.416	0.437
nbits8_m256	47.98	12.04	13.86
qlinear_u8_m1	2.234	0.079	0.092
qlinear_i8_m1	1.968	0.091	0.092

The benchmark models declare their weights as initializers only, so they are still constant under the stricter rule; the residual movement is host noise on a shared machine (ORT's own p50 for nbits4_m1 moved 0.125 → 0.077 ms between the two runs, so the ratio column is the noisier of the two and the absolute ours column is the honest comparison here).

Verification (re-run)

• cargo test -p onnx-runtime-ep-cpu-plugin --test plugin_ort_e2e — 52 passed, 1 ignored (was 51 + 1)
• cargo test -p onnx-runtime-ep-plugin --lib — 231 passed
• cargo fmt --all -- --check clean (rebased onto style: reformat the int4 tail-block slice #1104 left unformatted #1109, now merged)
• cargo clippy -p onnx-runtime-ep-cpu-plugin -p onnx-runtime-ep-plugin --all-targets clean with and without --features onnx-runtime-ep-cpu/mlas

[codecov]

Codecov Report

❌ Patch coverage is 90.90909% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.86%. Comparing base (805890e) to head (cc873b5).

Files with missing lines	Patch %	Lines
crates/onnx-runtime-ep-plugin/src/graph_reader.rs	78.18%	5 Missing and 7 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1110      +/-   ##
==========================================
- Coverage   80.40%   79.86%   -0.55%     
==========================================
  Files         369      367       -2     
  Lines      160217   157553    -2664     
  Branches   160217   157553    -2664     
==========================================
- Hits       128827   125823    -3004     
- Misses      26656    27007     +351     
+ Partials     4734     4723      -11     

Flag	Coverage Δ
cli-ort-linux	83.79% <ø> (ø)
cli-ort-windows	83.40% <ø> (+0.09%)	⬆️
mlas	?
offline	79.71% <90.90%> (-0.49%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
crates/onnx-runtime-ep-cpu-plugin/src/lib.rs	76.19% <100.00%> (+1.83%)	⬆️
crates/onnx-runtime-ep-plugin/src/ep.rs	83.39% <100.00%> (+0.89%)	⬆️
crates/onnx-runtime-ep-plugin/src/graph_reader.rs	69.63% <78.18%> (+0.70%)	⬆️

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

✅ Benchmarks — No Regression

Comparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).

ℹ️ Absolute times are informational only — they vary with runner load. The % change column is the reliable signal because both sides ran under identical conditions.

Status	Scenario	Base	PR	Change
✅	matmul/large_generic_bf16_threads=8/32x1024x1024	2.13 ms	2.40 ms	+12.7%
✅	reduce_mean/small_f32_threads=1-internal/4096	18.84 µs	19.08 µs	+1.3%
✅	sampling_latency/min_p_per_token	206.94 µs	203.63 µs	-1.6%
✅	gather/large_bf16_threads=1-internal/131072	15.17 µs	14.51 µs	-4.3%
✅	tokenization/encode_tokens_per_second	419.55 µs	400.49 µs	-4.5%
✅	kv_cache/alloc_dealloc_pages	39.07 µs	37.00 µs	-5.3%
✅	logit_processing/seven_processor_chain_per_step	315.36 µs	297.67 µs	-5.6%
✅	qwen3_sampling_processors/top_k_top_p_fast	663.88 µs	626.62 µs	-5.6%
✅	qwen3_sampling_processors/top_p_fast_after_top_k	516.60 µs	485.80 µs	-6.0%
✅	grammar_masking/llguidance_compute_mask/32	87.06 µs	81.80 µs	-6.0%
✅	sampling_latency/top_p_per_token	391.09 µs	363.77 µs	-7.0%
✅	qwen3_sampling_processors/top_k_full_sort_baseline	2.14 ms	1.98 ms	-7.5%
✅	reduce_mean/large_f32_threads=1-internal/262144	1.13 ms	1.04 ms	-8.1%
✅	block_quantized_moe_cached_dense/mxfp4_cached_dense_expert_repeated_call/rows=1,H=256,I=256,E=4,top_k=1	176.28 µs	161.93 µs	-8.1%
✅	add/large_bf16_threads=1-internal/4194304	2.45 ms	2.24 ms	-8.7%
✅	qwen3_sampling_processors/top_k_top_p_full_sort_baseline	5.91 ms	5.32 ms	-10.0%
✅	qwen3_sampling_processors/top_k_partial_selection	152.14 µs	136.85 µs	-10.0%
✅	matmul/large_generic_bf16_threads=1/32x1024x1024	2.19 ms	1.94 ms	-11.4%
✅	matmul/large_generic_f32_threads=1/32x1024x1024	9.84 ms	8.68 ms	-11.8%
✅	qwen3_sampling_processors/top_p_full_sort_after_top_k_baseline	3.74 ms	3.28 ms	-12.2%
✅	matmul/medium_generic_f32_threads=1/32x512x512	2.46 ms	2.15 ms	-12.6%
✅	tokenization/decode_tokens_per_second	8.18 ms	7.10 ms	-13.2%
✅	add/small_f16_threads=1-internal/1024	707.3 ns	605.8 ns	-14.3%
🟢	add/medium_bf16_threads=1-internal/262144	124.38 µs	104.81 µs	-15.7%
🟢	matmul/small_generic_f32_threads=1/1x256x256	42.59 µs	35.88 µs	-15.8%
🟢	reduce_mean/medium_f32_threads=1-internal/65536	297.57 µs	249.11 µs	-16.3%
🟢	gather/small_f16_threads=1-internal/4096	617.7 ns	515.3 ns	-16.6%
🟢	gather/medium_f32_threads=1-internal/32768	4.80 µs	3.95 µs	-17.6%
🟢	matmul/medium_generic_f32_threads=8/32x512x512	1.13 ms	933.05 µs	-17.7%
🟢	add/small_f32_threads=1-internal/1024	261.4 ns	212.3 ns	-18.8%
🟢	gather/small_f32_threads=1-internal/4096	856.1 ns	686.9 ns	-19.8%
🟢	matmul/small_generic_bf16_threads=1/1x256x256	36.76 µs	29.08 µs	-20.9%
🟢	matmul/large_generic_f16_threads=8/32x1024x1024	100.59 µs	79.39 µs	-21.1%
🟢	matmul/large_generic_f16_threads=1/32x1024x1024	95.17 µs	74.09 µs	-22.2%
🟢	gather/small_bf16_threads=1-internal/4096	629.4 ns	484.0 ns	-23.1%
🟢	add/large_f16_threads=1-internal/4194304	2.63 ms	2.02 ms	-23.1%
🟢	sampling_latency/greedy_per_token	4.20 µs	3.16 µs	-24.8%
🟢	sampling_latency/top_k_per_token	67.63 µs	50.83 µs	-24.8%
🟢	gather/large_f32_threads=1-internal/131072	53.80 µs	39.58 µs	-26.4%
🟢	matmul/small_generic_f16_threads=1/1x256x256	39.43 µs	28.60 µs	-27.5%
🟢	add/small_bf16_threads=1-internal/1024	685.4 ns	493.1 ns	-28.1%
🟢	block_quantized_moe_cached_dense/mxfp4_uncached_expert_dequant_each_call/rows=1,H=256,I=256,E=4,top_k=1	654.51 µs	466.37 µs	-28.7%
🟢	gather/medium_bf16_threads=1-internal/32768	3.51 µs	2.44 µs	-30.5%
🟢	matmul/small_generic_f16_threads=8/1x256x256	40.67 µs	28.12 µs	-30.9%
🟢	add/large_f32_threads=1-internal/4194304	905.59 µs	616.52 µs	-31.9%
🟢	gather/medium_f16_threads=1-internal/32768	3.80 µs	2.49 µs	-34.6%
🟢	matmul/small_generic_bf16_threads=8/1x256x256	45.42 µs	29.26 µs	-35.6%
🟢	matmul/large_generic_f32_threads=8/32x1024x1024	5.82 ms	3.63 ms	-37.6%
🟢	matmul/medium_generic_f16_threads=1/32x512x512	45.85 µs	27.95 µs	-39.1%
🟢	matmul/medium_generic_bf16_threads=1/32x512x512	827.43 µs	492.56 µs	-40.5%
🟢	matmul/medium_generic_bf16_threads=8/32x512x512	608.27 µs	355.77 µs	-41.5%
🟢	add/medium_f16_threads=1-internal/262144	190.74 µs	106.91 µs	-43.9%
🟢	matmul/medium_generic_f16_threads=8/32x512x512	51.89 µs	28.54 µs	-45.0%
🟢	gather/large_f16_threads=1-internal/131072	21.84 µs	11.72 µs	-46.4%
🟢	block_quantized_matmul_cached_dense/mxfp4_preexpanded_dense_oncelock_like_proxy/1x1024x1024	97.06 µs	48.76 µs	-49.8%
🟢	add/medium_f32_threads=1-internal/262144	53.97 µs	25.39 µs	-53.0%
🟢	block_quantized_matmul_cached_dense/mxfp4_uncached_dequant_each_call/1x1024x1024	1.19 ms	522.87 µs	-56.1%
🟢	matmul/small_generic_f32_threads=8/1x256x256	81.29 µs	32.80 µs	-59.7%
🟢	block_quantized_matmul_cached_dense/mxfp4_cached_dense_repeated_call/1x1024x1024	128.47 µs	42.88 µs	-66.6%

Visual flags: ⚠️ ≥ 15% slower, 🔴 ≥ 30% slower — calibrated against measured runner noise (~27% worst-case on multi-threaded matmul)

Host info

CPU: Apple M1 (Virtual)
Cores: 3
OS: Darwin 25.5.0 arm64
Rust: rustc 1.97.1 (8bab26f4f 2026-07-14)
Load avg: { 3.33 3.38 4.96 }

What this cannot catch

• Regressions in code paths not covered by these benchmarks (e.g., end-to-end decode with a real model)
• Sub-threshold regressions that compound over multiple PRs
• Performance changes that only manifest under GPU execution
• Latency changes in the ORT integration path (these benchmarks exercise the native Rust kernels)

[justinchuby]

Rubber-duck review — falsification attempt on PR #1110

Verdict: Request changes. One major soundness bug survived (and was empirically reproduced): the fix marks ONNX IR≥4 non-constant / overridable initializers as session constants, which ORT explicitly forbids. Everything else in the PR held up under attack — the mechanism is real, the wiring is correct, the tests are genuine falsifiers, and the perf numbers reproduce.

I reviewed from a detached worktree at 41627b6e1, built the cdylib with --features onnx-runtime-ep-cpu/mlas, and ran the suites against it (NXRT_CPU_PLUGIN_PATH pointed at a uniquely-named copy so I never touched the author's artifacts).

---

MAJOR — Claim 1 is false: Graph_GetInitializers ≠ "constant for the session lifetime"

read_initializer_names (crates/onnx-runtime-ep-plugin/src/graph_reader.rs:776) records every name returned by Graph_GetInitializers, and constant_input_flags (crates/onnx-runtime-ep-plugin/src/ep.rs:976) marks any node input in that set as constant. But Graph_GetInitializers does not return only constants. From ORT's own header (onnxruntime_c_api.h:6180-6187):

Returns the graph's initializers as OrtValueInfo instances. Includes constant and non-constant initializers. … For ONNX IR version >= 4, an initializer with a name that matches a graph input is considered a non-constant initializer.

A non-constant (overridable) initializer is a real, legal ONNX construct: it is the default value of an optional graph input, and the caller may replace its bytes by name at every Run. ORT even ships a tool to strip them (remove_initializer_from_input.py). So the invariant the PR asserts in its own doc comments —

• graph_reader.rs:30 "…which lists only the constants ORT owns for the lifetime of the session"
• ep.rs constant_input_flags doc: "ORT owns the buffer, materializes it once at session creation and cannot change it between Run calls"

— is factually wrong, and it is exactly the invariant Kernel::set_constant_inputs demands (crates/onnx-runtime-ep-api/src/kernel.rs:603 "Runtime inputs must never be marked constant: caching them would return stale results").

Consequence is a stale-result bug, not just a perf quirk. When B/scales are marked constant, MatMulNBits builds a session-lifetime packed weight once and reuses it, ignoring the current input on later calls (crates/onnx-runtime-ep-cpu/src/kernels/matmul_nbits.rs:1208-1218, self.packed_nbits_weight cache; also flips on the MLAS SQNBit path via can_prepack, matmul_nbits.rs:1092-1095 / 1728-1731). If the caller overrides B or scales at Run, the kernel returns the answer for the first B.

I reproduced it. I added a temporary test that declares B and scales as both initializers and graph inputs (an overridable initializer), created a session through the same plugin path, and read the EP's own counter. ORT printed:

[W:onnxruntime:, graph.cc:1419 Graph] Initializer B appears in graph inputs and will not be treated
  as constant value/weight. … Move it out of graph inputs if there is no need to override it …
[W:onnxruntime:, graph.cc:1419 Graph] Initializer scales appears in graph inputs and will not be
  treated as constant value/weight. …
REVIEW1110: overridable B+scales reported constant = 2 (expected 0 if sound)

ORT's runtime says these are not constant; this EP reported them constant = 2. (I reverted the temporary test; the worktree is clean.)

Reachability is not gated away: the CPU EP has no weight/graph-input claim gate (node_inputs_all_routable at ep.rs:1007 is only consulted for GPU EPs at ep.rs:392), so it claims and compiles such a node.

Fix is cheap and the correct API is already bound. ValueInfo_IsConstantInitializer exists in the vtable (OrtApi, generated in ort-sys bindings) and returns false precisely for the overridable case (onnxruntime_c_api.h:6011; doc at 6001: "an initializer that serves as the default value for a matching graph input is not a constant initializer"). read_initializer_names already iterates the OrtValueInfos — add one ValueInfo_IsConstantInitializer call per initializer and only insert names where it returns true. It reads no tensor data, so the 1 GB-weight constraint is unaffected. (ValueInfo_IsOptionalGraphInput would work equally well.)

---

Claims that survived falsification

Claim 2 — the mechanism (SQNBit gate + repack). Verified. can_prepack is constant_inputs[1..=4] (matmul_nbits.rs:1092) and gates mlas_sqnbit_owns_fp32_compute (:1728-1731); QLinearMatMul gates its B prepack on constant_inputs[3] (qlinear_matmul.rs:71). Leaving the flags false really does both repack per-Run and drop the faster kernel.

"Plugin EP never called set_constant_inputs". Verified. Pre-PR call sites are only onnx-runtime-session/src/executor/{bindings.rs:464,kernel_cache.rs:739}; the ep.rs:804 call is new.

Positional-vs-ORT-index alignment (the check I most expected to break). Survives. constant_input_flags is positional over node_inputs (incl. None slots); kernels index constant_inputs positionally (matmul_nbits.rs:994-996, qlinear_matmul.rs:290-292); the runtime rebuilds kernel_inputs in the same node-position order (compute.rs:2425). input_slots' per-distinct-value dedup lives only in the tensor-fetch path and never mixes with the flag array, so a node that names one value twice (Mul(x,x), shared QLinear zero-point) or has an absent optional input stays aligned. The existing a_node_that_names_one_value_twice_is_bound_once test passes.

Claim 3 — the counter test is a real falsifier, and shares statics. Verified two ways. (a) The test reads the counter from the dlopened cdylib (plugin_ort_e2e.rs:5494-5500) at the same absolute path ORT loads, so it is the same mapping/statics. (b) I reverted constant_input_flags to the producer/is_graph_input form, rebuilt, and ran it:

nbits4_decode: this EP reported 0 constant inputs, expected 2   (left: 0, right: 2)  FAILED

Exactly the message the PR predicts. This also independently proves the PR's premise that ORT hands a fused node's initializers over as subgraph graph inputs — otherwise the producer/is_graph_input form would have reported 2 and passed.

Claim 4 — second-run staleness test. Genuine, but scoped: it rotates the activation and re-runs the same session, catching an activation cached as if constant; sensitive >= 5 (:5262) keeps it non-vacuous, and ort_moved uses ORT as the oracle for detectability. Note (see minor below) it uses only pure-constant initializers, so by construction it cannot catch the weight-override bug above — a weight cache "still matches ORT."

Claim 5 — performance. Reproduced on the same host class; p50/p90, interleaved, assignment asserted before timing, inputs built once for both sides (symmetric, fair). Spot checks (ours p50):

case	PR "ours after"	my p50	my p90	ratio p50
bench_nbits4_m128	7.79 ms	7.64 ms	12.31 ms	7.51
bench_qlinear_u8_m1	0.079 ms	0.0885 ms	0.110 ms	1.96

The load-bearing "ours" column matches; the ours/ORT ratio drifts (7.51 vs 6.36) because ORT's p50 is the noisy side, which the PR states explicitly. Honest.

Claim 6 — read_initializer_names is cheap on a 1 GB model. Verified: it calls only GetValueInfoName, never GetTensorData; allocations are O(#initializers), not O(bytes).

Suite results against the mlas cdylib: plugin_ort_e2e 51 passed / 1 ignored; onnx-runtime-ep-plugin --lib 231 passed. Matches the PR.

---

Minor

• No regression test for the override path. The three new tests never construct a non-constant initializer, and the "marks-everything-constant fails the dense=0 assertion" guard does not cover it. The temporary test I wrote (overridable B+scales ⇒ expect 0) is a ready-made regression once the flag is filtered.
• Second-run test can't see weight staleness (plugin_ort_e2e.rs:5176-5232) — it rotates only inputs[0]. Fine for its stated goal, but it means nothing in the suite would have caught the major bug.

Nit

• nxrt_ep_reset_constant_weight_inputs returns () but plugin_export_abi.rs:l1_required_symbols_resolve resolves it typed as fn() -> usize. Only the address is taken (never called through that type), so it's harmless and mirrors the pre-existing nxrt_ep_reset_compiled_node_count — but resolving the reset symbols as fn() would be tidier.
• CONSTANT_WEIGHT_INPUTS is a process-global cumulative AtomicUsize (Relaxed). Correct for the lock-serialized test hook; just note it aggregates across every compile in the process and is a test-only observable with no runtime effect.

---

Bottom line: the diagnosis and wiring are correct and I could not break them, but the constant predicate is too broad by exactly the set ORT tells you to exclude. Gate read_initializer_names on ValueInfo_IsConstantInitializer, fix the two doc comments that assert the false invariant, and add the override regression test. Then this is a clean approve.