perf(cuda): derive RMSNorm fold hidden floor from SM count (#1421)

[justinchuby]

perf(cuda): derive the RMSNorm-fold hidden floor from SM count (#1421)

Replaces the global RMSNORM_FUSION_MIN_HIDDEN = 1280 with a per-device floor
derived from the device's SM count, anchored on the single H200 calibration point
rather than a per-device lookup table.

Closes #1421.

What the change does

derived_min_hidden(sm) = round_nearest(10 * sm / 132) chunks * 128, clamped to >= 1 chunk

• Derived from multiprocessor_count() — the property that sets how much
  parallel slack the device has to keep the standalone norm "almost free" at M=1.
• Not a device table. A single H200 anchor (132 SM → 1280) fixes the slope;
  every other device gets its floor from its own SM count. A big datacenter part
  inherits the protective H200-class floor; a small edge part folds aggressively.
  Avoids the §40 "regime-tuned threshold" trap.
• Reproduces both known data points: H200 (132 SM) → 1280 (exactly);
  RTX 4060 Laptop (24 SM) → 256.
• M-invisibility finding: the optimizer cannot see M. This pass runs once at
  model load on a decode graph whose batch dim is symbolic and shared across all
  batch sizes, so M ≥ 2 cannot gate the fold per step. The residual uncovered
  case (small model batched on a large GPU) is documented and left to the
  ONNX_GENAI_RMSNORM_MIN_HIDDEN escape hatch.

Tests (measured)

cargo test -p onnx-runtime-ep-cuda --features "cuda,gpu-tests" --lib -- optimizer::tests
→ 85 passed, 0 failed, 0 ignored (10x default-parallel runs, 0 failures — the new tests route env access through EnvVarGuard per #1588, closing the process-wide env race; the +1 vs the earlier count is #1588's guard test). (cuda,gpu-tests, not cuda alone, so the
suite is not silently all-ignored.) Four new derived_min_hidden tests cover the
anchor self-reproduction (132→1280), 24→256, .max(1) clamp at sm 0/1,
monotonicity, whole-128-chunk output, the device-derived gate, and env-override
precedence.

The fold is NOT bit-identical — and this PR corrects that long-unverified claim

The SkipSimplifiedLayerNormalization → GEMV fold was documented in the code as
"byte-identical". That was never verified, and it is false. Folding the RMS
normalization into the following GEMV's prologue changes the fp16 reduction
order, so at a greedy-argmax near-tie it can flip a token. The residual epilogue
(fp16(fp16(acc) + residual) == __hadd2) remains bit-identical; the reordering
is in the normalization reduction only. The code comments are updated to say this
accurately (see the second commit).

Measured (binary SHA256-pinned across runs, machine quiet)

Switch efficacy proven with ONNX_GENAI_PROFILE_OPS=1:
SkipSimplifiedLayerNormalization op-count 23 (fold ON) vs 48 (fold OFF) — the
two sides genuinely run different graphs.

profile_native (native CUDA, greedy, --tokens 128 --decode-skip 8, no
--steady), qwen0.5B (hidden 896), fold ON (ONNX_GENAI_RMSNORM_MIN_HIDDEN=256)
vs OFF (=2048):

Prompt	ON vs OFF
default, "Once upon a time…", "Explain quantum computing…", "def fibonacci(n):", "The capital of France is"	identical
"The quick brown fox jumps over the lazy dog and then"	diverges at token 49 (ON=448, OFF=304)

granite-1b-a400m (hidden 1024): identical on the fox prompt. So the divergence is
real but rare and input-dependent (1 of 6 sampled prompts), consistent with a
reduction-order change tipping only near-ties.

Which side is correct? The folded side.

Divergence alone doesn't say which token is right. Adjudicated with a
high-precision oracle: take prompt (11 tok) + generated[0..48] (49 tok) = a
60-token prefix and run a full prefill on the CPU EP (a completely different
implementation and precision the CUDA fold never touches), dumping top-k:

n_prompt_tokens: 60
selected_token:  448
top: [[448, -1.58743], [304, -1.60305], [438, -1.82180], ...]

The reference top-1 is 448 — the token fold ON selects. Fold OFF (today's
default: keep the standalone norm) selects 304, the runner-up, only 0.0156
nats behind. So the fp16 reduction reorder rounded toward the more accurate
result here, not away from it. This is a reduction-order effect, not a fusion
bug — there is no evidence of a numerical defect in the N=896 fused GEMV.

Repro

# fold ON vs OFF, compare generated_token_ids (differ at index 49):
$env:ONNX_GENAI_RMSNORM_MIN_HIDDEN='256'
profile_native --model <qwen05b> --ep cuda --backend native --tokens 128 --decode-skip 8 --prompt "The quick brown fox jumps over the lazy dog and then"
$env:ONNX_GENAI_RMSNORM_MIN_HIDDEN='2048'
# (same command)

# oracle: --prompt-ids takes a FILE PATH to a JSON array, not inline JSON.
# Write the 60-token prefix [prompt..+generated[0..48]] to a file, then:
profile_native --model <qwen05b> --ep cpu --backend ort --prompt-ids <prefix.json> --tokens 1   # dump top-k

Not verified / out of scope

• Performance: not measured (box not guaranteed quiet across the session).
• Generality of the "folded side is more accurate" result: established at the
  single measured flip point only; not proven for every possible near-tie.
• H200 behavior (132 SM → floor 1280, so 896/1024 stay unfused) verified via the
  derived value in unit tests, not on H200 hardware.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.78%. Comparing base (4c1b594) to head (bf20e41).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1582       +/-   ##
===========================================
- Coverage   82.19%   80.78%    -1.42%     
===========================================
  Files          12      381      +369     
  Lines        5471   173786   +168315     
  Branches     5471   173786   +168315     
===========================================
+ Hits         4497   140392   +135895     
- Misses        775    28500    +27725     
- Partials      199     4894     +4695     

Flag	Coverage Δ
cli-ort-linux	82.60% <ø> (?)
cli-ort-windows	82.19% <ø> (ø)
mlas	85.19% <ø> (?)
offline	80.64% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 372 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.