[qwen] Port qwen qk_l2norm to LpNormalization (onnxruntime-genai#2127)

[Copilot]

Imports microsoft/onnxruntime-genai#2127 into this repo and adds a fast test. The Qwen3.5 qk_l2norm path emitted a 5-node Square + ReduceSum + Add(eps) + Rsqrt + Mul subgraph per Q/K head per linear-attention layer; replace it with a single LpNormalization(p=2, axis=-1) op, which is natively supported by all current EPs (CPU, CUDA, WebGPU, …).

Per the upstream PR, net Qwen3.5-0.8B node delta is −181 (−72 Mul, −36 ReduceSum, −36 Add, −36 Rsqrt, +36 LpNorm), with decode TPS gains of +13.0% (NV 0.8B), +12.5% (NV 4B), +6.2% (Intel 0.8B), +3.4% (Intel 4B). The +eps fallback is dropped — q/k come from RMSNorm + Proj so magnitudes far exceed 1e-6, keeping any divergence within fp16 noise.

Changes

• modelbuilder/builders/base.py: add make_lp_normalization(name, root_input, dtype, shape, axis=-1, p=2) helper.
• modelbuilder/builders/qwen.py: rewrite Qwen35TextModel._make_l2_normalize to emit one LpNormalization node instead of the 5-node subgraph:

  node_name = f"{basename}/LpNormalization"
  self.make_lp_normalization(node_name, input_name, self.io_dtype, full_shape, axis=-1, p=2)
  return f"{node_name}/output_0"

• tests/fast/test_random_qwen3_5.py: add test_qwen3_5_qk_l2norm_uses_lp_normalization which builds the hybrid (full_attention + linear_attention) config and asserts that the Q/K *l2norm* basenames emit LpNormalization nodes with axis=-1, p=2, and that none of the legacy node-name suffixes (/Square/Mul, /SumSq/ReduceSum, /AddEps/Add, /Rsqrt, /Normalize/Mul) remain.

Notes

• Qwen35TextModel._get_shared_l2_eps is now unused but kept (matching upstream) to minimize the diff.

[github-actions]

📂 Impacted Files

3 file(s) changed in this pull request:

🔵 modelbuilder/builders/base.py (+5 / -0)
🔵 modelbuilder/builders/qwen.py (+8 / -25)
🔵 tests/fast/test_random_qwen3_5.py (+45 / -0)

[xadupre]

@copilot fix style

[Copilot]

@copilot fix style

Applied black formatting in 7eda0a8; ruff check passes on all modified files.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.39%. Comparing base (849108b) to head (7eda0a8).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #365      +/-   ##
==========================================
- Coverage   78.44%   78.39%   -0.06%     
==========================================
  Files          26       26              
  Lines        7990     7984       -6     
==========================================
- Hits         6268     6259       -9     
- Misses       1722     1725       +3     

Flag	Coverage Δ
fast-tests	78.39% <100.00%> (-0.06%)	⬇️
fast-tests-ort-nightly	78.34% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.