[desmond-intel]

Motivation

On Intel XPU, zai-org/GLM-4.7-Flash (Glm4MoeLiteForCausalLM) serves but gives wrong answers: gsm8k 5-shot drops to 0.25 (limit 20) with both --attention-backend intel_xpu and triton, against 84.2 on H200.

On XPU, the MoE layer already applies routed_scaling_factor on every path:

• sgl_kernel.fused_experts for unquantized (UnquantizedFusedMoEMethod.forward_xpu), fp8 and mxfp4 weights;
• the triton runner's moe_sum_reduce in the _is_xpu branch of fused_moe.py.

Glm4MoeLiteSparseMoeBlock.forward_normal multiplies the routed output by routed_scaling_factor again on every non-CUDA device, so XPU applies it twice (1.8^2 for GLM-4.7-Flash). DeepseekV2MoE.forward_normal already excludes XPU from this multiply. The unquantized path began passing the factor to the kernel in #39817, which aligned it with the fp8/mxfp4 paths and exposed the double scaling for the bf16 GLM-4.7-Flash checkpoint.

Modifications

• Add _is_xpu to the model-side scaling condition in Glm4MoeLiteSparseMoeBlock.forward_normal (python/sglang/srt/models/glm4_moe_lite.py), matching DeepseekV2MoE.forward_normal.
• CUDA, ROCm/aiter, NPU and CPU are unchanged: the condition is identical when _is_xpu is False.
• forward_normal_dual_stream is unchanged: alt_stream is only created on CUDA, so XPU never takes that path.
• Add test/registered/xpu/llm_models/test_xpu_glm_4_7_flash.py: GLM-4.7-Flash GSM8K on XPU at TP=4 in the nightly-xpu-kernel-main-4-gpu suite. Like the other tests in that folder it uses SimpleEvalGSM8KXPUMixin (sgl-eval gsm8k), with the cookbook settings: thinking on with --reasoning-parser glm45, temperature 0.7, top_p 0.95, max_tokens 2048.
• The XPU nightly jobs and the XPU test extra already install sgl-eval==0.1.3 and antlr4-python3-runtime==4.9.3 on main ([CI] [Intel] xpu-eval: Add sgl-eval to XPU test deps and move all GSM8K tests to it #47056), so this PR no longer changes nightly-test-intel.yml or pyproject_xpu.toml.

Glm4MoeSparseMoeBlock.forward_normal in glm4_moe.py (GLM-4.5/4.6/4.7) has the same condition and likely the same XPU issue. It is left out of this PR because those checkpoints could not be validated on the available XPU hardware.

Accuracy Tests

Intel Arc Pro B60, tp-size 4, --attention-backend intel_xpu, lm_eval gsm8k 5-shot (local-completions, greedy, max_gen_toks=8192):

Run	flexible-extract	strict-match
Before, limit 20	0.25	0.20
After, limit 20	0.75	0.75
After, full 1319	0.8408 +/- 0.0101	0.8415 +/- 0.0101
H200 reference	0.842	-

The new nightly test on 4x Arc Pro B60, sgl-eval GSM8K at 20 questions, three runs: 0.85, 0.80, 0.90 (1-2 answers per run truncated at 2048 tokens), threshold 0.75. The eval takes 5.5-7 min; the whole test took 18-26 min, mostly server start with the weights read from a network model mirror. An earlier 50-question run scored 0.90.

Prompt logprobs against a CPU fp32 HF transformers reference (4 prompts): mean absolute difference was 0.77-1.46 nats before the fix and 0.07-0.16 after. Greedy continuations match HF token-for-token on 2 of the 4 prompts; the other 2 diverge at a bf16 near-tie.

Reproduce (the last three serve flags are B60 concurrency settings, unrelated to this change):

SGLANG_USE_SGL_XPU=1 sglang serve --model-path zai-org/GLM-4.7-Flash --device xpu --tp-size 4 \
  --mem-fraction-static 0.85 --trust-remote-code --attention-backend intel_xpu --reasoning-parser auto \
  --disable-overlap-schedule --disable-radix-cache --max-running-requests 4

python -m lm_eval --model local-completions \
  --model_args pretrained=zai-org/GLM-4.7-Flash,base_url=http://127.0.0.1:30000/v1/completions,num_concurrent=16,max_gen_toks=8192,max_length=12288,trust_remote_code=True \
  --tasks gsm8k --num_fewshot 5 --batch_size 16

Speed Tests and Profiling

No speed impact expected. On XPU the change removes one redundant elementwise multiply per MoE layer; other platforms are untouched.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests. Added the XPU nightly GSM8K test above; the bug only shows with the full model on XPU.
• Update documentation according to Write documentations. Not needed.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ❌ Run #41987630259
Latest PR Test (Extra): ❌ Run #41987630128
Latest PR Test (AMD ROCm 10): ❌ Run #41987630040

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[airMeng]

please enable the related tests then I can include the PR in #45201

[desmond-intel]

@airMeng Added test/registered/xpu/llm_models/test_xpu_glm_4_7_flash.py in 3b5b261: GLM-4.7-Flash GSM8K on XPU at TP=4, in the nightly-xpu-kernel-main-4-gpu suite with the same mixin as the Qwen3-30B-A3B test. No existing test covers this path; the other GLM-4.7-Flash tests are CUDA only or CPU unit tests of the gate.

On Arc Pro B60 (main 6e667c4 plus this fix) it scores 0.96 on its 50 questions against a threshold of 0.90, in 1431 s. With the factor applied twice the model scores about 0.25. --max-running-requests 4 is the B60 setting that keeps concurrent requests from losing the device.

Could you take it into the next weekly consolidation ?

[airMeng]

@desmond-intel — forwarding @mingfeima’s comment from #46623:
#46623 (comment)

This new test inherits a mixin that calls deprecated run_eval.

Now the BKM is to use the sgl-eval helper/mixin with explicit generation settings and recalibrate the accuracy threshold under that harness.

[desmond-intel]

@mingfeima @airMeng Switched in 15d1c41. The test now uses the shared GSM8KMixin with gsm8k_backend = "sgl_eval" and explicit generation settings from the GLM-4.7-Flash cookbook: thinking on with --reasoning-parser glm45, temperature 0.7, top_p 0.95, max_tokens 2048. Recalibrated on Arc Pro B60 (tp 4, main 8301033 plus this PR): 0.92 on the 50 questions in 793 s, and the threshold is now 0.80.

The XPU nightly container does not ship sgl-eval, and the sgl-eval kits skip their test when it is missing. aa20bd2 installs sgl-eval==0.1.3 and antlr4-python3-runtime==4.9.3 (the pins of the main test extra) in the four XPU nightly jobs and adds them to the XPU test extra. The branch also merges current main (2e58c3c), since the nightly workflow was split after this PR's base.

[arathi-hlab]

can you confirm sgl-eval is used ? if yes so please collect the number based on that rest is LGTM

[desmond-intel]

Yes, it uses sgl-eval. SimpleEvalGSM8KXPUMixin calls run_sgl_eval(eval_name="gsm8k") (simple_eval_gsm8k_xpu_mixin.py:110), and each run logs sgl-eval gsm8k generation: {'max_tokens': 2048, 'temperature': 0.7, 'top_p': 0.95, ...}. The numbers in the description are from sgl-eval: three runs at 20 questions on 4x B60 scored 0.85, 0.80 and 0.90 (threshold 0.75), and an earlier 50-question run scored 0.90.