Skip to content

[Fix] [XPU] glm4_moe_lite: Skip model-side routed scaling on XPU (fix GLM-4.7-Flash accuracy) #43891

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
desmond-intel wants to merge 8 commits into
base: main
Choose a base branch
from
+43 −2
Open

[Fix] [XPU] glm4_moe_lite: Skip model-side routed scaling on XPU (fix GLM-4.7-Flash accuracy) #43891

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
90c2a39
glm4_moe_lite: skip model-side routed scaling on XPU (fix GLM-4.7-Fla…
desmond-intel Oct 12, 2026
7ebced6
Add GLM-4.7-Flash GSM8K nightly test on XPU (TP=4)
desmond-intel Oct 23, 2026
772599d
Use sgl-eval for the GLM-4.7-Flash XPU GSM8K test
desmond-intel Oct 28, 2026
b7ba8c6
Drop the XPU-availability skip from the GLM-4.7-Flash XPU test
desmond-intel Oct 28, 2026
a48ecd5
Use the XPU sgl-eval GSM8K mixin for the GLM-4.7-Flash XPU test
desmond-intel Nov 2, 2026
0d5f866
Run 20 GSM8K examples in the GLM-4.7-Flash XPU test
desmond-intel Nov 3, 2026
97b1356
Lower the GLM-4.7-Flash XPU test threshold to 0.75
desmond-intel Nov 4, 2026
1f92cc2
Merge branch 'main' into model-serve/glm47-flash-xpu
arathi-hlab Nov 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions python/sglang/srt/models/glm4_moe_lite.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@
from sglang.srt.models.deepseek_common.deepseek_weight_loader import (
DeepseekV2WeightLoaderMixin,
)
from sglang.srt.models.deepseek_common.utils import _is_cuda, _use_aiter
from sglang.srt.models.deepseek_common.utils import _is_cuda, _is_xpu, _use_aiter
from sglang.srt.models.deepseek_v2 import DeepseekV2AttentionMLA
from sglang.srt.runtime_context import get_exec, get_parallel, get_stream
from sglang.srt.utils import (
Expand Down Expand Up @@ -354,7 +354,7 @@ def forward_normal(
topk_output = self.topk.empty_topk_output(hidden_states.device)

final_hidden_states = self.experts(hidden_states, topk_output)
if not _is_cuda and not _use_aiter:
if not _is_cuda and not _is_xpu and not _use_aiter:
final_hidden_states *= self.routed_scaling_factor
if shared_output is not None:
with use_symmetric_memory(
Expand Down
41 changes: 41 additions & 0 deletions test/registered/xpu/llm_models/test_xpu_glm_4_7_flash.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
"""GLM-4.7-Flash GSM8K accuracy on Intel XPU (TP=4).

Scored by sgl-eval's ``gsm8k`` benchmark with the GLM-4.7-Flash cookbook
sampling settings.
"""

import unittest

from sglang.test.ci.ci_register import register_xpu_ci
from sglang.test.test_utils import CustomTestCase
from sglang.test.xpu.simple_eval_gsm8k_xpu_mixin import SimpleEvalGSM8KXPUMixin

register_xpu_ci(est_time=2400, suite="nightly-xpu-kernel-main-4-gpu", nightly=True)


class TestGLM4_7FlashXPU(SimpleEvalGSM8KXPUMixin, CustomTestCase):
model = "zai-org/GLM-4.7-Flash"
tp_size = 4
# 3 runs on 4x B60 scored 0.80-0.90; one question of margin below the lowest.
accuracy = 0.75
timeout_for_server_launch = 3600
num_examples = 20
num_threads = 4
# Cookbook sampling; the chat template thinks by default.
temperature = 0.7
top_p = 0.95
max_tokens = 2048

other_args = SimpleEvalGSM8KXPUMixin.other_args + [
"--reasoning-parser",
"glm45",

@arathi-hlab arathi-hlab Nov 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you confirm sgl-eval is used ? if yes so please collect the number based on that rest is LGTM

@desmond-intel desmond-intel Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it uses sgl-eval. SimpleEvalGSM8KXPUMixin calls run_sgl_eval(eval_name="gsm8k") (simple_eval_gsm8k_xpu_mixin.py:110), and each run logs sgl-eval gsm8k generation: {'max_tokens': 2048, 'temperature': 0.7, 'top_p': 0.95, ...}. The numbers in the description are from sgl-eval: three runs at 20 questions on 4x B60 scored 0.85, 0.80 and 0.90 (threshold 0.75), and an earlier 50-question run scored 0.90.

"--mem-fraction-static",
"0.85",
# More concurrent requests lose the device on Arc Pro B60.
"--max-running-requests",
"4",
]


if __name__ == "__main__":
unittest.main()
Loading