Skip to content

Update glm-5 b200 sglang image to nightly-dev-cu13-20260523-c112f762 #1567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hshrivastava-droid merged 13 commits into from
Jun 12, 2026
Merged

Update glm-5 b200 sglang image to nightly-dev-cu13-20260523-c112f762 #1567

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d06b958
Update glm-5 b200 sglang container image to nightly-dev-cu13-20260523…
Ankur-singh May 26, 2026
12cae11
remove cutlass gemm backend
Ankur-singh May 28, 2026
d2263d5
Merge branch 'main' into glm-update
Ankur-singh May 28, 2026
e80ab6e
glm5-b200-sglang: bump SGLang image to nightly-dev-cu13-20260605-7dc7…
Ankur-singh Jun 5, 2026
c23bea7
Merge remote-tracking branch 'origin/main' into glm-update
Ankur-singh Jun 5, 2026
bf52ead
remove transformers version ping
xinli-sw Jun 8, 2026
1469214
Merge branch 'main' into glm-update
xinli-sw Jun 8, 2026
bd62328
remove transformers version ping and remove deepgemm war
xinli-sw Jun 8, 2026
b14879a
glm5-fp4-b200-sglang-mtp: drop --mem-fraction-static to 0.8 for 8k1k …
Ankur-singh Jun 8, 2026
49cfc1f
Merge remote-tracking branch 'origin/main' into glm-update
Ankur-singh Jun 8, 2026
5b06baf
Lower mem-fraction-static at high concurrency for glm5 fp4 b200 mtp
hshrivastava-droid Jun 10, 2026
07220de
Merge branch 'main' into glm-update
hshrivastava-droid Jun 12, 2026
7f01c81
Update SGLang image and memory fraction settings
hshrivastava-droid Jun 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2205,7 +2205,7 @@ qwen3.5-fp4-b200-sglang-mtp:
- { tp: 2, ep: 1, conc-start: 4, conc-end: 64, spec-decoding: mtp }

glm5-fp8-b200-sglang:
image: lmsysorg/sglang:v0.5.12-cu130
image: lmsysorg/sglang:nightly-dev-cu13-20260605-7dc73766
model: zai-org/GLM-5-FP8
model-prefix: glm5
runner: b200
Expand All @@ -2224,7 +2224,7 @@ glm5-fp8-b200-sglang:
- { tp: 8, ep: 1, conc-start: 4, conc-end: 256 }

glm5-fp8-b200-sglang-mtp:
image: lmsysorg/sglang:v0.5.12-cu130
image: lmsysorg/sglang:nightly-dev-cu13-20260605-7dc73766
model: zai-org/GLM-5-FP8
model-prefix: glm5
runner: b200
Expand Down Expand Up @@ -2304,7 +2304,7 @@ glm5-fp8-b300-sglang-mtp:
- { tp: 8, ep: 1, conc-start: 4, conc-end: 256, spec-decoding: mtp }

glm5-fp4-b200-sglang:
image: lmsysorg/sglang:v0.5.12-cu130
image: lmsysorg/sglang:nightly-dev-cu13-20260605-7dc73766
model: nvidia/GLM-5-NVFP4
model-prefix: glm5
runner: b200
Expand All @@ -2325,7 +2325,7 @@ glm5-fp4-b200-sglang:
- { tp: 4, ep: 1, conc-start: 4, conc-end: 256 }

glm5-fp4-b200-sglang-mtp:
image: lmsysorg/sglang:v0.5.12-cu130
image: lmsysorg/sglang:nightly-dev-cu13-20260605-7dc73766
model: nvidia/GLM-5-NVFP4
model-prefix: glm5
runner: b200
Expand Down
1 change: 0 additions & 1 deletion benchmarks/single_node/fixed_seq_len/glm5_fp8_b200.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ PYTHONNOUSERSITE=1 python3 -m sglang.launch_server --model-path=$MODEL --host=0.
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype fp8_e4m3 --quantization fp8 \
--fp8-gemm-backend cutlass \

@cursor cursor Bot Jun 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

B200 DeepGemm workaround removed

Medium Severity

This change drops --fp8-gemm-backend cutlass from the GLM-5 FP8 B200 launch scripts while SGL_ENABLE_JIT_DEEPGEMM stays 1. The repo previously added that flag to avoid DeepGemm CUDA graph capture failures on B200; without an equivalent mitigation or a confirmed fix in the new nightly image, the server may fail during startup or warmup.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c23bea7. Configure here.

--attention-backend nsa \
--nsa-decode-backend trtllm --nsa-prefill-backend trtllm \
--moe-runner-backend flashinfer_trtllm \
Expand Down
1 change: 0 additions & 1 deletion benchmarks/single_node/fixed_seq_len/glm5_fp8_b200_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ PYTHONNOUSERSITE=1 python3 -m sglang.launch_server --model-path=$MODEL --host=0.
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype fp8_e4m3 --quantization fp8 \
--fp8-gemm-backend cutlass \
--attention-backend nsa \
--nsa-decode-backend trtllm --nsa-prefill-backend trtllm \
--moe-runner-backend flashinfer_trtllm \
Expand Down
8 changes: 8 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3502,3 +3502,11 @@
- "Update GPT-OSS model for MI355X vLLM from amd/gpt-oss-120b-w-mxfp4-a-fp8 to openai/gpt-oss-120b"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1670

- config-keys:
- glm5-fp4-b200-sglang
- glm5-fp4-b200-sglang-mtp
- glm5-fp8-b200-sglang
- glm5-fp8-b200-sglang-mtp
description:
- "Update SGLang image from v0.5.12-cu130 to nightly-dev-cu13-20260605-7dc73766"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1567
Loading