Skip to content

[Klaud Cold] Update glm5-fp8-b200-sglang (+mtp) SGLang image to v0.5.12-cu130 #1447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
functionstackx merged 9 commits into from
May 18, 2026
Merged

[Klaud Cold] Update glm5-fp8-b200-sglang (+mtp) SGLang image to v0.5.12-cu130 #1447

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
09a23b0
Update glm5-fp8-b200-sglang (+mtp) SGLang image to v0.5.12-cu130
functionstackx May 17, 2026
6c3182d
chore: fill pr-link for #1447
functionstackx May 17, 2026
ffce090
Re-trigger sweep (abandon in-flight run with transient infra failures)
May 18, 2026
e5d0df5
Merge remote-tracking branch 'origin/main' into HEAD
May 18, 2026
739585b
fix(glm5_fp8_b200): use --fp8-gemm-runner-backend cutlass
May 18, 2026
0034473
Merge remote-tracking branch 'origin/main' into HEAD
May 18, 2026
d40fd27
fix(glm5_fp8_b200): correct flag name --fp8-gemm-backend (not --fp8-g…
May 18, 2026
f210160
Merge remote-tracking branch 'origin/main' into HEAD
May 18, 2026
aeeed4b
Merge branch 'main' into update-glm5-fp8-b200-sglang-v0.5.12
functionstackx May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2189,7 +2189,7 @@ qwen3.5-fp4-b200-sglang-mtp:
- { tp: 2, ep: 1, conc-start: 4, conc-end: 64, spec-decoding: mtp }

glm5-fp8-b200-sglang:
image: lmsysorg/sglang:nightly-dev-cu13-20260317-1eea7448
image: lmsysorg/sglang:v0.5.12-cu130
model: zai-org/GLM-5-FP8
model-prefix: glm5
runner: b200
Expand All @@ -2208,7 +2208,7 @@ glm5-fp8-b200-sglang:
- { tp: 8, ep: 1, conc-start: 4, conc-end: 256 }

glm5-fp8-b200-sglang-mtp:
image: lmsysorg/sglang:nightly-dev-cu13-20260317-1eea7448
image: lmsysorg/sglang:v0.5.12-cu130
model: zai-org/GLM-5-FP8
model-prefix: glm5
runner: b200
Expand All @@ -2235,7 +2235,7 @@ glm5-fp8-b200-sglang-mtp:
# (either main had none or had a different conc/offload sweep).
# The original glm5-fp8-b200-sglang entry stays byte-identical to origin/main.
glm5-fp8-b200-sglang-agentic:
image: lmsysorg/sglang:nightly-dev-cu13-20260317-1eea7448
image: lmsysorg/sglang:v0.5.12-cu130
model: zai-org/GLM-5-FP8
model-prefix: glm5
runner: b200
Expand Down
1 change: 1 addition & 0 deletions benchmarks/single_node/glm5_fp8_b200.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ PYTHONNOUSERSITE=1 python3 -m sglang.launch_server --model-path=$MODEL --host=0.
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype fp8_e4m3 --quantization fp8 \
--fp8-gemm-backend cutlass \
--attention-backend nsa \
--nsa-decode-backend trtllm --nsa-prefill-backend trtllm \
--moe-runner-backend flashinfer_trtllm \
Expand Down
1 change: 1 addition & 0 deletions benchmarks/single_node/glm5_fp8_b200_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ PYTHONNOUSERSITE=1 python3 -m sglang.launch_server --model-path=$MODEL --host=0.
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype fp8_e4m3 --quantization fp8 \
--fp8-gemm-backend cutlass \
--attention-backend nsa \
--nsa-decode-backend trtllm --nsa-prefill-backend trtllm \
--moe-runner-backend flashinfer_trtllm \
Expand Down
8 changes: 8 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2847,3 +2847,11 @@
description:
- "Add MTP/EAGLE speculative-decoding sibling of dsr1-fp8-mi325x-sglang on lmsysorg/sglang:v0.5.12-rocm700-mi30x"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1500

- config-keys:
- glm5-fp8-b200-sglang
- glm5-fp8-b200-sglang-mtp
Comment on lines +2851 to +2853

@claude claude Bot May 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The PR also bumps the image for glm5-fp8-b200-sglang-agentic at nvidia-master.yaml:2237 (same change as the two siblings), but the new perf-changelog entry only lists glm5-fp8-b200-sglang and glm5-fp8-b200-sglang-mtp under config-keys. Add - glm5-fp8-b200-sglang-agentic to the list (and ideally mention it in the title/description) so the changelog reflects all three bumps and downstream changelog-driven sweep generation picks up the agentic variant on merge.

Extended reasoning...

What the bug is

This PR bumps the image tag for three config entries in .github/configs/nvidia-master.yaml from lmsysorg/sglang:nightly-dev-cu13-20260317-1eea7448 to lmsysorg/sglang:v0.5.12-cu130:

  • glm5-fp8-b200-sglang (line 2191)
  • glm5-fp8-b200-sglang-mtp (line 2210)
  • glm5-fp8-b200-sglang-agentic (line 2237)

However, the new perf-changelog.yaml entry (lines 2632-2638) only lists the first two under config-keys. The PR title ("Update glm5-fp8-b200-sglang (+mtp)"), description ("33/29 days old" — only two age values), and commit message also omit the agentic sibling, strongly suggesting this is an oversight rather than an intentional exclusion.

Why it matters

Per AGENTS.md lines 113-124, every image bump in a *-master.yaml must be paired with a perf-changelog.yaml entry ("required - triggers benchmarks"). The utils/process_changelog.py helper get_config_keys_from_master() resolves the listed config-keys and feeds them to generate_sweep_configs.py test-config --config-keys .... Configs that are missing from the config-keys list are simply skipped by the changelog-driven sweep generation.

The full-sweep-enabled label on this PR happens to cover all three configs for the PR-time sweep, so functionally the PR itself will benchmark the agentic variant. But the merge-time / post-merge baseline refresh path triggered by run-sweep.yml on paths: perf-changelog.yaml keys off the added entry's config-keys — so once merged, the agentic variant's image bump will not get a baseline benchmark refresh from this changelog entry, and any downstream consumer reading the changelog history (e.g. for release notes or change tracking) will miss it.

Step-by-step proof

  1. git show 09a23b0 -- .github/configs/nvidia-master.yaml — three blocks are modified, including the one at line 2237 for glm5-fp8-b200-sglang-agentic. The replacement string is identical to the one applied to the two siblings.
  2. perf-changelog.yaml lines 2632-2638 list only glm5-fp8-b200-sglang and glm5-fp8-b200-sglang-mtp under config-keys.
  3. Compare with the previous dsr1 entry at lines 2614-2619 (also a multi-sibling bump from PR Update dsr1-fp8-b200-sglang and -mtp SGLang image to v0.5.12-cu130 #1416): it correctly lists both dsr1-fp8-b200-sglang and dsr1-fp8-b200-sglang-mtp. The convention is to list every config whose image was bumped.
  4. utils/process_changelog.py reads added entries from this file and uses config-keys directly as the --config-keys argument to generate_sweep_configs.py. A config absent from the list is silently not benchmarked from that entry.

How to fix

Append - glm5-fp8-b200-sglang-agentic to the config-keys list in the new entry, and update the PR title/description (and optionally the changelog description string) to mention the agentic variant. The diff would be:

- config-keys:
    - glm5-fp8-b200-sglang
    - glm5-fp8-b200-sglang-mtp
    - glm5-fp8-b200-sglang-agentic
  description:
    - "Update SGLang image from nightly-dev-cu13-20260317-1eea7448 (33d/29d old) to v0.5.12-cu130"
  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1447

description:
- "Update SGLang image from nightly-dev-cu13-20260317-1eea7448 (33d/29d old) to v0.5.12-cu130"
- "Add --fp8-gemm-runner-backend cutlass to bypass DeepGemm illegal-memory-access on CUDA-graph capture (KLAUD_DEBUG §4a, B200 sm_100 variant)"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1447
Loading