[Klaud Cold] glm5.2-fp8-b200-sglang-agentic-mtp: day-zero GLM-5.2 FP8 SGLang AgentX MTP on B200 / B200 上 GLM-5.2 FP8 SGLang AgentX MTP 首发配方 - #2863
Conversation
… SGLang AgentX MTP on B200 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
60fcce4 to
60760a6
Compare
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
2 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 60760a6. Configure here.
| export SRT_SLURM_MODEL_PREFIX="glm5.2-fp4" | ||
| elif [[ $MODEL_PREFIX == "glm5.2" && $PRECISION == "fp8" ]]; then | ||
| export MODEL_PATH="${MODEL_PATH:-/scratch/models/GLM-5.2-FP8}" | ||
| export SRT_SLURM_MODEL_PREFIX="glm5.2-fp8" |
There was a problem hiding this comment.
Unstaged path breaks container mount
High Severity
The new glm5.2/fp8 branch points MODEL_PATH at /scratch/models/GLM-5.2-FP8 without creating that directory, while the launcher still bind-mounts MODEL_PATH into the container. Pyxis/enroot fail when the host source is missing, so the script's day-zero hf download never runs and every cell dies before serve.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 60760a6. Configure here.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Looks good, mechanical port of the validated FP4 sibling with narrow, correctly-scoped deltas.
What was reviewed: the new glm5.2_fp8_b200_sglang_mtp.sh script (checkpoint-completeness check, flock-serialized hf download, EAGLE/MTP and HiCache args) against its FP4 sibling; the nvidia-master.yaml and launcher routing additions, which mirror existing entries exactly. Checked the pr-link: PRLINK_PLACEHOLDER in perf-changelog.yaml — this matches the repo's documented create-PR-then-amend-with-real-URL workflow (.claude/commands/nuke.md), not a leftover bug. Also confirmed the missing --use-chat-template is consistent with every other agentic/*.sh script (that invariant applies only to fixed_seq_len scripts using run_benchmark_serving, which agentic scripts don't call).
Extended reasoning...
Overview
The diff adds a new single-node agentic benchmark script (glm5.2_fp8_b200_sglang_mtp.sh, 387 lines) that is explicitly a port of an already-validated FP4 sibling with two functional deltas (HF model id and --quantization fp8), plus a mirrored nvidia-master.yaml recipe entry, a matching perf-changelog.yaml append, and a small elif branch in the B200 launcher routing glm5.2+fp8 to a new model path/prefix. No shared library code, CI, or security-relevant paths are touched.
Security risks
None identified. There is no user input handling, auth, or crypto in this diff; it's benchmark tooling that downloads a public HF checkpoint via hf download and launches a local inference server for the sweep.
Level of scrutiny
Moderate scrutiny is appropriate given the file count and structural similarity to an already-validated, merged sibling script. I independently re-checked the two items the bug hunter's earlier passes flagged and ruled out: (1) the perf-changelog.yaml pr-link: PRLINK_PLACEHOLDER value, which I traced to a documented two-step workflow in .claude/commands/nuke.md (create PR with placeholder, then amend+force-push with the real URL) rather than a stray leftover; and (2) the unchanged 6h MODEL_DOWNLOAD_LOCK_TIMEOUT default despite the larger (~756GB vs ~433GB) checkpoint, which remains env-overridable and only risks an explicit failure (not silent corruption) if too short. I also independently checked the "Conventions noted" concern about --use-chat-template/run_benchmark_serving and confirmed no agentic script in the repo (including the validated FP4 sibling) calls that helper, so its absence here is not a regression.
Other factors
The configs/nvidia-master.yaml and runners/launch_b200-nscale-compat.sh diffs are minimal, mirror existing sibling patterns byte-for-byte in structure, and are easy to verify by eye. The bug hunting system reported no findings after a full multi-round review. Given the mechanical nature of the port, the narrow and well-justified deltas, and that my own spot checks did not surface anything beyond what was already investigated and ruled out, I'm comfortable approving.
This review covers commit 60fcce4, which is no longer the latest commit on this pull request; later commits are not covered by it.
…GLE DSA fp8 read-door None guard) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34169297260 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34179006158 |
1 participant
Summary
Day-zero GLM-5.2 FP8 AgentX (agentic-coding) recipe on B200 with SGLang native EAGLE/MTP:
glm5.2-fp8-b200-sglang-agentic-mtp, the FP8 precision sibling ofglm5.2-fp4-b200-sglang-agentic-mtp.zai-org/GLM-5.2-FP8(~756 GB, 141 shards,quant_method=fp8, 128x128 weight blocks, dynamic e4m3 activations). Staged on the day-zero run by the script's completeness-checked, flock-serializedhf downloadif not already under/scratch/models/GLM-5.2-FP8.benchmarks/single_node/agentic/glm5.2_fp8_b200_sglang_mtp.sh, a port of the validated FP4 B200 script. Only two functional deltas:HF_MODEL_ID=zai-org/GLM-5.2-FP8and--quantization fp8in place ofmodelopt_fp4. TP8 low-latency arm with fp8_e4m3 KV, cutedsl bf16 GEMM, chunked-prefill 8192, mem-fraction-static 0.83, glm47/glm45 parsers, HiCache (ratio 0.75 at c1-c8, 169 GB/rank target pool at c12/c16). EAGLE num-steps 3 / topk 1 / 4 draft tokens; acceptance pinned to golden AL 2.99, which was measured on this FP8 checkpoint.lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00(verified on Docker Hub, pushed 2026-09-07T01:43:42Z, digestsha256:19b8fa1223cc339c1eae7a5b703f1a8c2543b5b119155bf3d7efaef18f77f007, sgl-project/sglang@30705c00). Same tag as the Qwen3.5 B200 SGLang bumps in [Klaud Cold] Update qwen3.5-fp8-b200-sglang-agentic-mtp SGLang image to nightly-dev-cu13-20260907-30705c00 #2861 / [Klaud Cold] Update qwen3.5-fp4-b200-sglang-agentic-mtp SGLang image to nightly-dev-cu13-20260907-30705c00 #2862.glm5.2/fp8routing torunners/launch_b200-nscale-compat.sh(MODEL_PATH default/scratch/models/GLM-5.2-FP8,SRT_SLURM_MODEL_PREFIX=glm5.2-fp8); it previously hard-failed withUnsupported model prefix/precisionfor this model.Validation:
bash -non the script and launcher, both YAMLs parse,generate_sweep_configs.py test-configyields the expected 5 cells, and the launcher's script-name construction resolves to the new file.Recipes touched:
glm5.2-fp8-b200-sglang-agentic-mtpTest plan
🤖 Generated with Claude Code
Note
Low Risk
Changes are limited to benchmark scripts, master config, changelog, and B200 launcher model-path routing; no production serving or auth paths are modified.
Overview
Introduces
glm5.2-fp8-b200-sglang-agentic-mtp, the FP8 counterpart to the existing NVFP4 B200 AgentX MTP recipe, so agentic-coding sweeps can compare precision curves on the same TP8 + HiCache + EAGLE/MTP shape.Adds
benchmarks/single_node/agentic/glm5.2_fp8_b200_sglang_mtp.shas a port of the FP4 script:zai-org/GLM-5.2-FP8,--quantization fp8, completeness-checked flock-serialized staging for the large checkpoint, and the same serve knobs (golden AL 2.99 simulation, HiCache, glm47/glm45 parsers).configs/nvidia-master.yamlregisters the recipe with search space conc [1, 4, 8, 12, 16] and imagelmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b(re-pinned from 2026-09-07 for sglang #38318 DSA fp8/EAGLE crash guard).runners/launch_b200-nscale-compat.shnow routesglm5.2+fp8to/scratch/models/GLM-5.2-FP8.perf-changelog.yamldocuments the new config key and image bump.Reviewed by Cursor Bugbot for commit 4e726bf. Bugbot is set up for automated code reviews on this repo. Configure here.
Update: re-pinned to the 2026-09-08 nightly
The 2026-09-07 cu13 nightly (
nightly-dev-cu13-20260907-30705c00) carries an unguardedkv_index_translator.translate_dcp_read_idscall on the DSA fp8 KV read path; the EAGLE draft backend never binds a translator, so GLM-5.2 MTP runs crash intermittently withAttributeError: 'NoneType' object has no attribute 'translate_dcp_read_ids'(seen on the MI355X FP8 sibling, run 34173459478, after 74 minutes of clean serving). sgl-project/sglang#38318 (merged 2026-09-07T20:03Z) adds the None guard. Re-pinned tolmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b(pushed 2026-09-08T01:40:59Z, digestsha256:9a352a35c973a2357372e85f3bcb5388b6b3c46c1329165987260f3b089647dc), which is six commits past that fix. Recipe otherwise unchanged.