[NV] add glm5-fp4-gb200-dynamo-sglang / 新增 GLM-5 NVFP4 GB200 Dynamo SGLang 分离式基准测试 - #1780
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27575447726 |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit aa5f207. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27652197300 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27652967695 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27653968669 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28065782305 |
4 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28065782305 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28065782305 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28065782305 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28065782305 |
# Conflicts: # .github/configs/nvidia-master.yaml # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28483902187 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28615855931 |
functionstackx
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28828457171 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28828457171 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28828457171 |
|
/reuse-sweep-run |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
Ankur-singh
changed the title
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
# Conflicts: # perf-changelog.yaml
…LM-5.1 NVFP4 GB200 分离式 dynamo-sglang MTP 配置 (#2115) * Add glm5-fp4-gb200-dynamo-sglang-mtp config * Update perf-changelog pr-link for #2115 * Pass full CONFIG_FILE selector to srtctl in launch_gb200-nv.sh * Point gb200 MTP config at GLM-5.1 NVFP4 weights * Use glm5.1 model-prefix for gb200 MTP config (match PR #1780) * Align gb200 recipe model.container with image string; drop launcher alias --------- Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com>
…LM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 (#2114) * Add glm5-fp4-gb300-dynamo-sglang-mtp config * Update perf-changelog pr-link for #2114 * Strip zip_override selector before name-override sed in launch_gb300-nv.sh * Point gb300 MTP config at GLM-5.1 NVFP4 weights * Use glm5.1 model-prefix for gb300 MTP config (match PR #1780 convention) * Skip srtctl preflight for gb300 glm5.1 (weights on compute-only /scratch)
…runtime Ports the 23 STP recipes (13 ISL1K_OSL1K + 10 ISL8K_OSL1K) from NVIDIA/srt-slurm sa-submission-q2-2026 into benchmarks/multi_node/srt-slurm-recipes/GLM5/... so that the version consumed by the sweep is pinned to this PR rather than tracking upstream drift. Mirrors the same overlay convention PR #1780 used for the dynamo-sglang GLM5.1 recipes. launch_gb200-nv.sh adds `mkdir -p recipes/GLM5 && cp -rT ...` to the existing glm5.1 dynamo-trt clone case so the local tree overrides the upstream copy in the srt-slurm checkout before srtctl reads any recipe.
…增 GLM-5 NVFP4 GB200 分离式 Dynamo TRT-LLM STP 基准测试 (#1803) * Add GLM-5.1 NVFP4 GB200 disaggregated TRT-LLM benchmarks via Dynamo (non-MTP) Adds `glm5-fp4-gb200-dynamo-trt` covering 13 ISL1K/OSL1K + 10 ISL8K/OSL1K STP recipes from NVIDIA/srt-slurm `sa-submission-q2-2026` under `recipes/GLM5/disagg/trtllm_dynamo/gb200_nvfp4/{ISL1K_OSL1K,ISL8K_OSL1K}/STP`. Model naming aligned with the GLM-5.1 convention already used by the GB200 dynamo-sglang entry: `model: nvidia/GLM-5.1-NVFP4`, `model-prefix: glm5.1`, `MODEL_PATH=/mnt/lustre01/models/GLM-5.1-NVFP4`. `SRT_SLURM_MODEL_PREFIX` matches the upstream recipe's `model.path` alias (`nvidia/GLM-5-NVFP4`). `launch_gb200-nv.sh` gains a `glm5.1 + fp4` case in the dynamo-trt dispatch block and a matching srt-slurm clone entry. * Check in GLM5 trtllm_dynamo GB200 NVFP4 recipes locally + overlay at runtime Ports the 23 STP recipes (13 ISL1K_OSL1K + 10 ISL8K_OSL1K) from NVIDIA/srt-slurm sa-submission-q2-2026 into benchmarks/multi_node/srt-slurm-recipes/GLM5/... so that the version consumed by the sweep is pinned to this PR rather than tracking upstream drift. Mirrors the same overlay convention PR #1780 used for the dynamo-sglang GLM5.1 recipes. launch_gb200-nv.sh adds `mkdir -p recipes/GLM5 && cp -rT ...` to the existing glm5.1 dynamo-trt clone case so the local tree overrides the upstream copy in the srt-slurm checkout before srtctl reads any recipe. * Switch glm5-fp4-gb200-dynamo-trt from GLM-5.1 to GLM-5 - nvidia-master.yaml: model nvidia/GLM-5-NVFP4, model-prefix glm5 - launch_gb200-nv.sh: dynamo-trt glm5/fp4 case uses MODEL_PATH=/home/sa-shared/glm-model/GLM-5-NVFP4, SERVED_MODEL_NAME=glm-5-nvfp4; matching srt-slurm overlay branch keyed on MODEL_PREFIX=glm5 - perf-changelog: describe as GLM-5 with the new model path * Point glm5-fp4-gb200-dynamo-trt MODEL_PATH at Lustre Prior /home/sa-shared path was not present on the watchtower login node (the launcher's model-path pre-check surfaced FileNotFoundError immediately). Move MODEL_PATH to /mnt/lustre01/slurm-shared/glm-model/GLM-5-NVFP4, which is on Lustre and cross-mounted to every watchtower compute node. * Pin srt-slurm to v1.0.17 + adopt recipes/trtllm/glm5 overlay layout - Rename benchmarks/multi_node/srt-slurm-recipes/GLM5/disagg/trtllm_dynamo/ gb200_nvfp4/** -> benchmarks/multi_node/srt-slurm-recipes/trtllm/glm5/... . - launch_gb200-nv.sh dynamo-trt+glm5 branch: git checkout v1.0.17, mkdir -p recipes/trtllm/glm5, cp -rT src/dst updated. - configs/nvidia-master.yaml: 23 CONFIG_FILE= paths under glm5-fp4-gb200-dynamo-trt retargeted to recipes/trtllm/glm5/... ; matching doc-URL comments repointed at v1.0.17. - perf-changelog: PR 1803 description updated to v1.0.17 + new overlay path. gb300 sibling entries (recipes/GLM5/.../gb300_nvfp4/**) intentionally untouched — separate PR. * fix: bump srt-slurm pin from v1.0.17 to v1.0.26 Picks up NVIDIA/srt-slurm#261, which sets the dynamo.trtllm publish_events + metrics flags to false by default. Applies the bump in three places: the git checkout in launch_gb200-nv.sh, the 23 source-URL comments in configs/nvidia-master.yaml, and the perf-changelog description bullets for glm5-fp4-gb200-dynamo-trt. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Update perf-changelog.yaml * fix(config): declare NIXL for GLM-5 disaggregation Set the top-level kv-p2p-transfer field required by disaggregated matrix validation. The TRT-LLM recipes retain UCX as their internal cache transceiver backend.\n\n中文：为 GLM-5 分离式配置声明顶层 NIXL KV 传输方式，以满足矩阵校验要求；TRT-LLM recipe 继续使用 UCX 作为内部缓存传输后端。 * Update perf-changelog.yaml --------- Co-authored-by: hshrivastava-droid <hshrivastava@nvidia.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Cameron Quilici <cjquilici@gmail.com>
4 participants
Note
Low Risk
Benchmark and CI launcher/config only; no application runtime or auth changes. Main review surface is recipe/topology correctness and cluster resource assumptions.
Overview
Adds GLM-5 NVFP4 disaggregated Dynamo + SGLang benchmark coverage on GB200, mirroring the existing GB300 glm5 entry pattern.
nvidia-master.yamlintroducesglm5-fp4-gb200-dynamo-sglangwithfixed-seq-lenscenarios for 8k1k and 1k1k: wide-EP decode (TP=32) max-throughput topologies (4p–10p prefill variants) and per-node TP=4 low-latency decode workers, each wired to a concreteCONFIG_FILEunderrecipes/sglang/glm5/gb200-fp4/.New srt-slurm recipe YAMLs (ported from upstream
gb200-fp4/glm5.yaml, one file per topology) live underbenchmarks/multi_node/srt-slurm-recipes/sglang/glm5/gb200-fp4/with Slurm resources, Dynamo frontend, nixl disagg, and tunedsglang_config/sa-benchconcurrency per recipe.runners/launch_gb200-nv.shmapsglm5+fp4to lustreGLM-5-NVFP4and overlays the glm5 recipe tree ontoNVIDIA/srt-slurm(sa-submission-q2-2026).perf-changelog.yamldocuments the new config key.Reviewed by Cursor Bugbot for commit ba74df2. Bugbot is set up for automated code reviews on this repo. Configure here.
中文说明
新增 GLM-5 NVFP4 在 GB200 上通过 Dynamo + SGLang 的分离式多节点基准测试配置，与现有 GB300 配置模式一致。在
nvidia-master.yaml中添加glm5-fp4-gb200-dynamo-sglang条目，覆盖 8k1k 和 1k1k 的定长序列场景，包括宽 EP 高吞吐拓扑和单节点 TP=4 低延迟解码。新增 srt-slurm 配方文件，更新launch_gb200-nv.sh启动脚本和perf-changelog.yaml。