Add DSV4 B200 disaggregated Dynamo SGLang STP configuration / 新增 DSV4 B200 分离式 Dynamo SGLang STP 配置 - #2560
Conversation
新增 DSV4 B200 Dynamo-SGLang STP 配置，包含十个分离式拓扑并接入 B200 多节点运行环境。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
将性能变更日志条目关联到 PR #2560。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| @@ -0,0 +1,145 @@ | |||
| name: "disagg-b200-1p1d-dep8-tp8-4-c64" | |||
There was a problem hiding this comment.
🟡 The name: field in disagg-b200-1p1d-dep8-tp8.yaml is "disagg-b200-1p1d-dep8-tp8-4-c64", with a stray -4- segment that doesn't match this recipe's topology (1 prefill + 1 decode = 2 nodes) or the naming convention used by every other new B200 recipe in this PR (e.g. disagg-b200-1p4d-dep8-tp8-c64, disagg-b200-1p1d-tp8-tp8-c1). It looks like a copy-paste leftover from a GB200-style template where that suffix denotes total node count; harmless at runtime since launch_b200-nscale-slurm.sh overwrites the name via sed before srtctl reads it, but worth cleaning up for consistency.
Extended reasoning...
The name: field on line 1 of disagg-b200-1p1d-dep8-tp8.yaml reads "disagg-b200-1p1d-dep8-tp8-4-c64". Every other new B200 recipe added in this PR follows a clean <topology>-c<concurrency> naming pattern with no extra numeric segment: disagg-b200-1p1d-dep8-dep8.yaml -> disagg-b200-1p1d-dep8-dep8-c512, disagg-b200-1p1d-tp8-tp8.yaml -> disagg-b200-1p1d-tp8-tp8-c1, disagg-b200-1p4d-dep8-tp8.yaml -> disagg-b200-1p4d-dep8-tp8-c64, and so on through the 3p2d/4p2d/5p2d/6p2d files. This file alone inserts a stray -4- between the topology and the concurrency suffix.\n\nOn the existing GB200 recipes in this codebase, that kind of numeric segment is a real, intentional convention: it denotes total node count (prefill_nodes + decode_nodes), e.g. a GB200 disagg-gb200-1p4d-dep8-tp8-10-c64.yaml-style file uses -10- because it sums to 10 nodes. Applying that same mental model here breaks down: this recipe's resources: block sets prefill_nodes: 1 and decode_nodes: 1, i.e. 2 total nodes, not 4. So -4- isn't just inconsistent with sibling files in this PR, it's also the wrong number under the very convention it appears to be borrowed from — strong evidence this is a copy-paste leftover from a GB200-style template rather than a deliberate choice.\n\nProof by walkthrough: (1) open disagg-b200-1p1d-dep8-tp8.yaml, line 1 is name: \"disagg-b200-1p1d-dep8-tp8-4-c64\"\". (2) Check resources: in the same file: prefill_nodes: 1, decode_nodes: 1 -> topology is 1p1d, 2 nodes total. (3) Compare to the sibling disagg-b200-1p4d-dep8-tp8.yaml in the same PR, which has prefill_nodes: 1, decode_nodes: 4 (5 nodes total) and is named disagg-b200-1p4d-dep8-tp8-c64 — no numeric node-count segment despite actually having more nodes. (4) There is therefore no consistent rule under which -4- belongs in the 1p1d file's name.\n\nWhy nothing currently prevents or catches this: recipe names are free-form YAML string values with no schema validation, and nothing in configs/nvidia-master.yaml or the sweep matrix reads the name: field for routing — the matrix entry in nvidia-master.yaml references the recipe purely by its CONFIG_FILE path, not by the embedded name. And at runtime, runners/launch_b200-nscale-slurm.sh executes sed -i \"s/^name:.*/name: \\\"\\\\"/\" \"\\"\" before \"srtctl apply\"\" ever runs, unconditionally replacing whatever is on that line with the runner-generated name. So this typo has zero effect on job submission, GPU allocation, or benchmark results — it is purely a checked-in metadata string that a human reading the repo (or grepping recipe names) would see and find confusing.\n\nFix is trivial: change line 1 to name: "disagg-b200-1p1d-dep8-tp8-c64"`, matching the convention of every sibling file added in this PR. Given the complete absence of functional impact (confirmed by tracing the sed override in the launcher) and the triviality of the fix, this is cosmetic — nit severity, not a merge blocker.
|
|
||
| - config-keys: | ||
| - dsv4-fp4-b200-dynamo-sglang | ||
| description: | ||
| - "Add a DeepSeek-V4-Pro FP4 Dynamo-SGLang STP configuration for B200 with ten 8k/1k disaggregated topologies." | ||
| - "Use Mooncake for KV transfer with chat-formatted benchmark inputs." | ||
| - "Image: lmsysorg/sglang:nightly-dev-cu13-20260708-b3632494" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX |
There was a problem hiding this comment.
🟡 The new perf-changelog.yaml entry for dsv4-fp4-b200-dynamo-sglang has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, a literal unresolved placeholder. Every other entry in the file uses the real PR number, and since this PR is #2560, XXX should be replaced with 2560.
Extended reasoning...
The tail end of the newly-appended perf-changelog.yaml entry (config-keys: [dsv4-fp4-b200-dynamo-sglang]) reads:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXXXX is a literal, unresolved placeholder rather than an actual PR number.
Why this happened: perf-changelog.yaml entries are appended before the PR number is known (the PR doesn't exist yet at authoring time), so authors commonly stub the link with a placeholder and are expected to swap in the real number once GitHub assigns it. That final substitution step was missed here.
Evidence this is the convention, not a one-off: every other entry in the file uses a real, resolvable PR number — the immediately preceding entry (the B300 DSV4 AgentX MTP retune) links to /pull/2536, and entries further back link to /pull/2309, /pull/2238, and /pull/2206. This PR's own metadata identifies it as #2560, so the correct value is unambiguous.
Step-by-step proof:
- Open
perf-changelog.yamlat the new entry (added at the physical end of the file, lines 5779-5786). - Read the last field:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. - Compare against the pr-metadata for this change:
<pr number="2560">. XXX != 2560— the link is broken/dangling and does not resolve to this PR (or any PR).- Scan sibling entries in the same file (2536, 2309, 2238, 2206, ...) — all use real numbers, confirming XXX is an anomaly, not an established pattern.
Impact: This is a documentation/traceability defect, not a functional one — it doesn't affect any benchmark execution, config resolution, or CI job. But it does violate the PR's own checklist item ("I have appended a new entry to the physical end of perf-changelog.yaml ... accurate changelog entry") and breaks the changelog's utility as an audit trail back to the originating PR, since anyone auditing changelog history via pr-link will hit a 404 for this entry.
Fix: Replace XXX with 2560 in the pr-link field before merge.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31604708072 |
将 DSV4 B200 STP 配方恢复为原始合成输入路径，并让 nscale 启动器同时支持 STP 与 MTP。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31634521055 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31634521055 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31634521055 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31634521055 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31634521055 |
将 DSV4 B200 STP 分离式配置切换到 NIXL，并同步更新 SGLang 镜像、Dynamo 版本、运行时参数和聊天模板。
同步最新的 main 分支。
为 DSV4 B200 STP 的十个分离式配方禁用聊天模板，并同步最新的 main 分支与性能变更日志。
合并 main，并解决性能变更日志和 B200 Nscale 启动脚本冲突。
更新基础分支并保留 DSV4 B200 SGLang 配置矩阵。
更新 DSV4 B200 SGLang 镜像至可用版本。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33432328618 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
将十个 DSV4 B200 SGLang 配方的容器镜像与主配置保持一致。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33432820769 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33438053394 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33444064622 |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 99d1561. Configure here.
为十个 B200 分离式 STP 配方启用必需的 DCGM 功耗遥测，并同步更新功耗契约测试中的配方计数。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33445712857 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33447526958 |
|
/reuse-sweep-run 33447526958 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
…precation
MODELS.md set 2026-09-08 as the last day for the Single-turn 8k1k scenario on
DeepSeek-V4-Pro 1.6T (dsv4). Today is 2026-09-09, so this carries it out.
Agentic coding is unaffected and stays active for dsv4, including its MTP and
DSpark arms; the model is not retired.
Removes 46 dsv4 8k1k config keys from the active master configs and archives
them under configs/deprecated/ (bodies byte-identical to main):
- nvidia-dsv4-8k1k-master.yaml (33): fp4 b200 sglang/vllm/trt (+mtp), fp4
b300 sglang/vllm/trt (+mtp), fp8 h200 vllm/sglang (+mtp), and every
b200/b300/gb200/gb300 dynamo-vllm, dynamo-sglang, dynamo-trt and
llmd-vllm multi-node 8k1k key
- amd-dsv4-8k1k-master.yaml (13): fp8 mi300x/mi325x vllm (+mtp), fp4 mi355x
vllm/sglang/atom (+mtp), fp4 mi355x sglang-disagg (+mtp) and atom-disagg
The count is 46 rather than the 45 quoted in the notice because
dsv4-fp4-b200-dynamo-sglang (#2560) landed after the notice was written.
The 19 dsv4 agentic-coding keys are untouched.
Their 28 benchmark scripts move to the sibling deprecated/ directories (25
under single_node/fixed_seq_len/, 3 under multi_node/). The agentic
dsv4_fp4_mi355x_vllm_mtp.sh comment that cites the fixed_seq_len recipe now
points at its deprecated/ path.
Removes test_gb300_dsv4_recipe_images_match_their_master_configs and its two
private helpers: it asserted on the gb300 dynamo-sglang dsv4 8k1k keys that are
archived here, and no remaining gb300 dynamo-sglang key pins its recipe
container to the master image.
Deliberately kept, per #2493 and #2527: the dsv4 SPEED-Bench scripts
(driven by speedbench-al.yml, not the master configs), the srt-slurm and llm-d
recipe YAMLs the archived multi-node keys reference, ci-priority.yaml,
thresholds.yaml, and perf-changelog.yaml (removals do not trigger benchmarks).
中文：执行 MODELS.md 中 2026 年 9 月 8 日 DeepSeek-V4-Pro 1.6T（dsv4）单轮 8k1k
场景的下线计划；今日为 2026-09-09，故予以执行。智能体编码不受影响并继续启用，
含 MTP 与 DSpark 分支；该模型不退役。
从启用的主配置中移除 46 个 dsv4 8k1k 配置项并归档至 configs/deprecated/
（NVIDIA 33 个、AMD 13 个，内容与 main 逐字节一致）。数量为 46 而非公告所述的
45，是因为 dsv4-fp4-b200-dynamo-sglang（#2560）在公告发布后合入。19 个 dsv4
智能体编码配置项未做改动。对应的 28 个基准测试脚本移入同级 deprecated/ 目录
（单节点 fixed_seq_len/ 25 个、多节点 3 个）。
移除 test_gb300_dsv4_recipe_images_match_their_master_configs 及其两个私有
辅助函数：该测试针对的 gb300 dynamo-sglang dsv4 8k1k 配置项已在此归档，且余下
的 gb300 dynamo-sglang 配置项均未将配方容器固定为主配置镜像。
按 #2493 与 #2527 的先例保留：dsv4 SPEED-Bench 脚本、已归档多节点配置项引用的
srt-slurm 与 llm-d 配方 YAML、ci-priority.yaml、thresholds.yaml 以及
perf-changelog.yaml（移除操作不触发基准测试）。
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5 participants
Description
Update the DSV4 B200 multinode disaggregated Dynamo SGLang STP configuration for 8k/1k.
中文说明
更新 DSV4 在 B200 上的多节点分离式 Dynamo SGLang STP 8k/1k 配置。
Related Issue
Supersedes #2251.
替代 #2251。
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PRNote
Low Risk
Benchmark and launcher configuration only; changes affect official perf/power recipe selection and Slurm job definitions, not application runtime logic.
Overview
Adds
dsv4-fp4-b200-dynamo-sglangtonvidia-master.yamlwith ten fixed 8k/1k disaggregated scenarios (varying prefill/decode worker counts and TP vs DEP8 layouts), each pointing at a new srt-slurm recipe undersglang/deepseek-v4/8k1k/.The recipes wire B200 Slurm resources, Dynamo KV routing, SGLang prefill/decode with
nixldisaggregation transfer,use_chat_template: false, and required DCGM power telemetry. Image and Dynamo revision are pinned tolmsysorg/sglang:nightly-dev-cu13-20260821-f825d729and hash86f84b9486905c2bfaa4b756c9fa39f04ecf5ebd.launch_b200-nscale-slurm.shnow treats DSV4 FP4 Dynamo-SGLang as in-scope forSPEC_DECODINGnone(STP) as well as MTP.utils/test_b200_b300_power_official_contract.pyexpects 47 power-contract recipes (+10 oncluster:b200-nscale).perf-changelog.yamldocuments the new config key and NIXL/image/telemetry updates.Reviewed by Cursor Bugbot for commit 23ff29f. Bugbot is set up for automated code reviews on this repo. Configure here.