[NV] add dsv4-fp4-gb300-dynamo-sglang-mtp-1k1k / 新增 DeepSeek-V4-Pro FP4 GB300 Dynamo SGLang MTP 1k1k 基准测试配置 - #1697
[NV] add dsv4-fp4-gb300-dynamo-sglang-mtp-1k1k / 新增 DeepSeek-V4-Pro FP4 GB300 Dynamo SGLang MTP 1k1k 基准测试配置#1697hshrivastava-droid wants to merge 32 commits into
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you
PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
|
||
| model: | ||
| path: "dsv4-pro" | ||
| container: "lmsysorg/sglang:nightly-dev-cu13-20260510-2473659e" |
There was a problem hiding this comment.
Low-latency recipe container missing
High Severity
The two low-latency recipes still pin model.container to lmsysorg/sglang:nightly-dev-cu13-20260510-2473659e, while dsv4-fp4-gb300-dynamo-sglang-mtp-1k1k imports squash only for lmsysorg/sglang:nightly-dev-cu13-20260603-83bc7766. Workers resolve the recipe tag, which is not mapped in srtslurm.yaml and is documented as absent from Docker Hub, so those matrix points can fail at enroot import.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 2aeafb4. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27237009377 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27242563991 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27242563991 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27253510982 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27253510982 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27253510982 |
| image: lmsysorg/sglang:nightly-dev-cu13-20260603-83bc7766 | ||
| model: deepseek-ai/DeepSeek-V4-Pro | ||
| model-prefix: dsv4 | ||
| runner: gb300-nv |
There was a problem hiding this comment.
Wrong runner for SGLang recipes
High Severity
The new dsv4-fp4-gb300-dynamo-sglang-mtp-1k1k entry uses runner: gb300-nv, while sibling DeepSeek-V4 GB300 dynamo-sglang configs use gb300-cw. launch_gb300-nv.sh never copies staged recipes/sglang/deepseek-v4 into srt-slurm (only glm5 gets that path), and its srtslurm.yaml omits the dsv4-pro alias many new recipes use—so srtctl apply is likely to fail on missing recipes or model preflight.
Reviewed by Cursor Bugbot for commit 47460ef. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27366225297 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27366364441 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27372623348 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27372623348 |
|
@hshrivastava-droid The GB300 CW are back |
Resolve conflicts: - perf-changelog.yaml: take main's version and append the PR 1697 entry (dsv4-fp4-gb300-dynamo-sglang-mtp) at the end - runners/launch_gb300-nv.sh: keep both concurrent dynamo-sglang elif branches (our dsv4 + main's qwen3.5)
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28613151268 |
functionstackx
changed the title
Previous tag lmsysorg/sglang:nightly-dev-20260527-14f81a67 expired on Docker Hub (404), causing every job in run 28613151268 (PR #1697) to fail at enroot import. Bump the 8 MTP recipes and the nvidia-master entry to lmsysorg/sglang:nightly-dev-cu13-20260706-8673e85e (verified live). Update the PR's perf-changelog entry to reflect the bump. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts: # perf-changelog.yaml # runners/launch_gb300-nv.sh
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28822342442 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28822342442 |
…26) for dsv4+dynamo-sglang
The new 1k/1k STP recipes use benchmark: {type: custom, command: ...},
a schema feature that only exists on NVIDIA/srt-slurm main. Pinning
sa-submission-q2-2026 caused srtctl to reject the recipe with
"Invalid config ... {'benchmark': {'command': ['Unknown field.']}}"
before any benchmark could run (see failing sweep run 28977862941).
Same launcher fix PR #1697 already carries; applying it here so the
dynamo-sglang + dsv4 elif clones NVIDIA/srt-slurm@main.
Pin master image: and all 19 (11 x 1k1k + 8 x 8k1k) recipe model.container values to lmsysorg/sglang:nightly-dev-cu13-20260710-cfc66e05, aligning the whole config on the newer nightly and clearing the prior split between 1k1k (v0.5.13.post1-cu130) and 8k1k (07-06 nightly).
The runners/launch_gb300-nv.sh dynamo-sglang + dsv4 branch clones NVIDIA/srt-slurm and previously ran 'git checkout main', so every sweep picked up whatever srtctl 'main' happened to be at that moment. Between the last green run (2026-06-16) and the current failing head (2026-07-06), main advanced meaningfully, which is a plausible driver of the recent 'decode worker exit 137 / Server did not become healthy' failures on the byte-identical 1k1k recipes. Pin to v1.0.17 so all future sweeps in this PR run against a fixed srtctl SHA and drift can be ruled in or out cleanly.
Resolve the PR #1697 merge conflicts by keeping main's changelog entries and appending the branch entry, while preserving both launcher paths. Also remove recipe-level 3h Slurm limits from the new 1k1k disagg recipes so GitHub Actions' 8h cap governs the sweep. 中文：将 main 同步到 dsv4 GB300 配置分支。通过保留 main 的 changelog 条目并在末尾追加本分支条目来解决 PR #1697 的合并冲突，同时保留两个启动器路径；并移除新增 1k1k 分离式配方中的 3 小时 Slurm 限制，让 GitHub Actions 的 8 小时上限控制扫描运行。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29129150548 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29129150548 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29129150548 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29540585164 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29540585164 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29540585164 |
|
/reuse-sweep-run |
hshrivastava-droid
changed the title
|
/reuse-sweep-run 29540585164 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
* Add DeepSeek-V4-Pro NVFP4 GB300 1k1k STP disagg scenarios Appends 7 new fixed-seq-len entries (isl=1024, osl=1024) to the existing dsv4-fp4-gb300-dynamo-sglang key: 1P1D DEP8/DEP16, 1P4D and 1P6D TP4, 2P1D DEP12/DEP16/DEP48. Recipes under benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/1k1k/, container aligned with the existing GB300 8k1k image (lmsysorg/sglang:nightly-dev-cu13-20260520-425dffbd). launch_gb300-nv.sh gains a dynamo-sglang + dsv4 overlay branch (srt-slurm sa-submission-q2-2026 base, in-tree recipes overlaid on top). * Update perf-changelog pr-link for #2099 * Bump sglang nightly to nightly-dev-cu13-20260707-b4155233 Previous tag nightly-dev-cu13-20260520-425dffbd was GC'd from Docker Hub; every multi-node dyn-sgl job 404'd at enroot import. Also refresh dynamo commit ref in stack comment and fix repo URL in dep8 config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Bump 8k1k recipes and master image to sglang nightly-dev-cu13-20260707-b4155233 The earlier bump (3c9ee2e) only updated the 1k1k recipes, leaving the dsv4-fp4-gb300-dynamo-sglang master image: and all 8k1k recipes on the missing 20260520-425dffbd tag. The launcher's enroot import 404s on the master image tag and every matrix job dies before srt-slurm runs. This pulls the whole entry (master + 14 referenced recipes) onto the same tag. * launch_gb300-nv.sh: check out srt-slurm main (not sa-submission-q2-2026) for dsv4+dynamo-sglang The new 1k/1k STP recipes use benchmark: {type: custom, command: ...}, a schema feature that only exists on NVIDIA/srt-slurm main. Pinning sa-submission-q2-2026 caused srtctl to reject the recipe with "Invalid config ... {'benchmark': {'command': ['Unknown field.']}}" before any benchmark could run (see failing sweep run 28977862941). Same launcher fix PR #1697 already carries; applying it here so the dynamo-sglang + dsv4 elif clones NVIDIA/srt-slurm@main. * launch_gb300-nv.sh: pin srt-slurm to v1.0.17 for dsv4+dynamo-sglang Replace `git checkout main` with a pinned tag `git checkout v1.0.17`. v1.0.17 already carries the BenchmarkType.CUSTOM enum + benchmark.command schema field that the new 1k/1k STP recipes need, and pinning avoids drift as NVIDIA/srt-slurm main moves forward. * update result path in recipes 中文：更新配方中的结果路径 * Remove 1k1k STP disagg scenarios * fix: bootstrap GB300 DeepSeek-V4 source build Use srt-slurm v1.0.25 for hash-pinned Dynamo source builds so cargo and maturin are installed before compilation. 使用 srt-slurm v1.0.25 处理固定提交的 Dynamo 源码构建，确保编译前安装 cargo 和 maturin。 --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Rohit Pujar Nagraj <rpujarnagraj@nvidia.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com>
4 participants
Summary
dsv4-fp4-gb300-dynamo-sglang-mtpentry with eleven fixed-sequence 1k/1k scenarios for DeepSeek-V4-Pro on GB300 using FP4, multi-node disaggregated SGLang, and MTP/EAGLE speculative decoding.use_chat_template: true.lmsysorg/sglang:nightly-dev-cu13-20260710-cfc66e05.runners/launch_gb300-nv.shto useNVIDIA/srt-slurm@v1.0.17for DeepSeek-V4 Dynamo-SGLang and overlay the checked-in DeepSeek-V4 recipe tree.perf-changelog.yamlentry.Validation
Run Sweep 29540585164, attempt 4 completed successfully on in-PR commit
6d4c42c2606d62f1e222f500f6bea7441bdf66c9. It executed all 11 new 1k/1k benchmark jobs, all 8 affected 8k/1k benchmark jobs, 7 multi-node eval jobs, and the result/eval collection and comparison jobs. The DeepSeek-V4-Pro GSM8K results were 0.9613–0.9659em_strict(1,319 samples each), above the configured 0.91 threshold, using the pinned July 10 SGLang image.Current disposition
This PR predates PR #2263, which merged on July 18, 2026 and deprecated fixed-sequence 1k/1k scenarios from the active NVIDIA and AMD master configs. The current diff reintroduces active 1k/1k coverage, so it should not merge unchanged. The shortest disposition is to close/retire this PR; if the image, launcher, or 8k/1k recipe updates are still wanted, they should be resubmitted as a freshly validated 8k/1k-only change.
中文说明
dsv4-fp4-gb300-dynamo-sglang-mtp配置中新增 11 个固定序列长度 1k/1k 场景，用于 DeepSeek-V4-Pro 在 GB300 上通过 FP4、多节点 SGLang 分离式推理和 MTP/EAGLE 投机解码进行基准测试。use_chat_template: true。lmsysorg/sglang:nightly-dev-cu13-20260710-cfc66e05。runners/launch_gb300-nv.sh：DeepSeek-V4 Dynamo-SGLang 路径固定使用NVIDIA/srt-slurm@v1.0.17，并在运行时覆盖仓库内的 DeepSeek-V4 配方目录。perf-changelog.yaml末尾追加对应记录。验证结果
Run Sweep 29540585164，第 4 次运行 已在当前 PR 内的提交
6d4c42c2606d62f1e222f500f6bea7441bdf66c9上成功完成。该运行执行了全部 11 个新增 1k/1k 基准测试任务、全部 8 个受影响的 8k/1k 基准测试任务、7 个多节点评估任务，以及结果/评估汇总与对比任务。DeepSeek-V4-Pro 的 GSM8Kem_strict得分为 0.9613–0.9659（每项 1,319 个样本），高于配置中的 0.91 阈值，并使用了固定的 7 月 10 日 SGLang 镜像。当前处置建议
本 PR 早于 PR #2263 创建；后者已于 2026 年 7 月 18 日合并，并将固定序列长度 1k/1k 场景从 NVIDIA 和 AMD 的活动主配置中弃用。当前差异会重新引入活动的 1k/1k 配置，因此不应按现状合并。最短路径是关闭/退役本 PR；如果仍需保留镜像、启动器或 8k/1k 配方更新，应另行提交仅包含 8k/1k 范围的变更并重新完成验证。