[NV] Refresh GB300 DeepSeek-V4-Pro AgentX with SGLang DSpark6 / 使用 SGLang DSpark6 更新 GB300 DeepSeek-V4-Pro AgentX - #2623
Conversation
b8b5b29 to
75c1056
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟡
perf-changelog.yaml:5452-5461— AGENTS.md line 7 requires bilingual PR titles/descriptions (<English title> / <中文标题>plus a Chinese mirror section like## 中文说明), but PR #2623's title ("[NV] Refresh GB300 DeepSeek-V4-Pro AgentX with SGLang DSpark6") and body are English-only. Recently merged PRs (#2493, #2486) follow this bilingual convention — please add a Chinese title suffix and a## 中文说明section mirroring the summary.Extended reasoning...
What the rule is:
AGENTS.md(the single source of truth this repo'sCLAUDE.mdpoints to for all guidance) states verbatim in line 7: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format:<English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a## 中文说明section mirroring the summary...)." This applies to every PR in the repository, not an optional style suggestion.Where it is violated: PR #2623's title is
[NV] Refresh GB300 DeepSeek-V4-Pro AgentX with SGLang DSpark6— no/ <中文标题>suffix. The description contains only## Summary,## Validation, and## Referencesections in English, with no## 中文说明(or equivalent) mirroring the summary.Why this is not a false positive: The convention is actively followed elsewhere in this same repo's recent history.
git logshows PR #2493 titled"[Klaud Cold] Enact the August 3, 2026 scenario and precision deprecations / 执行 2026 年 8 月 3 日场景与精度下线"and PR #2486 titled"[Klaud Cold] minimaxm3-fp8-gb300-dynamo-vllm-mtp: day-zero GB300 MXFP8 EAGLE3 MTP + FULL_DECODE_ONLY CG / 新增 GB300 MXFP8 EAGLE3 MTP 配方，解码启用 FULL_DECODE_ONLY 图模式"— both bilingual per the mandated format. This confirms the rule is a live, enforced practice, and PR #2623 is an outlier that skipped it.Step-by-step proof:
- Open
AGENTS.mdat the repo root, line 7 — read the bilingual title/description requirement. - Open PR #2623's title:
[NV] Refresh GB300 DeepSeek-V4-Pro AgentX with SGLang DSpark6. Search for/followed by CJK characters — none found. - Open PR #2623's description: three English-only
##sections (Summary,Validation,Reference). Search for中文or## 中文说明— none found. - Compare against
git log --oneline -5: #2493 and #2486 both show the<English> / <中文>pattern in their subject lines, confirming this is the repo's actual practice, not a dead-letter rule. - Conclusion: PR #2623 is out of compliance with an explicit, currently-enforced repository convention.
Impact and fix: This does not affect the functional correctness of the recipe/config changes in the diff — it is purely a process/documentation gap. The fix is mechanical: rename the PR title to append
/ <中文标题>(e.g.,/ 刷新 GB300 DeepSeek-V4-Pro AgentX 至 SGLang DSpark6) and add a## 中文说明section to the body mirroring the English summary bullets. Since every original source bug for this finding is a[quality]-prefixed convention/documentation issue with no correctness impact, it should be gradednit. - Open
c8261b2 to
e5935c1
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31924886695 |
e5935c1 to
033ef73
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31925668643 |
033ef73 to
ab65e7f
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31925892725 |
ab65e7f to
c68dad8
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31926958601 |
weireweire
changed the title
Root cause: - The GB300 DeepSeek-V4-Pro AgentX matrix still used the older EAGLE/MTP serving path and an outdated SGLang nightly. - The previous draft targeted vLLM recipes instead of the existing SGLang topology ladder. Fix: - Replace the SGLang aggregate and disaggregated GB300 recipes with DSpark block size 6 on DeepSeek-V4-Pro-DSpark. - Use nightly-dev-cu13-20260816-4a6dc267, which already includes the required DSpark performance and MegaMoE support, without a runtime patch overlay. - Force thinking-on AL 3.82 with static ragged verification, retain MegaMoE on DEP workers, and set the 8P4D throughput endpoint to concurrency 1536. Validation: - Passed srtctl dry-run for all seven recipes. - Passed 224 matrix and schema tests. - Generated the eight targeted matrix jobs at concurrency 1, 4, 8, 32, 192, 400, 640, and 1536. 中文：将 GB300 DeepSeek-V4-Pro AgentX 的 SGLang 配置从 EAGLE/MTP 更新为 DSpark6，使用已包含所需上游修复的 20260816 nightly，不再注入补丁；固定 AL 3.82 与 static ragged verify，并将 8P4D 吞吐端点设为并发 1536。七份 recipe dry-run、224 项矩阵与 schema 测试以及八个目标矩阵任务生成均已通过。
c68dad8 to
8a4a0a0
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31929326055 |
Root cause: - The InferenceX port hard-coded SGLANG_SIMULATE_ACC_* in every recipe instead of carrying NVIDIA InferenceMAX PR #214 throughput-only master-config injection. - Eval-only launch therefore skipped the injector but still inherited synthetic acceptance from the checked-in recipe. - The previous AL 3.82 curve was measured for a different DeepSeek-V4-Pro variant and is not the model-specific golden value for the 0813 checkpoint. Fix: - Keep SGLANG_RAGGED_VERIFY_MODE=static in all six recipes while moving synthetic AL into each throughput search-space entry. - Preserve the earlier DSpark curve and commit a separate DeepSeek-V4-Pro-0813 probabilistic curve, selecting thinking-on DSpark6 AL 3.77 for throughput. - Add tests covering all six recipes, throughput injection, eval preservation, launcher ordering, DSpark6 parameters, tokenizer paths, and the model-specific golden value. Validation: - Passed 235 matrix, schema, runner, and synthetic-acceptance tests. - Generated seven throughput points and four eval-only points at the intended concurrencies. - Passed YAML parsing, formatting, and git diff checks. 中文：模拟接受长度仅在吞吐测试启动时注入，精度评测继续使用真实 DSpark 验证；保留旧模型曲线并新增 DeepSeek-V4-Pro-0813 概率曲线，DSpark6 吞吐 AL 改为 3.77。
8a4a0a0 to
d9a6672
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31950260070 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31958640004 |
044f8dc to
91e9096
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31959753035 |
91e9096 to
a3adb18
Compare
Root cause: Dynamo 1.4.0 still imported the legacy SGLang config-parser path, so the GB300 recipes wrote a compatibility module into the installed SGLang package at runtime. Fix: Pin all GB300 DSV4 AgentX recipes and router metadata to the public Dynamo 1.5.0.dev20260902 wheels containing ai-dynamo/dynamo#14054, then remove the setup hook, launcher copy, and site-packages patch. Validation: Confirmed the public aarch64 CPython 3.12 ai-dynamo and ai-dynamo-runtime wheels are downloadable and contain the new compatibility import; all seven recipes pass srtctl dry-run; 250 focused matrix and synthetic-acceptance tests pass; YAML, Bash syntax, changelog generation, and diff checks pass. 根因：Dynamo 1.4.0 仍导入旧的 SGLang 配置解析器路径，因此 GB300 配方会在运行时向已安装的 SGLang 包写入兼容模块。修复：固定到包含上游 #14054 的公开 Dynamo 1.5.0.dev20260902 wheels，并删除 setup hook、launcher 复制逻辑和 site-packages patch。验证：公开 aarch64 wheels、7 个 dry-run、250 项测试及静态检查均通过。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33763892158 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33769235286 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33769235286 |
Root cause: The DEP16 AgentX ladder assigned only 320 requests per prefill worker. Profiling showed frequent empty pending queues and partially filled prefill batches, so the measured points underused the added prefill capacity.\n\nFix: Raise the one- through four-worker DEP16 ladder to concurrency 480, 960, 1440, and 1920, and rename the corresponding recipes consistently. Keep the independently tuned DEP12 efficiency point at concurrency 1152.\n\nValidation: YAML parsing and matrix generation pass; all four updated recipes pass srtctl dry-run; 114 changelog tests pass; the concurrency-1920 four-worker topology completed a full-duration GB300 AgentX run with saturated prefill batches.\n\n中文：原 DEP16 阶梯每个 prefill worker 只有约 320 个请求，profiling 显示 pending queue 经常为空且 prefill batch 未填满。将一到四个 prefill worker 的并发调整为 480、960、1440 和 1920，并同步重命名 recipe；独立调优的 DEP12 c1152 效率点保持不变。YAML、矩阵生成、四份 recipe dry-run、114 项 changelog 测试以及 c1920 完整时长 GB300 AgentX 测试均通过。
# Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33769235286 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33844797283 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33844797283 |
…-final # Conflicts: # perf-changelog.yaml
Root cause: The standalone 2P DEP8 prefill plus DEP12 decode point failed repeated formal runs and did not produce a reliable benchmark result.\n\nFix: Remove the c1152 master-matrix arm and its dedicated recipe, and append a changelog entry recording the removal.\n\nValidation: Generated the filtered GB300 DSV4 Dynamo-SGLang AgentX matrix (7 entries, no c1152) and ran all 246 matrix-logic tests.\n\n中文：删除不稳定的 DSV4 c1152 拓扑。该独立的 2P DEP8 预填充加 DEP12 解码点在正式运行中连续失败，无法产生可靠结果。本提交删除对应矩阵条目和专用 recipe，并追加变更记录。验证后的矩阵包含 7 个条目且不再包含 c1152，矩阵逻辑测试 246 项全部通过。
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 322307a. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33933793466 |
|
/reuse-sweep-run 33933793466 |
删除未被引用且包含过时配方数量断言的合成接受长度测试。
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
|
Preserve the main-branch changelog byte-for-byte and append the three existing PR entries unchanged. Retain the DSpark recipes and both targeted master-config definitions without changes. Validation: 607 changelog, matrix, gating, and runner tests passed; changed YAML parses, launcher Bash syntax, exact-key matrix generation, and diff checks against main passed. 中文：合并 main 并解决 PR #2623 的更新日志冲突。逐字节保留 main 的历史内容，将本 PR 的三条记录原样追加到末尾；DSpark 配方和两个目标主配置保持不变。607 项本地测试及 YAML、Bash 语法、矩阵生成和差异检查通过。
Preserve the main-branch changelog byte-for-byte and append the three existing PR entries unchanged. Retain the DSpark recipes and both targeted master-config definitions without changes. Validation: 607 changelog, matrix, gating, and runner tests passed; changed YAML parses, launcher Bash syntax, exact-key matrix generation, and diff checks against main passed. 中文：合并 main 并解决 PR #2623 的更新日志冲突。逐字节保留 main 的历史内容，将本 PR 的三条记录原样追加到末尾；DSpark 配方和两个目标主配置保持不变。607 项本地测试及 YAML、Bash 语法、矩阵生成和差异检查通过。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
f06bc4d to
5dd01f8
Compare
Resolve conflicts in configs/nvidia-master.yaml and perf-changelog.yaml. - nvidia-master: keep this branch's rewrite of minimaxm3-fp4-gb200-dynamo-vllm-agentic-agg-mtp (nightly image, TP4 resident / TP4 lazy SimpleCPU offload / two-node TP8 resident, the -nightly-native recipe files), which supersedes main's PR #2807 tuning of the same entry. Main's disagg-mtp rewrite and the new Qwen3.5 GB300 Pareto entries are retained unchanged. - perf-changelog: keep both sides; main's #2812/#2807/#2623 entries stay in place and this branch's #2925 entry is appended at the end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5 participants
Summary / 概要
Refresh the complete GB300 DeepSeek-V4-Pro Dynamo-SGLang AgentX family from EAGLE/MTP to DSpark block size 6 on the DeepSeek-V4-Pro-0813 checkpoint.
将完整的 GB300 DeepSeek-V4-Pro Dynamo-SGLang AgentX 配置从 EAGLE/MTP 更新为基于 DeepSeek-V4-Pro-0813 checkpoint 的 DSpark block size 6。
Use
lmsysorg/sglang:nightly-dev-cu13-20260829-89816a21, public Dynamo1.5.0.dev20260902wheels, and NVIDIA/srt-slurm v1.0.40.Cover aggregate TP8 concurrency 1/4 and TP4 concurrency 8.
Cover disaggregated concurrency 480, 960, 1440, and 1920.
Enable the current DSV4 prefill, allocator, MegaMoE, routing, and hierarchical-cache gates used by the production configuration.
Use the committed DeepSeek-V4-Pro-0813 thinking-on golden synthetic acceptance length 3.77 for throughput; eval-only runs retain real DSpark verification.
使用
lmsysorg/sglang:nightly-dev-cu13-20260829-89816a21、公开 Dynamo1.5.0.dev20260902wheels 和 NVIDIA/srt-slurm v1.0.40。Aggregate 覆盖 TP8 并发 1/4 和 TP4 并发 8。
Disaggregated 覆盖并发 480、960、1440 和 1920。
启用生产配置当前使用的 DSV4 prefill、allocator、MegaMoE、routing 和 hierarchical-cache gates。
吞吐测试使用已提交的 DeepSeek-V4-Pro-0813 thinking-on golden synthetic acceptance length 3.77；eval-only 保留真实 DSpark verification。
Upstream compatibility / 上游兼容性
Dynamo
1.5.0.dev20260902contains ai-dynamo/dynamo#14054 and imports the current SGLang config-parser location directly. The recipes therefore run the published packages as shipped: no setup hook, site-packages modification, monkey patch, private wheel, or mounted source tree is used.Dynamo
1.5.0.dev20260902已包含 ai-dynamo/dynamo#14054，可直接导入当前 SGLang 配置解析器路径。因此 recipe 直接运行公开发布的软件包，不再使用 setup hook、site-packages 修改、monkey patch、私有 wheel 或挂载源码树。Validation / 验证
Verified that the public
ai-dynamoandai-dynamo-runtime1.5.0.dev20260902wheels are downloadable for Grace/aarch64 with Python 3.12 and thatai-dynamocontains the #14054 import logic.The six configured recipe files were exercised across seven successful throughput points in Run Sweep 33933793466.
The generated matrix contains the intended seven throughput points and four real-verification eval points.
The remaining runner test suite passes (15 tests); current-head CodeQL and Cursor Bugbot checks also pass.
Changelog validation, YAML parsing, Bash syntax, and diff checks passed.
已确认公开的
ai-dynamo和ai-dynamo-runtime1.5.0.dev20260902wheels 可用于 Grace/aarch64 与 Python 3.12，且ai-dynamo包含 #14054 的导入逻辑。6 个已配置 recipe 文件已在 Run Sweep 33933793466 的 7 个吞吐点中成功执行。
生成矩阵包含预期的 7 个吞吐点和 4 个真实 verification eval 点。
剩余 runner 测试套件通过（15 项测试）；当前 head 的 CodeQL 和 Cursor Bugbot 检查也通过。
Changelog 校验、YAML 解析、Bash 语法及 diff 检查通过。
Note
Medium Risk
Large benchmark-matrix and cluster-runner changes affect which GB300 jobs run and at what concurrency; misconfiguration could waste multi-node GPU time but does not touch auth or user data paths.
Overview
Refreshes the GB300 DeepSeek-V4-Pro Dynamo-SGLang AgentX benchmark family from EAGLE/MTP to DSpark block size 6 on
DeepSeek-V4-Pro-0813, with newer SGLang (20260829) and Dynamo1.5.0.dev20260902wheels. Aggregate TP4/TP8 low-latency recipes and all retained disaggregated Mooncake+hicache recipes pick up DSpark flags, FP4 indexer / MegaMoE gates, Slurm time limits, and related SGLang env tuning.configs/nvidia-master.yamlswitchesspec-decodingfrommtptodraft_model, raises synthetic acceptance length to 3.77, drops the c80 2P/1D point, retunes the disagg ladder to 480 / 960 / 1440 / 1920 (renamed recipe files), and shrinks the top point from 10P→8P prefill.disagg-gb300-2p1d-dep8-tp4-c80-mtp-kvoffload.yamlis deleted.Infra wiring:
runners.yamlexpandsgb300-nvfrom three to eighteen runners;launch_gb300-nv.shmaps the 0813 checkpoint on/scratch, bumps srt-slurm to v1.0.40, and documents nginx keepalive for AgentX.perf-changelog.yamldocuments the DSpark migration and concurrency retune.Reviewed by Cursor Bugbot for commit 5dd01f8. Bugbot is set up for automated code reviews on this repo. Configure here.