perf(agentx): add Qwen3.5 GB300 SGLang Pareto curve / 性能：新增 Qwen3.5 GB300 SGLang Pareto 曲线 - #2812
Conversation
Add two PP4 disaggregated and three aggregate AgentX deployments with matching srt-slurm recipes. 中文：新增两个 PP4 分离式和三个聚合式 AgentX 部署，并添加匹配的 srt-slurm 配置。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Backfill the public pull request URL in the appended performance changelog entry. 中文：在追加的性能变更日志条目中回填公开拉取请求链接。
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33814978888 |
Update the PP master configuration and both matching srt-slurm recipes to the current ai-dynamo main commit. 中文：将 PP 主配置及两个匹配的 srt-slurm 配方更新到当前 ai-dynamo main 提交。
Preserve the current main performance changelog byte-for-byte and re-append the PR 2812 entry with the refreshed ai-dynamo commit. 中文：逐字节保留当前 main 的性能变更日志，并在末尾重新追加包含最新 ai-dynamo 提交的 PR 2812 条目。
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
There are 4 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1a33e29. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34271862204 |
同步最新 main 并保留性能变更日志历史；按推测步骤修正五个配置的 Golden AL， 将双节点 TP8 配方设为多节点基准模式，保持镜像和 Dynamo 固定版本不变。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34345050756 |
|
/reuse-sweep-run 34345050756 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
|
/stage-results 34345050756 |
|
@cquil11 staged run 34345050756: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-09-09~r34345050756 This run remains available across future |
Resolve conflicts in configs/nvidia-master.yaml and perf-changelog.yaml. - nvidia-master: keep this branch's rewrite of minimaxm3-fp4-gb200-dynamo-vllm-agentic-agg-mtp (nightly image, TP4 resident / TP4 lazy SimpleCPU offload / two-node TP8 resident, the -nightly-native recipe files), which supersedes main's PR #2807 tuning of the same entry. Main's disagg-mtp rewrite and the new Qwen3.5 GB300 Pareto entries are retained unchanged. - perf-changelog: keep both sides; main's #2812/#2807/#2623 entries stay in place and this branch's #2925 entry is appended at the end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4 participants
Description
lmsysorg/sglang:nightly-dev-cu13-20260831-bb5e6198image.中文说明
lmsysorg/sglang:nightly-dev-cu13-20260831-bb5e6198镜像，为 GB300 新增五个 Qwen3.5 NVFP4 AgentX Pareto 点。Runtime details / 运行时详情
Corrected thinking-on Golden AL: C704 uses 4.41 at 5 speculative steps; C565 uses 3.97 at 4 steps; both TP2/C44 recipes and TP8/C7 use 4.8 at 6 steps. SGLang draft-token counts include the bonus token and are not the Golden curve index.
The two-node TP8/C7 recipe sets
IS_MULTINODE=trueto avoid selecting local-only power monitoring. This flag correction does not establish complete cross-node power collection.All five points use MTP and HiCache. Throughput jobs inject the matching committed Qwen3.5 golden acceptance lengths;
EVAL_ONLYremoves simulated acceptance and retains real MTP verification.The PP recipes pin Dynamo router/source identity to ai-dynamo/dynamo
maincommit61c37bc33c3e88da3facaa39fa2b5caf616740ba, selected during the September 8 refresh and unchanged by the acceptance-length correction.The SGLang image includes the support merged in sgl-project/sglang#36248. The recipes run the SGLang image as shipped; no engine source is mounted or patched.
已修正 thinking-on Golden AL：C704 的 5 个推测步骤对应 4.41；C565 的 4 个步骤对应 3.97；两个 TP2/C44 配方和 TP8/C7 的 6 个步骤对应 4.8。SGLang draft-token 数量包含额外 token，不能用作 Golden 曲线索引。
双节点 TP8/C7 配方设置
IS_MULTINODE=true，避免选择仅本地功率监控；此标志修正本身不代表完整跨节点功率采集已得到验证。五个点均使用 MTP 和 HiCache。吞吐量任务注入匹配的已提交 Qwen3.5 Golden AL；
EVAL_ONLY会移除模拟接受率并保留真实 MTP 验证。PP 配方将 Dynamo 路由器和源码标识固定到 ai-dynamo/dynamo
main提交61c37bc33c3e88da3facaa39fa2b5caf616740ba，该版本在 9 月 8 日更新时选定，本次接受长度修正不改变此固定版本。SGLang 镜像包含已在 sgl-project/sglang#36248 合并的支持。配方直接运行未修改的 SGLang 镜像，不挂载或修补引擎源码。
Recipe applicability / 配方适用性
No separate SGLang cookbook PR is required: every added deployment uses the repository's multi-node srt-slurm path, including the one-node aggregate shapes, and the matching recipes are checked into this PR.
无需单独创建 SGLang cookbook PR：新增部署全部使用仓库的多节点 srt-slurm 路径（包括单节点聚合形态），匹配的配方已包含在本 PR 中。
Validation / 验证
Parsed the master config, runner config, changelog, and all five recipe YAML files.
Generated both exact config keys and the affected Qwen3.5 FP4 GB300 Dynamo-SGLang AgentX family; the selected changelog expands to five throughput jobs and four eval-only jobs with positive node counts
5/7/1/1/2.Verified master-to-recipe parity for image, worker counts, TP/PP/EP, data-parallel attention, and node allocation.
Ran
srtctlv1.0.38 preflight and dry-run successfully for all five recipes using a local model-directory fixture. These checks validate configuration/rendering, not GPU execution or cluster model availability.Checked all seven speculative worker roles against the committed Golden AL table, all five topology/power flags, and throughput injection followed by exact eval restoration.
Ran the synthetic-acceptance injection/removal check for all five recipes and
runners/test_slurm_utils.py(21 passed).Validated the committed changelog against
origin/main, preserved all historical changelog bytes, and passedgit diff --check.已解析主配置、runner 配置、变更日志及全部五个配方 YAML 文件。
已生成两个精确配置键及受影响的 Qwen3.5 FP4 GB300 Dynamo-SGLang AgentX 配置族；变更日志选择结果包含五个吞吐量任务和四个仅评测任务，节点数
5/7/1/1/2均为正数。已验证主配置与配方在镜像、worker 数、TP/PP/EP、数据并行注意力和节点分配方面保持一致。
全部五个配方均使用本地模型目录占位通过
srtctlv1.0.38 preflight 和 dry-run；这些检查验证配置和渲染，不验证 GPU 执行或集群模型可用性。已对照已提交 Golden AL 表检查全部七个推测 worker 角色、全部五个拓扑/功率标志，并验证吞吐量注入后 eval 能精确恢复原配方。
已对全部五个配方验证模拟接受率的注入与移除，并运行
runners/test_slurm_utils.py（21 项通过）。已基于
origin/main验证已提交的变更日志，保留全部历史字节，并通过git diff --check。Related Issue
No separate issue; this update corrects the acceptance targets and TP8 node mode in this PR.
无单独 issue；本次更新修正本 PR 的接受目标和 TP8 节点模式。
Type of Change
配置变更和问题修正；验证范围见上文。
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.Local checks cover configuration and rendering only; GPU sweep/eval results are pending. No reuse authorization is requested here.
本地检查仅涵盖配置和渲染；GPU sweep/评测结果仍待完成。此处不请求复用授权。
Follow-up / 后续
A fresh full sweep and evals are required on the corrected head. The existing
full-sweep-enabledlabel is retained; earlier runs used the superseded acceptance targets and do not validate these corrections. Consult the current PR checks for the new run.修正后的提交需要重新运行 full sweep 和评测。保留现有
full-sweep-enabled标签；旧运行使用已被替换的接受目标，不能验证本次修正。新运行请查看本 PR 的当前检查。Note
Low Risk
Benchmark and Slurm recipe configuration only; no application runtime or auth/data-path changes.
Overview
Adds five GB300 Qwen3.5 NVFP4 AgentX Pareto points on
lmsysorg/sglang:nightly-dev-cu13-20260831-bb5e6198, with matching checked-in srt-slurm recipes and master-config search entries.Disaggregated (Dynamo + Mooncake): two PP4-prefill / DEP4-decode shapes at concurrency 704 (3P+2D) and 565 (3P+4D), with router pinned to Dynamo commit
61c37bc. Throughput jobs use synthetic acceptance lengths 4.41 and 3.97 aligned to speculative step counts.Aggregated: two distinct TP2 @ concurrency 44 fingerprints (batch-1 vs batch-2 / HiCache 32 vs 128) and one two-node TP8 @ concurrency 7 recipe (
IS_MULTINODE=truefor benchmark power mode). All three use synthetic acceptance 4.8 at six speculative steps. Every point enables MTP + HiCache and runsagentic_srt.sh.perf-changelog.yamlrecords the new keysqwen3.5-fp4-gb300-dynamo-sglang-agentic-pp-paretoandqwen3.5-fp4-gb300-dynamo-sglang-agentic-agg-pareto.Reviewed by Cursor Bugbot for commit 1b6b207. Bugbot is set up for automated code reviews on this repo. Configure here.