Refresh GB200 Qwen3.5 AgentX Pareto / 刷新 GB200 Qwen3.5 AgentX Pareto 配置 - #2660
Conversation
更新 GB200 Qwen3.5 AgentX 配置，加入 TP2/EP2 HiCache 分支并刷新 Pareto 配置点。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
在性能变更日志中补充 PR #2660 链接。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
|
||
| - config-keys: | ||
| - qwen3.5-fp4-gb200-dynamo-sglang-agentic-mtp | ||
| scenario-type: | ||
| - agentic-coding | ||
| description: | ||
| - "Refresh GB200 Qwen3.5 NVFP4-V2 AgentX on lmsysorg/sglang:nightly-dev-20260818-c0b6474b without engine patches." | ||
| - "Add a TP2/EP2 HiCache branch and update the resident recipes with extra_buffer_lazy, ReplaySSM speculative verification, and checkpoint page-cache release." | ||
| - "Select TP4 C1/C2/C4/C8/C12/C20/C24, TP2/EP2 resident C4/C8/C12/C20, and TP2/EP2 HiCache C22/C24/C28/C32/C40 with golden synthetic acceptance length 3.39." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX |
There was a problem hiding this comment.
🟡 The new changelog entry's pr-link uses the placeholder pull/XXX instead of the real PR number, even though this PR already exists as #2660. Per docs/configuration-procedures.md:250-254, the only sanctioned pre-PR placeholder is pr-link: TBD, to be replaced with the real URL immediately after PR creation; please change this to pull/2660 so the entry is traceable back to its PR.
Extended reasoning...
Bug: The changelog entry appended at the end of perf-changelog.yaml (lines 6089-6098) sets:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXThis is PR #2660 (per the PR metadata), so the link should read pull/2660. Every other entry among the 100+ historical entries in this file uses a concrete pull/<number> link.
Documented convention: docs/configuration-procedures.md:250-254 explicitly documents the workflow for this field: the only sanctioned placeholder to use before the PR exists is the literal string pr-link: TBD, and the author is instructed to replace it with the real URL immediately after creating the PR. XXX is neither the real number nor the sanctioned TBD placeholder — it's an ad hoc stand-in that falls outside the documented convention.
Why it's not caught elsewhere: The changelog-key generation and CI ingest logic only read config-keys and scenario-type from each entry (used to build lookup keys for benchmark results); pr-link is purely a human-facing traceability field and isn't validated or parsed by any downstream tooling. That's why this passes YAML parsing, git diff --check, and CI without complaint — nothing enforces the field's format at build time. One earlier historical entry (line 5598) also uses pull/XXX, so this isn't the first occurrence, but a single unfixed prior instance doesn't make it a sanctioned convention; it's more likely the same slip happening twice.
Impact: Low — this doesn't affect benchmark behavior, CI, or sweep execution. It's a traceability/documentation issue: anyone auditing perf-changelog.yaml later (e.g. to find which PR introduced a given config change) will hit a dead link for this entry instead of being routed to PR #2660.
Proof / walkthrough:
- Open
perf-changelog.yaml, jump to the last entry (lines 6089-6098, added by this PR). - Read
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX—XXXis not a valid PR number. - Check the actual PR: per this PR's metadata, it is PR Refresh GB200 Qwen3.5 AgentX Pareto / 刷新 GB200 Qwen3.5 AgentX Pareto 配置 #2660.
- Check
docs/configuration-procedures.md:250-254: the documented process says usepr-link: TBDbefore the PR exists, then swap in the real URL right after creating it — never a placeholder string likeXXX. - Fix: replace the line with
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2660.
Fix: Change line 6098 to pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2660.
| disagg: false | ||
| scenarios: | ||
| agentic-coding: | ||
| - dram-utilization: 0.70 | ||
| - dram-utilization: 0.90 | ||
| search-space: | ||
| # Measured Pareto points from the complete fast sweep. TP4 covers the | ||
| # low-latency branch; HiCache extends the high-throughput branch. | ||
| # TP4 covers the low-latency branch. TP4 HiCache is omitted because the | ||
| # half-GPU TP2/EP2 branch already dominates that operating range. | ||
| - spec-decoding: mtp | ||
| kv-offloading: none | ||
| conc-list: [1, 4, 8, 12, 16, 20, 32, 40] | ||
| conc-list: [1, 2, 4, 8, 12, 20, 24] | ||
| prefill: | ||
| num-worker: 1 | ||
| tp: 4 |
There was a problem hiding this comment.
🟡 This PR removes the last reference to agg-gb200-tp4-mtp-hicache.yaml from configs/nvidia-master.yaml (the TP4 HiCache search-space branch is replaced by the new TP2/EP2 resident + HiCache branches), leaving that recipe file orphaned and unable to pick up the new nightly image/flags its siblings received. Since unreferenced recipe files are common in this directory already, this is a minor nit — consider deleting it (or noting it as intentionally parked) as part of this cleanup.
Extended reasoning...
What changed: configs/nvidia-master.yaml previously wired one of the agentic-coding search-space branches under qwen3.5-fp4-gb200-dynamo-sglang-agentic-mtp to CONFIG_FILE=recipes/sglang/qwen3.5/gb200-fp4/agentic/agg-gb200-tp4-mtp-hicache.yaml. This PR removes that branch entirely and replaces it with two new branches — a TP2/EP2 resident arm and a TP2/EP2 HiCache arm — pointing at agg-gb200-tp2ep2-mtp.yaml and the newly-added agg-gb200-tp2ep2-mtp-hicache.yaml.
The orphan: benchmarks/multi_node/srt-slurm-recipes/sglang/qwen3.5/gb200-fp4/agentic/agg-gb200-tp4-mtp-hicache.yaml (added in PR #2568) is not touched by this PR's diff at all. A repo-wide grep for agg-gb200-tp4-mtp-hicache after this diff applies returns zero matches anywhere in configs/ or the recipe tree — the file simply stops being reachable from the master config. It still carries the old lmsysorg/sglang:v0.5.17-cu130 image and the pre-refresh flag set (no extra_buffer_lazy, no ReplaySSM, no checkpoint page-cache release) that its siblings (agg-gb200-tp4-mtp.yaml, agg-gb200-tp2ep2-mtp.yaml) just received in this same PR.
Why this isn't caught elsewhere: there's no CI check that cross-references recipe files against master-config CONFIG_FILE= references, so nothing flags an orphaned recipe automatically — it just sits in-tree, increasingly stale relative to its siblings, until someone notices.
Counter-consideration (from a reviewer who refuted this as a blocking issue): a repo-wide check shows roughly 191 of 423 recipe YAMLs under srt-slurm-recipes/ are already unreferenced by any master-config CONFIG_FILE=, so parking an inactive recipe in-tree is the established norm for this directory (it functions as a reusable recipe library, not a strict 1:1 mirror of the active sweep). That's a fair point, and it's why this is filed as a nit rather than a blocker — leaving the file is consistent with existing practice, and a future TP4-HiCache revival could reuse it as a starting point. The residual concern is narrower: this file was actively wired moments before this PR and became orphaned by this PR's own change (not pre-existing drift), and it now silently diverges from the nightly image/flag refresh its direct siblings received in the same diff, which could mislead a future reader into assuming the TP4-HiCache path is still validated against current settings.
Proof of orphaning:
- Before:
configs/nvidia-master.yamlline ~7266 (pre-diff) containsCONFIG_FILE=recipes/sglang/qwen3.5/gb200-fp4/agentic/agg-gb200-tp4-mtp-hicache.yamlunder thekv-offloading: dram/kv-offload-backend: hicachebranch withtp: 4. - After: that entire branch is deleted; the diff hunk at
configs/nvidia-master.yaml:7232-7244shows it replaced by atp: 2, ep: 2, kv-offloading: nonebranch and atp: 2, ep: 2, kv-offloading: dram/hicachebranch, both pointing atagg-gb200-tp2ep2-mtp*.yaml. grep -r "agg-gb200-tp4-mtp-hicache" .post-diff → no hits outside the file's own path.- The file on disk still declares
container: { image: lmsysorg/sglang:v0.5.17-cu130 }while its siblings now readlmsysorg/sglang:nightly-dev-20260818-c0b6474b— confirming the drift.
Suggested fix: delete benchmarks/multi_node/srt-slurm-recipes/sglang/qwen3.5/gb200-fp4/agentic/agg-gb200-tp4-mtp-hicache.yaml as part of this cleanup, or leave a one-line note if it's intentionally kept for a future TP4-HiCache revival.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32177976542 |
3 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32177976542 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32177976542 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32177976542 |
|
/reuse-sweep-run 32177976542 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
|
/reuse-sweep-run |
# Conflicts: # perf-changelog.yaml
3 participants
Description
Refresh the GB200 Qwen3.5 NVFP4-V2 AgentX configuration on
lmsysorg/sglang:nightly-dev-20260818-c0b6474b.extra_buffer_lazy, ReplaySSM speculative verification, and checkpoint page-cache release.EVAL_ONLYruns.cluster:gb200-nvlauncher and routing unchanged.Local validation completed:
git diff --checkpassed.Recipe requirement: N/A — this is a multi-node
dynamo-sglangsubmission with in-repository srt-slurm recipes.中文说明
刷新 GB200 Qwen3.5 NVFP4-V2 AgentX 配置，使用
lmsysorg/sglang:nightly-dev-20260818-c0b6474b镜像。extra_buffer_lazy、ReplaySSM 推测验证和检查点页缓存释放。EVAL_ONLY运行中禁用该设置。cluster:gb200-nv启动器与路由不变。已完成本地验证：
git diff --check检查。配方要求：不适用——这是使用仓库内 srt-slurm 配方的多节点
dynamo-sglang提交。Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.