Repository navigation
feat(config): add B200 vLLM AgentX DSpark configuration / 新增 B200 vLLM AgentX DSpark 配置 - #2946
RohitNagraj wants to merge 4 commits into
Conversation
新增 B200 vLLM AgentX DSpark 配置。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
2 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
在性能变更日志中补充 PR #2946 链接。
RohitNagraj
added
the
full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. Because it adds a large new benchmark script with nontrivial speculative-decoding and CUDA-graph-sizing logic, plus a routing change in the shared B200 NScale launcher, a human look would still be worthwhile.
What was reviewed:
- The new
dsv4_fp4_b200_vllm_dspark.shscript's KV-offload branches, DP-attention/DEP sizing, and the synthetic-AL-vs-real-DSparkSPEC_CONFIGsplit. perf-changelog.yaml— confirmed the new entry is appended only at the physical tail with no historical bytes changed, and thatgolden_al_distribution/dsv4-pro-0813-dspark.yamlexists backing the pinned AL 3.77.- The launcher change in
runners/launch_b200-nscale-compat.shis scoped tospec-decoding: draft_model, so the existing native-MTP (spec-decoding: mtp) route is untouched.
Extended reasoning...
Overview
This PR is a config/benchmark-infra addition: a new 349-line single-node launch script for a DeepSeek-V4-Pro-0813 FP4 agentic-coding benchmark on B200 with vLLM + native DSpark speculative decoding, a new top-level key in configs/nvidia-master.yaml, one append-only entry in perf-changelog.yaml, and a small conditional in runners/launch_b200-nscale-compat.sh to route draft_model spec-decoding jobs to the new _dspark script when it exists.
Security risks
None identified. This is benchmark-launcher shell/YAML for an internal performance-testing pipeline; no auth, crypto, or externally-reachable surface is touched. The script only shells out to vllm serve, mooncake_master, and hf download with locally-controlled arguments.
Level of scrutiny
Medium. The change is not a one-line config tweak — it introduces a sizable new bash script with speculative-decoding branch logic (synthetic vs. real DSpark acceptance), CUDA-graph capture-size arithmetic tied to num_speculative_tokens, and two KV-offload backend code paths (vllm-simple, mooncake), plus a shared-launcher routing conditional whose correctness affects an existing script (the native-MTP recipe) if scoped incorrectly. I traced the routing conditional and confirmed it only fires for SPEC_DECODING == draft_model and only when the _dspark script is present, so the existing mtp route is unaffected. I also confirmed the changelog append is at the physical tail and the referenced golden-AL file exists. No correctness issues surfaced, but the script's capture-size math, offload byte-budget arithmetic, and the router-suffix precedence in the shared launcher are exactly the kind of logic where a subtle off-by-one or scoping mistake would only show up at runtime on real hardware, which the automated review cannot exercise.
Other factors
The PR description itself states a blocking dependency ("A matching official vLLM recipe must be merged before this PR can merge") and the changelog entry still carries the placeholder pr-link: .../pull/XXX, both of which are process items for a human/maintainer to track before merge rather than code bugs. No CODEOWNERS restriction or third-party CHANGES_REQUESTED review was evident in the provided timeline. Given the complexity of the new script and the shared-launcher edit, and that this is a first-pass automated review, a human look is warranted before merge even though no concrete bug was found.
This review covers commit 13f8cdc, which is no longer the latest commit on this pull request; later commits are not covered by it.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34419226666 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34431969750 |
合并当前 main，并将 PR #2946 的性能变更日志条目重新追加到文件末尾。
|
Required vLLM recipe dependency opened: vllm-project/recipes#951 已创建所需的 vLLM recipe 依赖 PR：vllm-project/recipes#951 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34517767665 |
|
InferenceX has switched away from unmaintainable bash scripts to YAML files that don't repeat the same stuff over and over again. Please merge the latest |
|
Sorry, over the weekend, there was 2 major refactors to clean up the technical debt accumalated over the past 11 months of moving at the speed of light. We don't see any major refactors in the forthseeable future besides cleaning up AMD multinode AgentX pile of bash. As much, due to the refactors, u would need to ask your agent to rebase from remote main@latest. Thank you in advance for ur understanding |
|
Heads-up: #3576 (merged) replaced the bash launchers with a Python launcher, so this PR will conflict when you merge |
|
Closing this as superseded by #3274, which already runs the same B200 DeepSeek-V4-Pro-0813 vLLM AgentX DSpark6 configuration on main (newer image, TP8 plus DEP8). This is not a quality judgement on the work, and thanks for it, Rohit. I compared the two sweeps at TP8, concurrency 1/4/14 (and c8 against #3274's c6/c10): run 34517767665 here vs run 35434571418 for #3274.
The results are mixed and within run-to-run noise, #3274 has more points (TP8 c1-c16 and DEP8 up to c192), and the runs differ in image, attention backend and KV offload, so they do not isolate FlashMLA vs FlashInfer. A dedicated A/B can be opened later if we want that comparison. Please reopen if you have evidence that this adds a distinct curve. 中文由于 #3274 已在 main 上以更新的镜像（TP8 加 DEP8）运行相同的 B200 DeepSeek-V4-Pro-0813 vLLM AgentX DSpark6 配置，此 PR 作为重复项关闭，并非对工作的否定，感谢 Rohit。 对比了两次 sweep（本 PR 的 run 34517767665 与 #3274 的 run 35434571418）在 TP8 并发 1/4/14 上的结果（c8 对照 #3274 的 c6/c10）：结果互有高低，处于运行间噪声范围；#3274 的数据点更多（TP8 c1-c16，DEP8 至 c192）；两次运行的镜像、注意力后端与 KV offload 均不同，因此无法单独得出 FlashMLA 与 FlashInfer 的对比结论。如需该对比，可另开专门的 A/B。若有证据表明此 PR 提供了不同的曲线，请重新打开。 |
Description
Adds
dsv4-fp4-b200-vllm-agentic-dspark, a B200 vLLM AgentX configuration fordeepseek-ai/DeepSeek-V4-Pro-0813using its native DSpark draft heads.Throughput runs use the committed thinking-on golden synthetic acceptance target for six speculative tokens. Evaluation runs remove synthetic acceptance and use real DSpark verification. The serving path selects the FlashMLA sparse DeepSeek-V4 attention backend and CPU DRAM key-value cache offloading.
The B200 NScale launcher now selects the DSpark-specific benchmark script for this draft-model configuration while preserving the existing native-MTP route.
Local validation passed Bash and YAML syntax checks, exact-key matrix generation, evaluation matrix generation, and all 297 matrix tests.
A matching official vLLM recipe must be merged before this PR can merge. The currently published DeepSeek-V4-Pro recipe does not yet match this configuration’s major DSpark arguments.
中文说明
新增
dsv4-fp4-b200-vllm-agentic-dspark：面向 B200 的 vLLM AgentX 配置，使用deepseek-ai/DeepSeek-V4-Pro-0813检查点及其原生 DSpark 草稿头。吞吐量运行使用已提交的、对应六个推测 token 的 thinking-on 黄金合成接受目标；评测运行会移除合成接受设置，并使用真实 DSpark 验证。服务路径选择 FlashMLA 稀疏 DeepSeek-V4 注意力后端，并启用 CPU DRAM 键值缓存卸载。
B200 NScale 启动脚本现在会为该 draft-model 配置选择 DSpark 专用基准脚本，同时保留现有原生 MTP 路径。
本地验证已通过 Bash 与 YAML 语法检查、精确配置键矩阵生成、评测矩阵生成，以及全部 297 项矩阵测试。
本 PR 合并前，必须先合并与该配置匹配的官方 vLLM recipe。当前已发布的 DeepSeek-V4-Pro recipe 尚未匹配本配置的主要 DSpark 参数。
Related Issue
N/A / 不适用
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.Note
Low Risk
Benchmark and matrix configuration only; no changes to production serving paths or shared runtime libraries.
Overview
Adds DeepSeek-V4-Pro-0813 FP4 on B200 agentic benchmarking with native DSpark speculative decoding, separate from the existing MTP agentic path.
A new benchmark script drives vLLM with FlashMLA sparse DSv4 attention (not FlashInfer MLA), DSpark (
num_speculative_tokens=6), token-based CUDA graph capture, and the same agentic/KV-offload patterns as sibling recipes. Throughput runs pin synthetic acceptance length 3.77;EVAL_ONLYuses real DSpark verification.dsv4-fp4-b200-vllm-agentic-dsparkis registered in the master config (TP8, SimpleCPU DRAM offload,spec-decoding: draft_model, agentic-coding search space). The B200 NScale launcher now picks_*_dspark.shwhendraft_modelis set and that script exists, instead of reusing the MTP benchmark script.perf-changelog.yamldocuments the new config key.Reviewed by Cursor Bugbot for commit 9df882f. Bugbot is set up for automated code reviews on this repo. Configure here.