Repository navigation
[NVIDIA][AgentX] Refresh B200 DeepSeek-V4.1-Flash SGLang aggregate curve / [NVIDIA][AgentX] 更新 B200 DeepSeek-V4.1-Flash SGLang 聚合曲线 - #4111
Conversation
Repin dsv41flash-fp4-b200-sglang-agentic-dspark to lmsysorg/sglang:nightly-dev-cu13-20261014-684fea8a (multiarch digest) with eleven points: TP4/EP1 C1/C4/C8, TP4/EP4 C16/C24, and TP4/EP4 with DP attention C48-C256 behind sglang-router with MegaMoE, TRT-LLM sparse attention, HiCache host-DRAM KV offload and shortest-prefill-first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the contribution!
中文感谢你的贡献！
|
shyeh25
added
the
full-sweep-fail-fast
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline finding, I also checked the drop in tokenizer-worker-num from 8 to 1 across all six new DP4 arms (agentic.yaml, e.g. line 191) — since it's applied uniformly to every new conc point rather than isolated to one, it reads as a deliberate setting for this image/topology rather than a leftover regression, so I'm not flagging it as a bug.
Extended reasoning...
This PR bumps the SGLang nightly pin and restructures the DSv4.1-Flash FP4 B200 AgentX benchmark curve across nvidia-master.yaml, the matching srt-slurm recipe, and a perf-changelog entry; it touches no auth/crypto/permission surface, only benchmark scheduling and recipe config. Both changed config/recipe files fall under the nvidia-master.yaml and b200-* srt-slurm-recipes CODEOWNERS entries, and one CONFIRMED comment-style nit was already surfaced for inline posting, so approval is not appropriate here regardless. I additionally verified the tokenizer-worker-num change flagged as a candidate concern is applied consistently across all new DP4 points, which is why it was ruled out rather than treated as a bug.
| # TP4 covers C1-C24 (EP1 at C1-C8, EP4 at C16/C24) with a GPU-resident KV cache. | ||
| # DP4 (TP4 x DP-attention 4 + EP4, MegaMoE) covers C48-C256 behind sglang-router | ||
| # and offloads KV to host DRAM through HiCache. |
There was a problem hiding this comment.
🟡 nit (optional): Added lines 8659-8661 describing the whole TP4/DP4 topology, and the matching block in agentic.yaml:51-56, are header/overview comments above a config key and above a set of recipe overrides, which AGENTS.md's "Code comments (mandatory)" section (no header comments; only technical explanation of a specific line) forbids. Fix: replace both with per-line technical notes (e.g. why hicache-ratio: 2 or prefill-decode-interval falls with concurrency) or drop them; the pre-existing comment block above this key already had this shape, but this diff extends it with more topology narration instead of trimming it. [also at: inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/sglang/b200-fp4-mtp/agentic.yaml:56 - Maintainers reading this recipe (and nvidia-master.yaml) again get a banned header/overview comment, which AGENTS.md 'Code comments (mandatory)' explicitly forbids and which the base branch did not have in this form.]
Why this was flagged
AGENTS.md:60-67 ("Code comments (mandatory)") states: no header comments describing the file/topology/config, only succinct technical explanation of a specific line. nvidia-master.yaml:8652-8661 and agentic.yaml:51-56 both describe the overall TP4/DP4 search-space shape and scheduling story, not a single line's rationale. This is a pre-existing pattern (the prior comment block already summarized the topology) that this diff extends rather than removes, so a maintainer following the rule literally would need to pare both blocks down before merge.
Verification: nit. Candidate is factually accurate at both cited sites. AGENTS.md:60-67 ("Code comments (mandatory)") forbids header/overview comments: "No header comments ... Do not describe what the file, topology, or config is" and "Only technical explanation. A comment explains a specific line or block." nvidia-master.yaml:8659-8661 and agentic.yaml:51-56 describe the overall topology and override set rather than per-line rationale, which the rule prohibits.
Remove the topology overview comments above the B200 SGLang master-config key and the recipe overrides, per the AGENTS.md code-comment rules. No configuration change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38749770474 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38749770474 |
|
/use 38749770474 |
|
@nvpohanh staged run 38749770474. This run remains available across future |
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX config ( The commands mirror the launch measured in run 38749770474, pinned by this PR's |
|
[by Claude Code] sgl-project/sglang#44654 has been merged. This PR's AgentX config ( The commands mirror the launch measured in run 38749770474, pinned by this PR's |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — ✅ Check 1 (Passing sweep on in-PR commit): PASS — Run 38749770474 (attempt 1, ✅ Check 2 (Evals pass): PASS — ✅ Check 3 (Upstream recipe): PASS — sgl-project/sglang#44654 is ✅ Check 4 (Reuse command): PASS — ✅ Check 5 (Latest checklist): PASS — the sign-off contains all 18 current template items (including the sub-item), all checked, with a filled additional detail section. ✅ Check 6 (Upstream image / ordering): PASS — ✅ Check 7 (Deprecated models): PASS — MODELS.md lists ✅ Check 8 (No architecture hacks): PASS — the recipe has no ✅ Check 9 (Chat template): PASS — the AIPerf export for the run ( ✅ Check 10 (No engine patches): PASS — the diff touches only the recipe YAML, the master row and the changelog. There are no patch files, heredoc rewrites, site-packages edits or wheel installs. ✅ Check 11 (Golden simulated AL): PASS — the C48 worker env logs ➖ Check 12 (Append-only): N/A — the single new changelog entry does not set ✅ Check 13 (Draft precision): PASS — the draft is the DSpark head bundled in ✅ Check 14 (Pareto coverage): PASS — one curve, ✅ Check 15 (PR description): PASS — the body's claims match the pinned code and run: image/digest, the 11 points across TP4/EP1, TP4/EP4 and DP4, MegaMoE, ✅ Check 16 (NVIDIA overview): PASS — the NVIDIA overview Framework Recipes table has a DeepSeek-V4.1-Flash row whose SGLang cell links to the V4_1 cookbook page. ➖ Check 17 (Dynamo metrics): N/A — this is a direct Assessed commit: |
…glang-agentx-1014 Resolve the inferencex-e2e/perf-changelog.yaml append conflict by keeping main's entries and re-appending this PR's entry. [skip-sweep] Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nvpohanh
removed
the
full-sweep-fail-fast
3 participants
Summary
Refresh the DeepSeek-V4.1-Flash FP4 B200 SGLang AgentX aggregate curve (
dsv41flash-fp4-b200-sglang-agentic-dspark):lmsysorg/sglang:nightly-dev-cu13-20261014-684fea8a, pinned by multiarch digest;kv-offloading: dramwith thehicachebackend), and route them through sglang-router 0.3.2 with cache-aware, correlation-ID-affine routing.The deployment remains single-node and aggregate. No cluster-route or runner configuration changes are included. DSpark remains the checkpoint-bundled draft at its shipped precision with block size 5.
AI model disclosure
claude-opus-5-5)PR checklist and change type
Type of Change
Checklist
inferencex-e2e/perf-changelog.yamland have not edited historical entriesfull-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
更新 DeepSeek-V4.1-Flash FP4 B200 SGLang AgentX 聚合曲线（
dsv41flash-fp4-b200-sglang-agentic-dspark）：lmsysorg/sglang:nightly-dev-cu13-20261014-684fea8a，并以 multiarch digest 固定；kv-offloading: dram、后端hicache），并通过 sglang-router 0.3.2 进行缓存感知及 correlation-ID 亲和路由。部署仍为单节点聚合式部署。本 PR 不包含集群路由或 runner 配置改动。DSpark 继续使用 checkpoint 自带的 draft 及其原始精度，block size 为 5。
AI 模型披露：本 PR 使用 Claude Opus 5.5（
claude-opus-5-5）进行 benchmark 调优与分析、recipe/矩阵整合、验证与 PR 准备。🤖 Generated with Claude Code