Revert "[AgentX] DeepSeek-V4 B300 SGLang update" (#2701) / 回滚 "[AgentX] DeepSeek-V4 B300 SGLang update"（#2701） - #2736
Revert "[AgentX] DeepSeek-V4 B300 SGLang update" (#2701) / 回滚 "[AgentX] DeepSeek-V4 B300 SGLang update"（#2701）#2736Oseltamivir wants to merge 2 commits into
Conversation
This reverts commit 26265c7.
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
中文：为 #2701 回滚条目填入规范的 pr-link。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline finding on the stale comment block, I checked whether restoring the DEP4 (tp:4, ep:4, dp-attn:true) search-space rows was safe given PR #2701's rationale that DEP4 "cannot serve this trace" — that constraint was tied to #2701's staging-image megamoe path, and this PR fully reverts the script back to the pre-#2701 flashinfer_mxfp4 path, so it doesn't carry over. Not a new bug.
Extended reasoning...
This PR is a full revert of PR #2701 (image, search-space, dram-utilization, and recipe script) because #2701 was validated only on an unmerged staging SGLang build. One inline finding was already confirmed: the new comment block in configs/nvidia-master.yaml describing the recipe's serving flags is stale/wrong on several points (mem-fraction, chunked-prefill-size, EAGLE params, max-running-requests) relative to the reverted script. I independently re-derived the script's actual defaults (dsv4_fp4_b300_sglang_mtp.sh lines 113-133, 149, 204-206) and confirmed the mismatches described in that finding. Separately, I checked the one other candidate concern noted from this run — that reintroducing the DEP4 search-space rows might conflict with #2701's own documented rationale that DEP4 could not serve this trace (perf-changelog.yaml line 6426) — and ruled it out: that rationale was specific to #2701's megamoe/staging-image serving path, and since this PR reverts the script back to the pre-#2701 flashinfer_mxfp4 configuration wholesale, the constraint that blocked DEP4 does not apply to the restored code path.
| # dp-attn: false -> TP-only + flashinfer_mxfp4 + chunked-prefill 8192 | ||
| # + mem-fraction 0.88 + swa-full-tokens-ratio 0.1 | ||
| # dp-attn: true -> DP-attn + megamoe + fp4 indexer | ||
| # + chunked-prefill 65536 + mem-fraction 0.90 | ||
| # + swa-full-tokens-ratio 0.075 | ||
| # + prefill-decode-interval 20 | ||
| # Both paths share EAGLE (3,1,4) and max-running-requests 2*CONC. | ||
| # + EAGLE (3,1,4) + mem-fraction 0.90 | ||
| # dp-attn: true -> DP-attn + flashinfer_mxfp4 + chunked-prefill 32768 | ||
| # + EAGLE (1,1,2) + mem-fraction 0.92 + max-running 256 |
There was a problem hiding this comment.
🟡 New comment block documenting benchmarks/single_node/agentic/dsv4_fp4_b300_sglang_mtp.sh no longer matches the script it describes. It claims dp-attn:false uses mem-fraction 0.90 (script default is 0.88), dp-attn:true uses chunked-prefill 32768 (script sets 16384), EAGLE (1,1,2) for dp-attn:true (script always uses --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4, unconditionally), mem-fraction 0.92 for dp-attn:true (script sets 0.95, or 0.94 when CONC>=512), and a fixed max-running-requests of 256 (script computes MAX_RUNNING_REQUESTS=2*CONC, not a fixed value).
Extended reasoning...
An engineer re-landing the reverted #35880 tuning (as perf-changelog.yaml instructs) or debugging this recipe reads this comment to understand current serving parameters, tunes against the wrong baseline values (e.g. assumes 32768 chunked-prefill or a fixed 256 max-running-requests), and either misconfigures the re-land or wastes time reconciling the mismatch with the actual script.
Verification: nit — documentation-only mismatch introduced by this diff; no runtime effect. The rewritten comment in configs/nvidia-master.yaml (dp-attn:false -> mem-fraction 0.90; dp-attn:true -> chunked-prefill 32768 + EAGLE (1,1,2) + mem-fraction 0.92 + max-running 256) contradicts the script benchmarks/single_node/agentic/dsv4_fp4_b300_sglang_mtp.sh: MEM_FRACTION_STATIC=0.88 default (dp-attn:false), `MEM_
1 participant
Reverts #2701 for
dsv4-fp4-b300-sglang-agentic-hicache-mtp.#2701 served every point on
lmsysorg/sglang-staging:dev-cu13-pr-35880, a staging image carrying the unmerged draft sgl-project/sglang#35880, so its results do not correspond to any released SGLang build. Its ingested run (32695861783) has already been purged from the dashboard in InferenceX-app#873.This restores the image to
lmsysorg/sglang:v0.5.17-cu130, the 42-point search space (TP4 and DEP4 rows included),dram-utilization0.80, and the pre-#2701 serving flags in the recipe script.perf-changelog.yamlis append-only, so the #2701 entry stays and a new entry records the revert. Re-land the tuning once #35880 merges upstream and ships in a released tag.中文说明
回滚 #2701 对
dsv4-fp4-b300-sglang-agentic-hicache-mtp的改动。#2701 的所有数据点均运行在
lmsysorg/sglang-staging:dev-cu13-pr-35880上，该 staging 镜像包含尚未合并的草稿 PR sgl-project/sglang#35880，因此结果不对应任何已发布的 SGLang 版本。其 ingest 的 run（32695861783）已在 InferenceX-app#873 中从仪表板清除。本 PR 将镜像恢复为
lmsysorg/sglang:v0.5.17-cu130，恢复 42 点搜索空间（含 TP4 与 DEP4 行）、dram-utilization0.80，以及 recipe 脚本中 #2701 之前的服务参数。perf-changelog.yaml为 append-only，因此保留 #2701 的条目，并新增一条记录本次回滚的条目。待 #35880 合并进上游并随正式版本发布后，再重新引入该调优。Note
Low Risk
Benchmark and recipe configuration only; no application runtime or auth paths change, though published AgentX numbers for this key will shift until tuning is re-landed on a released SGLang build.
Overview
Reverts #2701 for
dsv4-fp4-b300-sglang-agentic-hicache-mtpbecause that work ran on an unreleased staging image (dev-cu13-pr-35880); dashboard run32695861783is purged and tuning is deferred until upstream sglang#35880 ships in a release tag.Config (
nvidia-master.yaml): Container image back tolmsysorg/sglang:v0.5.17-cu130,dram-utilization0.80 (from 0.95), and the 42-point agentic search space restored—including TP4 and DEP4 rows with HiCache and router variants that #2701 had narrowed.Recipe (
dsv4_fp4_b300_sglang_mtp.sh): Serving flags return to the pre-#2701 stack: FlashInfer MXFP4 MoE instead of MegaMoE / FP4 indexer / prefill-delayer; simpler mem-fraction and chunked-prefill (e.g. DP path16384, ratio 2 for TP8 HiCache); CUDA graph batch size tied toCONC(cap 64) instead of a fixed decode graph of 544; router--disable-retriesinstead of an 8-retry backoff policy; MegaMoE-specific env exports removed.perf-changelog.yaml: Append-only entry documents the revert while keeping the #2701 changelog row.Reviewed by Cursor Bugbot for commit d6afb3a. Bugbot is set up for automated code reviews on this repo. Configure here.