Add GB300 DeepSeek V4.1 Flash AgentX / 新增 GB300 配方 - #2961
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline pr-link finding, I checked two other candidates and ruled them out: select_available_server_port() in benchmarks/benchmark_lib.sh has a check-then-bind race against vllm serve's own bind, but the window is tiny and the failure mode is just a benchmark restart, not a security issue. The new recipe's dram-utilization: 0.80 in configs/nvidia-master.yaml has no effect at all here — agentic_dram_offload_gb() only applies when kv-offloading: dram, and this recipe sets kv-offloading: none — so the comment about bounding Engram's UVA DRAM offload is misleading but functionally inert.
Extended reasoning...
A confirmed finding (perf-changelog.yaml using pr-link: TBD, which validate_added_pr_link() rejects) is already being posted as an inline comment, so the top-level review just needs to record anything else examined. I independently verified two candidates from the ruled-out list by reading the code rather than restating them uncritically: (1) select_available_server_port() in benchmarks/benchmark_lib.sh does probe-then-close-then-let-vllm-bind, which is a textbook TOCTOU, but for a benchmark launcher in a CI/cluster context the exploitation surface is nil and the worst case is a failed benchmark run needing a retry — not worth blocking on. (2) I traced dram-utilization through infx/matrix/generate.py's agentic_dram_offload_gb() and confirmed it early-returns 0 unless kv-offloading == "dram"; the new dsv41flash-fp4-gb300-vllm-agentic-dspark recipe sets kv-offloading: none, so the field is fully dead code for this recipe despite the comment implying it governs Engram's UVA DRAM offload — misleading but harmless. Since the actionable bug is already covered by the inline comment and my own checks only turned up benign items, a short informational note is appropriate rather than a full approve/defer verdict.
| description: | ||
| - "Add GB300 TP4 AgentX through concurrency 128 with Engram UVA DRAM offload and native five-token DSpark" | ||
| - "Use deepseekv41-flash-0909 with adaptive verification and available-port selection" | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The new entry uses pr-link: TBD, which is not a value validate_added_pr_link() in utils/validate_perf_changelog.py accepts (only "XXX", the full .../pull/XXX placeholder, or the exact .../pull/<PR_NUMBER> URL). compare_entries(), called from utils/prepare_perf_changelog_merge.py's canonicalize_appended_links/resolve_conflict_bytes (the reuse-assisted merge automation), will raise ChangelogValidationError on this entry and block the automated merge for this very PR. Fix: replace TBD with XXX (the accepted placeholder) or the real PR URL before merge; the fix must ensure every new changelog entry's pr-link is one of the tool's accepted placeholder/URL forms.
Extended reasoning...
validate_added_pr_link(link, pr_number) in utils/validate_perf_changelog.py checks link in PR_LINK_PLACEHOLDERS (={'XXX', '.../pull/XXX'}) or link == expected (.../pull/<pr_number>) — 'TBD' matches neither branch (pr_number set or None), so it raises ChangelogValidationError(f"new PR entry must use {expected!r} or an XXX placeholder; found {link!r}"). This call happens inside compare_entries, invoked by prepare_perf_changelog_merge.py::canonicalize_appended_links (line ~92) before it even reaches its own placeholder substitution logic, so the merge-prep tool aborts instead of canonicalizing the link. No other entry in perf-changelog.yaml uses 'TBD' (grep confirms only this one), so this is a fresh, unprecedented violation of the pr-link contract introduced by this diff.
Verification: normal. The added entry perf-changelog.yaml:7079 uses pr-link: TBD. The reuse-assisted merge automation calls this into failure: merge_with_reuse.sh:18 sets set -euo pipefail and line 187 runs prepare_perf_changelog_merge.py canonicalize --pr-number "$PR" unguarded. canonicalize_appended_links (prepare_perf_changelog_merge.py:92) calls compare_entries(base, head, pr_number), which at…
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34461499451 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34463351592 |
为 GB300 单独配置 DeepSeek V4.1 Flash AgentX 扫描，启用 UVA DRAM offload 和 DSpark，并将 concurrency 扩展至 128。
补充 GB300 扫描的 PR 链接。
对齐官方单节点 TP 配方的模型 runner 和调度批处理默认值，保留 UVA offload、DSpark 及 AgentX 并发扫描。
DSv4.1 Flash 吞吐测试使用 thinking 开启、五个草稿 token 对应的黄金合成 AL 3.51，关闭自适应验证；准确率评测保留真实验证。同步中英文文档并追加性能变更记录。
bd916c4 to
9750276
Compare
同步最新 main，保留性能变更日志历史字节，并在末尾保留本 PR 的变更记录。
GB300 冷启动的权重加载和 CUDA graph capture 超过一小时，导致 Rust frontend 提前退出。将该 launcher 的引擎就绪期限设为两小时，共享脚本保留传入值；合成 AL、eval 验证和测试时长保持不变。
同步最新 main，保留历史性能变更记录及本 PR 的追加条目。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34504969146 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34512984806 |
Registers DeepSeek-V4.1-Flash as its own model bucket, sourced from the GB300 AgentX sweep in SemiAnalysisAI/InferenceX#2961 (run 34463351592): DB key `dsv41flash`, HF path `deepseek-ai/DeepSeek-V4.1-Flash`, FP4, vLLM, TP4, MTP. It is a Causal Encoder-Decoder / CSA2 architecture, not a V4-Pro point release, so it gets its own DB bucket and dropdown entry rather than folding into `dsv4`. The label carries the 552B backbone total; the 196B Engram conditional-memory table is sparsely accessed via token lookup and is excluded, matching how separate MTP heads are excluded elsewhere. Release date 2026-09-10 comes from the Hugging Face repo's initial commit (02:17Z) — the `deepseekv41-flash-0909` container tag is the image build date, not the weights date. The OpenRouter id is verified against the live `/api/v1/models` catalog. Curated as AgentX-only in the overview matrix: only agentic traces exist today, so the matrix must not claim a fixed-sequence row that will never fill. The compare-slug entry is structurally required — INFERENCE_MODEL_SLUGS, MODEL_ROUTES, /rankings and /run all derive from it and their tests demand coverage for every non-hidden model. No new GPU, precision, framework, or sequence is needed; all are already registered. MODEL_ARCHITECTURES is left for a follow-up (no diagram renders until added), as is the MTP engine exclusion — the guard only becomes meaningful once a second engine family sweeps the same SKU. 中文：新增 DeepSeek-V4.1-Flash 模型，数据来自 InferenceX#2961 的 GB300 AgentX sweep（run 34463351592）：DB key 为 `dsv41flash`，HF 路径 `deepseek-ai/DeepSeek-V4.1-Flash`，FP4 + vLLM + TP4 + MTP。要点： - 它采用 Causal Encoder-Decoder / CSA2 架构，并非 V4-Pro 的小版本更新， 因此单独建桶，不并入 `dsv4`。 - 标签中的参数量取 552B backbone；196B 的 Engram 条件记忆表按 token 稀疏查表 访问，不计入总量，与此前排除独立 MTP head 的处理一致。 - 发布日期 2026-09-10 依据 Hugging Face 仓库首次提交（02:17Z）；容器 tag 中的 `0909` 是镜像构建日期，而非权重发布日期。OpenRouter id 已对照线上目录核实。 - 在 overview 矩阵中标记为仅 AgentX：目前只有 agentic traces 数据，不应保留 永远不会填充的定长序列行。 - compare-slug 条目是结构性必需项：INFERENCE_MODEL_SLUGS、MODEL_ROUTES、 /rankings 与 /run 均由其派生，相关测试要求覆盖每个非 hidden 模型。 - 无需新增 GPU、精度、框架或序列长度。MODEL_ARCHITECTURES 与 MTP engine exclusion 留待后续处理。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Registers DeepSeek-V4.1-Flash as its own model bucket, sourced from the GB300 AgentX sweep in SemiAnalysisAI/InferenceX#2961 (run 34463351592): DB key `dsv41flash`, HF path `deepseek-ai/DeepSeek-V4.1-Flash`, FP4, vLLM, TP4, MTP. It is a Causal Encoder-Decoder / CSA2 architecture, not a V4-Pro point release, so it gets its own DB bucket and dropdown entry rather than folding into `dsv4`. The label carries the 552B backbone total; the 196B Engram conditional-memory table is sparsely accessed via token lookup and is excluded, matching how separate MTP heads are excluded elsewhere. Release date 2026-09-10 comes from the Hugging Face repo's initial commit (02:17Z) — the `deepseekv41-flash-0909` container tag is the image build date, not the weights date. The OpenRouter id is verified against the live `/api/v1/models` catalog. Curated as AgentX-only in the overview matrix: only agentic traces exist today, so the matrix must not claim a fixed-sequence row that will never fill. The compare-slug entry is structurally required — INFERENCE_MODEL_SLUGS, MODEL_ROUTES, /rankings and /run all derive from it and their tests demand coverage for every non-hidden model. No new GPU, precision, framework, or sequence is needed; all are already registered. MODEL_ARCHITECTURES is left for a follow-up (no diagram renders until added), as is the MTP engine exclusion — the guard only becomes meaningful once a second engine family sweeps the same SKU. 中文：新增 DeepSeek-V4.1-Flash 模型，数据来自 InferenceX#2961 的 GB300 AgentX sweep（run 34463351592）：DB key 为 `dsv41flash`，HF 路径 `deepseek-ai/DeepSeek-V4.1-Flash`，FP4 + vLLM + TP4 + MTP。要点： - 它采用 Causal Encoder-Decoder / CSA2 架构，并非 V4-Pro 的小版本更新， 因此单独建桶，不并入 `dsv4`。 - 标签中的参数量取 552B backbone；196B 的 Engram 条件记忆表按 token 稀疏查表 访问，不计入总量，与此前排除独立 MTP head 的处理一致。 - 发布日期 2026-09-10 依据 Hugging Face 仓库首次提交（02:17Z）；容器 tag 中的 `0909` 是镜像构建日期，而非权重发布日期。OpenRouter id 已对照线上目录核实。 - 在 overview 矩阵中标记为仅 AgentX：目前只有 agentic traces 数据，不应保留 永远不会填充的定长序列行。 - compare-slug 条目是结构性必需项：INFERENCE_MODEL_SLUGS、MODEL_ROUTES、 /rankings 与 /run 均由其派生，相关测试要求覆盖每个非 hidden 模型。 - 无需新增 GPU、精度、框架或序列长度。MODEL_ARCHITECTURES 与 MTP engine exclusion 留待后续处理。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
/reuse-sweep-run |
Resolve conflicts with the H200 AgentX arm (#2963): keep both master-config entries and both docs sections, take main's Chinese wording in MODELS_zh.md, keep the overridable engine-readiness default in the shared script, and append this branch's changelog entries at the tail. 将 main 合并入 GB300 DSv4.1 Flash AgentX 分支，解决与 H200 分支（#2963）的冲突： 保留两个 master 配置条目与两段文档，MODELS_zh.md 采用 main 的中文表述，共享脚本 保留可覆盖的引擎就绪默认值，并将本分支的 changelog 条目追加到文件末尾。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 597c06a. Configure here.
| --container-mounts="$GITHUB_WORKSPACE:/ix,$HF_HUB_CACHE_HOST_PATH:/hf-cache" \ | ||
| --no-container-mount-home --container-remap-root \ | ||
| --container-workdir=/ix --no-container-entrypoint \ | ||
| --export=ALL,PORT=8888 bash "$BENCH_SCRIPT" |
There was a problem hiding this comment.
GB300 launch omits aiperf mmap cache
Medium Severity
The new GB300 direct vLLM path mounts the Hugging Face cache but does not bind AIPERF_MMAP_CACHE_HOST_PATH or set AIPERF_DATASET_MMAP_CACHE_DIR. AgentX replay then rebuilds the tokenized mmap corpus on every job instead of reusing the persistent cache this launcher already documents as required.
Reviewed by Cursor Bugbot for commit 597c06a. Configure here.
Resolve conflicts with the merged H200 (#2963) and GB300 (#2961) arms: keep all three master-config entries, take main's docs and MODELS_zh.md wording, keep the overridable engine-readiness default in the shared script, and append this branch's changelog entries at the tail. 将 main 合并入 B200 DSv4.1 Flash AgentX 分支，解决与已合入的 H200（#2963）和 GB300（#2961）分支的冲突：保留三个 master 配置条目，文档与 MODELS_zh.md 采用 main 的表述，共享脚本保留可覆盖的引擎就绪默认值，并将本分支的 changelog 条目追加到末尾。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolve conflicts with the merged H200 (#2963), GB300 (#2961), and B200 (#2960) arms: keep all four master-config entries, take main's docs and MODELS_zh.md wording, keep the overridable engine-readiness default in the shared script, and append this branch's changelog entries at the tail. 将 main 合并入 B300 DSv4.1 Flash AgentX 分支，解决与已合入的 H200（#2963）、 GB300（#2961）和 B200（#2960）分支的冲突：保留四个 master 配置条目，文档与 MODELS_zh.md 采用 main 的表述，共享脚本保留可覆盖的引擎就绪默认值，并将本分支的 changelog 条目追加到末尾。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolve conflicts with the merged H200 (#2963), GB300 (#2961), and B200 (#2960) arms: keep all four master-config entries, take main's docs and MODELS_zh.md wording, keep the overridable engine-readiness default in the shared script, and append this branch's changelog entries at the tail. 将 main 合并入 GB200 DSv4.1 Flash AgentX 分支，解决与已合入的 H200（#2963）、 GB300（#2961）和 B200（#2960）分支的冲突：保留四个 master 配置条目，文档与 MODELS_zh.md 采用 main 的表述，共享脚本保留可覆盖的引擎就绪默认值，并将本分支的 changelog 条目追加到末尾。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2 participants
Add GB300 TP4 DSv4.1 Flash AgentX at concurrency 1–128, using the full
semianalysis_cc_traces_weka_062126corpus. Five-token DSpark throughput uses golden AL 3.51 (thinking on), with adaptive verification disabled; accuracy evals retain real block verification. Engram stays in host DRAM through UVA; KV stays on GPU. Validation: Bash, exact-key matrix, changelog, launcher/eval tests, and both serving-mode command checks pass. GPU validation pending.新增 GB300 TP4 DSv4.1 Flash AgentX，并发 1–128，使用完整
semianalysis_cc_traces_weka_062126语料。五 token DSpark 吞吐测试采用 thinking 开启时的黄金 AL 3.51，并关闭自适应验证；准确率 eval 保留真实块验证。Engram 通过 UVA 访问主机 DRAM，KV 保留在 GPU。验证：Bash、精确配置矩阵、changelog、launcher/eval 测试及两种服务模式的命令检查通过；GPU 验证待完成。Startup fix: GB300 now allows 7200 seconds for engine readiness. The previous 3600-second deadline killed the frontend during graph capture in run 34504969146. Launcher and command checks pass; GPU startup validation is pending.
启动修复：GB300 的引擎就绪期限现为 7200 秒。上述运行中，原 3600 秒期限在 graph capture 期间导致 frontend 退出。launcher 和命令检查通过，GPU 启动验证待完成。
Note
Medium Risk
Changes GB300 runner routing and startup timeouts for a new recipe; mis-routing or timeout tweaks could affect unrelated jobs if model-prefix checks are wrong, but scope is gated to single-node dsv41flash vLLM MTP.
Overview
Adds
dsv41flash-fp4-gb300-vllm-agentic-dsparkto the master config: TP4 vLLM oncluster:gb300-nv, AgentX coding at concurrency 1–128, native five-token DSpark, Engram UVA DRAM offload, and GPU-resident KV (kv-offloading: none).launch_gb300-nv.shnow short-circuits single-nodedsv41flash+ vLLM + MTP agentic runs: one-GPU-traysrunwith repo at/ix, shared HF cache, thedsv41flash_fp4_gb300_vllm_mtp.shentry (wrapper to the shared MTP script), andVLLM_ENGINE_READY_TIMEOUT_S=7200so cold load/graph capture is not cut off by the Rust frontend’s one-hour default. The shared MTP launcher only defaults that timeout to 3600 unless the environment overrides it.Documentation (EN/ZH), perf-changelog entries, and
test_dsv41flash_gb300.pylock in launcher flags (1 node, 4 GPUs, mounts, no nginx path) and exit-code propagation.Reviewed by Cursor Bugbot for commit 597c06a. Bugbot is set up for automated code reviews on this repo. Configure here.