[Klaud Cold] Update dsv41flash-fp4-b200-vllm-agentic-dspark to vLLM nightly-36768d1b / [Klaud Cold] 将 dsv41flash-fp4-b200-vllm-agentic-dspark 更新至 vLLM nightly-36768d1b by Klaud-Cold · Pull Request #3579 · SemiAnalysisAI/InferenceX

Klaud-Cold · 2026-09-29T12:49:56Z

Goal: Update vLLM image from vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4 to vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89@sha256:0ad6fa4178c34f4e04402452bbc4294df6dd2fb2785a0ecd402b2861aea2a189.
Baseline: 2026-09-29 · vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4
Mean latency · Sources: API 1, API 2, API 3

Point	Total tok/s/GPU ↑	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP2 EP1 8e063c	13,128.03	84.9	386.48	2.24
AgentX c1 TP4 EP1 b63216	7,282.72	44.48	350.25	1.84
AgentX c2 TP2 EP1 ea3f1c	13,448.65	97.96	318.09	2.42
AgentX c2 TP4 EP1 b3cdef	7,450.13	53.33	284.92	1.95
AgentX c4 TP2 EP1 2686b6	18,432.23	125.66	303.61	2.58
AgentX c4 TP4 EP1 2949fa	9,703.12	66.26	273.16	2.16
AgentX c8 TP2 EP1 152d84	31,009.14	222.67	273.9	3.01
AgentX c8 TP4 EP1 fc3c2c	16,127.35	113.98	228.87	2.43
AgentX c16 TP2 EP1 38d983	55,804.04	350.29	326.01	4.23
AgentX c16 TP4 EP1 91af93	28,773.9	183.91	256.72	3.24
AgentX c32 TP2 EP1 bb038d	93,351.73	584.14	1,854.08	8.42
AgentX c32 TP4 EP1 ce771f	60,400.18	347.08	326.09	4.49
AgentX c64 TP2 EP1 894236	20,502.5	152.16	181,986.65	24.08
AgentX c64 TP4 EP1 da1ba7	107,014.21	762.5	372.28	7.97
AgentX c128 TP2 EP1 affca5	18,694.92	213.91	447,179.47	35.82
AgentX c128 TP4 EP1 61d9ee	141,302.58	1,149.69	615.34	14.83

Eval	Score ↑	Samples
gsm8k/em_strict · c128	97.5%	1,319

中文

**目标：**将 vLLM 镜像从 vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4 更新为 vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89@sha256:0ad6fa4178c34f4e04402452bbc4294df6dd2fb2785a0ecd402b2861aea2a189。
**基线：**2026-09-29 · vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4
平均延迟 · 来源： API 1, API 2, API 3；数值及异常说明见上表。

将 B200 DeepSeek-V4.1-Flash vLLM AgentX DSpark 镜像更新为 nightly-36768d1b（按 digest 固定）。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-09-29T12:50:08Z

Thanks for the contribution!

Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

**审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
**PR 验证：**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast；仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
**合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

Klaud-Cold · 2026-09-29T12:51:24Z

Initial attempt · Failed · Run 36570824317 / attempt 1 · 2026-09-29 12:57 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89@sha256:0ad6fa4178c34f4e04402452bbc4294df6dd2fb2785a0ecd402b2861aea2a189 · f437f7bfd164 · Mean latency
Change: Measure PR head e4bd6ab: pin the family and its recipe to vLLM nightly 36768d1b by digest with no flag changes; the ddd6fbca...36768d1b source diff (vllm-project/vllm@ddd6fbc...36768d1) adds automatic DSV4.1 SM100 fusions (#58586 MoE finalize into all-reduce + mHC, #58634 fused WO-A) and leaves the used attention, DSpark, Engram and parser options unchanged; image-to-commit provenance rests on the nightly tag name.

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP2 EP1 8e063c	N/A	N/A	N/A
AgentX c1 TP4 EP1 b63216	N/A	N/A	N/A

Note: AgentX c1 TP2 EP1 8e063c: failed.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.
Note: AgentX c1 TP4 EP1 b63216: cancelled.

Eval	Score ↑	Samples
gsm8k/em_strict · c128	N/A	1,319/N/A (old/new)

Note: All rows: failed.
Note: All rows: Δ N/A: eval score unavailable.

Next: Retry with the commit-specific nightly tag without the digest suffix.

中文

初次尝试 · 失败 · Run 36570824317 / attempt 1 · 2026-09-29 12:57 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89@sha256:0ad6fa4178c34f4e04402452bbc4294df6dd2fb2785a0ecd402b2861aea2a189 · f437f7bfd164 · 平均延迟
**变更：**测量 PR 提交 e4bd6ab：将该配置族及其配方按 digest 固定到 vLLM nightly 36768d1b，不修改任何参数；ddd6fbca...36768d1b 源码差异新增自动启用的 DSV4.1 SM100 融合（#58586、#58634），所用 attention、DSpark、Engram 与 parser 选项未变；镜像与提交的对应关系依据 nightly 标签名。实测数值及异常说明见上表。
**下一步：**改用不带 digest 后缀的提交专属 nightly 标签重试。

Pyxis on the native single-node path cannot import digest-qualified references, so reference the commit-specific tag directly. 原生单节点路径上的 Pyxis 无法导入带 digest 的镜像引用，因此直接使用提交专属标签。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold · 2026-09-29T12:57:59Z

Repair 1/5 · Failed · Run 36571637678 / attempt 1 · 2026-09-29 13:30 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89 · f437f7bfd164 · Mean latency
Change: Measure PR head d89ef43: reference the commit-specific tag nightly-36768d1b without the digest suffix, because Pyxis on the native single-node path rejected the digest form; the tag resolved to multi-arch digest sha256:0ad6fa4178c34f4e04402452bbc4294df6dd2fb2785a0ecd402b2861aea2a189 at dispatch.

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP4 EP1 b63216	N/A	N/A	N/A
AgentX c1 TP2 EP1 8e063c	N/A	N/A	N/A

Note: AgentX c1 TP4 EP1 b63216: failed.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.
Note: AgentX c1 TP2 EP1 8e063c: cancelled.

Eval	Score ↑	Samples
gsm8k/em_strict · c128	97.35% (-0.15 pp)	1,319 each

Next: Keep the TP4 MXFP8 GEMM on FlashInfer's untuned base tactic through VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS and rerun the smoke.

中文

修复 1/5 · 失败 · Run 36571637678 / attempt 1 · 2026-09-29 13:30 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89 · f437f7bfd164 · 平均延迟
**变更：**测量 PR 提交 d89ef43：改用不带 digest 后缀的提交专属标签 nightly-36768d1b，因为原生单节点路径上的 Pyxis 拒绝 digest 形式；派发时该标签解析为多架构 digest sha256:0ad6fa4178c34f4e04402452bbc4294df6dd2fb2785a0ecd402b2861aea2a189。实测数值及异常说明见上表。
**下一步：**通过 VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS 让 TP4 的 MXFP8 GEMM 使用 FlashInfer 未调优的基础 tactic，并重新运行 smoke。

vLLM nightly-36768d1b fuses the DSV4.1 MoE finalize into the TP4 all-reduce + mHC and autotunes a 16-token pass, where FlashInfer picks an invalid MXFP8 split-K tactic. Leave mxfp8_gemm on its base tactic. vLLM nightly-36768d1b 在 TP4 上将 DSV4.1 MoE finalize 融合进 all-reduce + mHC，并新增 16 token 的 autotune，FlashInfer 在其中选出无效的 MXFP8 split-K tactic。 TP4 让 mxfp8_gemm 使用基础 tactic。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold · 2026-09-29T13:31:54Z

Repair 2/5 · Passed · Run 36575711395 / attempt 1 · 2026-09-29 14:57 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89 · 7804bc48924f · Mean latency
Change: Measure PR head 6414bbc: set VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS=mxfp8_gemm on the TP4 points only, because vLLM PR 58586 (vllm-project/vllm#58586) enables the TP4 fused all-reduce + mHC MoE finalize and its 16-token autotune pass, where FlashInfer 0.7.0 chose a split-K tactic its own forward check rejects (https://github.com/flashinfer-ai/flashinfer/blob/v0.7.0/flashinfer/gemm/gemm_base.py#L5847).

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP2 EP1 8e063c	85.47 (+0.7%)	396.82 (+2.7%)	2.18 (-2.7%)
AgentX c1 TP4 EP1 b63216	45.41 (+2.1%)	383.27 (+9.4%)	1.72 (-6.5%)

Eval	Score ↑	Samples
gsm8k/em_strict · c128	97.42% (-0.08 pp)	1,319 each

Next: Append the changelog entry and run the final full sweep.

中文

修复 2/5 · 已通过 · Run 36575711395 / attempt 1 · 2026-09-29 14:57 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89 · 7804bc48924f · 平均延迟
**变更：**测量 PR 提交 6414bbc：仅在 TP4 点设置 VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS=mxfp8_gemm，因为 vLLM PR 58586 在 TP4 启用 all-reduce + mHC 融合的 MoE finalize 及其 16 token autotune，FlashInfer 0.7.0 在其中选出了自身 forward 校验拒绝的 split-K tactic。实测数值及异常说明见上表。
**下一步：**追加 changelog 条目并运行最终完整 sweep。

记录 B200 DeepSeek-V4.1-Flash vLLM nightly-36768d1b 更新。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold · 2026-09-29T14:59:58Z

Final full sweep · Running · Run 36586650146 / attempt 1 · 2026-09-29 15:31 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89 · 9abd27774e84
Change: Run the complete family on vLLM nightly-36768d1b with the TP4 MXFP8 autotune skip and the changelog entry, after syncing main.

Eval	Score ↑	Samples
gsm8k/em_strict · c128	97.35% (-0.15 pp)	1,319 each

Next: Finish all 16 benchmark points.

中文

最终完整 sweep · 运行中 · Run 36586650146 / attempt 1 · 2026-09-29 15:31 UTC
vllm/vllm-openai:nightly-36768d1bfd39094681cdbc8cb37d4b31c0729c89 · 9abd27774e84
**变更：**同步 main 后，在 vLLM nightly-36768d1b 上运行完整配置族，包含 TP4 MXFP8 autotune 跳过设置与 changelog 条目。实测数值及异常说明见上表。
**下一步：**完成全部 16 个基准测试点。

github-actions · 2026-09-29T19:02:34Z

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=36586650146

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=36586650146

Klaud-Cold · 2026-10-25T06:42:56Z

closed-by-maintainer · Cleanup pending. Stop and confirm owned runs before closing.

中文

closed-by-maintainer · 清理待完成。先停止并确认自有运行结束，再关闭 PR。

Klaud-Cold · 2026-10-25T06:43:16Z

closed-by-maintainer · Repairs: unknown · Runs: 36570824317, 36571637678, 36575711395, 36586599507, 36586650146
Reason: Closed by a maintainer.
All owned runs ended. PR closed; branch deleted.

中文

closed-by-maintainer · 修复次数：未知 · 运行：36570824317, 36571637678, 36575711395, 36586599507, 36586650146
**原因：**已由维护者关闭。
所有自有运行均已结束。PR 已关闭；分支已删除。

perf(dsv41flash): bump B200 vLLM AgentX DSpark to nightly-36768d1b

e4bd6ab

将 B200 DeepSeek-V4.1-Flash vLLM AgentX DSpark 镜像更新为 nightly-36768d1b（按 digest 固定）。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-project-automation Bot added this to InferenceX Board Sep 29, 2026

github-actions Bot and others added 2 commits September 29, 2026 14:57

Merge origin/main into klaud/auto-49afb78d16a935e1-e91ce41127921588

9010b2b

chore(changelog): record B200 DeepSeek-V4.1-Flash vLLM nightly-36768d1b

9abd277

记录 B200 DeepSeek-V4.1-Flash vLLM nightly-36768d1b 更新。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Sep 29, 2026

adibarra closed this Sep 30, 2026

Oseltamivir deleted the klaud/auto-49afb78d16a935e1-e91ce41127921588 branch October 18, 2026 08:43

Klaud-Cold removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 25, 2026

Navigation Menu

Repository navigation

[Klaud Cold] Update dsv41flash-fp4-b200-vllm-agentic-dspark to vLLM nightly-36768d1b / [Klaud Cold] 将 dsv41flash-fp4-b200-vllm-agentic-dspark 更新至 vLLM nightly-36768d1b - #3579

[Klaud Cold] Update dsv41flash-fp4-b200-vllm-agentic-dspark to vLLM nightly-36768d1b / [Klaud Cold] 将 dsv41flash-fp4-b200-vllm-agentic-dspark 更新至 vLLM nightly-36768d1b#3579
Klaud-Cold wants to merge 5 commits into
mainfrom
klaud/auto-49afb78d16a935e1-e91ce41127921588

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Uh oh!

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Uh oh!

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Uh oh!

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Klaud-Cold commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

Klaud-Cold commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 29, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Klaud-Cold commented Sep 29, 2026 •

edited

Loading

Klaud-Cold commented Sep 29, 2026 •

edited

Loading