[Klaud Cold] Update Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX MTP image to v0.5.22-cu130 / [Klaud Cold] 将 Qwen3.8-Flash-Next FP4 GB300 SGLang AgentX MTP 镜像更新为 v0.5.22-cu130 by Klaud-Cold · Pull Request #4512 · SemiAnalysisAI/InferenceX

Klaud-Cold · 2026-10-25T16:27:08Z

Goal: Update SGLang image from lmsysorg/sglang:v0.5.21-cu130@sha256:1f176b3a1dd1205d2a462ab80ea9fc17769a32e53cb6f3c690efa2dc2c384977 to lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6.
Baseline: 2026-10-09 · lmsysorg/sglang:v0.5.21-cu130@sha256:1f176b3a1dd1205d2a462ab80ea9fc17769a32e53cb6f3c690efa2dc2c384977
AgentX · TP1/EP1 · Mean latency · Sources: API 1, API 2, API 3, API 4

Concurrency	Total tok/s/GPU ↑	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	14,605.77	160.23	682.15	3.62
2	16,968.63	190.18	663.54	3.96
4	23,650.43	254.99	590.23	4.4
8	39,482.77	375.02	699.41	5.19
16	79,913.49	702.84	889.76	8.26
32	127,572.15	1,059.17	1,336.5	14.66
48	141,336.53	1,329.06	1,819.24	23.44
64	134,313.48	1,394.08	9,091.99	32.3

Eval	Score ↑	Samples
gsm8k/em_strict · c64	97.35%	1,319

Stop: Close this PR at any time to stop this candidate now; Klaud cancels its runs and deletes its branch.

中文

**目标：**将 SGLang 镜像从 lmsysorg/sglang:v0.5.21-cu130@sha256:1f176b3a1dd1205d2a462ab80ea9fc17769a32e53cb6f3c690efa2dc2c384977 更新为 lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6。
**基线：**2026-10-09 · lmsysorg/sglang:v0.5.21-cu130@sha256:1f176b3a1dd1205d2a462ab80ea9fc17769a32e53cb6f3c690efa2dc2c384977
AgentX · TP1/EP1 · 平均延迟 · 来源： API 1, API 2, API 3, API 4；数值及异常说明见上表。
**停止：**随时关闭此 PR 即可立即停止该候选：Klaud 会取消其运行并删除其分支。

…22-cu130 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-25T16:27:16Z

Thanks for the contribution!

Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

**审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
**PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
**合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

Klaud-Cold · 2026-10-25T16:29:54Z

Initial attempt · Passed · Run 40211815811 / attempt 1 · 2026-10-25 18:32 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6 · 88588aaf353a · AgentX · TP1/EP1 · Mean latency
Change: Only the master image changes from SGLang v0.5.21-cu130 to v0.5.22-cu130 (arm64 digest) measured at ref 4dbd41c; every recipe flag and env var keeps its v0.5.22 definition and default (server-arg fields compared at tags e00930c and baa9bef), while coupled deps move torch 2.13.0 to 2.14.1 and flashinfer 0.6.18 to 0.7.1.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	170.55 (+6.4%)	608.68 (-10.8%)	3.54 (-2.2%)

Eval	Score ↑	Samples
gsm8k/em_strict · c64	97.8% (+0.45 pp)	1,319 each

Next: Smoke c1 throughput (0 request errors) and c64 gsm8k passed, so append the perf-changelog entry and dispatch the full-sweep-fail-fast final sweep.

中文

初次尝试 · 已通过 · Run 40211815811 / attempt 1 · 2026-10-25 18:32 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6 · 88588aaf353a · AgentX · TP1/EP1 · 平均延迟
**变更：**仅将主镜像从 SGLang v0.5.21-cu130 更新为 v0.5.22-cu130（arm64 digest，测量 ref 4dbd41c）；配方使用的所有参数和环境变量在 v0.5.22 中定义与默认值不变（对比标签 e00930c 与 baa9bef 的 server-arg 字段），耦合依赖 torch 从 2.13.0 升至 2.14.1，flashinfer 从 0.6.18 升至 0.7.1。实测数值及异常说明见上表。
**下一步：**smoke c1 吞吐（0 个请求错误）与 c64 gsm8k 评测均通过，接下来追加 perf-changelog 条目并通过 full-sweep-fail-fast 发起最终扫描。

Klaud-Cold · 2026-10-25T16:50:04Z

parked · Session 1 is waiting on 40211815811. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 1 个会话正在等待 40211815811。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

Klaud-Cold · 2026-10-25T18:30:22Z

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束；第 2 个会话将在其规划运行的新运行中继续。

…5.22 update Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold · 2026-10-25T18:34:01Z

Final full sweep · Running · Run 40221796954 / attempt 1 · 2026-10-25 18:33 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6 · b3c469ec2cdf · AgentX · TP1/EP1 · Mean latency
Change: Appended the perf-changelog entry for the SGLang v0.5.22-cu130 image update; recipe flags and env vars are unchanged.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	N/A	N/A	N/A
2	N/A	N/A	N/A
4	N/A	N/A	N/A
8	N/A	N/A	N/A
16	N/A	N/A	N/A
32	N/A	N/A	N/A
48	N/A	N/A	N/A
64	N/A	N/A	N/A

Note: All rows: pending.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Next: Session parks while the full-sweep-fail-fast final sweep runs all 8 AgentX points (c1-c64) and the c64 gsm8k eval on gb300-nv.

中文

最终完整 sweep · 运行中 · Run 40221796954 / attempt 1 · 2026-10-25 18:33 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6 · b3c469ec2cdf · AgentX · TP1/EP1 · 平均延迟
**变更：**为 SGLang v0.5.22-cu130 镜像更新追加 perf-changelog 条目；配方参数与环境变量保持不变。实测数值及异常说明见上表。
**下一步：**会话暂停，等待 full-sweep-fail-fast 最终扫描在 gb300-nv 上完成全部 8 个 AgentX 点（c1-c64）与 c64 gsm8k 评测。

Klaud-Cold · 2026-10-25T18:34:10Z

parked · Session 2 is waiting on 40221796954. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 2 个会话正在等待 40221796954。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

github-actions · 2026-10-25T18:49:29Z

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40221796954

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40221796954

Klaud-Cold · 2026-10-26T08:48:34Z

Final full sweep · Passed · Run 40221796954 / attempt 4 · 2026-10-26 08:48 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6 · b3c469ec2cdf · AgentX · TP1/EP1 · Mean latency
Change: Validate the complete updated-image family at this head.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	168.73 (+5.3%)	601.8 (-11.8%)	3.5 (-3.3%)
2	192.17 (+1%)	535.18 (-19.3%)	3.66 (-7.6%)
4	261.99 (+2.7%)	487.83 (-17.3%)	4.31 (-2%)
8	384.51 (+2.5%)	553.51 (-20.9%)	5.48 (+5.6%)
16	697.01 (-0.8%)	636.78 (-28.4%)	9.42 (+14%)
32	1,024.84 (-3.2%)	888.09 (-33.6%)	17.38 (+18.6%)
48	1,298.02 (-2.3%)	1,180.79 (-35.1%)	27.89 (+19%)
64	1,347.15 (-3.4%)	10,021.25 (+10.2%)	34.16 (+5.8%)

Eval	Score ↑	Samples
gsm8k/em_strict · c64	97.73% (+0.38 pp)	1,319 each

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 40221796954 / attempt 4 · 2026-10-26 08:48 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6 · b3c469ec2cdf · AgentX · TP1/EP1 · 平均延迟
**变更：**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步：**标记为就绪，等待维护者审查。

Klaud-Cold · 2026-10-26T08:48:45Z

validated · Repairs: unknown · Runs: 40211815811, 40221777227, 40221796954
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数：未知 · 运行：40211815811, 40221777227, 40221796954
所有自有运行均已结束。完整 sweep 已验证；已就绪，等待审查。

claude

Code review found no issues

No high-confidence issues detected in this change.

adibarra · 2026-10-26T23:26:28Z

/use 40221796954

github-actions · 2026-10-26T23:26:46Z

@adibarra staged run 40221796954.

Qwen3.8-Flash-Next · agentic-traces · FP4 · gb300_sglang · 2026-10-25

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

Resolve perf-changelog.yaml by taking main's file and re-appending this PR's entry. 中文：合并 origin/main；perf-changelog.yaml 采用 main 版本，并在末尾重新追加本 PR 的条目。

github-actions · 2026-10-26T23:36:41Z

Ingest report — ✅ Results ingested into the production database.

Ingest workflow · Merge Ingest run · Benchmark source run

Category	new	duplicate	skipped
benchmark	8	0	0
eval	1	1	4
trace	8	0	0
power	8	0	0

_{benchmarkFiles 8, rawBenchmarkRows 8, coveredByPerJobFiles 0, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 1319, gpuMetricSamplesNew 41567, powerPublicationPoints 8, changelogEntriesWritten 1}

Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun.

中文

Ingest 报告 — ✅ 结果已写入 production 数据库。

上表仅统计本次运行写入的数据，各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。

chore(qwen3.8next): update GB300 FP4 SGLang AgentX MTP image to v0.5.…

4dbd41c

…22-cu130 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-project-automation Bot added this to InferenceX Board Oct 25, 2026

chore(qwen3.8next): add perf-changelog entry for GB300 FP4 SGLang v0.…

b3c469e

…5.22 update Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 25, 2026

Klaud-Cold marked this pull request as ready for review October 26, 2026 08:48

Klaud-Cold requested a review from a team October 26, 2026 08:48

Klaud-Cold requested review from Ankur-singh, RohitNagraj, kedarpotdar-nv, nvpohanh and wzhao18 as code owners October 26, 2026 08:48

claude Bot reviewed Oct 26, 2026

View reviewed changes

Merge origin/main [skip-sweep]

d6c1ec7

Resolve perf-changelog.yaml by taking main's file and re-appending this PR's entry. 中文：合并 origin/main；perf-changelog.yaml 采用 main 版本，并在末尾重新追加本 PR 的条目。

adibarra merged commit 73bc855 into main Oct 26, 2026
27 checks passed

adibarra deleted the klaud/auto-bea43d7bd4014cd3-710e80f323fb0245 branch October 26, 2026 23:31

github-project-automation Bot moved this to Done in InferenceX Board Oct 26, 2026

nvpohanh mentioned this pull request Oct 27, 2026

[Doc] cookbook: Refresh SemiAnalysis AgentX launches with DeepSeek-V4 B200 DSpark6 disagg and Muse Spark GB300 points sgl-project/sglang#46807

Merged

Conversation

Klaud-Cold commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

github-actions Bot commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 26, 2026

Uh oh!

Klaud-Cold commented Oct 26, 2026

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Uh oh!

adibarra commented Oct 26, 2026

Uh oh!

github-actions Bot commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Klaud-Cold commented Oct 25, 2026 •

edited

Loading

Klaud-Cold commented Oct 25, 2026 •

edited

Loading

github-actions Bot commented Oct 26, 2026 •

edited

Loading