Skip to content

[Klaud Cold] Update glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp SGLang image to v0.5.19-cu130 / 将 glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp 的 SGLang 镜像更新至 v0.5.19-cu130 - #2992

Closed
Klaud-Cold wants to merge 3 commits into
mainfrom
klaud/auto-aa043d5cc89d9d3e-f7a6259862406d3d
Closed

[Klaud Cold] Update glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp SGLang image to v0.5.19-cu130 / 将 glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp 的 SGLang 镜像更新至 v0.5.19-cu130#2992
Klaud-Cold wants to merge 3 commits into
mainfrom
klaud/auto-aa043d5cc89d9d3e-f7a6259862406d3d

Conversation

@Klaud-Cold

Copy link
Copy Markdown
Collaborator

Refresh glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp from the SGLang nightly-dev-cu13-20260805-211ee642 nightly to the digest-pinned v0.5.19-cu130 release, moving the three disaggregated recipes' Dynamo install from source commit 71eb001e to the 1.5.0.dev20260910 wheel that Dynamo pairs with SGLang 0.5.19.

Baseline

  • Published date: 2026-08-19 (/api/v1/benchmarks?model=GLM-5.2&date=2026-08-19&exact=true, /api/v1/workflow-info?date=2026-08-19&benchmarkType=agentic_traces, /api/v1/evaluations?model=GLM-5.2&date=2026-08-19)
  • Old image: lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642 (SGLang commit 211ee642, 2026-08-05) with Dynamo built from source commit 71eb001e and NVIDIA/srt-slurm v1.0.50. The published rows carry router_version: 1.2.1 from the master metadata, which does not match the Dynamo source commit the recipes actually installed.
  • Producer: run 32207758126 attempt 2 at 6130a8b5b671be6fc9695e8d159197f2cd275482 (PR feat(config): add GLM-5.2 FP4 GB200 SGLang AgentX MTP / 添加 GLM-5.2 FP4 GB200 SGLang AgentX MTP 配置 #2642 sweep, curve 2336)
  • Workload: GLM-5.2 NVFP4 on GB200, agentic-coding trace replay, native MTP (EAGLE, prefill 2 / decode 3 draft tokens, golden acceptance length 2.5), NIXL KV transfer, prefill HiCache DRAM offload; 1P6D (TP8/DEP8 prefill, 6×TP4 decode, 32 GPUs) at concurrency 45, 1P4D (TP8/DEP8 prefill, 4×TP4 decode, 24 GPUs) at concurrency 48, 2P1D (2×TP8/DEP8 prefill, DEP16 decode, 32 GPUs) at concurrency 128
  • Source: SGLang 211ee642…v0.5.19 (1509 commits), SGLang #36972, Dynamo #14234, Dynamo #14151
Point Output tok/s/GPU Total tok/s/GPU Median interactivity (tok/s/user) Mean TTFT (s) Mean TPOT (ms) gsm8k (strict EM)
1P6D conc 45 (id 440078) 56.96 8901.6 225.2 3.346 4.72 0.9568 (eval id 9501)
1P4D conc 48 (id 440082) 73.10 10499.0 204.9 6.815 5.22 0.9545 (eval id 9502)
2P1D conc 128 (id 440079) 137.32 16607.7 102.8 7.095 10.18 0.9401 (eval id 9503)
  • Power: N/A. The published rows contain no power metrics.
  • Evals: the gsm8k scores above come from the same producer run. The current generator emits eval jobs for this family only under --all-evals or --evals-only; the default test-config matrix marks all three points run-eval: false.

glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp 从 SGLang nightly-dev-cu13-20260805-211ee642 夜间构建更新到按摘要固定的 v0.5.19-cu130 正式版,并将三个分离式配方中的 Dynamo 安装方式从源码提交 71eb001e 切换为 Dynamo 与 SGLang 0.5.19 配套的 1.5.0.dev20260910 wheel。

基线

  • 发布日期:2026-08-19(/api/v1/benchmarks?model=GLM-5.2&date=2026-08-19&exact=true/api/v1/workflow-info?date=2026-08-19&benchmarkType=agentic_traces/api/v1/evaluations?model=GLM-5.2&date=2026-08-19
  • 旧镜像:lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642(SGLang 提交 211ee642,2026-08-05),Dynamo 从源码提交 71eb001e 构建,NVIDIA/srt-slurm v1.0.50。已发布数据中的 router_version: 1.2.1 来自主配置元数据,与配方实际安装的 Dynamo 源码提交不一致。
  • 生产运行:run 32207758126 attempt 2,提交 6130a8b5b671be6fc9695e8d159197f2cd275482(PR feat(config): add GLM-5.2 FP4 GB200 SGLang AgentX MTP / 添加 GLM-5.2 FP4 GB200 SGLang AgentX MTP 配置 #2642 的 sweep,曲线 2336
  • 工作负载:GB200 上的 GLM-5.2 NVFP4,agentic-coding 轨迹回放,原生 MTP(EAGLE,prefill 2 / decode 3 个草稿 token,黄金接受长度 2.5),NIXL KV 传输,prefill HiCache DRAM 卸载;1P6D(TP8/DEP8 prefill,6×TP4 decode,32 GPU)并发 45,1P4D(TP8/DEP8 prefill,4×TP4 decode,24 GPU)并发 48,2P1D(2×TP8/DEP8 prefill,DEP16 decode,32 GPU)并发 128
  • 源码:SGLang 211ee642…v0.5.19(1509 个提交)、SGLang #36972Dynamo #14234Dynamo #14151
点位 输出 tok/s/GPU 总 tok/s/GPU 交互性中位数(tok/s/用户) 平均 TTFT(秒) 平均 TPOT(毫秒) gsm8k(严格 EM)
1P6D 并发 45(id 440078) 56.96 8901.6 225.2 3.346 4.72 0.9568(评测 id 9501)
1P4D 并发 48(id 440082) 73.10 10499.0 204.9 6.815 5.22 0.9545(评测 id 9502)
2P1D 并发 128(id 440079) 137.32 16607.7 102.8 7.095 10.18 0.9401(评测 id 9503)
  • 功耗:N/A。已发布数据中不包含功耗指标。
  • 评测:上表 gsm8k 分数来自同一生产运行。当前生成器仅在 --all-evals--evals-only 下为该配置族生成评测任务;默认 test-config 矩阵中三个点位均为 run-eval: false

🤖 Generated with Claude Code

Update glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp and its three disaggregated
srt-slurm recipes from lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642 to
the digest-pinned lmsysorg/sglang:v0.5.19-cu130 release. Move the recipes'
Dynamo install from the source commit 71eb001e to the ai-dynamo
1.5.0.dev20260910 wheel, which carries the SGLang #36972 compatibility fix
(ai-dynamo/dynamo#14234) and pins sglang==0.5.19, and align the master router
metadata and recipe identity to those versions.

将 glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp 及其三个分离式 srt-slurm 配方的
镜像从 lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642 更新为按摘要固定的
lmsysorg/sglang:v0.5.19-cu130 正式版。将配方中的 Dynamo 安装方式从源码提交
71eb001e 改为 ai-dynamo 1.5.0.dev20260910 wheel(包含针对 SGLang #36972 的
兼容性修复 ai-dynamo/dynamo#14234,并固定 sglang==0.5.19),并同步更新主配置
路由器元数据与配方身份信息。

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

@Klaud-Cold

Klaud-Cold commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator Author

Initial attempt

  • Image: lmsysorg/sglang:v0.5.19-cu130@sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9 (Docker Hub tag pushed 2026-09-04T22:50Z, multi-arch amd64/arm64; SGLang annotated tag v0.5.19 → commit 0bcd8223, 2026-09-04T19:40Z)
  • Head: 10aa0fedbededb4aa2fbb46fc064541cc7686cd4
  • Targeted smoke: run 34548744713 (test-config --config-files configs/nvidia-master.yaml --config-keys glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp --trim-conc, fail-fast, Klaud background priority). The trimmed matrix keeps all three deployment shapes at their single concurrency: 1P6D conc 45 (8 nodes), 1P4D conc 48 (6 nodes), 2P1D conc 128 (8 nodes); the test-config default selects no eval jobs for this family.
  • Changes: master image and the three disaggregated recipes' identity.container.image moved to the digest-pinned release; recipe dynamo block moved from hash: 71eb001e (source build) to wheel: "1.5.0.dev20260910" with install: true; recipe identity.frameworks and master router.version aligned to 1.5.0.dev20260910 / 0.5.19. Model, precision, topology, MTP settings, HiCache, workloads, resources and recipe references are unchanged.
  • Upstream deltas (SGLang 211ee642…v0.5.19, 1509 commits): every sglang_config flag the recipes pass still exists at v0.5.19 with unchanged option lists (nsa-*-backend trtllm, moe-runner-backend flashinfer_cutlass/flashinfer_trtllm/flashinfer_cutedsl, moe-a2a-backend deepep, speculative-moe-runner-backend deep_gemm, HiCache flags); every SGLANG_* env var the recipes set is still registered in environ.py. Coupled dependencies move from torch 2.11.0 / FlashInfer 0.6.15.post1 / sgl-kernel 0.4.5 / CUDA 13.0.1 / Mooncake 0.3.12.post1 to torch 2.13.0 / FlashInfer 0.6.18 / sgl-kernel 0.4.6.post1 / CUDA 13.0.3 / Mooncake 0.3.13.
  • Dynamo: SGLang #36972 (merged 2026-08-29, in v0.5.19) renamed ServerArgs.get_model_config() / use_mla_backend(); the recipes' pinned Dynamo commit 71eb001e (2026-07-31) predates the compatibility shim in Dynamo #14234 (merged 2026-09-03), so the source pin cannot start dynamo.sglang against this image. The 1.5.0.dev20260910 nightly wheel follows both #14234 and Dynamo #14151 (sglang==0.5.19, merged 2026-09-09), matches the pairing validated in [Klaud Cold] Update dsv4-fp4-gb300-dynamo-sglang-agentic-agg SGLang image to v0.5.19-cu130 / 将 dsv4-fp4-gb300-dynamo-sglang-agentic-agg 的 SGLang 镜像更新到 v0.5.19-cu130 #2969, and ships cp310-abi3 manylinux_2_28_aarch64 runtime wheels. NVIDIA/srt-slurm v1.0.50 (already pinned by the GB200 launcher) loads all three edited recipes and renders the staged-wheel install path; no launcher or shared-code change.
  • Provenance note: the nightly wheel's exact Dynamo commit is not published on PyPI, so its content is inferred from the nightly build date relative to the merged PRs.
  • Outcome (2026-09-11 04:30Z): run 34548744713 completed with all three multi-node agentic jobs successful (c48 01:00–02:31Z, c128 01:00–02:50Z, c45 02:30–04:28Z). Identity banners report frameworks.dynamo 1.5.0.dev20260910 and frameworks.sglang 0.5.19; the staged aarch64 CPython 3.12 Dynamo wheels installed without errors. Power: N/A (power_valid 0, matching the baseline rows).
  • Smoke results vs the 2026-08-19 baseline (same topology, concurrency and dataset; smoke points, not a full curve). Mean TTFT at 1P6D conc 45 regressed while its interactivity and TPOT improved:
Point Output tok/s/GPU Total tok/s/GPU Median interactivity (tok/s/user) Mean TTFT (s) Mean TPOT (ms)
1P6D conc 45 56.96 → 56.80 (−0.3%) 8901.6 → 8800.9 (−1.1%) 225.2 → 238.6 (+5.9%) 3.346 → 3.721 (+11.2%) 4.72 → 4.46 (−5.5%)
1P4D conc 48 73.10 → 75.01 (+2.6%) 10499.0 → 10603.5 (+1.0%) 204.9 → 214.8 (+4.8%) 6.815 → 6.558 (−3.8%) 5.22 → 4.97 (−4.8%)
2P1D conc 128 137.32 → 141.57 (+3.1%) 16607.7 → 17459.1 (+5.1%) 102.8 → 105.8 (+3.0%) 7.095 → 5.939 (−16.3%) 10.18 → 9.71 (−4.6%)
  • Next step: changelog entry appended and pushed (15fdf33a), then main merged into the branch because the PR conflicted on perf-changelog.yaml (main appended four entries after the base) and GitHub runs no pull_request workflow on a conflicting PR; main's changelog bytes were taken verbatim and the entry re-appended (head e621c6274d9e466731b0e3cc92fa0fadf1af2133). Final full sweep started with full-sweep-enabled on that head (see the Final full sweep comment).

初始尝试

  • 镜像:lmsysorg/sglang:v0.5.19-cu130@sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9(Docker Hub 标签推送于 2026-09-04T22:50Z,amd64/arm64 多架构;SGLang 附注标签 v0.5.19 → 提交 0bcd8223,2026-09-04T19:40Z)
  • 头提交:10aa0fedbededb4aa2fbb46fc064541cc7686cd4
  • 定向冒烟:run 34548744713test-config --config-files configs/nvidia-master.yaml --config-keys glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp --trim-conc,fail-fast,Klaud 后台优先级)。裁剪后的矩阵保留三种部署形态各自唯一的并发:1P6D 并发 45(8 节点)、1P4D 并发 48(6 节点)、2P1D 并发 128(8 节点);test-config 默认不为该配置族选择评测任务。
  • 改动:主配置镜像与三个分离式配方的 identity.container.image 切换为按摘要固定的正式版;配方 dynamo 块从 hash: 71eb001e(源码构建)改为 wheel: "1.5.0.dev20260910" 并保留 install: true;配方 identity.frameworks 与主配置 router.version 同步为 1.5.0.dev20260910 / 0.5.19。模型、精度、拓扑、MTP 设置、HiCache、工作负载、资源与配方引用均未改动。
  • 上游差异(SGLang 211ee642…v0.5.19,1509 个提交):配方传入的每个 sglang_config 参数在 v0.5.19 中仍存在且选项列表未变(nsa-*-backend trtllmmoe-runner-backend flashinfer_cutlass/flashinfer_trtllm/flashinfer_cutedslmoe-a2a-backend deepepspeculative-moe-runner-backend deep_gemm、HiCache 参数);配方设置的每个 SGLANG_* 环境变量仍在 environ.py 中注册。耦合依赖从 torch 2.11.0 / FlashInfer 0.6.15.post1 / sgl-kernel 0.4.5 / CUDA 13.0.1 / Mooncake 0.3.12.post1 变为 torch 2.13.0 / FlashInfer 0.6.18 / sgl-kernel 0.4.6.post1 / CUDA 13.0.3 / Mooncake 0.3.13。
  • Dynamo:SGLang #36972(2026-08-29 合并,包含于 v0.5.19)重命名了 ServerArgs.get_model_config() / use_mla_backend();配方原先固定的 Dynamo 提交 71eb001e(2026-07-31)早于 Dynamo #14234(2026-09-03 合并)中的兼容层,因此源码固定版本无法在该镜像上启动 dynamo.sglang1.5.0.dev20260910 夜间 wheel 晚于 #14234 与 Dynamo #14151sglang==0.5.19,2026-09-09 合并),与 [Klaud Cold] Update dsv4-fp4-gb300-dynamo-sglang-agentic-agg SGLang image to v0.5.19-cu130 / 将 dsv4-fp4-gb300-dynamo-sglang-agentic-agg 的 SGLang 镜像更新到 v0.5.19-cu130 #2969 中已验证的组合一致,并提供 cp310-abi3 manylinux_2_28_aarch64 运行时 wheel。GB200 启动器已固定的 NVIDIA/srt-slurm v1.0.50 可加载三个修改后的配方并渲染分阶段 wheel 安装路径;未改动启动器或共享代码。
  • 溯源说明:PyPI 未公布该夜间 wheel 对应的具体 Dynamo 提交,其内容依据夜间构建日期与相关 PR 合并时间推断。
  • 结果(2026-09-11 04:30Z):run 34548744713 的三个多节点 agentic 任务全部成功(c48 01:00–02:31Z,c128 01:00–02:50Z,c45 02:30–04:28Z)。身份校验横幅显示 frameworks.dynamo 1.5.0.dev20260910frameworks.sglang 0.5.19;分阶段暂存的 aarch64 CPython 3.12 Dynamo wheel 安装无错误。功耗:N/A(power_valid 0,与基线行一致)。
  • 冒烟结果与 2026-08-19 基线对比(相同拓扑、并发与数据集;为冒烟点位,非完整曲线)。1P6D 并发 45 的平均 TTFT 出现回退,而其交互性与 TPOT 有所改善:
点位 输出 tok/s/GPU 总 tok/s/GPU 交互性中位数(tok/s/用户) 平均 TTFT(秒) 平均 TPOT(毫秒)
1P6D 并发 45 56.96 → 56.80(−0.3%) 8901.6 → 8800.9(−1.1%) 225.2 → 238.6(+5.9%) 3.346 → 3.721(+11.2%) 4.72 → 4.46(−5.5%)
1P4D 并发 48 73.10 → 75.01(+2.6%) 10499.0 → 10603.5(+1.0%) 204.9 → 214.8(+4.8%) 6.815 → 6.558(−3.8%) 5.22 → 4.97(−4.8%)
2P1D 并发 128 137.32 → 141.57(+3.1%) 16607.7 → 17459.1(+5.1%) 102.8 → 105.8(+3.0%) 7.095 → 5.939(−16.3%) 10.18 → 9.71(−4.6%)
  • 下一步:变更日志条目已追加并推送(15fdf33a);由于 PR 在 perf-changelog.yaml 上与 main 冲突(main 在基线之后追加了四个条目),而 GitHub 不会为冲突中的 PR 运行任何 pull_request 工作流,因此将 main 合并进分支:原样采用 main 的变更日志字节并重新追加条目(头提交 e621c6274d9e466731b0e3cc92fa0fadf1af2133)。已在该头提交上通过 full-sweep-enabled 启动最终完整 sweep(见“最终完整 sweep”评论)。

Record the glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp update to
lmsysorg/sglang:v0.5.19-cu130 and the ai-dynamo 1.5.0.dev20260910 wheel
after the trimmed smoke run passed on all three disaggregated points.

为 glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp 更新至 lmsysorg/sglang:v0.5.19-cu130
与 ai-dynamo 1.5.0.dev20260910 wheel 追加性能变更日志条目;裁剪后的冒烟运行
已在三个分离式点位全部通过。

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Take main's perf-changelog.yaml bytes verbatim and re-append the
glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp entry at the tail so the PR is
mergeable and pull_request sweeps can run.

合并 main:原样采用 main 的 perf-changelog.yaml 字节,并在末尾重新追加
glm5.2-fp4-gb200-dynamo-sglang-agentic-mtp 条目,使 PR 可合并并允许
pull_request sweep 运行。

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Klaud-Cold

Klaud-Cold commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator Author

Final full sweep

  • Head: e621c6274d9e466731b0e3cc92fa0fadf1af2133 (image and Dynamo changes from the initial attempt, the appended perf-changelog.yaml entry, and a merge of main taking main's changelog bytes verbatim)
  • Sweep: run 34563376407 (run-sweep.yml, full-sweep-enabled applied at 2026-09-11 04:45Z after a passing capacity check; PR kept draft)
  • Expected matrix from utils/process_changelog.py: the complete family, three multi-node agentic benchmark points (1P6D conc 45 on 8 nodes, 1P4D conc 48 on 6 nodes, 2P1D conc 128 on 8 nodes) plus three lm-eval GSM8K jobs, one per deployment shape at its concurrency. No trimming, no eval-selection modifiers.
  • Status: in progress with one failure (2026-09-11 06:38Z). c48 GSM8K eval succeeded (05:20–05:41Z). The c48 benchmark job (04:48–06:29Z) FAILED in its benchmark stage: all prefill/decode/frontend servers started and the sweep reached the AgentX benchmark step, which exited with code 1 at 06:28Z. Root cause from the job's multinode_server_logs artifact (benchmark.out): the AgentX client bootstrap on the compute node failed while uv installed aiperf 0.12.0, with Failed to download pyarrow==25.0.1 … network timeout (UV_HTTP_TIMEOUT 30s), so no requests were sent and no ..._conc*.json result was produced; the servers themselves were healthy. This is a network/infrastructure flake unrelated to the SGLang image or the Dynamo wheel (the same point passed in smoke run 34548744713 and the c48 eval job on the same head bootstrapped fine). c45 and c128 benchmarks and their eval jobs are still queued and were not cancelled. The synchronize-event run 34563356459 from the merge push was cancelled by the sweep concurrency group before dispatching any job. This Klaud session reaches its Actions job limit at about 06:44Z; the final sweep is therefore not validated in-session and no repair was pushed (repairs used: 0/5).
  • Eval results vs the 2026-08-19 baseline (lm-eval GSM8K, strict exact match, 1319 samples):
Point GSM8K strict EM
1P4D conc 48 0.9545 → 0.9719 (+1.7 pt; flexible-extract 0.9742)
1P6D conc 45 N/A (queued)
2P1D conc 128 N/A (queued)
  • Next step: no recipe or image repair is indicated. Once run 34563376407 completes, re-running its failed c48 benchmark job (gh run rerun 34563376407 --failed) on the same head is the appropriate retry; the still-queued c45/c128 benchmark and eval jobs were left running. Session interrupted by the Actions job limit before completion; repairs used 0/5. This Klaud session's own Actions job started at 00:44Z and reaches the default 360-minute job limit at about 06:44Z, before all six sweep jobs can finish; if the session is cut there, the sweep keeps running on the exact head and the next autosweep's recovery step reconciles this recorded run (verifying coverage on success or reporting and closing on failure). On completion within the session, run the Klaud finish verification.

最终完整 sweep

  • 头提交:e621c6274d9e466731b0e3cc92fa0fadf1af2133(初始尝试的镜像与 Dynamo 改动、追加的 perf-changelog.yaml 条目,以及原样采用 main 变更日志字节的 main 合并)
  • Sweep:run 34563376407run-sweep.yml,容量检查通过后于 2026-09-11 04:45Z 添加 full-sweep-enabled;PR 保持草稿)
  • 依据 utils/process_changelog.py 的预期矩阵:完整配置族,三个多节点 agentic 基准点位(1P6D 并发 45、8 节点;1P4D 并发 48、6 节点;2P1D 并发 128、8 节点)以及三个 lm-eval GSM8K 任务(每种部署形态各一,取其并发)。不裁剪,不带评测选择修饰符。
  • 状态:进行中且有一项失败(2026-09-11 06:38Z)。c48 GSM8K 评测成功(05:20–05:41Z)。c48 基准任务(04:48–06:29Z)在基准阶段失败:所有 prefill/decode/frontend 服务均已启动,sweep 进入 AgentX 基准步骤后于 06:28Z 以退出码 1 结束。根据该任务 multinode_server_logs 产物中的 benchmark.out,根因是计算节点上 AgentX 客户端引导阶段 uv 安装 aiperf 0.12.0 时失败:Failed to download pyarrow==25.0.1 … network timeout (UV_HTTP_TIMEOUT 30s),因此未发送任何请求、未生成 ..._conc*.json 结果;服务端本身健康。这是与 SGLang 镜像或 Dynamo wheel 无关的网络/基础设施偶发故障(同一点位在冒烟运行 34548744713 中通过,同一头提交上的 c48 评测任务也正常完成引导)。c45、c128 基准及其评测任务仍在排队,未被取消。合并推送触发的 synchronize 事件运行 34563356459 在派发任何任务前已被 sweep 并发组取消。本 Klaud 会话将在约 06:44Z 达到 Actions 任务上限;因此最终 sweep 未能在会话内完成验证,也未推送修复(已用修复次数:0/5)。
  • 评测结果与 2026-08-19 基线对比(lm-eval GSM8K,严格精确匹配,1319 个样本):
点位 GSM8K 严格 EM
1P4D 并发 48 0.9545 → 0.9719(+1.7 个百分点;flexible-extract 0.9742)
1P6D 并发 45 N/A(排队中)
2P1D 并发 128 N/A(排队中)
  • 下一步:无需修改配方或镜像。待 run 34563376407 完成后,在同一头提交上重跑其失败的 c48 基准任务(gh run rerun 34563376407 --failed)即为合适的重试;仍在排队的 c45/c128 基准与评测任务保持运行。会话因 Actions 任务上限在完成前中断;已用修复次数 0/5。本 Klaud 会话自身的 Actions 任务于 00:44Z 开始,将在约 06:44Z 达到默认 360 分钟任务上限,早于六个 sweep 任务全部完成;若会话在此被中断,sweep 仍会在该头提交上继续运行,下一次自动扫描的恢复步骤将核对此已记录的运行(成功则验证覆盖,失败则报告并关闭)。若在会话内完成,则运行 Klaud finish 验证。

@github-actions

Copy link
Copy Markdown
Contributor

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Klaud Cold: unexpected-error. Finishing cleanup; owned child runs will be stopped and checked before closure.


Klaud Cold:unexpected-error。正在完成清理;将先停止并确认自有子运行的状态,再关闭 PR。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Klaud Cold: unexpected-error. All owned runs are terminal. Repairs: unknown. Runs: 34548744713, 34563356459, 34563376407.

PR closed; the exact-candidate branch is retained for manual review. The interruption does not prove image incompatibility.


Klaud Cold:unexpected-error。所有自有运行均已结束。修复次数:未知。运行:34548744713, 34563356459, 34563376407

PR 已关闭;保留该候选的分支,等待人工审查。运行中断不能证明镜像不兼容。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant