[WIP][AMD][AGENTX] GLM-5.2 FP4 MI355X SGLang v0.5.19: full Pareto re-sweep - #3014
[WIP][AMD][AGENTX] GLM-5.2 FP4 MI355X SGLang v0.5.19: full Pareto re-sweep#3014giovanniguastiamd wants to merge 2 commits into
Conversation
Image: v0.5.16-rocm720-mi35x-20260728 → v0.5.19-rocm720-mi35x-20260908 Includes SGLang #37124 (fused DSA metadata kernels, +18-19% norm_intvty_p90 at c4/c6 vs v0.5.16 baseline) and #37133 (e_score_correction_bias fp32). Search-space updated to 3-arm Pareto sweep: - TP4/EP4 no-KV: conc-list [1,2,4,6,8] (GPU-resident KV, low latency) - TP8/EP1 no-KV: conc-list [1,2,4] (interactivity arm) - TP4/EP4 HiCache: conc-list [1,2,4,8,10] (high-concurrency fallback) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
giovanniguastiamd
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 01b8cc2. Configure here.
…mtp (PR #3014) Image bump v0.5.16 → v0.5.19 (+18-19% norm_intvty_p90 at c4/c6 via SGLang #37124 fused DSA metadata kernels). Restore 3-arm Pareto search-space: TP4/EP4 no-KV, TP8/EP1 no-KV, TP4/EP4 HiCache. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - { tp: 4, ep: 4, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [1, 2, 4, 8, 10], spec-decoding: mtp } | ||
| # GLM-5.2 FP4 agentic-coding benchmark on MI355X via ATOM with MTP speculative | ||
| # decoding. TP4 uses LMCache DRAM offload; TP8 is GPU-resident with no KV offload. | ||
| # Recipe is from PR https://github.com/ROCm/ATOM/pull/1877 |
There was a problem hiding this comment.
🔴 This diff bumps the sglang image (v0.5.16->v0.5.19) and restructures the agentic-coding search-space (2 arms -> 3 arms, changed conc-lists) for glm5.2-fp4-mi355x-sglang-agentic-mtp, but perf-changelog.yaml is not touched anywhere in the commit. AGENTS.md's non-negotiable invariant requires every image bump and recipe search-space modification to append a new perf-changelog.yaml entry (append-only, tail). Fix: add a perf-changelog.yaml entry documenting the image bump (old/new versions) and the search-space restructuring before merge.
Extended reasoning...
AGENTS.md line 31 states every change that can affect benchmark performance or recipe modification requires a new perf-changelog.yaml entry, appended at the tail, and the file is treated as byte-sensitive/append-only history of perf-relevant changes. This PR changes the container image (affecting kernel/runtime behavior per the PR's own claimed +18-19% perf impact) and rewrites the search-space arms/conc-lists for a benchmark recipe, yet git diff shows no changes to perf-changelog.yaml at all. The PR description even lists 'Update perf-changelog.yaml with final Pareto values before merge' as an unchecked TODO in the test plan, confirming the entry is missing at HEAD. Merging as-is leaves the changelog silently out of sync with the recipe history, breaking downstream tooling or reviewers that rely on perf-changelog.yaml as the authoritative record of performance-affecting changes.
Verification: normal. The diff (git diff e5f3e41..HEAD) touches only configs/amd-master.yaml. Lines 1707-1708 change image: lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260728 -> v0.5.19-rocm720-mi35x-20260908 (an image bump), and lines 1720-1722 rewrite the agentic-coding search-space from two arms ([1,2,4,8,10,12] hicache + [1,2,4,10] TP8) to three arms (TP4/EP4 no-KV [1,2,4,6,8], TP8/EP1 no-KV [1,2,4],…
| # concurrency; GPU-resident KV keeps latency independent of DRAM bandwidth. | ||
| # c6 and c8 removed after sweep validation: dominated by TP4/EP4 arm. | ||
| # SA selects the Pareto-optimal arm per concurrency point. | ||
| # v0.5.19 image bump: includes SGLang #37124 (fused DSA metadata kernels) + #37133. | ||
| # Full Pareto re-sweep: TP4/EP4 no-KV, TP8/EP1 no-KV, TP4/EP4 HiCache. |
There was a problem hiding this comment.
🟡 nit (optional): The stale comment block above this recipe (lines 1697-1705) still describes the old 2-arm structure with conc-lists [1,2,4,8,10,12] and [4,10], directly contradicting both the new 3-arm search-space and the new explanatory lines just added (1706-1707) above it. A maintainer reading top-to-bottom gets conflicting arm counts/conc values. Fix: update or remove the stale block to match the new 3-arm TP4/EP4-noKV, TP8/EP1-noKV, TP4/EP4-HiCache structure and its actual conc-lists.
Extended reasoning...
Lines 1697-1705 (untouched by this diff) narrate 'Two arms' with specific concurrency sweeps [1,2,4,8,10,12] and [4,10], and a rationale ('c6 and c8 removed... dominated by TP4/EP4 arm') that no longer matches the code below. The diff appends two new comment lines (1706-1707) directly beneath it summarizing the real 3-arm change, so the two comment blocks now sit back-to-back describing mutually exclusive designs for the same recipe. A future engineer editing this recipe (e.g. per AGENTS.md's 'follow the nearest existing pattern') could reasonably follow the stale block's numbers instead of the actual search-space, since nothing marks it superseded.
Verification: nit: The comment block at lines 1697-1705 (untouched) still narrates the old 2-arm design: "Two arms: (1) TP4/EP4 with HiCache... concurrency sweep [1, 2, 4, 8, 10, 12]; (2) TP8/EP1 without KV offloading, concurrency sweep [4, 10];" and "c6 and c8 removed after sweep validation: dominated by TP4/EP4 arm." The new search-space (lines 1720-1722) is 3 arms: TP4/EP4 no-KV conc [1,2,4,6,8],…
1 participant
Summary / 摘要
Bump SGLang image from v0.5.16 to v0.5.19 for the GLM-5.2 FP4 MI355X agentic recipe and re-run the full Pareto sweep with 3 arms.
v0.5.19 includes:
Measured impact (pre-sweep, run 34333529180 on mi355x-amds_03)
Changes / 变更内容
configs/amd-master.yamlv0.5.16-rocm720-mi35x-20260728→v0.5.19-rocm720-mi35x-20260908TP4/EP4 no-KV: conc-list[1, 2, 4, 6, 8]TP8/EP1 no-KV: conc-list[1, 2, 4]TP4/EP4 HiCache: conc-list[1, 2, 4, 8, 10]Rationale for 3 arms: same structure as PR #2853 — covers GPU-resident KV regime (no-KV up to c8), interactivity-optimised TP8/EP1, and HiCache fallback for high concurrency.
Test plan / 测试计划
full-sweep-fail-fastlabel)progress.csv, update Pareto plotperf-changelog.yamlwith final Pareto values before mergeNote
Low Risk
Benchmark and changelog config only; no runtime application code or auth/data-path changes.
Overview
Updates the GLM-5.2 FP4 MI355X SGLang agentic-MTP benchmark to
lmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260908, which pulls in fused DSA metadata kernels (#37124) and related fixes noted in the config comments.The agentic-coding search space is expanded from the prior two-arm setup back to a three-arm Pareto sweep: TP4/EP4 without KV offload (conc 1–8), TP8/EP1 GPU-resident KV (conc 1–4), and TP4/EP4 HiCache DRAM offload (conc 1–10, with c12 dropped on that arm). Inline comments document the re-sweep rationale and expected throughput gains from the image bump.
perf-changelog.yamladds a matching entry forglm5.2-fp4-mi355x-sglang-agentic-mtp/ agentic-coding, citing measured norm_intvty_p90 improvements at c4/c6 and linking PR #3014.Reviewed by Cursor Bugbot for commit 6d0ef75. Bugbot is set up for automated code reviews on this repo. Configure here.