[Klaud Cold] Muse Spark Open MI355X: HiCache DRAM offload above conc 8 / [Klaud Cold] Muse Spark Open MI355X：并发大于 8 时启用 HiCache DRAM 卸载

[functionstackx]

Summary

Muse Spark Open MXFP4 on MI355X (SGLang + DFlash2, from #4127): add SGLang HiCache CPU DRAM KV offloading above concurrency 8.

• Conc 12 and 16 at TP4/EP4 and TP8/EP8 now set enable-hierarchical-cache, hicache-size: 180 GB per rank and hicache-write-policy: write_through, giving 720 GB at TP4 and 1.44 TB at TP8. These match the GLM-5.2 MXFP4 MI355X SGLang sibling.
• dram-utilization: 0.85 puts the matrix DRAM budget at 1274 GB for TP4 and 2549 GB for TP8, so both tiers fit within the AgentX proportional limit of 3 TB per server.
• Conc 1, 4 and 8 stay GPU-resident (kv-offloading: none). The image, serve flags, DFlash2 draft and golden AL are unchanged.
• The search space is still 10 points: 6 GPU-resident and 4 HiCache.

Context: the last #4127 sweep lost its TP8/EP8 c12 point to a host_setup failure on node mia1-p01-g14, not to the recipe. This PR's sweep re-measures every point.

AI model disclosure

• Model/version: claude-opus-5-5[1m] (Claude Opus 5.5, via Claude Code)
• Role: wrote the recipe, config and changelog change, validated the matrix, opened this PR and is monitoring the sweep.

PR checklist and change type

Type of Change

• Configuration change

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• One new perf-changelog entry is appended for the affected config key
• One primary sweep label: full-sweep-fail-fast

中文

摘要

为 MI355X 上的 Muse Spark Open MXFP4（SGLang + DFlash2，来自 #4127）在并发大于 8 时启用 SGLang HiCache CPU DRAM KV 卸载。

• TP4/EP4 与 TP8/EP8 的并发 12 和 16 点新增 enable-hierarchical-cache、每 rank hicache-size: 180 GB 及 hicache-write-policy: write_through，TP4 合计 720 GB，TP8 合计 1.44 TB，与 GLM-5.2 MXFP4 MI355X SGLang 同类配置一致。
• dram-utilization: 0.85 使矩阵 DRAM 预算为 TP4 1274 GB、TP8 2549 GB，两者均在 AgentX 每台服务器 3 TB 的按比例上限之内。
• 并发 1、4、8 仍保持 KV 常驻 GPU（kv-offloading: none）。镜像、启动参数、DFlash2 draft 和黄金 AL 均不变。
• 搜索空间仍为 10 个点：6 个常驻 GPU，4 个使用 HiCache。

背景：#4127 上一次 sweep 中 TP8/EP8 c12 点失败的原因是节点 mia1-p01-g14 的 host_setup 故障，与配方无关。本 PR 的 sweep 会重新测量所有点。

AI 模型使用说明

• 模型/版本：claude-opus-5-5[1m]（Claude Opus 5.5，通过 Claude Code）
• 工作内容：编写配方、配置与 changelog 修改，校验矩阵，创建本 PR 并监控 sweep。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

Findings marked 🟡 are optional suggestions and need no follow-up push.

[claude]

🔴 Every srt-slurm job, not only Muse Spark, would run on srt-slurm v2.51.1 again, undoing the v2.59.1 upgrade from #4167. The pointer goes back from ae3d687 to 545233a0. That commit is the exact v2.51.1 pin #4167 replaced, so this looks like a stale-branch merge artifact. It drops NVIDIA/srt-slurm#584, #573 and #602, and #4167 also deleted the #602 backport patch, so nothing restores that fix. Fix: reset the submodule to ae3d6877c46f3147e3a5dafa46ee4cd89322e52f so this PR only changes the recipe, config and changelog.

Why this was flagged

Commit 22b39af (#4167) moved inferencex-e2e/utils/srt-slurm from v2.51.1 (545233a0) to v2.59.1 (ae3d6877). It also deleted the #602 backport patch, and runners/srt-slurm/patches/ now holds only 573-participating-gpus.patch. This diff moves the gitlink back to 545233a0d36e7d21cf74bb01a4365475c9214003, and the PR description gives no reason. After merge, every srt-slurm launch on every cluster checks out v2.51.1. SGLang attention-DP workers sharing a node lose the per-DP-rank KV-event port reservation from NVIDIA/srt-slurm#584 and can collide on base port + DP rank again. The #602 fix is also gone, and no patch replaces it. On the base branch all of these fixes are present. The HiCache recipe change does not need this bump.

Verification: Trigger: any srt-slurm launch after merge, on every cluster, because the launchers check out the shared submodule pin. The diff changes inferencex-e2e/utils/srt-slurm from Subproject commit ae3d6877c46f3147e3a5dafa46ee4cd89322e52f to 545233a0d36e7d21cf74bb01a4365475c9214003. So this PR rolls the shared pin back to the exact v2.51.1 commit that #4167 replaced.

[claude]

🔴 The supported merge script will fail on this PR because the new changelog entry uses pr-link: TBD, a placeholder the tooling does not accept. compare_entries only allows XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or the PR's own link (validate_perf_changelog.py:147). Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, the form CONTRIBUTING.md:206 documents, or to this PR's canonical URL. Then canonicalize_appended_links can rewrite it at merge.

Why this was flagged

The new block ends with pr-link: TBD at perf-changelog.yaml:10045. PR_LINK_PLACEHOLDERS holds only "XXX" and "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX" (validate_perf_changelog.py:22-25). validate_added_pr_link raises "new PR entry must use ... or an XXX placeholder; found 'TBD'" for any other value (validate_perf_changelog.py:147-150). merge_with_reuse.py:631 calls canonicalize_changelog, then canonicalize_appended_links and compare_entries(..., pr_number) (prepare_perf_changelog_merge.py:88). It does this after it has already posted /reuse-sweep-run and merged origin/main. Nothing catches the error at merge_with_reuse.py:477-488, so the documented merge path aborts. resolve_changelog_conflict takes the same compare_entries path (prepare_perf_changelog_merge.py:146). PR CI does not catch this: validate_perf_changelog.main() only runs the planner (lines 330-336), never compare_entries, so the sweep goes green and the failure shows up only at merge time.

Verification: Trigger: the PR is merged through the supported merge script, merge_with_reuse. The diff appends a block ending in pr-link: TBD at inferencex-e2e/perf-changelog.yaml:10045. merge_with_reuse.py:631 calls canonicalize_changelog(pr, repo). The PR-time CI check in run-sweep.yml:97-117 (validate_matrix_compatible_change) does not call compare_entries, so it will not flag this.

[claude]

🟡 nit (optional): Both new comments break the AGENTS.md comment rules. Each is an overview placed above a section and restates values already in the code. The recipe comment above the overrides at agentic.yaml:53-54 repeats hicache-size: 180 and multiplies it into totals. The rewritten master-config comment above musespark-fp4-mi355x-sglang-agentic-dflash (amd-master.yaml:1607-1608) restates the conc split and per-rank size. Fix: delete both, or replace each with one technical line on a specific value, such as why 180 GB per rank was chosen.

Why this was flagged

AGENTS.md lines 64-67 apply to every recipe and config. They forbid header comments above a recipe, override or section, and they forbid comments that restate the code. The comment added at agentic.yaml:53-54 sits above the whole override list and restates hicache-size: 180 times TP. The rewritten comment at amd-master.yaml:1607-1608 sits above the config key and describes its topology and conc split. That information is already in the search-space rows. Maintainers end up with comments that drift from the values they repeat, and the base branch does not have this duplication in the recipe.

Verification: Nothing breaks at runtime. Both comments break the repository's mandatory comment rules in AGENTS.md:60-67. In agentic.yaml:53-54 the diff adds a comment above the whole override list, not next to a value, that restates hicache-size: 180 and the conc split. The comment at amd-master.yaml:1607-1608 sits above the config key and restates the search-space split and the recipe's hicache-size.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38947852942

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38947852942

[functionstackx]

/use 38947852942

[github-actions]

@functionstackx staged run 38947852942.

• Muse-Spark-Open · agentic-traces · FP4 · mi355x_sglang · 2026-10-16

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow