[AMD][AgentX] MINIMAX-M3 FP4 MI355X agentX vLLM disagg - #2326
Closed
charxwu wants to merge 4 commits into
Closed
Conversation
… offload Register minimaxm3-fp4-mi355x-vllm-disagg-agentic (1P1D TP8, conc-list [2, 4, 8, 16, 32]) on MI355X amd-aim: agentic trace replay, runtime Mooncake v0.3.11.post1 install, MultiConnector (MoRIIO + MooncakeStore), vLLM #46240 and batch32 patches, and ionic RDMA bind-mount for vllm-disagg. 中文：新增 MiniMax-M3 MXFP4 MI355X vLLM 分离式 agentic 配方（1P1D TP8，并发 [2, 4, 8, 16, 32]）：支持 agentic trace replay、job 运行时安装 Mooncake v0.3.11.post1、MultiConnector（MoRIIO + MooncakeStore）、vLLM #46240 与 batch32 patch，以及 vllm-disagg 的 ionic RDMA bind-mount。 Co-authored-by: Cursor <cursoragent@cursor.com>
charxwu
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
July 24, 2026 06:50
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
中文：将 perf-changelog 中的 pr-link 更新为 PR SemiAnalysisAI#2326。 Co-authored-by: Cursor <cursoragent@cursor.com>
Keep kimik2.5-fp4-mi355x-vllm-disagg (SemiAnalysisAI#2301) from main and append minimaxm3-fp4-mi355x-vllm-disagg-agentic (SemiAnalysisAI#2326) at the tail. 中文：解决 perf-changelog.yaml 与 main 的冲突，保留 main 上的 kimik2.5 条目并在末尾追加 MiniMax-M3 agentic 条目。 Co-authored-by: Cursor <cursoragent@cursor.com>
Comment on lines
+299
to
+307
| rm -rf "$mooncake_src" "$mooncake_stage" | ||
| git clone --depth 1 --branch "$mooncake_tag" --recurse-submodules \ | ||
| --shallow-submodules https://github.com/kvcache-ai/Mooncake.git "$mooncake_src" | ||
| cmake -S "$mooncake_src/extern/yalantinglibs" \ | ||
| -B "$mooncake_src/extern/yalantinglibs/build" \ | ||
| -DBUILD_EXAMPLES=OFF -DBUILD_BENCHMARK=OFF -DBUILD_UNIT_TESTS=OFF | ||
| cmake --build "$mooncake_src/extern/yalantinglibs/build" -j "$build_jobs" | ||
| cmake --install "$mooncake_src/extern/yalantinglibs/build" | ||
| cmake -S "$mooncake_src" -B "$mooncake_src/build" -G Ninja \ |
Collaborator
There was a problem hiding this comment.
thanks for the contribution, does ROCm have an prebuilt wheel for mooncake?
Collaborator
Author
There was a problem hiding this comment.
Thanks for the question. As of now there is no official ROCm/HIP Mooncake wheel on PyPI.
Collaborator
There was a problem hiding this comment.
thanks for the quick reply @charxwu what is the timeline for mooncake wheel? is there an ticket to track this?
+viz @chunfangamd
| # MiniMax-M3 MXFP4 MI355X vLLM disaggregated agentic (1P1D TP8, Mooncake DRAM KV offload). | ||
| # Mooncake v0.3.11.post1 is installed at job runtime by server_vllm.sh (not baked into image). | ||
| minimaxm3-fp4-mi355x-vllm-disagg-agentic: | ||
| image: rocm/vllm-dev:vllm-0.23.1-rocm723-mi35x-mori-0625 |
Collaborator
There was a problem hiding this comment.
is there an vllm upstream image
functionstackx
requested changes
Jul 24, 2026
Switch minimaxm3-fp4-mi355x-vllm-disagg-agentic from the Mori-specific rocm/vllm-dev image to vllm/vllm-openai-rocm nightly with MiniMax-M3 MXFP4 support (vllm#45794). 中文：将 MiniMax-M3 MI355X vLLM 分离式 agentic 基准配置的镜像从 Mori 专用 rocm/vllm-dev 切换为支持 MiniMax-M3 MXFP4 的 vLLM ROCm nightly 镜像。 Co-authored-by: Cursor <cursoragent@cursor.com>
Collaborator
|
@charxwu the AgentX/AIPerf harness has been updated, please merge origin/main into your branch and refresh your submission. Additional tuning may be necessary depending on the config. I apologize for any inconvenience. This is an automated message. |
Collaborator
|
stale PR & probably superceded by other PRs. please reopen when ready |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
minimaxm3-fp4-mi355x-vllm-disagg-agentictoamd-master.yaml: MiniMax-M3 MXFP4 on MI355X, 1P1D TP8 disaggregated vLLM with Mooncake DRAM KV offload, agentic-coding scenario,conc-list[2, 4, 8, 16, 32].benchmarks/multi_node/agentic/minimaxm3_fp4_mi355x_vllm-disagg.sh(CI-style sibling of the fixed-seq disagg recipe).server_vllm.shfor Mooncake runtime install (v0.3.11.post1), MultiConnector + MoRIIO P/D KV transfer, and MiniMax-M3 MXFP4 bring-up on ROCm.apply_vllm_46240_scheduler_patch.py— skip stale async KV recv/send completions (v1 P/D: _update_from_kv_xfer_finished AssertionError (kills EngineCore) when an aborted request's finished_recving + finished_sending land in the same step vllm-project/vllm#46240).apply_vllm_mooncake_transfer_batches.py— raise Mooncake max transfer batch keys (default 32 viaINFERENCEX_MOONCAKE_MAX_TRANSFER_BATCH_KEYS).job.slurmfor the new agentic vLLM-disagg path; appendperf-changelog.yamlentry.Image:
rocm/vllm-dev:vllm-0.23.1-rocm723-mi35x-mori-0625Model:
amd/MiniMax-M3-MXFP4KV offload: MooncakeStoreConnector (DRAM,
kv-offload-backend: mooncake v0.3.11.post1)中文说明
amd-master.yaml新增minimaxm3-fp4-mi355x-vllm-disagg-agentic：MI355X 上 MiniMax-M3 MXFP4 的 1P1D TP8 分离式 vLLM，配合 Mooncake DRAM KV 卸载，agentic-coding 场景，conc-list为 [2, 4, 8, 16, 32]。benchmarks/multi_node/agentic/minimaxm3_fp4_mi355x_vllm-disagg.sh（与 fixed-seq 分离式 recipe 对应的 CI 入口）。server_vllm.sh：运行时安装 Mooncake（v0.3.11.post1）、MultiConnector + MoRIIO P/D KV 传输，以及 ROCm 上 MiniMax-M3 MXFP4 的 bring-up。apply_vllm_46240_scheduler_patch.py— 跳过过期的 async KV recv/send 完成事件（v1 P/D: _update_from_kv_xfer_finished AssertionError (kills EngineCore) when an aborted request's finished_recving + finished_sending land in the same step vllm-project/vllm#46240）。apply_vllm_mooncake_transfer_batches.py— 提高 Mooncake max transfer batch keys（默认 32，INFERENCEX_MOONCAKE_MAX_TRANSFER_BATCH_KEYS）。job.slurm以支持 agentic vLLM-disagg 路径；在perf-changelog.yaml末尾追加 changelog 条目。镜像：
rocm/vllm-dev:vllm-0.23.1-rocm723-mi35x-mori-0625模型：
amd/MiniMax-M3-MXFP4KV 卸载： MooncakeStoreConnector（DRAM，
kv-offload-backend: mooncake v0.3.11.post1）