Skip to content

[AMD] Update MiniMax-M3 FP4 MI355X ATOM image and serving args (0630) #1967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Oseltamivir merged 4 commits into from
Jul 1, 2026
Merged

[AMD] Update MiniMax-M3 FP4 MI355X ATOM image and serving args (0630) #1967

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
027f08a
[AMD] Update MiniMax-M3 FP4 MI355X ATOM image and serving args
seungrokj Jul 1, 2026
064cba8
perf-changelog: update PR link for #1967
seungrokj Jul 1, 2026
06c19b1
Merge branch 'main' into amd/m3_atom_pd_fp4_0630
Oseltamivir Jul 1, 2026
009b63d
cleanup perfchangelog
Oseltamivir Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2695,7 +2695,7 @@ minimaxm3-fp4-mi355x-vllm-mtp:
# https://github.com/ROCm/ATOM/blob/5d42d49f9e4292e5b61475917e92e7ec1b1dacb7/recipes/MiniMax-M3.md
# block size 128 is mandatory for MSA. TP4 on a single gfx950 node, per the recipe.
minimaxm3-fp4-mi355x-atom:
image: rocm/atom-dev:MiniMax-M3-20260623
image: rocm/atom-dev:MiniMax-M3-20260630
model: amd/MiniMax-M3-MXFP4
model-prefix: minimaxm3
runner: mi355x
Expand All @@ -2714,7 +2714,7 @@ minimaxm3-fp4-mi355x-atom:
- { tp: 4, conc-start: 1, conc-end: 256 }

minimaxm3-fp4-mi355x-atom-mtp:
image: rocm/atom-dev:MiniMax-M3-20260623
image: rocm/atom-dev:MiniMax-M3-20260630
model: amd/MiniMax-M3-MXFP4
model-prefix: minimaxm3
runner: mi355x
Expand Down
5 changes: 3 additions & 2 deletions benchmarks/single_node/fixed_seq_len/minimaxm3_fp4_mi355x_atom.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@ if [ "$DP_ATTENTION" = "true" ]; then
fi

SPEC_ARGS=()
OPT_ARGS=(--online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "vision_tower", "multi_modal_projector", "patch_merge_mlp", "*block_sparse_moe"]}' --hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}')

# Start GPU monitoring (power, temperature, clocks every second)
start_gpu_monitor
MEM_FRAC_STATIC=0.8

set -x
export AITER_QUICK_REDUCE_QUANTIZATION=INT4
export AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0
export ATOM_M3_SPARSE_USE_ASM_PA=1
export ATOM_FORCE_ATTN_TRITON=1
export MAX_MODEL_LEN=32768
export MAX_NUM_BATCHED_TOKENS=32768
export MAX_NUM_SEQS=256
Expand All @@ -48,6 +48,7 @@ python3 -m atom.entrypoints.openai_server \
--server-port $PORT \
"${PARALLEL_ARGS[@]}" \
"${SPEC_ARGS[@]}" \
"${OPT_ARGS[@]}" \
--block-size 128 \
--gpu-memory-utilization $MEM_FRAC_STATIC \
--max-model-len $MAX_MODEL_LEN \
Expand Down
5 changes: 3 additions & 2 deletions benchmarks/single_node/fixed_seq_len/minimaxm3_fp4_mi355x_atom_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@ if [ "$DP_ATTENTION" = "true" ]; then
fi

SPEC_ARGS=(--method eagle3 --draft-model Inferact/MiniMax-M3-EAGLE3 --num-speculative-tokens 3 )
OPT_ARGS=(--online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "vision_tower", "multi_modal_projector", "patch_merge_mlp", "*block_sparse_moe"]}' --hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}')

# Start GPU monitoring (power, temperature, clocks every second)
start_gpu_monitor
MEM_FRAC_STATIC=0.8

set -x
export AITER_QUICK_REDUCE_QUANTIZATION=INT4
export AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0
export ATOM_M3_SPARSE_USE_ASM_PA=1
export ATOM_FORCE_ATTN_TRITON=1
export MAX_MODEL_LEN=32768
export MAX_NUM_BATCHED_TOKENS=32768
export MAX_NUM_SEQS=256
Expand All @@ -48,6 +48,7 @@ python3 -m atom.entrypoints.openai_server \
--server-port $PORT \
"${PARALLEL_ARGS[@]}" \
"${SPEC_ARGS[@]}" \
"${OPT_ARGS[@]}" \
--block-size 128 \
--gpu-memory-utilization $MEM_FRAC_STATIC \
--max-model-len $MAX_MODEL_LEN \
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4351,3 +4351,12 @@
- "Use nvidia/MiniMax-M3-NVFP4 from /scratch/models/MiniMax-M3-NVFP4 with vllm/vllm-openai:vllm-minimax-m3-perf-x86_64-13.0.1-8b00f41, which includes vllm-project/vllm PR #46380; no runtime patch needed"
- "Reuse the existing MXFP8 B300 topology and concurrency matrix across 15 srt-slurm recipes, while dropping the FP8-only Marlin override from TP4 decode"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1966

- config-keys:
- minimaxm3-fp4-mi355x-atom
- minimaxm3-fp4-mi355x-atom-mtp
description:
- "Bump image to rocm/atom-dev:MiniMax-M3-20260630 for both fp4 atom entries"
- "Add OPT_ARGS: pass --online_quant_config '{\"global_quant_config\": \"ptpc_fp8\", \"exclude_layer\": [\"lm_head\", \"model.embed_tokens\", \"vision_tower\", \"multi_modal_projector\", \"patch_merge_mlp\", \"*block_sparse_moe\"]}' and --hf-overrides '{\"use_index_cache\": true, \"index_topk_freq\": 4}' to both scripts"
- "Replace AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0 and ATOM_M3_SPARSE_USE_ASM_PA=1 with ATOM_FORCE_ATTN_TRITON=1"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER

Check failure on line 4362 in perf-changelog.yaml

View check run for this annotation

Claude / Claude Code Review

perf-changelog pr-link left as PLACEHOLDER 

The new perf-changelog entry appended for this PR sets `pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER` — a literal, un-substituted token. The changelog validator (`utils/validate_perf_changelog.py`) only accepts the canonical `/pull/<digits>` URL or the sentinel `XXX` / `.../pull/XXX`; `PLACEHOLDER` is neither, so `utils/prepare_perf_changelog_merge.py` will raise `appended entry N has unexpected pr-link` and block merge. Replace it with `1967` (or `XXX` to hit the auto-