Skip to content

[AMD] Update MiniMax-M3 FP8 MI355X ATOM image and serving args (0630) #1968

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Oseltamivir merged 2 commits into from
Jul 1, 2026
Merged

[AMD] Update MiniMax-M3 FP8 MI355X ATOM image and serving args (0630) #1968

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e0a3238
[AMD] Update MiniMax-M3 FP8 MI355X ATOM image and serving args
seungrokj Jul 1, 2026
9f4198d
perf-changelog: update PR link for #1968
seungrokj Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2733,7 +2733,7 @@ minimaxm3-fp4-mi355x-atom-mtp:
- { tp: 4, conc-start: 1, conc-end: 256, spec-decoding: mtp }

minimaxm3-fp8-mi355x-atom:
image: rocm/atom-dev:MiniMax-M3-20260623
image: rocm/atom-dev:MiniMax-M3-20260630
model: MiniMaxAI/MiniMax-M3-MXFP8
model-prefix: minimaxm3
runner: mi355x
Expand All @@ -2752,7 +2752,7 @@ minimaxm3-fp8-mi355x-atom:
- { tp: 4, conc-start: 1, conc-end: 256 }

minimaxm3-fp8-mi355x-atom-mtp:
image: rocm/atom-dev:MiniMax-M3-20260623
image: rocm/atom-dev:MiniMax-M3-20260630
model: MiniMaxAI/MiniMax-M3-MXFP8
model-prefix: minimaxm3
runner: mi355x
Expand Down
5 changes: 3 additions & 2 deletions benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_mi355x_atom.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@ if [ "$DP_ATTENTION" = "true" ]; then
fi

SPEC_ARGS=()
OPT_ARGS=(--online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "vision_tower", "multi_modal_projector", "patch_merge_mlp", "*block_sparse_moe"]}' --hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}')

# Start GPU monitoring (power, temperature, clocks every second)
start_gpu_monitor
MEM_FRAC_STATIC=0.8

set -x
export AITER_QUICK_REDUCE_QUANTIZATION=INT4
export AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0
export ATOM_M3_SPARSE_USE_ASM_PA=1
export ATOM_FORCE_ATTN_TRITON=1
export MAX_MODEL_LEN=32768
export MAX_NUM_BATCHED_TOKENS=32768
export MAX_NUM_SEQS=256
Expand All @@ -48,6 +48,7 @@ python3 -m atom.entrypoints.openai_server \
--server-port $PORT \
"${PARALLEL_ARGS[@]}" \
"${SPEC_ARGS[@]}" \
"${OPT_ARGS[@]}" \
--block-size 128 \
--gpu-memory-utilization $MEM_FRAC_STATIC \
--max-model-len $MAX_MODEL_LEN \
Expand Down
5 changes: 3 additions & 2 deletions benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_mi355x_atom_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@ if [ "$DP_ATTENTION" = "true" ]; then
fi

SPEC_ARGS=(--method eagle3 --draft-model Inferact/MiniMax-M3-EAGLE3 --num-speculative-tokens 3 )
OPT_ARGS=(--online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "vision_tower", "multi_modal_projector", "patch_merge_mlp", "*block_sparse_moe"]}' --hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}')

# Start GPU monitoring (power, temperature, clocks every second)
start_gpu_monitor
MEM_FRAC_STATIC=0.8

set -x
export AITER_QUICK_REDUCE_QUANTIZATION=INT4
export AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0
export ATOM_M3_SPARSE_USE_ASM_PA=1
export ATOM_FORCE_ATTN_TRITON=1
export MAX_MODEL_LEN=32768
export MAX_NUM_BATCHED_TOKENS=32768
export MAX_NUM_SEQS=256
Expand All @@ -48,6 +48,7 @@ python3 -m atom.entrypoints.openai_server \
--server-port $PORT \
"${PARALLEL_ARGS[@]}" \
"${SPEC_ARGS[@]}" \
"${OPT_ARGS[@]}" \
--block-size 128 \
--gpu-memory-utilization $MEM_FRAC_STATIC \
--max-model-len $MAX_MODEL_LEN \
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4351,3 +4351,12 @@
- "Use nvidia/MiniMax-M3-NVFP4 from /scratch/models/MiniMax-M3-NVFP4 with vllm/vllm-openai:vllm-minimax-m3-perf-x86_64-13.0.1-8b00f41, which includes vllm-project/vllm PR #46380; no runtime patch needed"
- "Reuse the existing MXFP8 B300 topology and concurrency matrix across 15 srt-slurm recipes, while dropping the FP8-only Marlin override from TP4 decode"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1966

- config-keys:
- minimaxm3-fp8-mi355x-atom
- minimaxm3-fp8-mi355x-atom-mtp
description:
- "Bump image to rocm/atom-dev:MiniMax-M3-20260630 for both fp8 atom entries"
- "Add OPT_ARGS: pass --online_quant_config '{\"global_quant_config\": \"ptpc_fp8\", \"exclude_layer\": [\"lm_head\", \"model.embed_tokens\", \"vision_tower\", \"multi_modal_projector\", \"patch_merge_mlp\", \"*block_sparse_moe\"]}' and --hf-overrides '{\"use_index_cache\": true, \"index_topk_freq\": 4}' to both scripts"
- "Replace AITER_QUICK_REDUCE_CAST_BF16_TO_FP16=0 and ATOM_M3_SPARSE_USE_ASM_PA=1 with ATOM_FORCE_ATTN_TRITON=1"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1968

@claude claude Bot Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new perf-changelog entry uses pull/PLACEHOLDER as its pr-link, but utils/validate_perf_changelog.py only accepts either the canonical pull/1968 or the literal placeholder XXX (PR_LINK_PLACEHOLDERS at lines 24-27). This will fail the changelog validation gate — replace PLACEHOLDER with XXX (auto-canonicalized to pull/1968 on merge) or with 1968 directly.

Extended reasoning...

What the bug is

perf-changelog.yaml:4362 sets:

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER

utils/validate_perf_changelog.py defines the accepted pre-merge placeholders at lines 24-27:

PR_LINK_PLACEHOLDERS = {
    "XXX",
    "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX",
}

validate_added_pr_link() (lines 144-160) requires the link to be either the canonical https://github.com/SemiAnalysisAI/InferenceX/pull/{pr_number} or a member of PR_LINK_PLACEHOLDERS. The value .../pull/PLACEHOLDER is neither, so the check raises ChangelogValidationError.

Where it triggers

  • utils/merge_with_reuse.sh calls prepare_perf_changelog_merge.py (both resolve-conflict and canonicalize subcommands), which invoke compare_entries() → validate_added_pr_link(). canonicalize_appended_links() also explicitly checks if link not in PR_LINK_PLACEHOLDERS. Since merge_with_reuse.sh is the standard merge path, this blocks merging.
  • Convention across the file (see e.g. entries at lines 4337/4345/4353 with real PR numbers, and other in-flight entries with XXX) confirms that XXX is the accepted pre-merge placeholder — a real PR number is baked in on merge.

Step-by-step proof

  1. On merge, merge_with_reuse.sh runs prepare_perf_changelog_merge.py canonicalize.
  2. It reads the new entry with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER.
  3. validate_added_pr_link() computes canonical = 'https://github.com/SemiAnalysisAI/InferenceX/pull/1968'.
  4. It checks link == canonical → False (PLACEHOLDER ≠ 1968).
  5. It checks link in PR_LINK_PLACEHOLDERS = {'XXX', '.../pull/XXX'} → False.
  6. The check at line 156 raises ChangelogValidationError: new PR entry must use 'https://github.com/SemiAnalysisAI/InferenceX/pull/1968' or an XXX placeholder; found 'https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER'.

How to fix

Replace PLACEHOLDER with the literal string XXX:

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX

The canonicalize step will rewrite it to .../pull/1968 at merge time. Alternatively, bake in 1968 directly.