Skip to content

Update MiniMax M3 MXFP4 AITER command #609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
haic0 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update MiniMax M3 MXFP4 AITER command #609

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions models/MiniMaxAI/MiniMax-M3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -456,13 +456,19 @@ guide: |
or launch the tested TP8 baseline directly:

```bash
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_MOE=1
export VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1
Comment on lines +459 to +461

@gemini-code-assist gemini-code-assist Bot Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The command uses the $MAX_MODEL_LEN environment variable on line 469, but it is not defined or initialized anywhere in this bash snippet. If a user copy-pastes and runs this block directly, the variable will evaluate to an empty string, causing the vllm serve command to fail with a parsing error.

Please define MAX_MODEL_LEN in the environment setup lines above the command.

  export VLLM_ROCM_USE_AITER=1
  export VLLM_ROCM_USE_AITER_MOE=1
  export VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1
  export MAX_MODEL_LEN=131072

@tjtanaa tjtanaa Jul 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haic0 please address the concern of gemini. It is valid.


vllm serve amd/MiniMax-M3-MXFP4 \
--tensor-parallel-size 8 \
--trust-remote-code \
--block-size 128 \
--no-enable-prefix-caching \
--language-model-only \
--max-model-len "$MAX_MODEL_LEN" \
--attention-backend TRITON_ATTN \
--mm-encoder-tp-mode data \

@tjtanaa tjtanaa Jul 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we not want to support image?

--mm-encoder-attn-backend ROCM_AITER_FA \
--moe-backend aiter \
Comment on lines +467 to +471

@gemini-code-assist gemini-code-assist Bot Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The updated guide command now uses the text-only launch flags (--language-model-only, --no-enable-prefix-caching) and the ROCm AITER MoE path (--moe-backend aiter), and removes the multimodal encoder flags (--mm-encoder-tp-mode data, --mm-encoder-attn-backend ROCM_AITER_FA).

However, the mxfp4 variant definition under variants.mxfp4.hardware_overrides.mi355x (lines 123-132) still contains the old multimodal encoder arguments and lacks the new environment variables and arguments:

    hardware_overrides:
      mi355x:
        docker_image: "vllm/vllm-openai-rocm:nightly"
        extra_args:
          - "--trust-remote-code"
          # Keep the multimodal encoder loaded and run it data-parallel on MI355X.
          - "--mm-encoder-tp-mode"
          - "data"
          - "--mm-encoder-attn-backend"
          - "ROCM_AITER_FA"

To ensure that the automated command builder/UI generates the correct command for the mxfp4 variant, please update the variant definition in lines 123-132 to match the new guide command:

    hardware_overrides:
      mi355x:
        docker_image: "vllm/vllm-openai-rocm:nightly"
        extra_args:
          - "--trust-remote-code"
          - "--no-enable-prefix-caching"
          - "--language-model-only"
          - "--moe-backend"
          - "aiter"
        extra_env:
          VLLM_ROCM_USE_AITER: "1"
          VLLM_ROCM_USE_AITER_MOE: "1"
          VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS: "1"

@tjtanaa tjtanaa Jul 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gemini feedback here is also valid. Please comment on this @haic0

--tool-call-parser minimax_m3 \
--enable-auto-tool-choice \
--reasoning-parser minimax_m3
Expand Down