[ROCm] Support MiniMax-M3 NVFP4 SwiGLU-OAI

[wangjiaxin99]

MiniMax-M3 NVFP4 ROCm SwiGLU-OAI Support

Summary

• Allow the NVFP4 MoE emulation backend to be selected for models that set swiglu_limit.
• Thread MiniMax-M3 swiglu_alpha and swiglu_beta metadata from Quark MoE layers into FusedMoEQuantConfig as gemm1_alpha and gemm1_beta.
• Preserve existing behavior for backends and models that do not provide SwiGLU-OAI activation parameters.

Motivation

MiniMax-M3 uses a clamped SwiGLU-OAI activation in its MoE experts. On ROCm, Quark NVFP4 checkpoints rely on the NVFP4 emulation backend rather than native NVIDIA FP4 kernels. Before this change, models with swiglu_limit filtered the emulation backend out of the candidate backend list before support probing, so ROCm could not fall back to the path that can apply the activation parameters.

The Quark NVFP4 path also did not pass swiglu_limit, swiglu_alpha, or swiglu_beta from the layer into the shared NVFP4 MoE quant config helper. As a result, the execution backend could not receive the activation metadata needed to reproduce MiniMax-M3's MoE formula.

Implementation

The change updates the NVFP4 MoE backend oracle to treat NvFp4MoeBackend.EMULATION as clamp-capable. This lets ROCm deployments with swiglu_limit reach the emulation backend during normal backend selection.

The shared nvfp4_moe_quant_config helper now accepts optional gemm1_alpha and gemm1_beta scalar parameters and stores them in FusedMoEQuantConfig. The EMULATION branch of make_nvfp4_moe_quant_config maps layer-level swiglu_alpha and swiglu_beta to these gemm1_* fields, matching where the activation is applied: after the first MoE GEMM and before the down projection.

The Quark NVFP4 MoE method now reads swiglu_limit, swiglu_alpha, and swiglu_beta from the layer with getattr(..., None) and forwards them into the shared helper. Models without those attributes continue to use the existing None defaults.

Support Model and Accuracy

https://huggingface.co/amd/MiniMax-M3-NVFP4

Note

This is scoped to NVFP4 MoE quant config construction and Quark's NVFP4 MoE method. Existing models that do not define SwiGLU-OAI attributes continue passing None, preserving prior behavior. The main behavioral change is that EMULATION can now be selected for clamped SwiGLU NVFP4 MoE models.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[tjtanaa]

@wangjiaxin99 please provide the vllm serve command and gsm8k numshot 20 score. numshot 20 and high concurrency is to ensure that the sparse indexer logic is correct.

#!/bin/bash

MODEL=<MODEL>
NUM_FEWSHOT=${NUM_FEWSHOT:-20}
RESULT_TAG=${RESULT_TAG:-minimaxm3_numshot${NUM_FEWSHOT}_cc256}
lm_eval --model local-completions --model_args model=$MODEL,base_url=http://0.0.0.0:8001/v1/completions,num_concurrent=256,max_retries=10,max_gen_toks=2048,max_length=1048576,timeout=60000,trust_remote_code=True,temperature=0 --batch_size auto --tasks gsm8k --num_fewshot "${NUM_FEWSHOT}" \
  --output_path "./results_${RESULT_TAG}" \
  --log_samples \
| tee "lmeval_${RESULT_TAG}.log"

[wangjiaxin99]

@wangjiaxin99 please provide the vllm serve command and gsm8k numshot 20 score. numshot 20 and high concurrency is to ensure that the sparse indexer logic is correct.

#!/bin/bash

MODEL=<MODEL>
NUM_FEWSHOT=${NUM_FEWSHOT:-20}
RESULT_TAG=${RESULT_TAG:-minimaxm3_numshot${NUM_FEWSHOT}_cc256}
lm_eval --model local-completions --model_args model=$MODEL,base_url=http://0.0.0.0:8001/v1/completions,num_concurrent=256,max_retries=10,max_gen_toks=2048,max_length=1048576,timeout=60000,trust_remote_code=True,temperature=0 --batch_size auto --tasks gsm8k --num_fewshot "${NUM_FEWSHOT}" \
  --output_path "./results_${RESULT_TAG}" \
  --log_samples \
| tee "lmeval_${RESULT_TAG}.log"

Sure, here is the command I used. @tjtanaa

The vllm serve command matches the one documented in the model card:

vllm serve /mnt/amd/MiniMax-M3-NVFP4 \
  --trust-remote-code \
  --block-size 128 \
  --tensor-parallel-size 4 \
  --attention-backend TRITON_ATTN \
  --mm-encoder-tp-mode data \
  --mm-encoder-attn-backend ROCM_AITER_FA \
  --tool-call-parser minimax_m3 \
  --enable-auto-tool-choice \
  --reasoning-parser minimax_m3

The GSM8K accuracy for the regular evaluation is already reported in the model card:
https://huggingface.co/amd/MiniMax-M3-NVFP4#accuracy

I also ran the requested GSM8K 20-shot evaluation with high concurrency:

#!/bin/bash

MODEL=/mnt/amd/MiniMax-M3-NVFP4
NUM_FEWSHOT=${NUM_FEWSHOT:-20}
RESULT_TAG=${RESULT_TAG:-minimaxm3_numshot${NUM_FEWSHOT}_cc256}

lm_eval --model local-completions \
  --model_args model=$MODEL,base_url=http://0.0.0.0:8000/v1/completions,num_concurrent=256,max_retries=10,max_gen_toks=2048,max_length=1048576,timeout=60000,trust_remote_code=True,temperature=0 \
  --batch_size auto \
  --tasks gsm8k \
  --num_fewshot "${NUM_FEWSHOT}" \
  --output_path "./results_${RESULT_TAG}" \
  --log_samples \
  | tee "lmeval_${RESULT_TAG}.log"

Result:

Requesting API: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 1319/1319 [04:35<00:00,  4.79it/s]
fatal: not a git repository (or any of the parent directories): .git
2026-07-21:07:26:36 INFO     [loggers.evaluation_tracker:247] Saving results aggregated
2026-07-21:07:26:36 INFO     [loggers.evaluation_tracker:119] Saving per-task samples to results_minimaxm3_numshot20_cc256/__mnt__amd__MiniMax-M3-NVFP4/*.jsonl
local-completions ({'model': '/mnt/amd/MiniMax-M3-NVFP4', 'base_url': 'http://0.0.0.0:8000/v1/completions', 'num_concurrent': 256, 'max_retries': 10, 'max_gen_toks': 2048, 'max_length': 1048576, 'timeout': 60000, 'temperature': 0}), gen_kwargs: ({}), limit: None, num_fewshot: 20, batch_size: auto
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|    20|exact_match|↑  |0.9280|±  |0.0071|
|     |       |strict-match    |    20|exact_match|↑  |0.9287|±  |0.0071|

Please let me know if you would like me to run any additional settings.

[fxmarty-amd]

LGTM

[wangjiaxin99]

Hi @tjtanaa , thanks again for the guidance.

I’ve added the requested vllm serve command and GSM8K 20-shot high-concurrency result in the earlier comment. When you get a chance, could you please let me know if there is anything else needed from my side for the review?

Thanks!

[BowenBao]

LGTM thanks, @tjtanaa ptal

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wangjiaxin99.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[tjtanaa]

Thank you for pinging @wangjiaxin99 . LGTM.

[github-actions]

✅ @wangjiaxin99, CI is now available for this PR.

• /ci run starts upstream CI; /amd-ci run starts AMD CI only.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
• /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

[tjtanaa]

/ci run

[github-actions]

✅ Triggered Buildkite CI #84581 for commit 617dec5a1190.

[mergify]

Hi @wangjiaxin99, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[tjtanaa]

@wangjiaxin99 can you fix the pre-commit

[BowenBao]

duplicated from L503 above

[BowenBao]

maybe it's fixed in another PR? now the only thing new is unittest, the rest seems duplicated after solving merge conflict.

[mergify]

Hi @wangjiaxin99, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[mergify]

Hi @wangjiaxin99, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[wangjiaxin99]

I will close this PR since this issue has been fixed in #47106

[wangjiaxin99]

maybe it's fixed in another PR? now the only thing new is unittest, the rest seems duplicated after solving merge conflict.

You're right, this PR ended up duplicating changes that had already been merged. I verified that there wasn't any additional functional change beyond what's already in the codebase, so I've closed the PR. Thanks for pointing it out.

[wangjiaxin99]

@wangjiaxin99 can you fix the pre-commit

Thanks for the heads-up. I checked and found that this PR was duplicating changes that had already been merged, so I've closed it. Appreciate you taking a look!