[ischencheng]

Overview

MiniMax-M3.1-Flash fails to load on Hopper unless --moe-backend marlin is passed:

ValueError: Per-token NVFP4 activation scales are not supported by HUMMING.

Its NVFP4 experts always declare a per-token FC1 scheme (#64111). On SM90 the auto selection puts Humming first (#56997), and Humming runs NVFP4 experts as W4A16 there. make_nvfp4_moe_kernel already drops the per-token metadata for Marlin W4A16. This PR does the same for Humming when its quant config has no activation dtype.

The recipe's Hopper serve command pins --moe-backend marlin and already loads. Its Hopper strategy didn't pass that flag, so this PR adds it there, and the Hopper notes now say the experts run as W4A16 on Humming when no MoE backend is given. No open PR or issue covers this.

Claims

• MiniMax-M3.1-Flash's NVFP4 experts load and run on SM90 with the default MoE backend (Humming W4A16) or an explicit --moe-backend humming.
• Configs that loaded before are unchanged. Marlin keeps the same condition, and Humming with quantized activations still raises.

Validation

New test tests/kernels/moe/test_minimax_m3_hopper_nvfp4_moe.py, on 1x H100. It builds the experts the way M3.1 does (ModelOptMxFp8Config with NVFP4 MoE, qat_activation_key=kNvfp4DynamicToken), loads a random NVFP4 checkpoint, and compares the forward against a dequantized reference. It runs in a new H100 CI step.

case	main + new test only	this PR
[humming]	ValueError: Per-token NVFP4 activation scales are not supported by HUMMING.	passed
[marlin]	passed	passed

Real weights through LLM on 1x H200: MiniMax-M3.1-Flash cut to its first 4 layers (3 dense, 1 NVFP4 MoE; hf_overrides={"text_config": {"num_hidden_layers": 4}}, only the 3 shards holding them on disk), TP1, the recipe's Hopper flags, greedy 16 tokens with top-5 logprobs on 4 prompts. Same nightly 2b009fc with main 564ac0d's Python on top, the fixed run plus this PR:

source	MoE backend	load	greedy tokens vs Marlin	max logprob diff on those tokens
main	auto, picks HUMMING	fails with the error above	-	-
this PR	auto, picks HUMMING	ok	64 / 64 same	0.061
this PR	--moe-backend marlin	ok	reference	-

A 4-layer cut doesn't produce text (both arms repeat one token), so this run checks real-weight loading and that the two W4A16 backends agree.

Decode on the full layer count, 1x H200 with the MiniMax-M3.1-Flash config cut to one TP8 rank's shapes (8 q heads, 1 KV and index head, FFN widths and vocab / 8, all 60 layers and 128 experts, dummy weights) and the recipe's Hopper flags. Both rows run nightly 2b009fc with main 564ac0d's Python on top, the second one plus this PR:

source	MoE backend	load	decode C1	decode C8
main	auto, picks HUMMING	fails with the error above	-	-
this PR	auto, picks HUMMING	ok	5.33 ms/token	7.93 ms/token
this PR	--moe-backend marlin	ok	5.18 ms/token	6.74 ms/token

Decode is 4K-token prompts, 192 output tokens, greedy. Dummy weights make the outputs meaningless, so these runs only show that the model loads and decodes on each backend.

I haven't run the full checkpoint on Hopper (the recipe uses 8x H200), so there is no accuracy eval for M3.1 here. CI has no small M3.1 checkpoint for an LLM-level test, so the CI test works at the experts level and the 4-layer real-weight run above is local.

Commands

pytest -v tests/kernels/moe/test_minimax_m3_hopper_nvfp4_moe.py

# Real weights, first 4 layers (a local dir with the 3 shards and an index of just them)
LLM(model=local_dir, hf_overrides={"text_config": {"num_hidden_layers": 4}}, language_model_only=True,
    block_size=128, disable_hybrid_kv_cache_manager=True, kv_cache_dtype="fp8",
    attention_config={"indexer_kv_dtype": "fp8"}, compilation_config={"cudagraph_mode": "FULL_AND_PIECEWISE"},
    max_model_len=4096)  # + moe_backend="marlin" for the reference arm

# All 60 layers, one TP8 rank's shapes, dummy weights
LLM(model="MiniMaxAI/MiniMax-M3.1-Flash", load_format="dummy", language_model_only=True,
    hf_overrides={"text_config": {"num_attention_heads": 8, "num_key_value_heads": 1,
        "intermediate_size": 384, "dense_intermediate_size": 1536, "shared_intermediate_size": 384,
        "vocab_size": 25008, "sparse_attention_config": {..., "sparse_num_index_heads": 1}}},
    block_size=128, disable_hybrid_kv_cache_manager=True, kv_cache_dtype="fp8",
    attention_config={"indexer_kv_dtype": "fp8"}, compilation_config={"cudagraph_mode": "FULL_AND_PIECEWISE"},
    max_model_len=5120)

Details

• ModelOptMxFp8Config builds M3.1's experts with the model's qat_activation_key (kNvfp4DynamicToken) on every platform, so per_token_activation is set on Hopper too.
• Humming's quant config has quant_dtype is None when it runs A16. For Marlin's NVFP4 config that is the same check as use_nvfp4_w4a16.
• With the metadata dropped, the experts take the same Humming W4A16 path other NVFP4 checkpoints use on SM90.
• The ModelOpt NVFP4 nvfp4_per_token override doesn't reach this code on Hopper. ModelOptNvFp4FusedMoE rejects any backend other than FlashInfer TRTLLM first.

AI assistance (Claude Code) was used. I reviewed the change and ran the tests above.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

Recipe preview: https://vllm-recipes-git-preview-vllm-pr-64932-inferact-inc.vercel.app