[zyongye]

Overview

The first PR of the split of #63232. It adds the MiniMax-M3.1-Flash model code (everything under vllm/models/minimax_m3, NVIDIA and ROCm, including the DSpark draft) and all kernel changes. The model-runner, sampling and KV cache manager changes from #63232 are left out and will follow as separate PRs (plan under Details).

Claims

• Serve MiniMax-M3.1-Flash on Blackwell (SM100/SM103/SM107): the NVFP4 routed experts of the MXFP8 checkpoint load through the compressed-tensors NVFP4 MoE with per-token FC1 input scales. The KV cache and the index-K cache run in NVFP4 on the MSA CUTLASS Q8KV4 kernels.
• DSpark speculative decoding: a draft model that defaults to the checkpoint's dspark/ subfolder, with fused draft kernels and a FLASHINFER_TS backend for non-causal draft blocks.
• Decode kernels:
  • fused QKV+indexer MXFP8 GEMM epilogue;
  • SM100 fused dense GEMM1;
  • persistent QK-norm/RoPE/KV-insert kernel;
  • small-batch MoE finalize fused into all-reduce + Gemma norm + MXFP8 quant;
  • DeepGEMM NVFP4 MegaMoE (opt-in, with expert parallelism);
  • opt-in NVFP4 indexer decode context parallelism at TP4.
• ROCm:
  • NVFP4 KV + NVFP4 indexer on gfx950;
  • one-launch-per-layer FlyDSL mono decode;
  • small-M MXFP8 linear tuning;
  • gfx942 fallbacks.

Validation

Unit tests ran on GB300 (SM103) against this branch, with MSA 25c1447 (the pinned version) vendored and CUDA 13.4 for the Q8KV4 JIT:

Suite	Result
tests/kernels/attention -k minimax_m3	219 passed, 64 skipped (ROCm/other-SM)
fused QK-norm/RoPE/KV insert, DSpark draft kernels, MegaMoE, tests/models/minimax_m3, Qwen3-DFlash MXFP8 context KV, reasoning parser, speculative config, FLASHINFER_TS, DFlash/DSpark spec-decode unit tests, aux hidden states over PP, M3 multimodal processing, tests/quantization/test_modelopt.py	689 passed, 5 skipped, 3 failed (below)
tests/distributed/test_custom_all_reduce.py (2 GPUs)	57 passed, 5 skipped

The 3 failures (test_modelopt_fp8_pb_wo_checkpoint_setup, test_modelopt_mixed_dspark_checkpoint, test_modelopt_nested_mixed_checkpoint) fail identically on main in the same environment. They are not caused by this PR.

End-to-end on this branch. MiniMaxAI/MiniMax-M3.1-Flash served with the command below (TP4 on GB300, NVFP4 KV + NVFP4 indexer, DSpark k=7). Higher is better. MAL is the mean number of tokens accepted per verify step, out of 8.

Eval	Config	Score	DSpark MAL
gsm8k (1319)	chat path, 5-shot, thinking_mode: disabled, T=0, max_tokens 2048	0.9287 flexible / 0.9196 strict	4.77
aime26 mean@16 (30 × 16)	chat path, thinking_mode: enabled, T=1.0, top_p 0.95, max_tokens 98304	0.9271 (pass@16 0.9667)	3.48

Neither run had request errors. gsm8k had no truncated or unanswered rows. On aime26, 11 of 480 samples (2.3%) hit the max_tokens limit, and they are the only unanswered rows.

On #63232 (the full branch), same setup with thinking on (aime26 mean@16, MMMU-Pro strict-match over 3460 questions):

Arm	aime26	MMMU-Pro strict	DSpark MAL (aime26 / MMMU-Pro)
plain	0.9458	0.7936	–
DSpark k=7	0.9208	0.7861	3.33 / 3.21

The two arms ran on different revisions of #63232. The paired comparison on one revision was within noise: aime26 +0.00 pp (95% CI ±3.0 pp), MMMU-Pro McNemar p=0.876.

vllm serve MiniMaxAI/MiniMax-M3.1-Flash -tp 4 --reasoning-parser minimax_m3 \
  --block-size 128 --kv-cache-dtype nvfp4 \
  --attention-config '{"indexer_kv_dtype": "nvfp4"}' \
  --speculative-config '{"method": "dspark", "num_speculative_tokens": 7, "attention_backend": "FLASHINFER_TS", "kv_cache_dtype": "fp8", "draft_sample_method": "probabilistic"}' \
  --disable-hybrid-kv-cache-manager

Not covered:

• ROCm was not run (no AMD hardware here); the ROCm code is from the AMD contributors' branch and tests.
• VLLM_GPU_SYNC_CHECK=error was not run.

Details

Shared code this PR touches. All of it is opt-in or a no-op for other models:

• FusedMoEConfig.qat_activation_key (default None): a model declares the activation scheme its experts were QAT'd against. make_nvfp4_moe_kernel now raises instead of silently dropping per_token_activation for a backend that cannot honor it.
• ModelOptMxFp8Config.moe_quant_algo (default None): routes moe_quant_algo: NVFP4 experts to the compressed-tensors NVFP4 MoE.
• Attention(query_pre_quantized=False): lets the DSpark draft skip a second query quantization.
• DraftModelSpeculator.attn_vllm_config: the draft's metadata builders now see the draft overrides (attention backend, KV cache dtype). The copies are shallow, so engine-resolved state such as cache_config.kv_cache_layout survives. Covered by test_draft_attention_backend_override.py.
• New FLASHINFER_TS attention backend, attention_config.indexer_decode_cp, a MiniMax-M3 TP check for NVFP4 KV, and the MSA pin bump (968061a → 25c1447).
• The CuTe DSL primitives of the DeepSeek-V4.1 all-reduce and the Kimi-K3 latent MoE tail move to vllm/models/common/ops/moe_tail_fusion, shared with the M3 finalize fusion. Their callers only change import paths.

For reviewers:

• The ROCm mono decode here (VLLM_ROCM_MINIMAX_M3_MONO) predates and overlaps the upstream M3 MonoKernel ([ROCm][Perf][MiniMax-M3] MonoKernel decode for M3 MXFP4 on MI355X #62712, kernel_config.monokernel). The two need reconciling; it can also move to its own PR.
• attention_config.indexer_decode_cp (NVFP4 indexer, TP4) sits next to the existing minimax_m3_indexer_cp (ROCm bf16 Triton CP).
• The Q8KV4 prefill keeps a per-device split workspace that is only allocated on SM107. It was a stopgap on the source branch.

Split of #63232. Later PRs:

PR	Contents	Depends on
this	MiniMax-M3.1 model + DSpark draft + kernels	–
2	ROCm NVFP4 → MXFP8/MXFP4 online re-quantization and fused shared experts through the online quantization API (needed to serve the NVFP4-expert checkpoint on ROCm)	–
3	NIXL P/D for per-head-contiguous NVFP4 pages (AttentionSpec.kv_head_major_slots)	this
4	FA4: no descale tensors for a unit per-tensor scale	–
5 (deferred)	DSpark/DFlash sampling performance: Markov bias in the Gumbel kernel, batch-sharded draft sampling, wide DFlash padding, top-k/top-p	this
6 (deferred)	KV cache manager: keep KV zeroing for mixed-precision groups	–
7 (deferred)	SimpleCPUOffload HugeTLB host buffer (not MiniMax-specific)	–

Where the excluded files of #63232 go

• PR 2: quantization/online/* (incl. shared_expert.py, requantization.py), quantization/utils/config_utils.py, fused_moe/{utils,routed_experts}.py, quark/*, nvfp4_emulation_utils.py, base_config.py, model_loader/utils.py, the NVFP4 oracle's re-quantization hook, the dequantize-iterator change in the ModelOpt/CT NVFP4 MoE, and the ROCm NVFP4-expert guard and FSE resolver in minimax_m3/amd/model.py.
• PR 3: kv_cache_interface.py (kv_head_major_slots), nixl/base_worker.py, the model's kv_head_major_slots=True, and its insert test.
• PR 4: v1/attention/backends/flash_attn.py.
• PR 5: v1/worker/gpu/{sample/gumbel,sample/batch_shard,input_batch,model_runner}.py, the rest of the DFlash/DSpark speculators, v1/sample/ops/topk_topp_*, the scheduler's DFlash/DSpark draft slots, and Qwen3DSparkForCausalLM.compute_draft_logits_local.
• PR 6: v1/core/kv_cache_utils.py and KVCacheConfig.mixed_precision_kv_cache.
• PR 7: v1/simple_kv_offload/* and its connector.

AI assistance (Claude, Codex, Kimi) was used to write parts of these changes and this PR; the author reviewed the split and ran the tests above.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[github-actions]

✅ CI is already running for this commit: https://buildkite.com/vllm/ci/builds/98120

[zyongye]

/ci run

[github-actions]

✅ Triggered Buildkite CI #98139 for commit 53c2c6caa3a5.

[zyongye]

/ci run

[github-actions]

✅ Triggered Buildkite CI #98182 for commit b17b2ba7cc6a.

ℹ️ This PR is 16 commits behind upstream main. Merged upstream main into this branch as b17b2ba7cc6a before starting CI. Pull the branch before pushing more changes.

[khluu]

stamp for Buildkite changes