Skip to content

[Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness - #48929

Merged
vllm-bot merged 3 commits into
vllm-project:mainfrom
lucifer1004:feat/flashinfer-cutlass-swigluoai-uninterleave
Aug 5, 2026
Merged

vllm-bot merged 3 commits into
vllm-project:mainfrom
lucifer1004:feat/flashinfer-cutlass-swigluoai-uninterleave

Conversation

@lucifer1004

Copy link
Copy Markdown
Contributor

Purpose

Fix two MiniMax-M3 correctness issues exposed by the NVFP4 checkpoint.

First, the routed experts use packed SWIGLUOAI_UNINTERLEAVE with model-specific alpha, beta, and clamp values. FlashInfer CUTLASS already supports this math, but the vLLM adapter neither advertised the packed activation nor forwarded all three parameters. Marlin similarly replaced missing quant-config alpha/beta values with plain-SiLU defaults instead of falling back to the model's MoE config. This PR adds the missing FlashInfer capability and parameter plumbing, and gives both backends the same quant-config-first/model-config-second resolution rule.

Second, the MiniMax-M3 Triton indexer and SM100 MSA indexer write different physical top-k layouts. Treating the buffer shape as the layout source is ambiguous when the token and head dimensions are equal. The writer implementation now declares the layout: Triton uses head-major storage while SM100 MSA keeps its native token-major storage. Readers normalize from that explicit contract. The MSA sparse_topk_select output path is unchanged.

No weight format or low-level FlashInfer, CUTLASS, or Marlin kernel is changed.

Test Plan

prek run --files $(git diff --name-only upstream/main...HEAD)

inferlab run --stack vllm -- pytest -q \
  tests/kernels/moe/test_swigluoai_params.py \
  tests/kernels/moe/test_flashinfer_moe.py \
  -k 'swigluoai_params or swigluoai_activation_mapping'

inferlab run --stack vllm -- pytest -q \
  tests/kernels/attention/test_minimax_m3.py \
  -k 'indexer_output_buffer_layout or prefill_index_topk_correctness or decode_index_topk_correctness or prefill_sparse_attention_correctness or decode_sparse_attention_correctness'

Also run the MiniMax-M3-NVFP4 OpenAI smoke recipe with TP=4 and verify server cleanup.

Test Result

  • prek: all hooks passed.
  • MoE focused tests: 6 passed, 144 deselected.
  • Indexer/sparse-attention focused tests: 31 passed, 37 deselected.
  • TP4 OpenAI smoke: HTTP 200, one completion returned, recipe succeeded, and cleanup was verified.
  • With the same deterministic 16-token prompt, Marlin changed from garbled output to the United States of America. It is the cultural, commercial, and financial center after restoring the model's SwiGLU-OAI parameters.

The TP4 host selected the non-SM100 Triton indexer path. SM100-only performance was not measured locally; the MSA writer and sparse_topk_select(..., output=buf[:num_tokens]) hot path are unchanged, and the shared layout tests cover both token-major and head-major storage including the square-shape case.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the quantization label Jul 23, 2026
@pavanimajety pavanimajety added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 24, 2026
@ehfd

ehfd commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

#49149 has been merged first. Can we resolve the conflicts?

@mergify

mergify Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @lucifer1004.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Jul 25, 2026
@ehfd

ehfd commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Is this the combination of #49149 and #49941 ?

@jpezzulli

Copy link
Copy Markdown

Additional evidence from the compressed-tensors MXFP4 path:

  • model: olka-fi/MiniMax-M3-MXFP4
  • hardware: one RTX PRO 6000 (SM120)
  • runtime base: LvLLM v2.3.8 / vLLM v0.26.0 merge

Without forwarding MiniMax M3's activation parameters, the MXFP4 path hit the
mandatory SWIGLUOAI_UNINTERLEAVE requires clamp_limit assertion. Forwarding
alpha=1.702, beta=1.0, and limit=7.0 into the MXFP4 MoE quant config
allowed the server to load.

With that plumbing plus the independently merged MiniMax top-k layout fix, the
setup produced coherent cold/warm completions, completed an exact 16,000-token
prompt without corruption, and generated 500 coherent tokens in 21.09 s.

This does not directly test this PR's branch or its NVFP4 backend; it confirms
the same model-level SwiGLU-OAI parameter requirement through a separate MXFP4
entry point.

Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
@lucifer1004
lucifer1004 force-pushed the feat/flashinfer-cutlass-swigluoai-uninterleave branch from 539dcee to f9f780e Compare July 31, 2026 06:41
@mergify mergify Bot removed the needs-rebase label Jul 31, 2026
@pavanimajety
pavanimajety enabled auto-merge (squash) August 4, 2026 16:04

@pavanimajety pavanimajety left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few folks have tested the fix, thanks for your contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working nvidia quantization ready ONLY add when PR is ready to merge/full CI is needed

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants