Conversation
|
#49149 has been merged first. Can we resolve the conflicts? |
|
This pull request has merge conflicts that must be resolved before it can be |
|
Additional evidence from the compressed-tensors MXFP4 path:
Without forwarding MiniMax M3's activation parameters, the MXFP4 path hit the With that plumbing plus the independently merged MiniMax top-k layout fix, the This does not directly test this PR's branch or its NVFP4 backend; it confirms |
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
539dcee to
f9f780e
Compare
pavanimajety
left a comment
There was a problem hiding this comment.
A few folks have tested the fix, thanks for your contribution.
Purpose
Fix two MiniMax-M3 correctness issues exposed by the NVFP4 checkpoint.
First, the routed experts use packed
SWIGLUOAI_UNINTERLEAVEwith model-specificalpha,beta, and clamp values. FlashInfer CUTLASS already supports this math, but the vLLM adapter neither advertised the packed activation nor forwarded all three parameters. Marlin similarly replaced missing quant-config alpha/beta values with plain-SiLU defaults instead of falling back to the model's MoE config. This PR adds the missing FlashInfer capability and parameter plumbing, and gives both backends the same quant-config-first/model-config-second resolution rule.Second, the MiniMax-M3 Triton indexer and SM100 MSA indexer write different physical top-k layouts. Treating the buffer shape as the layout source is ambiguous when the token and head dimensions are equal. The writer implementation now declares the layout: Triton uses head-major storage while SM100 MSA keeps its native token-major storage. Readers normalize from that explicit contract. The MSA
sparse_topk_selectoutput path is unchanged.No weight format or low-level FlashInfer, CUTLASS, or Marlin kernel is changed.
Test Plan
Also run the MiniMax-M3-NVFP4 OpenAI smoke recipe with TP=4 and verify server cleanup.
Test Result
prek: all hooks passed.6 passed, 144 deselected.31 passed, 37 deselected.the United States of America. It is the cultural, commercial, and financial centerafter restoring the model's SwiGLU-OAI parameters.The TP4 host selected the non-SM100 Triton indexer path. SM100-only performance was not measured locally; the MSA writer and
sparse_topk_select(..., output=buf[:num_tokens])hot path are unchanged, and the shared layout tests cover both token-major and head-major storage including the square-shape case.