Update MiniMax M3 for B300 NVFP4 - #730
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Code Review
This pull request adds support and verification for B300 hardware to the MiniMax-M3 model configuration, configures FlashInfer TRT-LLM attention kernels for the NVFP4 variant, and adds a guide section for agentic coding on B300. The review feedback correctly identifies that the newly added CLI arguments in extra_args use underscores (e.g., --attention_config) instead of hyphens (e.g., --attention-config), which would lead to unrecognized argument errors in vLLM.
| - "--attention_config.backend" | ||
| - "FLASHINFER" | ||
| - "--attention_config.use_trtllm_attention" | ||
| - "true" |
There was a problem hiding this comment.
The CLI arguments should use hyphens instead of underscores to match the standard vLLM argument format (e.g., --attention-config instead of --attention_config). Using underscores will result in an unrecognized arguments error from argparse when launching the server.
Note: Please also update the other occurrences of --attention_config in this file (such as lines 170 and 227-232) to use hyphens for consistency and correctness.
- "--attention-config.backend"
- "FLASHINFER"
- "--attention-config.use_trtllm_attention"
- "true"Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
1818aaf to
6f70a3d
Compare
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
xinli-sw
left a comment
There was a problem hiding this comment.
thanks for the changes!
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
Mark B300 as verified for MiniMax-M3, run the NVFP4 variant on FlashInfer TRT-LLM attention with the FP8 indexer KV cache, and document the B300 agentic-coding launch with the EAGLE3-GQA draft head. Best configs based on SemiAnalysisAI/InferenceX#2328.