[NV] Add MiniMax M3 Blackwell marlin TP8 recipe - #558
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
a8518f7 to
5a83ca2
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces strategy overrides and guide documentation for running the MiniMax-M3 model on NVIDIA Blackwell hardware using the Marlin MoE backend. However, the reviewer identified critical issues where the Marlin MoE backend is applied to the unquantized BF16 model (both in the global strategy overrides and the guide commands), which will cause runtime failures since Marlin MoE kernels do not support unquantized BF16 weights.
| strategy_overrides: | ||
| single_node_tp: | ||
| hardware_overrides: | ||
| blackwell: | ||
| extra_args: | ||
| - "--moe-backend" | ||
| - "marlin" |
There was a problem hiding this comment.
Applying --moe-backend marlin globally under strategy_overrides for Blackwell will affect all variants, including the default BF16 model (MiniMaxAI/MiniMax-M3). Since Marlin MoE kernels in vLLM do not support unquantized BF16 weights, running the default BF16 recipe on Blackwell will fail at runtime.
If the schema supports it, please move this override under the mxfp8 variant configuration, or remove it from the global strategy_overrides to avoid breaking the BF16 path.
| ### NVIDIA Blackwell — TP8 (8x B200 / B300) | ||
|
|
||
| ```bash | ||
| vllm serve MiniMaxAI/MiniMax-M3 \ | ||
| --tensor-parallel-size 8 \ | ||
| --block-size 128 \ | ||
| --moe-backend marlin \ | ||
| --tool-call-parser minimax_m3 \ | ||
| --reasoning-parser minimax_m3 \ | ||
| --enable-auto-tool-choice | ||
| ``` |
There was a problem hiding this comment.
This guide section attempts to run the BF16 model (MiniMaxAI/MiniMax-M3) with --moe-backend marlin. Since Marlin MoE kernels do not support unquantized BF16 weights, this command will fail at runtime.
Since the BF16 model on Blackwell runs with the same parameters as Hopper, this section is redundant. Please remove this section or merge it with the Hopper section above.
5a83ca2 to
f86f6ea
Compare
Signed-off-by: jasonlizhengjian <jasonlizhengjian@gmail.com>
f86f6ea to
9a1400a
Compare
Summary
Adds
--moe-backend marlinonly to the MiniMax-M3 MXFP8 Blackwell TP8 recipe path.Changes
single_node_tpmarlin override under themxfp8variant so the BF16 default path is unaffected.--moe-backend marlin.Validation
git diff --checknode scripts/build-recipes-api.mjssingle_node_tpcommands do not include--moe-backend marlin.single_node_tpcommands include--moe-backend marlin.single_node_tepcommands do not include--moe-backend marlin.