Skip to content
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion models/MiniMaxAI/MiniMax-M3.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ meta:
provider: "MiniMax"
description: "MiniMax M3 vision-language MoE (427B total / 26B active) for frontier coding, agent toolchains, and 1M-token reasoning via MSA sparse attention — native multimodal (image + video + computer use); BF16 plus MXFP8, NVIDIA Blackwell NVFP4, and AMD MI355X MXFP4 variants. Runs on NVIDIA (Hopper/Blackwell) and AMD CDNA4/CDNA3."
date_added: 2026-06-12
date_updated: 2026-07-15
date_updated: 2026-08-03
difficulty: advanced
tasks:
- text
Expand All @@ -14,6 +14,7 @@ meta:
hardware:
h200: verified
b200: verified
b300: verified
mi300x: verified
mi355x: verified

Expand Down Expand Up @@ -72,6 +73,11 @@ features:
description: "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel."
args:
- "--language-model-only"
thinking_always_on:
description: "Pin `thinking_mode: enabled` server-side — every turn reasons before answering, including after tool results. Recommended for agentic workflows."
args:
- "--default-chat-template-kwargs"
- '{"thinking_mode": "enabled"}'
encoder_parallel:
description: "Run the vision encoder data-parallel instead of tensor-parallel — avoids TP comm overhead on the small encoder. Also enables the host-shared-memory multimodal processor cache and the per-brand encoder attention backend (FlashInfer on NVIDIA, AITER FlashAttention on AMD). Mutually exclusive with text_only."
# No platform-agnostic `args`: this recipe targets only NVIDIA + AMD, both
Expand Down Expand Up @@ -100,6 +106,7 @@ features:
opt_in_features:
- spec_decoding
- text_only
- thinking_always_on
- encoder_parallel

variants:
Expand Down Expand Up @@ -161,10 +168,16 @@ variants:
description: "NVIDIA-quantized NVFP4 weights (ModelOpt) for Blackwell (B200/B300) — native FP4 tensor cores, ~half the VRAM of MXFP8."
extra_args:
- "--trust-remote-code"
# FlashInfer TRT-LLM attention kernels with an FP8 indexer KV cache.
- "--attention_config.backend"
- "FLASHINFER"
- "--attention_config.use_trtllm_attention"
- "true"
Comment on lines +174 to +177

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The CLI arguments should use hyphens instead of underscores to match the standard vLLM argument format (e.g., --attention-config instead of --attention_config). Using underscores will result in an unrecognized arguments error from argparse when launching the server.

Note: Please also update the other occurrences of --attention_config in this file (such as lines 170 and 227-232) to use hyphens for consistency and correctness.

      - "--attention-config.backend"
      - "FLASHINFER"
      - "--attention-config.use_trtllm_attention"
      - "true"

- "--attention_config.indexer_kv_dtype"
- "fp8"
extra_env:
VLLM_FLOAT32_MATMUL_PRECISION: "high"
VLLM_FLASHINFER_ALLREDUCE_BACKEND: "trtllm"
mxfp4:
model_id: "amd/MiniMax-M3-MXFP4"
precision: mxfp4
Expand Down