feat(model): add MiniMax M3 language-model bridge and recipes

docs/fern/versions/_nav_order.yml

@@ -17,6 +17,7 @@
 "./versions/nightly/pages/models/llama/llama2.mdx": 1
 "./versions/nightly/pages/models/mimo/mimo.mdx": 1
 "./versions/nightly/pages/models/minimax/minimax-m2.mdx": 1
+"./versions/nightly/pages/models/minimax/minimax-m3.mdx": 2
 "./versions/nightly/pages/models/mistral/mistral.mdx": 1
 "./versions/nightly/pages/models/moonlight/moonlight.mdx": 1
 "./versions/nightly/pages/models/nemotron/llama-nemotron.mdx": 1

docs/fern/versions/nightly.yml

@@ -90,6 +90,8 @@ navigation:
   - contents:
     - page: MiniMax-M2
       path: ./nightly/pages/models/minimax/minimax-m2.mdx
+    - page: MiniMax-M3
+      path: ./nightly/pages/models/minimax/minimax-m3.mdx
     section: MiniMax
   - contents:
     - page: Mistral

docs/fern/versions/nightly/pages/models/minimax/index.mdx

@@ -5,3 +5,4 @@ MiniMax model documentation is organized by model variant.
 | Variant | Guide |
 |---------|-------|
 | MiniMax-M2 / M2.5 / M2.7 | [minimax-m2.md](minimax-m2.md) |
+| MiniMax-M3 | [minimax-m3.md](minimax-m3.md) |

docs/fern/versions/nightly/pages/models/minimax/minimax-m3.mdx

@@ -0,0 +1,54 @@
+# MiniMax-M3
+
+[MiniMax-M3](https://huggingface.co/MiniMaxAI/MiniMax-M3) is a natively multimodal sparse MoE model from MiniMaxAI (428B total, ~23B active parameters). Megatron Bridge supports the M3 *language model* through the `MiniMaxM3Bridge`: the text backbone of the `MiniMaxM3SparseForConditionalGeneration` checkpoint is converted to a Megatron-Core `GPTModel`.
+
+## Supported Variants
+
+| Variant | Hugging Face ID | Notes |
+|---------|-----------------|-------|
+| MiniMax-M3 | `MiniMaxAI/MiniMax-M3` | Language model only (bf16 weights) |
+
+## Architecture Notes
+
+- Mixed dense/MoE decoder: 60 layers, the first 3 dense, the rest with 128 routed experts (top-4) plus one shared expert.
+- Sigmoid router with expert-bias correction and `routed_scaling_factor` applied to the normalized top-k weights (DeepSeek-V3-style routing); the checkpoint's FP32 router weights remain FP32 during import.
+- SwiGLU-OAI activation in every MLP and expert: clamped gate/up projections with a `+1` linear offset, mapped to `activation_func_clamp_value` / `glu_linear_offset` (same mechanism as GPT-OSS).
+- Gemma-style RMSNorm (`x * (1 + w)`) on every norm, mapped to `layernorm_zero_centered_gamma`.
+- GQA attention (64 query heads, 4 KV heads) with per-head QK RMSNorm and partial RoPE (64 of 128 head channels rotated, theta 5e6).
+
+## Known Limitations
+
+- **Language model only.** The CLIP-style vision tower, multimodal projector, and patch-merge MLP are not mapped.
+- **Full attention only.** The lightning-indexer block-sparse attention branch (`self_attn.index_*` weights) is not mapped; the Megatron model runs full causal attention on every layer. Block selection keeps `index_topk_blocks * index_block_size` (2048) key tokens per query, so full attention is mathematically identical up to that sequence length and an approximation beyond it.
+- **MTP modules are not mapped.** The released checkpoint advertises `num_nextn_predict_layers` in its config but ships no `mtp.*` weights.
+- **Auxiliary-loss scoring differs for training.** The recipes use MCore's token-global load-balancing loss over normalized sigmoid scores. Hugging Face leaves its optional router loss disabled by default and uses softmax scores when enabled.
+- The MXFP8 variant (`MiniMaxAI/MiniMax-M3-MXFP8`) is not supported; use the bf16 checkpoint.
+
+## Conversion
+
+```python
+from megatron.bridge import AutoBridge
+
+bridge = AutoBridge.from_hf_pretrained("MiniMaxAI/MiniMax-M3", trust_remote_code=True)
+provider = bridge.to_megatron_provider()
+```
+
+The bridge imports the language backbone from the multimodal checkpoint.
+Standalone Hugging Face checkpoint export is not supported because the vision,
+projector, lightning-indexer, and MTP weights are intentionally not mapped;
+native Megatron checkpoints and in-memory weight round-trip verification are
+supported.
+
+## Examples
+
+For real-checkpoint Slurm conversion, inference, hardware requirements, and
+validated parallelism settings, see the [MiniMax-M3 examples README](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/minimax/minimax_m3/README.md).
+
+## Recipes
+
+Pretraining and packed-sequence (THD) SFT recipes are available under [`src/megatron/bridge/recipes/minimax`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/recipes/minimax) (`minimax_m3_pretrain_256gpu_h100_bf16_config`, `minimax_m3_sft_128gpu_h100_bf16_config`), using a TP=2 / PP=4 / EP=32 baseline layout.
+
+## Related Implementation
+
+- Bridge implementation: [`src/megatron/bridge/models/minimax_m3`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/minimax_m3)
+- Examples: [`examples/models/minimax/minimax_m3`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/minimax/minimax_m3)

docs/models/README.md

@@ -16,7 +16,7 @@ Megatron Bridge conversion, training recipe links, and model-specific notes.
 | **GPT-OSS** | [GPT OSS](gpt_oss/gpt-oss.md) |
 | **Kimi** | [Kimi K2](kimi/kimi-k2.md), [Kimi-K2.5-VL](kimi/kimi-k25-vl.md) |
 | **Llama** | [Llama 2](llama/llama2.md), [Llama 3](llama/llama3.md) |
-| **MiniMax** | [MiniMax-M2 / M2.5 / M2.7](minimax/minimax-m2.md) |
+| **MiniMax** | [MiniMax-M2 / M2.5 / M2.7](minimax/minimax-m2.md), [MiniMax-M3](minimax/minimax-m3.md) |
 | **Mistral** | [Mistral](mistral/mistral.md), [Ministral 3](mistral/ministral3.md) |
 | **Xiaomi-MiMo** | [Xiaomi-MiMo](mimo/mimo.md) |
 | **Moonlight** | [Moonlight](moonlight/moonlight.md) |

docs/models/minimax/index.md

@@ -6,8 +6,10 @@ MiniMax model documentation is organized by model variant.
 :hidden:
 
 minimax-m2.md
+minimax-m3.md
 ```
 
 | Variant | Guide |
 |---------|-------|
 | MiniMax-M2 / M2.5 / M2.7 | [minimax-m2.md](minimax-m2.md) |
+| MiniMax-M3 | [minimax-m3.md](minimax-m3.md) |

docs/models/minimax/minimax-m3.md

@@ -0,0 +1,54 @@
+# MiniMax-M3
+
+[MiniMax-M3](https://huggingface.co/MiniMaxAI/MiniMax-M3) is a natively multimodal sparse MoE model from MiniMaxAI (428B total, ~23B active parameters). Megatron Bridge supports the M3 *language model* through the `MiniMaxM3Bridge`: the text backbone of the `MiniMaxM3SparseForConditionalGeneration` checkpoint is converted to a Megatron-Core `GPTModel`.
+
+## Supported Variants
+
+| Variant | Hugging Face ID | Notes |
+|---------|-----------------|-------|
+| MiniMax-M3 | `MiniMaxAI/MiniMax-M3` | Language model only (bf16 weights) |
+
+## Architecture Notes
+
+- Mixed dense/MoE decoder: 60 layers, the first 3 dense, the rest with 128 routed experts (top-4) plus one shared expert.
+- Sigmoid router with expert-bias correction and `routed_scaling_factor` applied to the normalized top-k weights (DeepSeek-V3-style routing); the checkpoint's FP32 router weights remain FP32 during import.
+- SwiGLU-OAI activation in every MLP and expert: clamped gate/up projections with a `+1` linear offset, mapped to `activation_func_clamp_value` / `glu_linear_offset` (same mechanism as GPT-OSS).
+- Gemma-style RMSNorm (`x * (1 + w)`) on every norm, mapped to `layernorm_zero_centered_gamma`.
+- GQA attention (64 query heads, 4 KV heads) with per-head QK RMSNorm and partial RoPE (64 of 128 head channels rotated, theta 5e6).
+
+## Known Limitations
+
+- **Language model only.** The CLIP-style vision tower, multimodal projector, and patch-merge MLP are not mapped.
+- **Full attention only.** The lightning-indexer block-sparse attention branch (`self_attn.index_*` weights) is not mapped; the Megatron model runs full causal attention on every layer. Block selection keeps `index_topk_blocks * index_block_size` (2048) key tokens per query, so full attention is mathematically identical up to that sequence length and an approximation beyond it.
+- **MTP modules are not mapped.** The released checkpoint advertises `num_nextn_predict_layers` in its config but ships no `mtp.*` weights.
+- **Auxiliary-loss scoring differs for training.** The recipes use MCore's token-global load-balancing loss over normalized sigmoid scores. Hugging Face leaves its optional router loss disabled by default and uses softmax scores when enabled.
+- The MXFP8 variant (`MiniMaxAI/MiniMax-M3-MXFP8`) is not supported; use the bf16 checkpoint.
+
+## Conversion
+
+```python
+from megatron.bridge import AutoBridge
+
+bridge = AutoBridge.from_hf_pretrained("MiniMaxAI/MiniMax-M3", trust_remote_code=True)
+provider = bridge.to_megatron_provider()
+```
+
+The bridge imports the language backbone from the multimodal checkpoint.
+Standalone Hugging Face checkpoint export is not supported because the vision,
+projector, lightning-indexer, and MTP weights are intentionally not mapped;
+native Megatron checkpoints and in-memory weight round-trip verification are
+supported.
+
+## Examples
+
+For real-checkpoint Slurm conversion, inference, hardware requirements, and
+validated parallelism settings, see the [MiniMax-M3 examples README](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/minimax/minimax_m3/README.md).
+
+## Recipes
+
+Pretraining and packed-sequence (THD) SFT recipes are available under [`src/megatron/bridge/recipes/minimax`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/recipes/minimax) (`minimax_m3_pretrain_256gpu_h100_bf16_config`, `minimax_m3_sft_128gpu_h100_bf16_config`), using a TP=2 / PP=4 / EP=32 baseline layout.
+
+## Related Implementation
+
+- Bridge implementation: [`src/megatron/bridge/models/minimax_m3`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/minimax_m3)
+- Examples: [`examples/models/minimax/minimax_m3`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/minimax/minimax_m3)

examples/conversion/hf_megatron_roundtrip_multi_gpu.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2025, NVIDIA CORPORATION.  All rights reserved.
+# Copyright (c) 2025-2026, NVIDIA CORPORATION.  All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -93,6 +93,8 @@ def main(
     trust_remote_code: bool | None = None,
     strict: bool = False,
     skip_save: bool = False,
+    atol: float = 1e-1,
+    rtol: float = 1e-5,
 ) -> None:
     """Perform round-trip conversion between HuggingFace and Megatron-LM models on multiple GPUs."""
     if os.environ.get("WORLD_SIZE") is None:
@@ -209,11 +211,21 @@ def main(
             elif compare_param.dtype != compare_original.dtype or any(p in name for p in IGNORE_PRECISION_PARAMS):
                 compare_param = param.float()
                 compare_original = original_param.float()
-                match = torch.allclose(compare_param, compare_original.to(compare_param.device), atol=1e-1)
+                match = torch.allclose(
+                    compare_param,
+                    compare_original.to(compare_param.device),
+                    atol=atol,
+                    rtol=rtol,
+                )
 
             # --- Case 3: regular param → direct allclose ---
             else:
-                match = torch.allclose(compare_param, compare_original.to(compare_param.device), atol=1e-1)
+                match = torch.allclose(
+                    compare_param,
+                    compare_original.to(compare_param.device),
+                    atol=atol,
+                    rtol=rtol,
+                )
 
             all_match = all_match and match
             table.add_row(
@@ -286,6 +298,8 @@ def main(
     parser.add_argument(
         "--skip-save", action="store_true", help="Skip saving the model after comparison (verification only)"
     )
+    parser.add_argument("--atol", type=float, default=1e-1, help="Absolute tolerance for tensor comparison")
+    parser.add_argument("--rtol", type=float, default=1e-5, help="Relative tolerance for tensor comparison")
     args = parser.parse_args()
     main(
         args.hf_model_id,
@@ -298,6 +312,8 @@ def main(
         args.megatron_load_path,
         args.trust_remote_code,
         skip_save=args.skip_save,
+        atol=args.atol,
+        rtol=args.rtol,
     )
 
     if torch.distributed.is_initialized():

examples/conversion/hf_to_megatron_generate_text.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2025, NVIDIA CORPORATION.  All rights reserved.
+# Copyright (c) 2025-2026, NVIDIA CORPORATION.  All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -88,6 +88,30 @@ def loss_func(x, **kwargs):
     return model(**forward_args), loss_func
 
 
+def _tokenize_prompt(tokenizer, prompt: str, *, apply_chat_template: bool, thinking_mode: str) -> torch.Tensor:
+    """Tokenize a raw prompt, optionally formatting it as a user chat turn."""
+    if not apply_chat_template:
+        return tokenizer.encode(prompt, return_tensors="pt")
+
+    encoded = tokenizer.apply_chat_template(
+        [{"role": "user", "content": prompt}],
+        add_generation_prompt=True,
+        tokenize=True,
+        return_dict=True,
+        return_tensors="pt",
+        thinking_mode=thinking_mode,
+    )
+    return encoded["input_ids"]
+
+
+def _decode_completion(tokenizer, generated_ids: torch.Tensor, prompt_length: int) -> str:
+    """Decode generated tokens without echoing the prompt or special tokens."""
+    return tokenizer.decode(
+        generated_ids[0, prompt_length:].tolist(),
+        skip_special_tokens=True,
+    )
+
+
 def main(args) -> None:
     """Main function for text generation from HuggingFace or Megatron models.
 
@@ -202,7 +226,13 @@ def main(args) -> None:
 
     # Tokenize the input prompt
     prompt = args.prompt
-    input_ids = tokenizer.encode(prompt, return_tensors="pt").cuda()
+    input_ids = _tokenize_prompt(
+        tokenizer,
+        prompt,
+        apply_chat_template=args.apply_chat_template,
+        thinking_mode=args.thinking_mode,
+    ).cuda()
+    prompt_length = input_ids.size(1)
     position_ids = (
         torch.arange(input_ids.size(1), dtype=torch.long, device=input_ids.device).unsqueeze(0).expand_as(input_ids)
     )
@@ -267,8 +297,9 @@ def main(args) -> None:
             if next_token_ids.item() in stop_tokens:
                 break
 
-    # Decode the generated sequence
-    generated_text = tokenizer.decode(list(generated_ids[0]))
+    # Decode only the completion. Passing CUDA tensor objects directly to the
+    # tokenizer can produce corrupt text with some remote-code tokenizers.
+    generated_text = _decode_completion(tokenizer, generated_ids, prompt_length)
     print_rank_0("======== GENERATED TEXT OUTPUT ========")
     print_rank_0(f"Prompt: {prompt}")
     print_rank_0(f"Generated: {generated_text}")
@@ -295,6 +326,17 @@ def main(args) -> None:
         default=20,
         help="Maximum number of new tokens to generate.",
     )
+    parser.add_argument(
+        "--apply-chat-template",
+        action="store_true",
+        help="Format the prompt as a user turn using the tokenizer's chat template.",
+    )
+    parser.add_argument(
+        "--thinking-mode",
+        choices=("enabled", "adaptive", "disabled"),
+        default="adaptive",
+        help="Thinking mode passed to the chat template when --apply-chat-template is set.",
+    )
     parser.add_argument("--tp", type=int, default=1, help="Tensor parallelism size")
     parser.add_argument("--pp", type=int, default=1, help="Pipeline parallelism size")
     parser.add_argument("--ep", type=int, default=1, help="Expert parallelism size")

examples/models/minimax/minimax_m3/README.md

@@ -0,0 +1,76 @@
+# MiniMax-M3 Examples
+
+This directory contains real-checkpoint conversion and inference examples for
+[MiniMax-M3](https://huggingface.co/MiniMaxAI/MiniMax-M3). The bridge imports
+the language backbone from the multimodal checkpoint; the vision tower,
+projector, lightning-indexer weights, and MTP modules are not converted.
+
+## Hardware requirements
+
+MiniMax-M3 has about 428B parameters stored in bf16 (the published checkpoint
+is about 869 GB). The supplied Slurm jobs use 32 GPUs with `TP=1`, `PP=1`, and
+`EP=32`, which is a conservative layout for 80 GB GPUs. Hardware with larger
+GPU memory can reduce `EP` and the node count as long as `EP` divides 128.
+
+## Setup
+
+Set the container and mounts without placing credentials in the scripts:
+
+```bash
+export CONTAINER_IMAGE=/path/to/megatron-bridge.sqsh
+export CONTAINER_MOUNTS=/shared:/shared,/path/to/Megatron-Bridge:/opt/Megatron-Bridge
+export HF_HOME=/shared/cache/huggingface
+export UV_CACHE_DIR=/shared/cache/uv
+export HF_TOKEN=your_token_if_required
+export SLURM_ACCOUNT=your_slurm_account
+```
+
+The repository is mounted at `/opt/Megatron-Bridge` by default. Override
+`WORKDIR` if your mount uses a different path. Fully populate the shared model
+cache before starting either 45-minute compute job; the checkpoint download is
+about 869 GB:
+
+```bash
+hf download MiniMaxAI/MiniMax-M3
+```
+
+## Conversion round-trip
+
+Submit [slurm_conversion.sh](slurm_conversion.sh) to import the real HF
+checkpoint into a distributed Megatron model and export every bridged tensor
+back in memory. The job compares those tensors with the original checkpoint
+and skips writing a second 869 GB copy.
+
+```bash
+mkdir -p logs
+sbatch --account="${SLURM_ACCOUNT}" examples/models/minimax/minimax_m3/slurm_conversion.sh
+```
+
+Success is reported only when all bridged language-model parameters match the
+original checkpoint exactly (`atol=0`, `rtol=0`). This is an in-memory
+verification; standalone Hugging Face checkpoint export is not supported
+because the bridge intentionally omits the multimodal modules.
+
+## Inference
+
+Submit [slurm_inference.sh](slurm_inference.sh) to convert the real checkpoint,
+apply the checkpoint's chat template with thinking disabled, and greedily
+generate a short response with Megatron-Core:
+
+```bash
+mkdir -p logs
+sbatch --account="${SLURM_ACCOUNT}" examples/models/minimax/minimax_m3/slurm_inference.sh
+```
+
+The run is successful when it completes without missing-weight or forward
+errors and the generated answer is coherent for the prompt.
+
+## Validated configuration
+
+The real `MiniMaxAI/MiniMax-M3` checkpoint was validated on 32 H100 80 GB
+GPUs with `TP=1`, `PP=1`, `EP=32`, and `ETP=1`. All 1,053 mapped parameter
+tasks passed the exact in-memory HF → Megatron → HF round-trip check. With the chat
+template enabled, Megatron generated a coherent continuation beginning:
+
+> The sky appears blue because of Rayleigh scattering, where sunlight
+> interacts with Earth's atmosphere.