[MiniMax-M3.1] Enable Mono decode with fixed DSpark verify by wuhuikx · Pull Request #2686 · ROCm/ATOM

wuhuikx · 2026-10-23T08:18:35Z

Summary

Enables the existing MiniMax-M3 Mono decode path (PTPC FP8 attention + fused MXFP4 MoE) for MiniMax-M3.1-Flash with fixed DSpark verify. The kernel ABI is unchanged.

Builds on the MiniMax-M3.1-Flash support merged in #2678; this diff contains only the two Mono commits.

Sparse-attention prefix. M3.1 layers 0..2 are sparse attention + indexer + dense MLP, which neither the Mono dense kernels nor the MoE kernels serve. They now run on the original modules and Mono starts at layer 3. M3's dense full-attention prefix keeps its fused kernels and launches.
Fixed DSpark verify. Binds DSpark aux taps to Mono's residual stream (atom.mono.runtime.aux_taps). Confidence-scheduled and ragged DSpark remain refused.
Both switches resolve at deployment/bind time. Existing M3 and DSpark-off paths retain their kernel builders and ABIs.

Mono turns on only with all of the following:

ATOM_MONO_ENABLE=1
PTPC FP8 attention/dense (ptpc_fp8, excluding *block_sparse_moe)
ATOM_FUSE_MXFP8_SHARED_INTO_MXFP4=1
TP4, FP8 KV/index cache, block 128, fixed 7-draft DSpark

On the release checkpoint, whose NVFP4 routed experts do not use this FSE path, Mono refuses cleanly at layer 3.

Validation

Current head on latest main:

focused dispatch suite: 75 passed
upstream Black, Ruff, non-GPU unit tests and offline inference smoke: passed
all four ranks report MiniMax-M3 mono decode on; width-8/16 builds and graph capture complete

Existing Quark GSM8K runs on this PTPC/FSE path:

Mono off: flexible 0.955, strict 0.909, robust 0.9674
Mono on: flexible 0.9416, strict 0.8802, robust 0.9613
another Mono-off run was 0.938 / 0.878, robust 0.957; Mono-on is inside the observed run-to-run spread

Paired real-acceptance diagnostic on the current Quark checkpoint:

56/56 q=8 target steps had identical Mono/original accepted-prefix length
8/448 final top-1 rows differed, but none changed acceptance
the temporary diagnostic was fully reverted; production code is unchanged

Mono remains opt-in, so the default path is unchanged.

Performance

Current PR head, amd/MiniMax-M3.1-Flash-Quark-MXFP4, 4xMI355X TP4, PTPC/FSE and fixed 7-draft DSpark. Baseline and Mono arms differ only in ATOM_MONO_ENABLE.

SA-style AgentX validation (inferencex-agentx-mvp, SemiAnalysis/Weka 256K traces, prefix cache, 900 s arms) with synthetic accepted length 4.0 to isolate target-path performance, Mono off -> on:

C1: output 187.56 -> 213.44 tok/s (+13.8%), p90 ITL 2.68 -> 2.33 ms (-13.1%), p99 context 243.3K -> 243.2K
C2: output 224.52 -> 235.79 tok/s (+5.0%), p90 ITL 2.64 -> 2.31 ms (-12.5%), p99 context 243.3K -> 243.2K
no request errors or cancellations

These are 900-second PR-validation arms, not the 3600-second full InferenceX sweep.

Synthetic acceptance is performance-only and is not used for accuracy claims.

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

Port #2686 at ac8b0fb.

valarLip · 2026-10-27T13:30:29Z

        if getattr(self, "drafter", None) is not None:
            self.drafter.arm_aux_capture(self.model)
+            bind_mono_drafter = getattr(self.model, "bind_mono_drafter", None)
+            if bind_mono_drafter is not None:


align the mono drafter impl with other models?

Aligned in 53b8065. Both MiniMax architectures now use mono_decode_installers; the model-specific installer receives the drafter after arm_aux_capture and binds the in-model Mono router. The generic ModelRunner getattr(..., bind_mono_drafter) special case is removed, matching the Kimi/V4 lifecycle.

Post-change E2E also passed on the official Quark checkpoint: TP4 + fixed DSpark, all four ranks logged MiniMax-M3 mono decode on, width-8/16 graph capture completed, and the q=8/C2 smoke completed 8/8 requests.

Port #2686 at ac8b0fb.

valarLip · 2026-10-28T07:23:38Z

    "KimiK3ForConditionalGeneration": (
        "atom.models.kimi_k3.mono.dispatch.install_mono_decode"
    ),
+    "MiniMaxM3SparseForCausalLM": (


need rebase after this one land #2776

Rebased onto main at d1c7bb2 (#2776) in head a3171ee. The MiniMax entries now live in atom/models/registry.py, and its installer accepts the post-#2776 metadata_builder ABI. Black, Ruff, non-GPU unit tests, and offline inference smoke all pass.

…ules M3.1's layers 0..2 are sparse attention + indexer + dense MLP, which neither the mono dense kernels (full attention) nor the MoE kernels serve, so mono refused every M3.1 deployment: "layers 0..2 must be dense full-attention layers". ``dense_prefix_mono`` resolves the prefix's arm once at bind time from the layer stack. M3's dense full-attention prefix keeps the fused dense kernels and every launch it had; M3.1's runs on the original modules through the ``run_dense`` that was already there, which leaves the first mono layer the same (ar, res) pair in ars[0]. A model with no mono arm for its prefix binds no DenseLayer, builds no dense kernels and needs no swiglu parameters. ``run_dense`` takes its aux list the way ``run_dense_mono`` does rather than defaulting it to None: it is the production prefix on M3.1, where an Eagle3 aux layer among 0..2 would have appended to the default. The first layer mono runs still has to select its own sparse blocks. The refusal now names the layer, because M3.1 numbers its sparse layers from 0 and an index_topk_freq schedule can land a reusing layer there. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

wuhuikx mentioned this pull request Oct 23, 2026

[MiniMax-M3.1] Day-0 support for MiniMax-M3.1-Flash: Quark MXFP4 checkpoint, DSpark, image input #2678

Merged

wuhuikx force-pushed the hattie/minimax-m31-flash branch from 432d569 to 48aff05 Compare October 24, 2026 08:18

Base automatically changed from hattie/minimax-m31-flash to main October 26, 2026 08:21

XiaobingSuper force-pushed the hattie/minimax-m31-mono branch from 2736848 to 4ac438c Compare October 26, 2026 09:38

XiaobingSuper changed the title ~~[MiniMax-M3.1] Enable Mono decode with fixed DSpark verify (stacked on #2678)~~ [MiniMax-M3.1] Enable Mono decode with fixed DSpark verify Oct 26, 2026

XiaobingSuper mentioned this pull request Oct 26, 2026

[MiniMax-M3.1] Native-MXFP8 Mono decode through S32 (stacked on #2686) #2718

Draft

XiaobingSuper force-pushed the hattie/minimax-m31-mono branch from 4ac438c to ac8b0fb Compare October 26, 2026 10:01

XiaobingSuper marked this pull request as ready for review October 26, 2026 13:24

Copilot AI balanced review requested due to automatic review settings October 26, 2026 13:24

Copilot AI reviewed Oct 26, 2026

XiaobingSuper requested a review from valarLip October 26, 2026 13:24

MengqingCao added a commit that referenced this pull request Oct 27, 2026

feat(minimax): enable M3.1 Mono with fixed DSpark verify

9f0873e

Port #2686 at ac8b0fb.

MengqingCao mentioned this pull request Oct 27, 2026

[MiniMax-M3.1] Add TP4 P/D Mooncake Store benchmarks and NVFP4 recipe #2750

Draft

valarLip requested changes Oct 27, 2026

View reviewed changes

XiaobingSuper requested a review from valarLip October 28, 2026 02:01

ganyi1996ppo mentioned this pull request Oct 28, 2026

[MiniMax-M3.1] MXFP4 sparse KV cache and index cache #2770

Open

MengqingCao added a commit that referenced this pull request Oct 28, 2026

feat(minimax): enable M3.1 Mono with fixed DSpark verify

bc715c9

Port #2686 at ac8b0fb.

zhuyuhua-v mentioned this pull request Oct 27, 2026

[MiniMax-M3.1] FP4 index cache and NVFP4 sparse KV cache #2744

Open

valarLip reviewed Oct 28, 2026

View reviewed changes

Zhang and others added 3 commits October 28, 2026 19:13

feat(minimax-m3.1): enable Mono for fixed DSpark verify

10d2d16

Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(minimax-m3): bind drafter in mono installer

a3171ee

Co-authored-by: Cursor <cursoragent@cursor.com>

XiaobingSuper force-pushed the hattie/minimax-m31-mono branch from 53b8065 to a3171ee Compare October 28, 2026 11:24

XiaobingSuper requested a review from valarLip October 28, 2026 11:37

valarLip approved these changes Oct 28, 2026

View reviewed changes

valarLip merged commit d7f5da1 into main Oct 28, 2026
37 of 39 checks passed

valarLip deleted the hattie/minimax-m31-mono branch October 28, 2026 11:51

ganyi1996ppo mentioned this pull request Oct 29, 2026

[MiniMax-M3.1] Mono decode over the MXFP4 KV and index caches #2812

Draft

Navigation Menu

Repository navigation

[MiniMax-M3.1] Enable Mono decode with fixed DSpark verify - #2686

[MiniMax-M3.1] Enable Mono decode with fixed DSpark verify#2686
valarLip merged 3 commits into
mainfrom
hattie/minimax-m31-mono

wuhuikx commented Oct 23, 2026 •

edited by XiaobingSuper

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

valarLip Oct 27, 2026

Uh oh!

XiaobingSuper Oct 28, 2026

Uh oh!

XiaobingSuper Oct 28, 2026

Uh oh!

valarLip Oct 28, 2026

Uh oh!

XiaobingSuper Oct 28, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

wuhuikx commented Oct 23, 2026 • edited by XiaobingSuper Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Performance

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

valarLip Oct 27, 2026

Choose a reason for hiding this comment

Uh oh!

XiaobingSuper Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

XiaobingSuper Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

valarLip Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

XiaobingSuper Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wuhuikx commented Oct 23, 2026 •

edited by XiaobingSuper

Loading