Repository navigation
[MiniMax-M3.1] Native-MXFP8 Mono decode through S32 (stacked on #2686) - #2718
Draft
XiaobingSuper wants to merge 7 commits into
Draft
XiaobingSuper wants to merge 7 commits into
XiaobingSuper wants to merge 7 commits into
Conversation
XiaobingSuper
force-pushed
the
hattie/minimax-m31-mono
branch
from
October 26, 2026 10:01
4ac438c to
ac8b0fb
Compare
XiaobingSuper
force-pushed
the
hattie/minimax-m31-native-mxfp8
branch
2 times, most recently
from
October 28, 2026 01:59
b3c4cdf to
8953efb
Compare
…ules M3.1's layers 0..2 are sparse attention + indexer + dense MLP, which neither the mono dense kernels (full attention) nor the MoE kernels serve, so mono refused every M3.1 deployment: "layers 0..2 must be dense full-attention layers". ``dense_prefix_mono`` resolves the prefix's arm once at bind time from the layer stack. M3's dense full-attention prefix keeps the fused dense kernels and every launch it had; M3.1's runs on the original modules through the ``run_dense`` that was already there, which leaves the first mono layer the same (ar, res) pair in ars[0]. A model with no mono arm for its prefix binds no DenseLayer, builds no dense kernels and needs no swiglu parameters. ``run_dense`` takes its aux list the way ``run_dense_mono`` does rather than defaulting it to None: it is the production prefix on M3.1, where an Eagle3 aux layer among 0..2 would have appended to the default. The first layer mono runs still has to select its own sparse blocks. The refusal now names the layer, because M3.1 numbers its sparse layers from 0 and an index_topk_freq schedule can land a reusing layer there. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
XiaobingSuper
force-pushed
the
hattie/minimax-m31-mono
branch
from
October 28, 2026 11:24
53b8065 to
a3171ee
Compare
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
XiaobingSuper
force-pushed
the
hattie/minimax-m31-native-mxfp8
branch
from
October 28, 2026 11:26
8953efb to
5f4ec52
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #2686. This diff contains only the native-MXFP8/S32 kernel work; sparse-prefix and fixed-DSpark lifecycle changes remain in the base PR.
The native path is opt-in through
ATOM_MONO_ENABLE=1. For the Quark export, keep attention/shared-expert weights native (ATOM_FUSE_MXFP8_SHARED_INTO_MXFP4=0, noptpc_fp8conversion), with TP4, FP8 KV/index cache, block 128 and fixed 7-draft DSpark.Validation
Latest
mainplus #2686:183 passedamd/MiniMax-M3.1-Flash-Quark-MXFP4: all four ranks reportMiniMax-M3 mono decode on; width-8/16/24/32 builds and graph capture completeHistorical preview2 accuracy for the same native layouts:
0.962979.2 / 82.5 / 82.1%, mean81.3%Performance
Current PR head and
amd/MiniMax-M3.1-Flash-Quark-MXFP4, 4xMI355X TP4, native MXFP8 attention/shared expert and fixed 7-draft DSpark. Baseline and Mono arms differ only inATOM_MONO_ENABLE.SA-style AgentX validation (
inferencex-agentx-mvp, SemiAnalysis/Weka 256K traces, prefix cache, 900 s arms) with synthetic accepted length 4.0 to isolate target-path performance, Mono off -> on:137.83 -> 189.01 tok/s(+37.1%), p90 ITL4.32 -> 2.92 ms(-32.4%), p99 context243.4K -> 243.3K220.10 -> 264.58 tok/s(+20.2%), p90 ITL4.91 -> 4.01 ms(-18.3%), p99 context241.0K -> 242.5K246.32 -> 344.56 tok/s(+39.9%), p90 ITL5.03 -> 4.62 ms(-8.2%), p99 context221.5K -> 231.6KThese are 900-second PR-validation arms, not the 3600-second full InferenceX sweep. Synthetic acceptance is performance-only and is not used for accuracy claims.
S48/C6 was prototyped and excluded because the down path required private scratch, violating the persistent-kernel all-CTA residency requirement.