[Fix] MiniMax-M3: default to the breakable prefill CUDA graph

[thanhhao98]

Motivation

MiniMax-M3 defaults to the tc_piecewise prefill CUDA graph, and on main that launch dies at boot (reproduced on nightly-dev-cu13-20261005-f70e8c68, GB300 TP4):

Capture prefill CUDA graph failed: Unsupported object.__new__ user-defined class construction
  ... object.__new__(DeclaredSum) is not safe

tc_piecewise is being removed (#41634), so this PR moves M3 to breakable. That needed one fix: under breakable, RadixAttention ran M3's sparse attention inline instead of as an eager break, so the graph replayed the capture batch's metadata and the server died on its first requests with CUDA error: an illegal memory access was encountered.

Modifications

• Run sparse attention as an eager break under breakable, like dense attention and like DSA's indexer and QSA ([QSA] Enable breakable prefill CUDA graphs for text-only Qwen3.8 Flash-Next (capture-safe metadata, MTP side-channel padding) #41729).
• Add both M3 archs to the breakable allowlist, so M3 defaults to breakable.
• A routing test fails without the fix.

Accuracy Tests

End to end on GB300, MiniMax-M3-NVFP4: this PR at its default (breakable) vs main with the prefill graph disabled, each pair on one node.

config	GSM8K 1319 q, this PR	GSM8K 1319 q, main without prefill graph
TP4	0.870	0.864
TP4, EP4, DP attention 4	0.864	0.877

Across our other runs (two nightlies, with and without DP attention), both settings stay within 0.864-0.877. Image requests replay the graph too: a colored-shape probe (nine images, sent one at a time and all at once) answers 18/18 in every arm. Every arm also passes a bs=1 GSM8K canary and an EOS-termination check.

Speed Tests

Same pairs, ISL 8192 / OSL 1024, median TTFT and output throughput:

config	concurrency	TTFT (ms), this PR / main	tok/s, this PR / main
TP4	1	181 / 232	205 / 203
TP4	16	1502 / 1505	1441 / 1429
TP4	64	4868 / 4946	2838 / 2812
TP4, EP4, DP attention 4	1	505 / 438	112 / 112
TP4, EP4, DP attention 4	64	4313 / 4670	2854 / 2699

With DP attention the graph makes every DP rank replay a shared MAX_LEN bucket, so at concurrency 1 the idle ranks run the busy rank's length. TPOT is unchanged except DP attention at concurrency 64 (19.13 to 18.15 ms).

Reproduce

Container lmsysorg/sglang:nightly-dev-cu13-20261005-f70e8c68 with python/sglang at this PR's head (baseline: main efb62ce269), checkpoint nvidia/MiniMax-M3-NVFP4, one 4-GPU GB300 node.

# This PR (breakable is the default). Baseline: the same command on main plus --cuda-graph-backend-prefill disabled.
# DP-attention rows add: --ep-size 4 --enable-dp-attention --dp-size 4
SGLANG_DISABLE_MSA=1 python3 -m sglang.launch_server --model-path nvidia/MiniMax-M3-NVFP4 --trust-remote-code \
  --tp 4 --context-length 1048576 --reasoning-parser auto --tool-call-parser auto --port 30000

# GSM8K: 5-shot, temperature 0, 128 in parallel. The bs=1 canary runs --num-questions 40 with --parallel 1 and with --parallel 40.
python3 -m sglang.test.few_shot_gsm8k --num-shots 5 --num-questions 1319 --port 30000

# Speed: --num-prompts 12 / 64 / 256 for --max-concurrency 1 / 16 / 64.
python3 -m sglang.bench_serving --backend sglang --base-url http://127.0.0.1:30000 \
  --dataset-name random --random-input-len 8192 --random-output-len 1024 --random-range-ratio 1.0 \
  --flush-cache --max-concurrency 1 --num-prompts 12

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #37870659321
Latest PR Test (Extra): ❌ Run #37870659081
Latest PR Test (AMD ROCm 10): ❌ Run #37870659400

[nvpohanh]

MiniMax-M3 defaults to the tc_piecewise prefill CUDA graph

@thanhhao98 Could we try to set default to breakable? I think tc_piecewise is going to be deprecated/removed

[thanhhao98]

@nvpohanh should I fix for tc_piecewise and change to breakable; or just need to change default to breakable?

[nvpohanh]

@thanhhao98 tc_piecewose will be removed soon, so let's just change it to breakable if it works.
@ch-wan and @Oasis-Git will remove tc_piecewise

[thanhhao98]

@nvpohanh I updated this pr to move to breakable only.

[nvpohanh]

[by Claude Code] Severity: style | Confidence: High

Issue

The Accuracy Tests and Speed Tests tables give the model, GPU, and parallel configs, but no commands. Nobody can rerun the GSM8K scores or the TTFT and throughput rows without guessing the server flags, the eval settings, and the benchmark settings.

Fix

Add the exact commands for one config, and note what changes for the other:

• the sglang.launch_server command for this PR and for the main baseline, including the flag that turns off the prefill graph (for example --cuda-graph-backend-prefill disabled);
• the GSM8K eval command (script, number of shots, parallelism);
• the sglang.bench_serving command (dataset, ISL/OSL, number of prompts, concurrency).

Also give the checkpoint path for MiniMax-M3-NVFP4.

[nvpohanh]

/rerun-test test/registered/unit/layers/test_radix_attention.py test/registered/cuda_graph/breakable/test_breakable_cuda_graph.py test/registered/unit/server_args/test_server_args.py

[github-actions]

🚀 ubuntu-latest (2 tests): ✅ View workflow run

cd test/ && python3 registered/unit/layers/test_radix_attention.py
cd test/ && python3 registered/unit/server_args/test_server_args.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/cuda_graph/breakable/test_breakable_cuda_graph.py

[kevin-mii]

/tag-and-rerun-ci

[nvpohanh]

@ispobock could you review this? Thanks!

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

[by Claude Code] I've started babysitting the NVIDIA CI of this PR. I'll check it every 30 minutes until all NV pipelines pass. If a relevant job fails, I may rerun it, merge the latest main into this branch, push a small test-only fix, or post a note here.

Anyone can stop me at any time: just post a comment asking me to stop (for example, stop babysitting). I'll stop at my next check, within 30 minutes, and take no further action on this PR.

[nvpohanh]

/rerun-failed-ci

[nvpohanh]

[by Claude Code] All NV pipelines have passed.