[zhuyuhua-v]

Summary

This PR adds a MiniMax-M3.1-Flash — TP4 + DSpark section to recipes/MiniMax-M3-Agentic-InferenceX.md, plus a pointer to it at the top of the file. The M3 (EAGLE3) content is unchanged.

The recipe lists only the best-performing configuration. The sweeps that chose it are below.

CONC	configuration	PRs
1–4	FP8 KV / index + Mono + prefill delayer (ATOM_PREFILL_DECODE_INTERVAL=20000)	#2686, #2743
8–16	FP8 KV / index + Mono	#2686, #2743
32–48	NVFP4 KV + FP4 index + fused verify top-k	#2744, #2743

Every band uses DSpark K=7, --online_quant_config ptpc_fp8 (which #2743 also applies to the draft), dense cudagraph sizes and --gpu-memory-utilization 0.9.

Docs only. This stays a draft until #2686, #2743 and #2744 land.

Recipe numbers (3600 s per point, vs SGLang CI run 40110172586)

CONC	tok/s/GPU (SGLang)	P90 intvty (SGLang)	p50 intvty (SGLang)
1	43.6 (41.42)	558.3 (420.2)	588.5 (460.8)
2	51.0 (50.11)	566.7 (365.0)	611.1 (467.3)
4	63.2 (63.09)	556.2 (347.2)	616.6 (478.5)
8	113.0 (110.59)	295.7 (314.5)	456.0 (392.2)
16	181.6 (181.86)	185.7 (226.2)	313.7 (324.7)
32	324.1 (335.4)	106.6 (141.6)	178.5 (242.7)
48	434.8 (436.1)	67.8 (70.9)	116.0 (145.3)

gsm8k strict-match (InferenceX CI method, real acceptance):

• FP8 + Mono, K=7: 0.9682 ± 0.0048;
• NVFP4 ([MiniMax-M3.1] FP4 index cache and NVFP4 sparse KV cache #2744 + [MiniMax-M3.1] DSpark draft follows --online_quant_config #2743 on main): 0.9644 ± 0.0051;
• SGLang CI: 0.9598 ± 0.0054.

Sweep 1: speculative tokens K = 3 / 5 / 7, Mono on

Each K uses its InferenceX golden accepted length: K3 2.88, K5 3.46, K7 4.04, from infx/golden_al_distribution/minimaxm31flash_dspark.yaml.

Mono serves decode steps of up to 16 rows, so it covers bs ≤ 4 at K=3 and bs ≤ 2 at K=5 and K=7.

Each K passed a gsm8k gate with Mono serving the steps (real acceptance): K3 0.9659, K5 0.9712, K7 0.9682.

FP8 band, 3600 s, MI355X TP4:

CONC	tok/s/GPU, K=3 / 5 / 7	P90 intvty, K=3 / 5 / 7
1	43.0 / 43.2 / 44.1	488.4 / 506.6 / 539.5
2	49.2 / 49.7 / 50.6	431.6 / 438.6 / 450.4
4	64.2 / 65.1 / 65.4	383.7 / 389.3 / 426.3
8	112.8 / 111.2 / 113.0	292.0 / 275.5 / 295.7
16	179.7 / 179.3 / 181.6	154.7 / 162.6 / 185.7

K=7 is best on both metrics at every concurrency. Mono's wider coverage at K=3 does not make up for its lower accepted length (2.88 tokens per step against 4.04).

Sweep 2: prefill delayer, ATOM_PREFILL_DECODE_INTERVAL=20000 (K=7, Mono on)

Each cell is off → on.

CONC	tok/s/GPU	P90 intvty	TTFT p90	ticks held
1	44.1 → 43.6 (−1.1%)	539.5 → 558.3 (+3.5%)	0.92 → 0.94 s	3%
2	50.6 → 51.0 (+0.8%)	450.4 → 566.7 (+25.8%)	0.98 → 1.17 s	17%
4	65.4 → 63.2 (−3.4%)	426.3 → 556.2 (+30.5%)	0.88 → 2.10 s	30%
8	113.0 → 108.4 (−4.1%)	295.7 → 362.6 (+22.6%)	0.73 → 6.58 s	56%
16	181.6 → 161.4 (−11.1%)	185.7 → 239.9 (+29.2%)	0.77 → 14.2 s	83%

"Ticks held" is hold_decode_interval / total scheduler ticks, from the delayer's own stats.

With 20000, a new prefill waits until no request is decoding.

• CONC 1–4: this adds 3–31% P90 interactivity within 3.4% of throughput, so the recipe turns it on.
• c8 and c16: it holds 56–83% of ticks. Throughput drops 4–11% and TTFT p90 reaches 6.6–14.2 s against SGLang's 0.69–0.74 s, so the recipe leaves it off.

A smaller interval (2–8, with ATOM_PREFILL_DELAYER_MAX_QUEUE_MS) has not been tried at c8–c16.

Provenance

• c1–c16 and both sweeps: 2026-10-27, image rocm/atom-dev:nightly_202610261558.
• c32 and c48: 2026-10-26, image nightly_202610251529.
• All on the PRs' first revisions. The reworked [MiniMax-M3.1] FP4 index cache and NVFP4 sparse KV cache #2744 + [MiniMax-M3.1] DSpark draft follows --online_quant_config #2743 reproduce c48 over 1200 s: 397.6 tok/s/GPU and P90 71.4, against 396.7 and 70.5.
• Prefix hit is token-weighted (atom:prefix_cache_cached_tokens / atom:prompt_tokens).

🤖 Generated with Claude Code

[github-actions]

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

• ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

• ✅ Run after the PR is approved and Pre Checkin passes
• ✅ Run immediately when an approval review is submitted
• ✅ Can be requested before approval with labels

Label	Tests
ci:full	Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom	Run native ATOM model accuracy tests
ci:vllm	Run ATOM vLLM OOT model accuracy tests
ci:sglang	Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2747 --add-label <label>