[whx-sjtu]

Overview

Add opt-in MiniMax-M3 sparse decode through ATOM's tensor library, with shared native weights and per-batch eligibility. Companion: ROCm/ATOM#2492.

vLLM model: validate loaded weights + dispatch eligible batches + own lifetime
  -> AtomM3Mono(layer_specs, cache_specs, tp_context)
     prepare_step / forward_layer / close
ATOM: metadata expansion, compilation, scratch, IPC, fused kernels

This is an alternative dependency boundary to #59705 (vendored kernels) and #59653 (AITER port): ATOM owns the kernels, and both paths borrow the same loaded weights. AI assistance was used, with independent Standards, Spec, and router/kernel reviews before committing.

Claims

• Integration stays in the MiniMax-M3 ROCm model. The ordinary worker/model runner are used; no common worker, runner, core, or platform code changes. This addresses the review request to remove common-path changes.
• All sparse layers and TP ranks must have compatible prepared weights: AITER-preshuffled per-channel E4M3 QKV/O with FP32 scales, BF16 router weights with FP32 logits, and supported shuffled MXFP4 experts. ATOM creates no quantized/cast/shuffled weight copies. Existing vLLM online quantization supplies the projection format; Hipb storage is normalized through a transpose view.
• Incompatible weights/configurations collectively retain native execution. Compatible decode batches with at most four padded request rows can use mono; larger, prefill, mixed, and unsupported shapes run natively. A large batch does not disable a later eligible small batch.
• Router storage defaults to FP32. Explicit text_config.router_dtype=bfloat16 selects BF16 weights for the VL checkpoint while keeping logits, sigmoid, and correction bias FP32. The mono flag never changes router precision.

Validation

Before: the adapter/library produced private FP8 projection and BF16 router copies, and restricted configured concurrency to four. After: native and mono share prepared storage; compatibility and current batch shape determine dispatch.

October 8, gfx950/256 CU, GPUs 0–3, TP4, MiniMax-M3-MXFP4, EAGLE3 three speculative tokens with standard acceptance, online per-channel FP8 QKV/O:

Check	Result
Model eligibility/lifecycle tests	14 passed; selected by AMD basic-model CI
Paired ATOM runtime/layout/dispatch/library/build-key suites	182 passed
Native AITER preshuffled QKV/O GEMMs, token counts 1/4/16/64/8192	10 checks passed against dequantized FP32 reference
Current mono startup	All four ranks initialized temporary caches, closed them, initialized final caches, and captured CUDA graphs
Current GSM8K pilot	FP32 router/native fallback 62/64; BF16 router/native 63/64; same BF16/FP8 weights with mono 62/64
Current online concurrency/cache transition checks	12/12 requests completed through 8→4→8→4 live-cache transitions; GPU trace confirms native/mono execution. Broad 1K/8K smoke remains incomplete

All three pilots had no API errors, truncations or parse failures. A used the default FP32 router with mono requested and collectively refused; B and C shared BF16 router/FP8 projection weights. C used a 0.70 memory reservation before other jobs appeared; A/B used 0.25 with the same 1024 cache blocks and request settings. These 64-question samples do not establish accuracy parity or performance.

A fixed-activation CPU probe isolated router weight rounding on 112 token rows from layers 3 and 20. BF16-rounded weights with FP32 matrix multiplication changed two top-4 expert sets versus FP32 weights (maximum absolute logit difference 0.005854). This is evidence that rounding can affect routing, not an end-to-end accuracy guarantee.

Commands and evidence scope

Commands used the existing environment through uv run --no-project --python /usr/bin/python, with ATOM's vLLM plugin disabled:

# vLLM
python -m pytest -q --confcutdir=tests/models tests/models/test_minimax_m3_mono.py
pre-commit run --files vllm/models/minimax_m3/amd/model.py vllm/models/minimax_m3/amd/mono.py vllm/models/minimax_m3/amd/mono_weights.py tests/models/test_minimax_m3_mono.py docs/design/minimax_m3_atom_mono.md
pre-commit run mypy-3.12 --hook-stage manual --files vllm/models/minimax_m3/amd/model.py vllm/models/minimax_m3/amd/mono.py vllm/models/minimax_m3/amd/mono_weights.py

# ATOM
python -m pytest -q tests/test_mono_runtime.py tests/test_minimax_m3_mono_layout.py tests/test_minimax_m3_mono_dispatch.py tests/test_minimax_m3_mono_library.py tests/test_minimax_m3_mono_build_key.py

# External frozen-request evaluator, each arm
python scripts/rfc001/gsm8k.py --source artifacts/rfc001/baseline-audit-20261002/requests/enabled --output <arm-output> --limit 64
python scripts/rfc001/online_smoke.py --output <smoke-output> --concurrencies 8 5 4 2 1

Applicable pre-commit hooks, mypy 3.12, ATOM Ruff/Black, and whitespace checks passed. Current evidence: artifacts/rfc001/shared-weights-20261008/. External checkpoint/request fixtures are not included in these PRs. Frozen request-file SHA256: 94b69013c48651fced9936a139c3c80c7ed0548267c9e66b335577e7f093d270; temperature 0, max output 8192, checkpoint chat template with thinking enabled.

The design document contains the launch command. The tested native backend is AITER preshuffled; this environment's Hipb backend lacked a rowwise FP8 algorithm for the large QKV prefill shape. Local runs also pinned ROCR_VISIBLE_DEVICES=0,1,2,3 and TRITON_LIBHIP_PATH=/opt/rocm/lib/libamdhip64.so after startup diagnostics.

Details

Scope: gfx950/256 CU, TP4/PP1/DP1, V2 runner, 57 sparse layers, max length 16384, FP8 main/index cache block128, shared-expert fusion, EAGLE3 three tokens, prefix caching disabled. Scheduler concurrency may exceed four; each mono batch has at most four padded request rows. Explicit num_gpu_blocks_override reserves room for lazy allocation; the launch uses 1024 blocks.

The first eager forward with bound caches initializes the library. The existing model cache detach closes it collectively after graph teardown, including temporary profiling caches. Weight transfer, LoRA, KV transfer, sleep and unsupported parallel configurations disable mono. Weights and captured addresses must remain stable; direct weight writes, partial cache replacement and raw/kernel-format reload require rebuilding the execution state. Eager execution and CUDA graphs are supported; direct torch.compile of the library is rejected because functionalization can replace storage referenced by device descriptors.

Merge Danger

Door: two-way, opt-in, no checkpoint migration. Blast Radius: MiniMax-M3 ROCm and the paired ATOM library.

The profiler trace was saved, but profiler-summary export blocked subsequent requests and the diagnostic process was stopped. A later mono restart with external GPU jobs present and a 0.25 memory reservation stalled after final runtime preparation; native B and A completed under those conditions. The cause is unresolved. The persistent kernel assumes 256 CTAs can reside together; dedicated GPU capacity is required for further qualification. Long-input smoke and normal mono shutdown still need a dedicated-GPU rerun.

Still Draft: full 1319-question accuracy, performance, native ATOM engine requalification and human line-by-line review remain outstanding. Previous converted-weight model evaluations and saved-layer replay results do not qualify this shared-weight revision. Shared weights do not imply bitwise-identical kernels. The replay harness now requires new online-FP8/BF16-router captures.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).
• AI assistance was used during the creation of this PR.
• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
• Testing and Validation: Includes behavioral tests and local model validation; external GPU fixtures and qualification limits are documented.
• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
• Pull Request Contents: Includes relevant links, supporting evidence, tradeoffs, and contribution-guide requirements.

[mergify]

Documentation preview: https://vllm--60528.org.readthedocs.build/en/60528/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @whx-sjtu.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @whx-sjtu.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[whx-sjtu]

Might reopen when ATOM import path is needed again.