[KK] Load DS4.1 and Kimi-K3 lossless X4T checkpoints

[voipmonitor]

Behavior

Load DeepSeek V4.1 and Kimi-K3 lossless X4T checkpoints through B12X. X4T compresses MXFP4 scale grids without changing the weight nibbles or W4A16 arithmetic; it is not two-bit Trellis quantization.

Requires B12X #446.

• Register --load-format exact_mxfp4. Use --quantization exact_mxfp4 for DS4.1 or --quantization kimi_x4t for Kimi-K3.
• Load routed experts into prepared B12X weights, sharing caller-owned scale scratch between serialized layers. Kimi uses its SiTU(4,25) activation and native TP16 expert geometry without padding.
• Preserve dense/shared/draft source precision unless an independent online-quantization option is explicitly enabled. DS4.1 file-backed Engram and draft-only filtering remain intact.
• Bound CPU shard-copy threads while reading small expert slices, restoring the serving thread setting even when loading fails.

Set quantization_config.format_version=1 and checkpoint_root to the absolute mounted X4T directory. Kimi uses the same metadata in its nested text configuration. Native MXFP4 and EXL3 checkpoint paths are unchanged.

Compatibility

SM120/SM121, BF16 expert activations. Reader geometry: DS4.1 TP1/2/4/8; Kimi TP1/2/4/8/12/16. Qualified full-model topologies: DS4.1 TP4 and Kimi TP16. Other topologies are implemented geometry, not serving qualifications.

PP, ubatching, EP, DP and expert remapping are unsupported. Shared scratch requires serialized execution on one CUDA stream. No native CUDA extension, speculative algorithm or KV representation is added.

Validation

• Implemented and component-qualified: 112 tests on the published loader code and B12X [II] Reserve Kimi-K3 AttnRes prefill workspace #446 with CUTLASS DSL 4.7.1: tests/quantization/test_ds41_exact_mxfp4.py and tests/quantization/test_exl3.py. Coverage includes nested Kimi metadata, BF16 dense preservation, SiTU selection, thread restoration, compressed expert filtering, DS4.1 Engram and EXL3 invariants. The earlier DS4.1-only qualification also included loader-registry tests (122 total).
• Qualified serving composition: Kimi X4T TP16/DCP16, DSpark K7, CC8, p4096, FP8 KV and selected MXFP8 KDA projections. Three coding runs and eight simultaneous 32k-prompt requests pass without OOM, preemption or restart. Replaying this PR and the companion PRs produces the identical complete source tree used in the measured image, not a Python overlay. Source lock and artifact bundle: integration issue #906.
• Research-only timing: Kimi game-coding median 116.72 decode tok/s (three runs, 34.55% median acceptance); 32k prefill 3740.41 tok/s. Sixteen RTX PRO 6000 Blackwell GPUs, 600 W/+6000 memory offset, approximately 16365 MHz memory. Reliability event mask 0x400 prevents an unthrottled performance qualification. CUTLASS 4.6.2 was used for the full-model run; no 4.7.1 full-model claim is made.
• DS4.1 TP4 companion evidence remains at validation/x4t/serving.json: X4T/native pooled decode 355.6/357.5 tok/s and 32k prefill 17907/18041 tok/s under its recorded hardware limits.

Full-model KLD and 1M-token generation are not qualified. AI assistance: OpenAI Codex. Review requested for checkpoint identity, serialized scratch ownership and Kimi activation selection.

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 68422d0c-2aa7-480a-85da-507ec45b84e7

📥 Commits

Reviewing files that changed from the base of the PR and between ab86b70 and 120a909.

📒 Files selected for processing (12)

• .lil/changes/vllm-kimi-x4t-experts.json
• tests/quantization/test_ds41_exact_mxfp4.py
• vllm/config/load.py
• vllm/config/model.py
• vllm/model_executor/layers/fused_moe/b12x.py
• vllm/model_executor/layers/quantization/__init__.py
• vllm/model_executor/layers/quantization/kimi_x4t.py
• vllm/model_executor/model_loader/__init__.py
• vllm/model_executor/model_loader/exact_mxfp4_loader.py
• vllm/models/deepseek_v4_1/exact_mxfp4.py
• vllm/models/deepseek_v4_1/nvidia/model.py
• vllm/models/kimi_k3/nvidia/model.py

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Closing as not needed: no shipped docker model uses the DS4.1/Kimi-K3 X4T loaders. DeepSeek-V4.1 is served from its lossless MXFP4-CSF checkpoint. It can be reopened if X4T checkpoints become a target again.