[II] Serve projection-mixed EXL3 K3/K4/K5 weights

[voipmonitor]

Status

• Implementation: implemented
• Unit and kernel qualification: qualified
• GLM-5.2 checkpoint E2E qualification: pending

Purpose

Serve EXL3 checkpoints that assign MCG K3, K4, and K5 independently to routed-expert gate, up, and down projections. The B12X runtime reads packed checkpoint storage directly and executes all projection tiers through one CUDA-graph-capturable mixed-Trellis launch plan.

Resulting behavior

• Represent tier membership per expert and per projection with exact gate, up, and down bounds.
• Validate descriptor counts, packed extents, route identifiers, and projection counts before launch.
• Keep expert counts as runtime data so layer-dependent partitions reuse compiled kernels.
• Prewarm scheduler capacity, rank-sliced draft, route-identifier, and scalar-alignment specializations before graph capture.
• Preserve graph-stable scratch ownership for decode and prefill.
• Preserve QSRT atoms-v2 compact-offset pair modes and the qualified FC2 32x512 tile contract.

Compatibility

The MCG codebook accepts K3, K4, and K5 for this mixed runtime. SQG-XOR-Cheb-T12 retains its K2/K3/K4 contract. Existing QSRT paths and the removed route-major W4A8 interface are unchanged. Malformed or incomplete projection maps fail closed.

The vLLM checkpoint loader is provided by a companion pull request against dev/infernal-invocation.

Validation

Source base: ac07f6746edce4852f01cc11f34dbe7abae1fdaa.

• 58 host-runnable focused tests passed; 40 GPU-only cases skipped.
• 6 SM120 tests passed for K3/K4/K5 numerical parity, projection-tight payloads, CUDA graph replay, K6 small-M dispatch, one-grid prefill planning, and GLM-5.2 large-M execution.
• Ruff check and format: passed.
• git diff --check: passed.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3fd7a793-4955-485b-b42e-a85dbb114b09

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[lukealonso]

Closing without merge because this branch is incompatible with the master execution contract. It adds an entirely new projection-mixed Trellis kernel path, has unresolved conflicts in the shared W4A16 kernels, and adds dense K6 dispatch through b12x.gemm.trellis_linear._small_m, which is absent from master. The branch also depends on unmerged PR #146 and does not include checkpoint-level end-to-end qualification.

The host-only route-pack specialization prewarm idea is separable and should be proposed independently from master with focused capture evidence.