[GG] EXL3: consolidate mixed execution, MXFP8 overlay, and shared rotations

[voipmonitor]

Summary

Consolidate the three EXL3 changes that touched the same loader and planning code into one reviewable GG PR:

• retain the shape-aware mixed K3/K4 decode path and serial block-64 prefill path from [GG] exl3: shape-aware mixed decode and bounded serial prefill #222;
• retain the explicit MXFP8 overlay for BF16 linear/shared-expert weights from [GG] quant: add EXL3 BF16 MXFP8 online overlay #223;
• add the backward-compatible shared_h_v1 artifact layout from the original [GG] EXL3: consolidate mixed execution, MXFP8 overlay, and shared rotations #225.

Keeping one PR avoids merge-order conflicts and makes the combined memory, dispatch, and quantization contracts testable together.

Runtime contracts

Mixed K3/K4 execution

• Decode and small M use the one-grid mixed Trellis path.
• Larger prefills use homogeneous serial K3/K4 plans.
• VLLM_EXL3_PREFILL_CAPACITY optionally bounds persistent prefill scratch.
• Route weights, route IDs, output views, and CUDA graph behavior are preserved.

Online MXFP8 overlay

• EXL3 tensor-storage records remain controlled by Exl3LinearMethod/Exl3MoEMethod.
• Only BF16 dense/shared-expert weights are eligible for the MXFP8 overlay.
• Routed EXL3 experts reject incompatible MoE overlays.
• Existing ModelOpt/MXFP4 overlay behavior is unchanged.

Shared-H artifacts

• Missing rotation_layout remains legacy per_expert_v1.
• shared_h_v1 requires the exact shared_h_tensor_schema metadata.
• Gate/up SU and down SV are allocated as one physical [1,H] row and broadcast by stride/pointer contract without expansion.
• Mixed K3/K4 tiers retain the same single physical row in both one-grid decode and serial prefill.
• Invalid mixed legacy/shared tensor names fail closed.

For GLM-5.2, the shared-H layout removes 672.36 MiB/GPU of duplicated persistent rotations across 75 MoE layers.

Validation

• 47 passed: tests/quantization/test_exl3.py and tests/quantization/test_exl3_prefill_plan.py
• Includes an integration regression test for mixed K3/K4 + shared_h_v1 + serial prefill.
• Ruff check/format, py_compile, and git diff --check pass.
• The earlier matched TP4/DCP4 EXL3 validation from [GG] exl3: shape-aware mixed decode and bounded serial prefill #222 remains applicable:
  • 3k: 3,012 -> 3,792 tok/s
  • 32k: 2,855 -> 3,690 tok/s
  • 128k: 2,363 -> 3,324 tok/s
  • KV capacity: 804,864 -> 856,320 tokens

The shared-H loader is explicit and backward compatible. A complete newly encoded shared-H checkpoint still requires its own KLD and E2E release validation.

Supersedes #222 and #223.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds rank-sliced EXL3 support for shared H-side rotation tensors. It validates metadata, normalizes tensor names, stores shared rotations once per projection, supports broadcast pointer tables, and preserves shared rows during fused-MoE preparation.

Changes

Shared-H rank-sliced EXL3

Layer / File(s)	Summary
Metadata and tensor normalization vllm/model_executor/layers/quantization/exl3.py, tests/quantization/test_exl3.py	Rank-sliced metadata validates per-expert and shared-H layouts. Weight normalization maps valid shared-H tensors to canonical names and rejects invalid schemas and paths.
MoE broadcast storage and preparation vllm/model_executor/layers/quantization/exl3.py, tests/quantization/test_exl3.py	MoE layers allocate shared rotations with one row, validate broadcast storage, build expert pointer tables, select rotation rows, and preserve shared rows during fused-MoE preparation.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Exl3Config
  participant WeightNormalizer
  participant Exl3MoELayer
  participant FusedMoEPreparation
  Exl3Config->>WeightNormalizer: validate layout and normalize shared-H names
  WeightNormalizer->>Exl3MoELayer: provide canonical shared rotation parameters
  Exl3MoELayer->>FusedMoEPreparation: pass broadcast-aware rotation rows
  FusedMoEPreparation-->>Exl3MoELayer: retain shared rows and expert pointers

Loading

Possibly related PRs

• local-inference-lab/vllm#190: Extends the same EXL3 rank-sliced MoE implementation and test areas.
• local-inference-lab/vllm#222: Covers related EXL3 rank-sliced and fused-MoE rotation preparation.

Suggested reviewers: brandonmmusic-max

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title identifies shared rotations, which is a primary change, but it also mentions mixed execution and MXFP8 overlay without support in the provided changes.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/gg-exl3-shared-h-rotations-20260801

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Superseded by the direct-GG, self-contained integration in #228. The successor retains this EXL3 runtime and adds the qualified prefill policy plus persistent online K6 caching required by r20.