[jlqibm]

Motivation

Stacks on #41288 (fix/compressed-tensors-mxfp4-marlin), which fixed MXFP4 dense
linear. This fixes MXFP4 MoE.

A compressed-tensors mxfp4-pack-quantized MoE checkpoint whose experts are
stored split per expert (experts.{e}.{gate,up,down}_proj.*, as emitted by
llmcompressor) cannot load today. The A/B control, same node and same venv:

• On [Fix] compressed-tensors: Serve MXFP4 dense linear via FP4 Marlin #41288 : KeyError: 'model.layers.0.block_sparse_moe.experts.w2_weight_packed'
  at granitemoe.py:93.
• On this PR: loads, 0 tensors dropped, coherent output.

The cause is a layout mismatch, not a missing kernel. Mxfp4MoEMethod and
Mxfp4MarlinMoEMethod handle the fused GPT-OSS layout and register
w13/w2_weight; a split-expert checkpoint's per-expert tensors have no
destination there.

Modifications

Add CompressedTensorsW4A16Mxfp4MoE, a weight-only MXFP4 MoE scheme registering
the _packed parameter names and routing to the FP4 Marlin MoE runner. It serves
both mxfp4a16 and mxfp4 w4a4; the latter's activation quantization is
dropped, since the Marlin MoE kernel is weight-only. A w4a4 checkpoint still
ships input global scales and the split-expert loader raises KeyError on any
tensor with no registered destination, so they are registered and only then
dropped in process_weights_after_loading.

Unlike NVFP4 there is no weight_global_scale, so the compressed-tensors
reciprocal convention (CT stores 1/scale where Marlin wants scale) does not
arise here.

check_moe_marlin_supports_layer is deliberately not called: it tests
group_size against the GPTQ/AWQ set and reports on int4 zero-point parameters
this layout does not have. The shape constraints it would have covered
(hidden_size % 128 == 0, intermediate_size_per_partition % 64 == 0) are
validated explicitly instead.

How the two layouts are told apart

They cannot be told apart from the config. GraniteMoe and Kimi-K3 ship
indistinguishable mxfp4-pack-quantized configs — same quant_method, same
format, config_groups targeting ["Linear"] in both, both using
make_expert_params_mapping. The real difference lives in the model loader:
Kimi-K3 renames weight_packed→weight and substitutes Mxfp4Config before
building its experts.

So the selection is an explicit opt-out rather than config sniffing: FusedMoE
gains a serves_fused_mxfp4: bool = True constructor parameter, and
granitemoe.py passes False. It must be a constructor parameter, not a
post-construction attribute, because get_quant_method runs inside
FusedMoE.__init__.

Every existing fused MXFP4 MoE keeps the default and is unaffected. That is
asserted in both directions by a new test, since a silent kernel change on the
GPT-OSS path would be the worst failure mode here.

Accuracy Tests

Granite MoE (GraniteMoeForCausalLM), H100 80GB, all coherent text:

model	TP	mxfp4	mxfp4a16	nvfp4	nvfp4a16
20b	1	OK	OK	OK	OK
20b	2	OK	OK	OK	—
20b	4	OK	OK	OK	—
120b	4	OK	OK	—	—

13 [OK], rc=0, 0 tensors dropped. 120b is independently valuable: 128 experts
vs 56, hidden 3072 vs 2048.

18 CPU scheme-selection tests + 8 subtests, covering the MXFP4/NVFP4 predicate
boundary (both are FP4 group quant; only group_size and strategy separate
them, so a widened predicate would silently steal the other's checkpoints), the
registered-parameter contract against the checkpoint's names/shapes/dtypes, and
the fused-bypass gate in both directions.

Dense MXFP4 output is byte-identical to PR 2's, confirming the MoE scheme does
not perturb the dense path.

The fused GPT-OSS MXFP4 MoE path is covered by the CPU gate test rather than
end-to-end: GPT-OSS uses native Mxfp4Config, so it never enters
CompressedTensorsConfig.get_quant_method, and no compressed-tensors fused MXFP4
MoE checkpoint was available to test against. The gate test asserts the dispatch
branch directly, which is a stronger check than inferring it from generated text.

Speed Tests and Profiling

No benchmark run. Weight-only via the existing FP4 Marlin MoE runner — no new
kernel, and activations stay bf16. get_min_capability() is 80 and the routing
is capability-independent, so Blackwell takes the same MoE path as Hopper. The one
asymmetry: an NVFP4 w4a4 MoE checkpoint on SM100 still keeps the native
CompressedTensorsW4A4Nvfp4MoE scheme (capability 100); the weight-only fallback
must not lower that gate, which is why they are separate classes and is asserted
by test_nvfp4_w4a4_keeps_native_scheme_on_blackwell.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

🤖 Generated with Claude Code

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ✅ Run #40493897581
Latest PR Test (Extra): ❌ Run #40493897438
Latest PR Test (AMD ROCm 10): ⏳ Run #40493897389

[BBuf]

This branch includes the dense MXFP4 bias issue described in #41288 (comment). I reproduced the same Python-side mismatch at this PR's head, 8ba1730d08dda0bea18a29477db5f243d0482826: preparation permutes bias, but the dense apply path adds it to normal-order GEMM output.

Please carry the fix and bias regression coverage from #41288 into this PR as well. The detailed reproduction and its CPU-only validation scope are in the linked comment.

[BBuf]

Approving the MoE changes. Please also bring in the dense MXFP4 bias fix and regression coverage from #41288, as noted in my earlier comment.

I'll trigger run-ci after the fix is pushed. Holding off on CI for now.

[jlqibm]

@BBuf the bias fix for #41288 is merged here as well. Can you please issue run-ci?

[jlqibm]

@BBuf can you please trigger run-ci? Fixes are merged.