[Quant] ModelOpt mixed precision: dispatch block-FP8 MoE experts and derive the block size

[zhendonghua]

Motivation

ModelOptMixedPrecisionConfig has a Linear entry for FP8_PB_WO but no FusedMoE entry for any block-FP8 algorithm, and it hard-codes the block-FP8 sub-config to [128, 128].

nvidia/Qwen3.8-Flash-Next-NVFP4 lists its MTP experts (mtp.layers.0.mlp.experts) as block-FP8: FP8_BLOCK_SCALES in hf_quant_config.json, the canonical FP8_PB_WO in the inline config.json copy (SGLang currently reads the file because the inline copy carries no KV-cache key, but either name can win). Both resolve to "no quant method" for FusedMoE, so the experts are built unquantized: the loader casts the fp8 values to bf16 without their block scales and silently skips weight_scale_inv. The server starts and target accuracy is unaffected, but the draft is numerically wrong (accept length ~1.6 instead of ~3).

Mirrors vllm-project/vllm#55513.

Modifications

• _BLOCK_FP8_ALGOS = ("FP8_PB_WO", "FP8_BLOCK_SCALES"); Linear and FusedMoE both dispatch on it, FusedMoE to Fp8MoEMethod(self.fp8_pb_wo_config) (block scales via weight_scale_inv, dynamic activation).
• The block-FP8 sub-config's block size comes from the checkpoint's group_size (default 128); a MIXED_PRECISION map whose block-FP8 layers disagree raises.

Test

• test_modelopt_loader.py::TestModelOptMixedPrecisionConfig: both names dispatch to Fp8MoEMethod with [128,128]; block size follows group_size; conflicting sizes raise. Verified red on the pre-change code.
• Existing mixed-precision tests in that class still pass.
• E2E together with the follow-up Qwen4-Exp PR: nvidia/Qwen3.8-Flash-Next-NVFP4, 4x B300, TP4, NEXTN 3/1/4: sgl-eval GSM8K (200, thinking) 0.97, accept length ~3.2-3.6.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel if you have any questions.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #34401732103
Latest PR Test (Extra): ❌ Run #34401731840
Latest PR Test (AMD ROCm 10): ❌ Run #34401731988

[zhendonghua]

/tag-and-rerun-ci

[zhendonghua]

/rerun-failed-ci