quant(fp8): bridge MXFP8 dense and NVFP4 experts

[voipmonitor]

Summary

Adds the missing FP8 checkpoint bridge for GLM-style hybrid checkpoints:

• dense_format=mxfp8 routes dense LinearBase weights through the serialized MXFP8 compressed-tensors method.
• store_dtype=nvfp4 routes MoE experts through the ModelOpt NVFP4 fused MoE method instead of the generic FP8 MoE path.

This is the runtime patch already present in the GLM 5.2 sweep image and is required for offline checkpoints such as GLM-5.2-MXFP8dense-NVFP4experts / GLM-5.2-FP8-NVFP4experts.

Validation

• python3 -m py_compile vllm/model_executor/layers/quantization/fp8.py
• git diff --check

Summary by CodeRabbit

• New Features

  • Added support for serialized MXFP8 dense linear layers, enabling this format to be selected through configuration.
  • Added support for NVFP4-based Mixture-of-Experts models when using the matching serialized format.
  • Introduced a new configuration option to choose the dense weight format.

• Bug Fixes

  • Improved quantization method selection so supported layer types now route to the appropriate quantized implementation automatically.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 400e4d27-f326-4836-9e19-7adc3ba5b65a

📥 Commits

Reviewing files that changed from the base of the PR and between c382f1d and d005934.

📒 Files selected for processing (1)

• vllm/model_executor/layers/quantization/fp8.py

---

📝 Walkthrough

Walkthrough

Adds a new Mxfp8SerializedLinearMethod adapter delegating dense linear weight creation/loading/apply to CompressedTensorsW8A8Mxfp8, extends Fp8Config with a dense_format option to select it for LinearBase layers, and adds an nvfp4 store_dtype branch in get_quant_method returning ModelOptNvFp4FusedMoE for RoutedExperts.

Changes

FP8 Quantization Method Extensions

Layer / File(s)	Summary
MXFP8 dense linear method and config wiring vllm/model_executor/layers/quantization/fp8.py	Adds Mxfp8SerializedLinearMethod delegating to CompressedTensorsW8A8Mxfp8, adds dense_format field to Fp8Config init/from_config, and updates get_quant_method for LinearBase to select it when dense_format == "mxfp8".
NVFP4 MoE method branch vllm/model_executor/layers/quantization/fp8.py	Adds a store_dtype == "nvfp4" branch in get_quant_method for RoutedExperts that constructs ModelOptNvFp4Config and returns ModelOptNvFp4FusedMoE.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LinearBase
  participant Fp8Config
  participant Mxfp8SerializedLinearMethod
  participant CompressedTensorsW8A8Mxfp8

  LinearBase->>Fp8Config: get_quant_method(layer)
  Fp8Config->>Fp8Config: check dense_format == "mxfp8"
  Fp8Config-->>LinearBase: Mxfp8SerializedLinearMethod()
  LinearBase->>Mxfp8SerializedLinearMethod: create_weights / apply
  Mxfp8SerializedLinearMethod->>CompressedTensorsW8A8Mxfp8: delegate scheme methods

Loading

sequenceDiagram
  participant RoutedExperts
  participant Fp8Config
  participant ModelOptNvFp4Config
  participant ModelOptNvFp4FusedMoE

  RoutedExperts->>Fp8Config: get_quant_method(layer)
  Fp8Config->>Fp8Config: check store_dtype == "nvfp4"
  Fp8Config->>ModelOptNvFp4Config: construct nv_cfg
  Fp8Config-->>RoutedExperts: ModelOptNvFp4FusedMoE(nv_cfg, moe_config)

Loading

Possibly related PRs

• local-inference-lab/vllm#20: Both PRs touch the ModelOptNvFp4FusedMoE path, with this PR adding the code that returns it from Fp8Config and the related PR fixing its weight-loading/preparation logic.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/fp8-modelopt-bridge-20260706

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}