[Quant][Feature] Support online MXFP8 quantization for MoE and dense models

[EdalatiAli]

Purpose

Add support for online MXFP8 quantization (--quantization mxfp8), enabling BF16/FP16 models to be dynamically quantized to MXFP8 (microscaling FP8 with block-32 scales) at load time — for both linear layers and MoE expert layers.

This is powered by the FlashInfer kernels

• trtllm_fp8_block_scale_moe for MoE layers (see Add support for ModelOpt MXFP8 MoE models #35986)
• mm_mxfp8 for linear layers (see Integrate flashinfer mm_mxfp8 in ModelOpt MXFP8 #35053 )

This PR implements part of the online quantization support proposed in #32029 and #32412

Usage

# Serve any BF16 model with online MXFP8 quantization
vllm serve <model> --quantization mxfp8

# Or with layer skipping via config.json:
# "quantization_config": {
#     "quant_method": "mxfp8",
#     "modules_to_not_convert": ["lm_head", "mlp.gate"]
# }

Requires SM 100+ (Blackwell) GPU.

Test Plan

E2E tests for both a dense model (Qwen/Qwen3-0.6B) and a MoE model (Qwen/Qwen3-30B-A3B): logprobs comparison against BF16 baseline + generation smoke test.

python -m pytest tests/models/quantization/test_mxfp8.py -v

In addition, we report the the accuracy on MMLU_pro and GM8K using lm_eval_harness as well as performance benchmarks for Qwen/Qwen3-30B-A3B.

Test Result

Accuracy

lm_eval \
  --model vllm \
  --trust_remote_code \
  --model_args pretrained=Qwen/Qwen3-30B-A3B,quantization=mxfp8,enforce_eager=True \
  --tasks gsm8k,mmlu_pro \
  --batch_size auto

Task	BF16	MXFP8
GSM8K (flexible-extract)	85.7	87.8
GSM8K (strict-match)	89.7	88.2
MMLU-Pro	69.2	68.9

Performance

vllm bench throughput --model Qwen/Qwen3-30B-A3B \
--tensor-parallel-size 1 \
--trust-remote-code \
--async-scheduling \
--backend vllm \
--dataset-name random \
--random-prefix-len 0 \
--random-input-len 1024 \
--random-output-len 1024 \
--max-num-seqs 128 \
--num-prompts 512 \
--quantization mxfp8    ## Remove for the bf16 model

BF16 performance

Throughput: 7.73 requests/s, 15833.85 total tokens/s, 7916.92 output tokens/s

MXFP8 performance

Throughput: 10.34 requests/s, 21179.37 total tokens/s, 10589.68 output tokens/s

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request introduces support for online MXFP8 MoE quantization. The changes are comprehensive, adding a new Mxfp8Config and Mxfp8OnlineMoEMethod, updating backend selection logic, and modifying the flashinfer_fused_moe_blockscale_fp8 custom op. The implementation correctly handles the specifics of MXFP8, such as block shapes and scale types. I've identified a critical bug in the Mxfp8OnlineMoEMethod.create_weights method related to parameter naming that would cause a runtime error. My review includes a suggested fix for this issue. Overall, the changes are well-structured to integrate the new quantization mode.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @EdalatiAli.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @EdalatiAli.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mgoin]

It seems like you assert SILU but don't restrict selection in _supports_activation, is this necessary?

[EdalatiAli]

Yes, it's necessary — the monolithic class serves both block-scale (SILU only, hardcoded in flashinfer) and per-tensor (SILU + RELU2) paths, and _supports_activation can't distinguish them. Adding quant context to _supports_activation would require changing the abstract interface and every implementation across the codebase.

[mgoin]

We already have an mxfp8 oracle at https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/fused_moe/oracle/mxfp8.py, could we use that rather than overloading fp8?

[EdalatiAli]

Sure, I changed the code to use https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/fused_moe/oracle/mxfp8.py. I had to make a few other minor changes to correctly use it.

[mgoin]

Okay seems reasonable to me to accept. I'm not sure how much we should truly reuse with the fp8 methods but it is fair enough to follow that pattern for now

[mgoin]

It is a bit confusing to use Fp8MoeBackend here and elsewhere for mxfp8, but I guess it is needed to reuse the moe utils

[EdalatiAli]

Yes, using Fp8MoeBackend was needed to reuse the rest of the FP8 moe utils.
We can follow a better approach when more MXFP8 backend are available in the future.
Thank you for the feedback!