[Quantization] Support FP8 MoE bias for models like GPT-OSS

[jasperjiaguo]

Summary

GPT-OSS-120B has biased MoE layers (gate_up_proj_bias, down_proj_bias). When serving the BF16 model with --quantization fp8, Fp8MoEMethod doesn't register bias parameters, causing weight loading failures.

This PR adds bias support to Fp8MoEMethod:

• Register w13_bias / w2_bias in create_weights() when FusedMoEConfig.has_bias is set
• Pass biases through to fused_experts() in apply()
• Guard against unsupported FusedMoEModularKernel + bias combination (consistent with UnquantizedFusedMoEMethod)

Test plan

Tested on 4×H200 with GPT-OSS-120B BF16 model:

Metric	BF16 (no quant)	BF16 + --quantization fp8
GSM8K Accuracy	0.848	0.834
Output throughput	~9,600 tok/s	~14,000 tok/s

• vllm serve --quantization fp8 loads successfully with biased MoE
• FP8 fused_moe_kernel confirmed via nsys profiling

Related

• Companion SGLang PR: [GPT-OSS] support fp8 online quantization for gpt-oss bf16 sgl-project/sglang#18988

🤖 Generated with Claude Code

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jasperjiaguo.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[gemini-code-assist]

Code Review

This pull request introduces support for FP8 MoE bias in models like GPT-OSS-120B. The changes include registering bias parameters in Fp8MoEMethod.create_weights, passing them to the fused_experts kernel in apply, and adding a safety guard for the FusedMoEModularKernel.

While the logic is sound, there are a few critical issues regarding the data types used for bias parameters and potential typos in attribute names that could lead to runtime errors or precision loss. Specifically, biases should typically remain in the model's original high precision (e.g., BF16) rather than being quantized to FP8, and there are references to self.moe which should likely be self.moe_config.

[gemini-code-assist]

The attribute self.moe is not defined in Fp8MoEMethod. Based on the class initialization and vLLM conventions, this should be self.moe_config.

Suggested change

	if self.moe.has_bias:
	if self.moe_config.has_bias:

[simon-mo]

Can you double check this?

[jasperjiaguo]

This is false positive. self.moe is inherited from the parent class FusedMoEMethodBase

class FusedMoEMethodBase(QuantizeMethodBase):
    def __init__(self, moe: FusedMoEConfig):
        super().__init__()
        self.moe: FusedMoEConfig = moe
        self.moe_quant_config: FusedMoEQuantConfig | None = None
        self.moe_mk: mk.FusedMoEModularKernel | None = None

[jasperjiaguo]

vllm/vllm/model_executor/layers/fused_moe/fused_moe_method_base.py

Line 28 in b8d8b7e

self.moe: FusedMoEConfig = moe

[mergify]

Hi @jasperjiaguo, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[vkuzo]

@jasperjiaguo could you share the exact command to reproduce the test plan in this PR?