[Kernel] [Quantization] Add MXFP4 and bias support for marlin kernel

[jinzhen-lin]

This pr add mxfp4 and bias support for marlin kernel. (for gpt-oss)

Performance test on gpt-oss-20b + RTX4090 + TP1 (tokens / s)

bs	triton	marlin
1	97.9	97.8
2	190.2	191.8
4	348	379.6
8	551.2	622.3
16	849.5	1063.9
32	1513.5	1769.5
64	2380.4	3142
128	3659.5	4670.6

Performance test on gpt-oss-20b + A100 + TP1 (tokens / s)

bs	triton	marlin
1	124.4	125.8
2	210.8	246
4	322.8	477.6
8	515.2	879
16	814.4	1398.4
32	1395	2367.7
64	2349.1	3756.5
128	3533	5887.8

Performance test on gpt-oss-120b + A100 + TP2 (tokens / s)

bs	triton	marlin
1	110.4	117.5
2	193.8	227.1
4	345.2	427.1
8	515.3	731.9
16	833.6	1283.1
32	1280	2054.2
64	1868.7	3052.7
128	2777.8	4172.4

Note: not benchmark test, just copy the Avg generation throughput: value from vllm log

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request adds support for MXFP4 and bias in the Marlin kernel. The changes are extensive, touching kernel implementations, generation scripts, and Python bindings. The core logic for handling the new MXFP4 format and incorporating bias seems well-integrated. A new scalar type for MXFP4 scales is introduced, and the necessary dequantization logic is added. Tests have also been updated to cover these new features. My main concern is a critical inconsistency in the definition of the new scalar type between C++ and Python, which could lead to incorrect behavior.

[mgoin]

Very nice work! I think most of the places that need to updated look properly addressed. I left some comments from initial pass

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jinzhen-lin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[ehartford]

Hero!

[jinzhen-lin]

@mgoin The failure is fixed. I previously omitted the permute operation on the bias (Qwen1.5 has bias in attention layer).

The metric is ok in my local test. However, the metrics of the test results fluctuate significantly. I run the same test multi-times:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.4450|±  |0.0137|
|     |       |strict-match    |     5|exact_match|↑  |0.2752|±  |0.0123|

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.4738|±  |0.0138|
|     |       |strict-match    |     5|exact_match|↑  |0.3010|±  |0.0126|

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.4647|±  |0.0137|
|     |       |strict-match    |     5|exact_match|↑  |0.2835|±  |0.0124|

So the ground_truth 0.465 / 0.3 may be too high. Also see the test failure in another pr mentioned by @bnellnm .

[simon-mo]

Please run GPQA

pip install gpt-oss, and run OPENAI_API_KEY=empty python -m gpt_oss.evals --model openai/gpt-oss-120b --eval gpqa --n-threads 128

[simon-mo]

Looking forward to ship this!

[jinzhen-lin]

Please run GPQA

pip install gpt-oss, and run OPENAI_API_KEY=empty python -m gpt_oss.evals --model openai/gpt-oss-120b --eval gpqa --n-threads 128

The test result of gpt-oss-20b:

# reasoning_effort=low
{'chars': 53.39962121212121, 'chars:std': 208.81521548256066, 'score': 0.5650252525252525, 'score:std': 0.49575368534588543}

# reasoning_effort=medium
{'chars': 40.38762626262626, 'chars:std': 182.659349965226, 'score': 0.6647727272727273, 'score:std': 0.47206985536793966}

The reasoning_effort=high setting take too many time to run. The results of reasoning_effort=low/medium settings
seem ok.

[simon-mo]

I ran some GPQA on H100 as well. LGTM

20B

Low
{'chars': np.float64(49.21022727272727), 'chars:std': np.float64(199.19166667350962), 'score': np.float64(0.5669191919191919), 'score:std': np.float64(0.4955015860245882)}

Medium
{'chars': np.float64(40.28787878787879), 'chars:std': np.float64(184.79641529424072), 'score': np.float64(0.6843434343434344), 'score:std': np.float64(0.4647768262451001)}

120B

[
{'eval_name': 'gpqa', 'model_name': 'openai-gpt-oss-120b-low_temp1.0_20250813_150155', 'metric': 0.6407828282828283},
{'eval_name': 'gpqa', 'model_name': 'openai-gpt-oss-120b-medium_temp1.0_20250813_150155', 'metric': 0.7102272727272727},
{'eval_name': 'gpqa', 'model_name': 'openai-gpt-oss-120b-high_temp1.0_20250813_150155', 'metric': 0.7916666666666666}]

[simon-mo]

Forgot to post my local perf test record, putting here for reference

With Marlin: H100

vllm bench serve --dataset-name hf --dataset-path AI-MO/aimo-validation-aime --model openai/gpt-oss-120b --port 8001 --num-prompts 2048


============ Serving Benchmark Result ============
Successful requests:                     2048      
Benchmark duration (s):                  287.20    
Total input tokens:                      214965    
Total generated tokens:                  4051696   
Request throughput (req/s):              7.13      
Output token throughput (tok/s):         14107.40  
Total Token throughput (tok/s):          14855.88  
---------------Time to First Token----------------
Mean TTFT (ms):                          44399.41  
Median TTFT (ms):                        4974.10   
P99 TTFT (ms):                           160217.06 
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          57.83     
Median TPOT (ms):                        59.46     
P99 TPOT (ms):                           68.26     
---------------Inter-token Latency----------------
Mean ITL (ms):                           55.48     
Median ITL (ms):                         53.24     
P99 ITL (ms):                            169.69    
==================================================

============ Serving Benchmark Result ============
Successful requests:                     2048
Benchmark duration (s):                  288.44
Total input tokens:                      214965
Total generated tokens:                  4048325
Request throughput (req/s):              7.10
Output token throughput (tok/s):         14035.29
Total Token throughput (tok/s):          14780.56
---------------Time to First Token----------------
Mean TTFT (ms):                          45371.57
Median TTFT (ms):                        5395.25
P99 TTFT (ms):                           163403.50
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          58.56
Median TPOT (ms):                        60.77
P99 TPOT (ms):                           68.30
---------------Inter-token Latency----------------
Mean ITL (ms):                           56.11
Median ITL (ms):                         53.49
P99 ITL (ms):                            145.67
==================================================

With Triton matmul ogs

uv pip install -U --pre torch==2.9.0.dev20250804+cu128 triton==3.4.0+git663e04e8 triton-kernels --index-url https://wheels.vllm.ai/gpt-oss --extra-index-url https://download.pytorch.org/whl/nightly/cu128 --index-strategy unsafe-best-match




============ Serving Benchmark Result ============
Successful requests:                     2048
Benchmark duration (s):                  276.56
Total input tokens:                      214965
Total generated tokens:                  4086045
Request throughput (req/s):              7.41
Output token throughput (tok/s):         14774.54
Total Token throughput (tok/s):          15551.83
---------------Time to First Token----------------
Mean TTFT (ms):                          50542.82
Median TTFT (ms):                        11954.13
P99 TTFT (ms):                           164304.18
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          57.00
Median TPOT (ms):                        59.84
P99 TPOT (ms):                           82.02
---------------Inter-token Latency----------------
Mean ITL (ms):                           53.19
Median ITL (ms):                         46.46
P99 ITL (ms):                            186.68
==================================================