[Quant][Prototype] Manual fusion AR+RMS+Quant - #42597

Open

mgoin wants to merge 26 commits into

vllm-project:mainfrom

mgoin:fp8-manual-fusion-expand

Commits on May 12, 2026

[Quant][Prototype] Hoist NVFP4 input quant out of Linear apply

mgoin
and
claude
committed
989c78c

Commits on May 13, 2026

[Quant][Prototype] Add static FP8 + Llama; gate manual fusion behind env var

mgoin
and
claude
committed
cc0e21c
[Quant][Prototype] Wire fused rms_norm_static_fp8_quant on hoist path

mgoin
and
claude
committed
5165a8e
[Quant][Prototype] Drop VLLM_HOIST_INPUT_QUANT gate

mgoin
and
claude
committed
5d6d041
[Quant][Prototype] Drop NVFP4 scope; let QuantizedActivation flow into kernel

mgoin
and
claude
committed
d49d677
[Quant][Prototype] Move RMSNorm+quant dispatch into quant_fusion

mgoin
and
claude
committed
4cb7ecc
[Quant][Prototype] Pass quant_key/input_scale explicitly at call site

mgoin
and
claude
committed
5c84221
Revert "[Quant][Prototype] Pass quant_key/input_scale explicitly at call site"
mgoin
committed
1c6157f

Commits on May 14, 2026

[Quant][Prototype] Add allreduce to manual fusion path; wire Qwen3

mgoin
and
claude
committed
2994e39
[Quant][Prototype] Robust AR check + handle final norm

mgoin
and
claude
committed
44c5ad1
[Quant][Prototype] Wire FlashInfer fused AR+RMSNorm+FP8 single kernel

mgoin
and
claude
committed
6f5dcaf

Commits on May 20, 2026

[Quant][Prototype] Refactor to FusedAllReduceRMSQuant + NVFP4 impl
mgoin
committed
7321e88

Commits on May 21, 2026

Change to functional approach and no quant support
mgoin
committed
b11c165

Commits on May 26, 2026

Merge branch 'main' into fp8-manual-fusion-expand
mgoin
committed
8e14d9e
Query kernel.input_quant_key() instead of hard-coding it in schemes
mgoin
committed
34d692d
Guard ar_rms_quant flashinfer paths for CUDA-without-FlashInfer
mgoin
committed
ce8c613

Commits on May 27, 2026

Commits on May 28, 2026

Commits on May 29, 2026

Fall back to unfused AR+RMSNorm when batch exceeds flashinfer workspace
mgoin
committed
689bc13
Make AR+RMSNorm fusion opt-in to fix TP correctness in reused decoders
mgoin
committed
4dde8cd
Tests
mgoin
committed
c3cb1c7