[ROCm] Expose kFp8DynamicTokenSym on AITER PTPC linears - #54248
Merged
AndreasKaratzas merged 9 commits intoSep 16, 2026
Merged
AndreasKaratzas merged 9 commits into
AndreasKaratzas merged 9 commits into
Conversation
rebklee
requested review from
AndreasKaratzas,
mgoin,
pavanimajety,
robertgshaw2-redhat,
tjtanaa,
tlrmchlsmth,
yewentao256 and
zyongye
as code owners
August 28, 2026 18:13
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
okorzh-amd
reviewed
Aug 31, 2026
okorzh-amd
reviewed
Aug 31, 2026
okorzh-amd
reviewed
Aug 31, 2026
okorzh-amd
reviewed
Aug 31, 2026
okorzh-amd
reviewed
Aug 31, 2026
okorzh-amd
reviewed
Aug 31, 2026
okorzh-amd
reviewed
Aug 31, 2026
okorzh-amd
reviewed
Aug 31, 2026
rebklee
force-pushed
the
rebklee/aiter-ptpc-input-quant-key
branch
3 times, most recently
from
September 1, 2026 01:59
151da3d to
8598ebf
Compare
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
FP8ScaledMMLinearKernel.apply_weights already consumes a matching QuantizedActivation, but the AITER PTPC kernels never exposed input_quant_key(), so expose_input_quant_key left the layer unmarked. Return the key from the AITER PTPC kernels and expose it on Fp8PtpcOnlineLinearMethod. Do not override input_quant_key on FP8ScaledMMLinearKernel: torch fallbacks call get_output_padding() via get_current_vllm_config(), which is unset in profile_run. Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Move fusion.quant_activation below fused_moe (ruff I001). Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
rebklee
force-pushed
the
rebklee/aiter-ptpc-input-quant-key
branch
from
September 9, 2026 04:37
df9435a to
281f214
Compare
dllehr-amd
self-requested a review
September 9, 2026 17:33
AndreasKaratzas
added
the
verified
Member
|
/ci run |
|
✅ Triggered Buildkite CI #87971 for commit |
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
/ci run |
|
✅ Triggered Buildkite CI #89135 for commit |
Signed-off-by: Rebecca Lee <rebecca.lee@amd.com>
Contributor
Author
|
/ci run |
|
✅ Triggered Buildkite CI #89189 for commit |
Contributor
Author
|
@AndreasKaratzas this is approved and passing CI on |
rebklee
added a commit
to rebklee/vllm
that referenced
this pull request
Sep 16, 2026
This branch carried its own copy of vllm-project#54248: the three aiter.py input_quant_key() methods, the online/fp8.py QuantizedActivation plumbing, and the contract test's AITER kernel registrations. Those files now come from vllm-project#54248 alone, and this branch keeps only the KDA gated-RMSNorm fusion, inert until vllm-project#54248 lands -- the same dependency that vllm-project#55043 already has. Signed-off-by: Rebecca Lee <rebecca.lee@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
FP8ScaledMMLinearKernel.apply_weightsalready no-ops in-kernel quant on a matchingQuantizedActivation. The AITER PTPC kernels never exposedinput_quant_key(), soexpose_input_quant_keyleft the layer unmarked and producers could not skipdynamic_per_token_scaled_quant.Return
kFp8DynamicTokenSymfrominput_quant_key()on the AITER PTPC kernels (AiterPreshuffledPerTokenFp8ScaledMMLinearKernel, AiterPerTokenFp8ScaledMMLinearKernel, AiterHipbMMPerTokenFp8ScaledMMLinearKernel).Fp8PtpcOnlineLinearMethodandQuarkW8A8Fp8callexpose_input_quant_keyafter kernel init.Not a duplicate. #44260 landed the QuantizedActivation contract and CUDA static-FP8 consumers; AITER PTPC kernels still return None, so expose_input_quant_key never marks the layer. #44132 added Fp8PtpcOnlineLinearMethod without that call. #33773 landed the GEMMs, not the fusion key.
Test Plan
Extended
tests/fusion/test_quant_activation_contract.pywith the three AITER PTPC kernels.Added
test_quark_fp8_ptpc_exposes_kernel_input_quant_key:QuarkW8A8Fp8.create_weightscopies the kernel'sinput_quant_keyonto the layer.pytest tests/fusion/test_quant_activation_contract.py --noconftestpytest tests/quantization/test_quark.py::test_quark_fp8_ptpc_exposes_kernel_input_quant_keyTest Result
Image:
vllm/vllm-openai-rocm:nightly-1dc464d42681d22f38caf1fdc1eb632dc4421c45tests/fusion/test_quant_activation_contract.py— 4 passedtests/quantization/test_quark.py::test_quark_fp8_ptpc_exposes_kernel_input_quant_key— 1 passed (VLLM_ROCM_USE_AITER=1)