Repository navigation
[ROCm][Perf] gfx950: use aiter.mqa_logits for the sparse indexer prefill - #60294
Draft
amd-sriram wants to merge 1 commit into
Draft
amd-sriram wants to merge 1 commit into
amd-sriram wants to merge 1 commit into
Conversation
…fx950 Falls back to the existing path when AITER has no aiter.mqa_logits. Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com>
amd-sriram
marked this pull request as draft
October 7, 2026 08:58
6 tasks
This was referenced Oct 16, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
On gfx950, the sparse indexer prefill calls the Triton/Gluon
fp8_mqa_logitsdirectly, and moving it to another kernel takes a vLLM change each time (#59280 did this for FlyDSL). This callsaiter.mqa_logits(ROCm/aiter#6185) instead: AITER picks the backend and FlyDSL kernel variant per shape from its tuned table.aiter.mqa_logits, when the installed AITER has it.aiter.mqa_logits: the existing Triton/Gluon path, unchanged.Needs ROCm/aiter#6185. Until the AITER that vLLM pins includes it, the import fails and gfx950 keeps the Triton/Gluon path, so this is safe to merge first.
Test Plan
pytest tests/v1/attention/test_rocm_aiter_mqa_logits_dispatch.py: gfx942 uses FlyDSL; gfx950 usesaiter.mqa_logits, or Triton/Gluon when AITER lacks it; other archs use Triton/Gluon; AITER disabled uses the torch fallback.rocm_fp8_mqa_logitson gfx950 (seq_len 2048, seq_len_kv 16384, H=32, D=128), compared withfp8_mqa_logits_torch.Test Result
MI355X (gfx950), a vLLM 0.30.1rc1 dev build with this change applied to
rocm_aiter_mla_sparse.py, AITER from ROCm/aiter#6185, Triton 3.8.0, FlyDSL 0.3.4.1.aiter.mqa_logits, which used the tuned GLM-5 row (FlyDSLmfma32x32x64_bkv128_r2_w4_lds3). Same-infmask as the torch fallback,calc_diff1.4e-6.