Skip to content

[ROCm][Perf] gfx950: use aiter.mqa_logits for the sparse indexer prefill - #60294

Draft
amd-sriram wants to merge 1 commit into
vllm-project:mainfrom
amd-sriram:rocm/aiter-mqa-logits-dispatch
Draft

amd-sriram wants to merge 1 commit into
vllm-project:mainfrom
amd-sriram:rocm/aiter-mqa-logits-dispatch

Conversation

@amd-sriram

Copy link
Copy Markdown
Contributor

Purpose

On gfx950, the sparse indexer prefill calls the Triton/Gluon fp8_mqa_logits directly, and moving it to another kernel takes a vLLM change each time (#59280 did this for FlyDSL). This calls aiter.mqa_logits (ROCm/aiter#6185) instead: AITER picks the backend and FlyDSL kernel variant per shape from its tuned table.

Needs ROCm/aiter#6185. Until the AITER that vLLM pins includes it, the import fails and gfx950 keeps the Triton/Gluon path, so this is safe to merge first.

Test Plan

  • pytest tests/v1/attention/test_rocm_aiter_mqa_logits_dispatch.py: gfx942 uses FlyDSL; gfx950 uses aiter.mqa_logits, or Triton/Gluon when AITER lacks it; other archs use Triton/Gluon; AITER disabled uses the torch fallback.
  • A real call of rocm_fp8_mqa_logits on gfx950 (seq_len 2048, seq_len_kv 16384, H=32, D=128), compared with fp8_mqa_logits_torch.

Test Result

MI355X (gfx950), a vLLM 0.30.1rc1 dev build with this change applied to rocm_aiter_mla_sparse.py, AITER from ROCm/aiter#6185, Triton 3.8.0, FlyDSL 0.3.4.1.

…fx950

Falls back to the existing path when AITER has no aiter.mqa_logits.

Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rocm Related to AMD ROCm

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant