[ROCm][Perf] Dispatch Kimi-K3 KDA group64 projection

[JohnQinAMD]

Purpose

Prepack Kimi-K3's fixed TP8 KDA input projection after checkpoint loading and
dispatch it through the AITER gfx950 group-64 kernel.

The common KDA layer exposes one protected projection seam whose default calls
the same linear layer as before. The AMD subclass owns nonpersistent packed
buffers and overrides only that seam. The adapter fails closed on AITER
availability, HIP, gfx950, shape, dtype, device, layout, and packed-weight
support. It adds no public tuning option and does not change NVIDIA dispatch.

Depends on ROCm/aiter#4499.

Test plan

Tested on 8x MI355X (gfx950) with the public Kimi-K3 image and
moonshotai/Kimi-K3@9f62e4e9. The AITER #4499 kernel head is 5f5dd2b5; the
command below pins the vLLM base and PR revisions.

Fetch and verify the vLLM source:

git clone https://github.com/vllm-project/vllm.git vllm
git -C vllm fetch origin refs/pull/50666/head:pr-50666
test "$(git -C vllm rev-parse pr-50666)" = \
  41e41ea84d16c7396d986bb6487fc575109fd216
git -C vllm checkout --detach 6c91de36897932ba9b5adb11992235a2a789e009
git -C vllm diff 6c91de36897932ba9b5adb11992235a2a789e009..pr-50666 \
  --binary | git -C vllm apply -
git -C vllm diff --check

Use complete verified vLLM/AITER trees as overlays on the public image,
including AITER csrc/ and hsa/; do not copy individual files. Verify
import vllm, vllm._C, aiter before testing.

python -m pytest -q \
  ../aiter/op_tests/flydsl_tests/test_kimi_k3_kda_input_group64.py \
  tests/models/kimi_k3/test_amd_kda_group64.py
  --model /model --output /tmp/group64.json

Serve both source-isolated arms with identical flags:

VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_FP4BMM=1 \
AITER_SITUV2_A8W4=1 AITER_BF16_FP8_MOE_BOUND=0 \
VLLM_USE_BREAKABLE_CUDAGRAPH=0 \
vllm serve /model --served-model-name moonshotai/Kimi-K3 \
  --tensor-parallel-size 8 --trust-remote-code --moe-backend auto \
  --gpu-memory-utilization 0.95 --max-num-seqs 128 \
  --max-num-batched-tokens 4096 --max-model-len 1048576 \
  --enable-prefix-caching --kv-cache-dtype fp8 --reasoning-parser kimi_k3

After one 8K/128 warmup, run three 8K/1K batch-one trials with seeds 1--3,
temperature zero, and --ignore-eos. Run full lm-eval==0.4.12 GSM8K on both
arms: 1,319 questions, 5-shot, greedy completion, 2,048 generated tokens,
concurrency 128, and seed 42.

Test results

• Combined focused suite: 15/15 passed.
• Changed-file pre-commit, git diff --check, and DCO passed.

Measurement	Control	Candidate	Change
AITER real-weight microbenchmark	15.1033 us	9.2379 us	1.6349x
TP8 decode throughput	36.0825 tok/s/GPU	36.5483 tok/s/GPU	+1.291%
TPOT	27.7143 ms	27.3611 ms	-0.3532 ms

The real-weight operator check reported relative RMSE 1.25e-7 and cosine
similarity 1.0 against the dequantized group-64 oracle.

Full GSM8K: control 1273/1319, candidate 1279/1319, zero invalid or
transport failures; 14 wins/8 losses (p=0.2863). This paired run found no
statistically detectable accuracy difference.

Overlap and limits

The real-weight error measures implementation accuracy against the dequantized
packed-weight oracle; it does not measure BF16-to-FP8 quantization error or
model quality. GSM8K is the model-quality gate.

This projection is upstream of #50654's convolution/recurrent/gated-norm
fusion and is complementary to it. #50659's PR-specific implementation is
NVIDIA-only.

Tool assistance

OpenAI Codex assisted with implementation, tests, benchmarking, and drafting
this description.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[AndreasKaratzas]

Nobody is going to easily review a FULLY vibe coded PR.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @JohnQinAMD.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork