[ROCm][DSV4][Perf] Keep C4 decode metadata dense on gfx950

[Fangzhou-Ai]

Summary

Keep DeepSeek-V4 C4 decode top-k metadata dense on gfx950 and let the native
Triton sparse-attention kernel consume the [batch, topk] indices plus per-row
lengths directly.

Previously the C4 path mapped local top-k indices to global cache slots, built a
ragged indptr, packed every dense row into a second flat buffer, and then read
that ragged representation in every C4 attention layer. The new gfx950 path
removes that repack. gfx942 and other architectures retain the existing ragged
path.

The shared global-index mapper also now masks padded rows before reading their
request or block-table metadata. This is required because FULL graph padding can
carry sentinel request IDs and top-k values; the safety change is shared by the
existing NVIDIA/XPU callers.

Scope and C128

This PR intentionally changes C4 decode only. C128 metadata remains ragged.

C128 already exposes dense indices, but its active row width changes with
sequence length while FULL graphs capture fixed shapes and strides. Reusing the
C4 flag naively could capture width 128 and later clamp a longer C128 row during
replay. A safe C128 follow-up needs a graph-stable physical-capacity contract,
row-strided dense input support, and short-capture-to-long-replay coverage.
Also, C128 performs its conversion once per shared metadata build rather than
once per C128 layer, so its expected latency benefit is much smaller.

Correctness and memory

• Dense and ragged decode outputs are bit-exact in eager and graph replay tests.
• Tests cover 8, 16, and 32 local heads, invalid padded rows, boundary lengths,
  stale dense tails, and long-to-short-to-long graph replay.
• The dense [B, K] output replaces the same-capacity flat ragged allocation
  and removes the small [B + 1] indptr. It does not increase GPU memory.
• Model serving validation used TP8. Other TP settings are structurally covered
  by the local-head-count unit matrix, but were not run end to end.

Performance

MI355X/gfx950 graph-replay microbenchmark, 51 interleaved samples after warmup:

Decode rows	Integrated speedup	Saved per C4 layer	Estimated 30-layer saving
1	1.271x	5.888 us	0.177 ms
4	1.261x	5.800 us	0.174 ms
8	1.259x	5.632 us	0.169 ms
16	1.269x	6.196 us	0.186 ms

TP8 serving used 8k input / 1k output random requests. The comparator is the
arithmetic mean of two unchanged-code control runs with identical server and
prompt settings; the candidate was a third isolated run.

Concurrency	Output throughput	Mean TPOT	Mean TTFT	Median TTFT
1	+0.71%	-0.74%	+0.51%	-0.23%
4	+1.63%	-1.70%	-0.10%	-1.51%
8	+0.79%	-0.72%	-2.00%	-0.10%

Validation

HIP_VISIBLE_DEVICES=0 .venv/bin/python -m pytest \
  tests/kernels/attention/test_rocm_triton_attn_dsv4.py -q
# 62 passed

.venv/bin/pre-commit run --files \
  tests/kernels/attention/test_rocm_triton_attn_dsv4.py \
  vllm/models/deepseek_v4/amd/rocm.py \
  vllm/models/deepseek_v4/common/ops/cache_utils.py \
  vllm/v1/attention/ops/rocm_aiter_mla_sparse.py
# passed

git diff --check upstream/main...HEAD
# passed

Full GSM8K, 5-shot, deterministic no-thinking generation completed all 1,319
examples:

Filter	Exact match
strict-match	0.9553
flexible-extract	0.9545

Equivalent direct server command:

export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_MOE=1
export VLLM_ROCM_AITER_C4A_TOPK=aiter

vllm serve deepseek-ai/DeepSeek-V4-Pro \
  --port 8888 \
  --tensor-parallel-size 8 \
  --data-parallel-size 1 \
  --max-model-len 9472 \
  --max-num-seqs 512 \
  --max-num-batched-tokens 16384 \
  --async-scheduling \
  --no-enable-prefix-caching \
  --distributed-executor-backend mp \
  --gpu-memory-utilization 0.8 \
  --kv-cache-dtype fp8 \
  --trust-remote-code \
  --moe-backend aiter \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --compilation-config '{"mode":3,"cudagraph_mode":"FULL_AND_PIECEWISE"}'

Equivalent direct serving benchmark command:

for conc in 1 4 8; do
  vllm bench serve \
    --model deepseek-ai/DeepSeek-V4-Pro \
    --backend vllm \
    --base-url http://127.0.0.1:8888 \
    --endpoint /v1/completions \
    --dataset-name random \
    --random-input-len 8192 \
    --random-output-len 1024 \
    --random-range-ratio 0.8 \
    --num-prompts $((conc * 10)) \
    --max-concurrency "${conc}" \
    --num-warmups $((conc * 2)) \
    --request-rate inf \
    --ignore-eos \
    --percentile-metrics ttft,tpot,itl,e2el \
    --trust-remote-code \
    --save-result
done

Duplicate-work check

No open PR duplicates this change. #41105 is the closest conceptual neighbor,
but it fuses CUDA top-k generation with the page-table transform; it does not
change the ROCm gfx950 sparse-decode consumer or eliminate its post-transform
repack. #51714 adds an optional external AITER/Gluon decode backend and still
expects ragged metadata. #50566, #46172, and #52628 optimize different decode or
metadata stages. #52212 is merged and is the native gfx950 Triton baseline for
this PR.

AI assistance

OpenAI Codex assisted with investigation, implementation, testing, benchmarking,
review, and drafting. The human submitter directed and reviewed the work and is
responsible for understanding and defending every changed line and for rerunning
the relevant validation before merge.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Fangzhou-Ai.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork