[ROCm][Perf][DSV4] Enable split sparse decode on gfx942

[tuukkjs]

Purpose

Enable the existing split partial/reduce Triton sparse decode path for DeepSeek-V4 sparse MLA decode on AMD gfx942.

gfx950 already uses this path. gfx942 currently falls back to the monolithic _sparse_attn_decode_ragged_kernel, even though the split path runs correctly on gfx942. This PR extends the tuned-architecture guard to include gfx942 and updates the ROCm DeepSeek-V4 sparse attention tests so the split decode path is covered on both gfx942 and gfx950.

This is a targeted decode-path improvement. The latest MI300X/gfx942 profiles show the sparse decode work itself drops by about 40% with the split path. Serving-level impact is modest and workload dependent: in the repeated-pass validation, high-throughput output throughput was effectively at parity with lower observed TTFT, while the low-rate decode-heavy workload showed lower TPOT/ITL and lower observed TTFT.

AI assistance was used for this PR.

Test Plan

• Run ROCm DeepSeek-V4 sparse attention tests on gfx942:

python3 -m pytest -q tests/kernels/attention/test_rocm_triton_attn_dsv4.py

• Start a vLLM OpenAI-compatible server for deepseek-ai/DeepSeek-V4-Flash on an 8x gfx942 node and run:
  • deterministic completion smoke request
  • GSM8K smoke evaluation
  • vllm bench serve high-throughput random workload: input 128, output 96, 128 prompts, request rate inf
  • vllm bench serve low-rate decode-heavy random workload: input 17, output 96, 32 prompts, request rate 1
  • repeated-pass high-throughput and low-rate comparisons, reporting first pass separately from later warm passes

Testing environment for DeepSeek-V4-Flash prompt/e2e validation on gfx942:

Image:
  vllm/vllm-openai-rocm:nightly-9e57de7197f234f9d9187715d96e07e007048c0f
Hardware:
  8x AMD Instinct MI300X OAM, gfx942
Common server args:
  tensor_parallel_size=8
  kv_cache_dtype=fp8_e4m3
  max_model_len=4096
  tokenizer_mode=deepseek_v4
  moe_backend=triton_unfused
  max_num_batched_tokens=8192
  VLLM_ROCM_USE_AITER=1

Baseline used the installed gfx950-only guard, so gfx942 took _sparse_attn_decode_ragged_kernel. Proposed changed only the guard to include gfx942, so gfx942 took _sparse_attn_decode_partial_kernel plus _sparse_attn_decode_reduce_kernel.

Test Result

ROCm DeepSeek-V4 sparse attention tests passed on gfx942:

41 passed, 16 warnings in 26.43s

ROCm DeepSeek-V4 sparse attention tests also passed on MI355X/gfx950:

40 passed, 16 warnings in 26.35s
exit code: 0

DeepSeek-V4-Flash serving correctness on 8x gfx942:

Check	Baseline fallback	Proposed split
Deterministic smoke request	returned 42	returned 42
GSM8K limit-20 flexible exact match	0.95	0.95
GSM8K limit-20 strict exact match	0.95	0.95

Kernel-level sparse decode benchmark, MAIN_LEN=80, EXTRA_LEN=32, production DeepSeek-V4 cache packing:

NUM_QUERIES	Correctness max abs diff	Fallback ragged mean	Split partial+reduce mean	Improvement
1	0.000122	42.754 us	22.836 us	46.59%
8	0.000244	42.260 us	23.088 us	45.37%
32	0.000244	42.547 us	24.490 us	42.44%
128	0.000244	43.463 us	34.121 us	21.49%

High-throughput profiler comparison on gfx942 confirms the intended kernel replacement:

Category	Baseline CUDA ms	Proposed CUDA ms	Delta
Sparse decode total	2542.147	1535.915	-39.58%
_sparse_attn_decode_ragged_kernel	2542.147	0.000	replaced
_sparse_attn_decode_partial_kernel	0.000	1369.342	added
_sparse_attn_decode_reduce_kernel	0.000	166.573	added

Serving benchmarks used one baseline server and one proposed server per workload, with six sequential benchmark passes each. Pass 1 is reported separately from the warm average over passes 2-6.

High-throughput random workload, input 128/output 96, 128 prompts, request rate inf:

Variant	Pass 1 output tok/s	Pass 1 TTFT	Pass 1 TPOT	Warm passes	Warm output tok/s	Warm TTFT	Warm TPOT
Baseline fallback	1887.34	2585.36 ms	41.02 ms	2-6	2908.57	831.55 ms	35.42 ms
Proposed split	2022.96	2243.84 ms	39.89 ms	2-6	2930.17	777.70 ms	35.65 ms
Proposed delta	+7.19%	-13.21%	-2.75%	2-6	+0.74%	-6.48%	+0.63%

Low-rate decode-heavy random workload, input 17/output 96, 32 prompts, request rate 1:

Variant	Pass 1 output tok/s	Pass 1 TTFT	Pass 1 TPOT	Warm passes	Warm output tok/s	Warm TTFT	Warm TPOT
Baseline fallback	90.44	65.91 ms	22.02 ms	2-6	90.43	60.05 ms	21.94 ms
Proposed split	90.67	64.59 ms	21.05 ms	2-6	90.65	57.79 ms	20.95 ms
Proposed delta	+0.25%	-2.00%	-4.41%	2-6	+0.25%	-3.76%	-4.50%

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @tuukkjs.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @tuukkjs.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[tuukkjs]

@tjtanaa could we merge this PR? Even though the e2e improvement is modest, it seems to me it is still an improvement for the codebase.

[tjtanaa]

LGTM. Thanks for fixing the extra and fnuz semantics. It is clearer than the one introduced in #46080 .