[Bugfix][ROCm][CI] Restore the DeepSeek-V4 input GEMM override point

[stefankoncarevic]

Purpose

The GSM8K accuracy job for amd/DeepSeek-V4-Flash-NVFP4 on gfx950 reports 0.0000 against a
threshold of 0.92. It is not a near miss: the server comes up clean and then answers every one of
the questions with an unparsable run of repeated tokens, so the failure is in the numerics rather
than in the harness. A bisect over the window in which the job turned red lands on
79c865b, "[Perf] Narrow DeepSeek V4 eager CUDA graph region"
(#51430).

Alongside narrowing the graph region, that commit deleted
DeepseekV4Attention._fused_wqa_wkv_gemm and inlined self.fused_wqa_wkv(hidden_states)[0] at its
only call site. The deleted method is the override point the ROCm attention layer implements. On
ROCm the fused_wqa_wkv weight is preshuffled in place when the model loads and its block scale is
kept to the side, so the only thing that can multiply it correctly afterwards is AITER's
preshuffled-B GEMM. With the call inlined, the input projection of every attention layer runs the
plain linear over a shuffled weight, and the model produces garbage from the first layer on. The
same removal leaves the ROCm override's own fallback calling a super() method that no longer
exists, so any path that took the fallback would raise AttributeError.

Nothing about this is visible on CUDA, and the ROCm override still reads as if it were connected,
which is why the removal went through.

Relationship to the other #51430 follow-ups

Two open PRs address #51430, both for a different symptom on NVIDIA, and neither covers this one.
#51750 is an auto-generated full revert opened
after the B200 nightly saw GSM8K fall to 0.0432 with MTP acceptance collapsing to 4.1%; it would
undo this bug as a side effect, but it is still a draft and the author of #51430 has proposed a
narrower fix-forward instead. That fix-forward, #51768, attributes the NVIDIA failure to the legacy V1 model runner's breakable PIECEWISE CUDA graph path and guards that combination in
vllm/config/vllm.py, leaving the attention implementation exactly as #51430 left it. If #51768 is
the one that lands, ROCm stays broken.

The two failures are independent. This one does not need a model runner, a graph mode or MTP to
show up: it reproduces at compilation mode 0 with no graph captured, and at FULL_DECODE_ONLY,
which #51768 explicitly keeps allowed. It also cannot appear on CUDA, since nothing preshuffles the
weight there.

This is not an alternative to either of them. On CUDA the restored method computes exactly what the
inlined call computed, so this change is a no-op there and does nothing for the B200 failure —
#51768 is still needed for that. In the other direction, if #51750 lands after all, this one becomes
redundant and I will close it.

Searches for open PRs mentioning fused_wqa_wkv, and for open PRs referencing #51430, turned up
nothing else that restores this method.

The change

Restore _fused_wqa_wkv_gemm on the base class as a plain call into fused_wqa_wkv, and send the
call site back through it. That is the whole change, eight lines. The narrowed CUDA graph region and
everything else from #51430 are untouched.

Test Plan

gfx950 (MI355X), with the model, server arguments and environment taken verbatim from
tests/evals/gsm8k/configs/DeepSeek-V4-Flash-NVFP4.yaml: tensor parallel 8, fp8 KV cache,
compilation mode 3 with FULL_DECODE_ONLY, AITER enabled. GSM8K at 8 shots, shortened to 400
questions to keep the turnaround reasonable. The same evaluation was run before the change on the
unmodified checkout and after it.

To rule out the parts of #51430 that the title points at, the failure was also reproduced at tensor
parallel 1 and at compilation mode 0, where no graph is captured at all.

Test Result

Before the change, on current main, accuracy is 0.0000 with an invalid rate of 1.000 — no answer is
parsable. It stays 0.0000 at tensor parallel 1 and at compilation mode 0, so neither sharding nor
graph capture is what breaks it.

After the change, accuracy is 0.953 with an invalid rate of 0.000, comfortably above the 0.84 floor
the threshold and its tolerance imply. Reverting #51430 in full gives an equivalent result, which
confirms the method removal is the only part of that commit involved.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[zyongye]

/ci run

[github-actions]

✅ Triggered Buildkite CI #83584 for commit a5517ac07164.

[AndreasKaratzas]

/ci run

[github-actions]

✅ Triggered Buildkite CI #83591 for commit 50be23998769.

[AndreasKaratzas]

/ci run

[github-actions]

✅ Triggered Buildkite CI #83623 for commit d9b7f489baeb.