[GeorgeMA-Strong]

Summary

Fix two independently observed failures in the existing gfx1030 RDNA one-shot all-reduce path:

• Retained outputs were overwritten. The native collective returned views of a process-global output buffer. A later collective overwrote an earlier result still in use. Allocate independent output storage with at::empty_like.
• Python/native build mismatches disabled the intended backend. Check all six native operators before GPU allocation, coordinate fallback across ranks, and provide a strict V620 launcher preflight for import provenance. Keep existing timeout, self-test, and wedge-marker protections intact.

Add focused CPU regression coverage, a four-GPU eager/graph correctness and latency harness, a three-chat tool-return qualification script, and a detailed build/test report. The baseline launcher now respects explicit RDNA disable/cutoff overrides for A/B tests; existing defaults are unchanged. No systemd service or KV-offload implementation is changed.

Scope and related work

• Targets opengfx1030/vllm-rdna:rdna_extras. This clean branch excludes the earlier RAM KV-offload work in Fix ROCm RAM KV offload corruption and hybrid cache reloads #24.
• feat(rocm): replace forced custom allreduce with two-shot rdna_ar #22 introduces two-shot all-reduce but retains the shared-output allocation. This is a separate correctness/native-compatibility fix. Both touch the same implementation and will need integration according to merge order; this PR does not implement two-shot.
• [ROCm] Add native HIP RDNA3/RDNA4 custom all-reduce backend vllm-project/vllm#57767 introduces a separate RDNA3/RDNA4 backend, rather than fixing this existing gfx1030 path.
• The native file also receives repository formatter changes.

Validation

On this clean PR branch, based on rdna_extras at 3d67bdf49:

• python -m pytest --noconftest tests/distributed/test_rdna_ar.py -q: 16 passed, using the existing project virtual environment.
• Ruff checks on all changed Python files: passed.
• bash -n tools/rdna2/serve_v620_baseline.sh: passed.
• git diff --check: passed.

Hardware testing was performed on four Radeon PRO V620 GPUs in an isolated server Git checkout of the original investigation branch. The retained-output failure reproduced twice on all four ranks before the fix. After the fix, all nine FP16/FP32 payload cases passed: retained eager outputs, consecutive captured collectives, 16 changing-input graph replays, interleaved eager work, and oversized-input rejection. All four native startup self-tests passed. The latency harness also completed cleanup successfully after explicitly releasing captured graphs.

Provenance limitation: the hardware-tested investigation stack also included #24. GPU/model tests have not been repeated on this clean branch's newer base. RAM reload observations belong to that combined stack. Requalification on the final integrated revision is required before deployment. No production service was changed while preparing this PR.

Performance evidence — not a general speedup claim

Direct-vLLM RCCL comparison, microseconds per collective:

FP16 bytes	RDNA	vLLM RCCL
5,120	30.4	79.1
15,360	44.1	79.4
30,720	54.2	74.4
61,440	117.0	61.0

These results motivated a 32 KiB experimental cutoff, not a default change.

Intel AutoRound model, TP4/EP, MTP2, FULL_DECODE_ONLY graphs, 64 GiB RAM offload; llm-context-bench coding performance, one measured repetition, 1,024 output tokens:

Input tier	Actual prompt tokens	RDNA decode tokens/s	RCCL decode tokens/s	RDNA / RCCL TTFT seconds
8K	8,964	68.37	64.23	6.43 / 6.43
64K	71,991	65.94	65.93	49.97 / 49.63
128K	143,875	64.09	66.34	103.88 / 103.67

All six single-session trials passed benchmark validation, but there is no consistent end-to-end decode gain. Generated outputs and speculative acceptance differed; these are single-trial measurements, not controlled per-step attribution or code-quality evaluations.

Three concurrent short chats passed exact tool-name/argument checks and resumed after staggered synthetic tool returns, with RAM KV reloads observed. A separate three-chat coding run completed every request but one output failed the benchmark's dominant-token gate; that aggregate result is invalid and is not a throughput claim. Full-VRAM multi-session stress qualification remains outstanding.

At the tested 32 KiB cutoff, larger padded three-chat MTP target batches fall back to RCCL, so small-message microbenchmark gains should not be extrapolated to concurrent throughput.

Full commands, results, limitations, and activation gates are in docs/rdna2/V620-RDNA-AR-NATIVE-BUILD.md.

Review status and AI disclosure

Draft for human review and final hardware requalification. Implementation, investigation, test execution, and this description were assisted by OpenAI Codex under the submitter's direction. Human review/sign-off on the final diff and results is pending; this draft does not claim it has already occurred.