feat(rocm): replace forced custom allreduce with two-shot rdna_ar

[BlivionIaG]

Proposal

Stop using VLLM_FORCE_CUSTOM_ALL_REDUCE on gfx10x. That backend pulls peer memory and treats PCIe as if it were fully connected. On gfx1030, capture detection is also unreliable, so a breakable CUDA graph can all-reduce the wrong buffer.

rdna_ar already pushes into uncached staging and polls local flags, which matches this fabric. It was too narrow to replace custom all-reduce: one-shot only, fp16/fp32 only, contiguous only, and a 64KB cap. This PR widens that path so the serve scripts can leave the force flag off.

What changes

• Two-shot for messages above VLLM_RDNA_AR_ONESHOT_KB (default 32KB), up to VLLM_RDNA_AR_MAX_KB. Each rank pushes only its chunk, reduces in fixed fp32 order, then pushes the reduced chunk. Traffic is about 2*(W-1)/W * N instead of one-shot's (W-1)*N.
• bf16 next to fp16 and fp32.
• Flat non-contiguous spans, same storage-order rule as custom all-reduce.
• VLLM_RDNA_AR_ALGO=auto|oneshot|twoshot.
• Abort phases 3 and 4 name a stalled allgather barrier or a missing allgather flag.
• scripts/serve_gfx1030_full.sh and scripts/serve_gfx1030_flashnext.sh default to VLLM_RDNA_AR=1 and VLLM_FORCE_CUSTOM_ALL_REDUCE=0.

VLLM_RDNA_AR stays default-off in the library. A failed boot self-test still disables the backend and falls back to RCCL.

Why this is not a duplicate

gh pr list --repo opengfx1030/vllm-rdna --state open --search "rdna_ar OR allreduce" returned no other open PR.

Tests

/tmp/rdna-ar-test/bin/python -m pytest tests/distributed/test_rdna_ar.py -v --noconftest
7 passed

Those tests cover abort-record decode (including the new two-shot phases), the wedge marker, and the default-off gate. They do not launch the HIP kernel. No model eval: with VLLM_RDNA_AR unset, dispatch is unchanged.

AI assistance was used (Cursor Grok 4.7).

  

[BlivionIaG]

Validated on 4× Radeon PRO V620 (gfx1030, TP=4) — works, with one gate fix

Integrated locally as a clean merge onto rdna_extras@cd38a1d3, _rocm_C
rebuilt (only csrc changed), then measured against the pre-PR one-shot path
and against pure PYNCCL.

Tests

• tests/distributed/test_rdna_ar.py → 7 passed (incl. the new twoshot-phase
  abort test).
• Boot self-test at a 20 MiB gate: passes on all 4 ranks, two-shot ≈ 5.7 ms
  per call.
• Full matrix (MTP=0/2 × {1k/512, 16k/1k} × {c=1, c=8}, Flash-Next W4A16,
  FULL_AND_PIECEWISE, TunableOp rows): every cell 100% success, coherent
  probes, no wedge markers.

Results (details in the gfx1030_optimized repo,
docs/rdna2/pr22-twoshot-validation-2026-09-27.md)

	PYNCCL	pre-PR one-shot	this PR (20 MiB gate)
1k/512 c=1 TPOT	25.75 ms	23.26 ms	23.26 ms
1k/512 c=8 aggregate	162.4 tok/s	173.5	171.5
16k/1k c=1 cold prefill	1686 tok/s	—	1755 t/s (+4%)
16k/1k c=8 MTP=0 aggregate	74.0 tok/s	75.9	77.3

So: decode at parity with the one-shot path, prefill strictly better than
RCCL, and the forced-custom-AR path is no longer needed.

One real defect found — small gates self-disable the backend. The boot
self-test's two-shot trial fails nondeterministically when the gate is small:

VLLM_RDNA_AR_MAX_KB	trial 2 (one two-shot call at the gate)
64 KiB	FAIL — wrong result (mean 26.60/27.01/28.59 vs 30.0), 3/3 launches
128 KiB	FAIL — spin-cap: "peer rank 1's flag never arrived … at collective #10" (phase 4) + partial wrong result 29.88
1 MiB	PASS
20 MiB	PASS (3 launches)

With MAX_KB=64 (the previous default) the backend self-disables and silently
falls back to RCCL, which costs ~5–10% decode. The 128 KiB abort points at the
cross-rank allgather handshake on the second two-shot call — a cross-call
counter/parity sync bug in the two-shot path; follow-up material (standalone
probe sweep of the 64 KiB–1 MiB band).

Integrated in rdna_extras@0b8585723 with the gate default raised to
20480 KiB (library + launchers) so VLLM_RDNA_AR=1 passes the self-test out
of the box. Closing as integrated — reopen if you'd rather update this branch
directly instead.