[Bugfix] Coordinate UnifiedRadix abort cleanup across ranks

[KastanDay]

Note, we're not using UnifiedRadixCache currently, so skip.

TLDR: L3/HiCache storage prefetch abort path in UnifiedRadixCache

Motivation

UnifiedRadixCache.release_aborted_request() could take different cleanup paths on different attention ranks for the same request. One rank may already have allocated host pages and started storage I/O while another still sees a pending prefetch, or no ongoing_prefetch entry at all.

Before this change, the active rank entered the transfer barrier while the pending/missing rank returned early. That creates two related hazards:

1. a collective deadlock because only some ranks enter the barrier; and
2. divergent ownership cleanup, where host slots, locks, or occupancy can be released on one rank while a peer is still transferring.

The sidecar-length portion of the old fork PR was intentionally removed because equivalent coverage is already present on current SGLang main via upstream PR sgl-project#31443. This replacement contains only the still-unique abort-cleanup fix.

Modifications

• Classify each rank's local abort state as missing, pending, or active I/O.
• Reduce that state with MAX across attention groups before any rank returns.
• Require every rank to enter the abort barrier when any peer has active storage I/O.
• Preserve the existing local ownership rules:
  • missing ranks only join the collective;
  • pending ranks terminate and revoke their pending prefetch before joining;
  • active ranks drain the operation, join the barrier, then release locks and completed host pages.
• Add deterministic CPU unit coverage for all three asymmetric rank states.

This is intentionally one PR: the state reduction and the barrier participation rule are one atomic distributed invariant. Splitting them would leave an intermediate revision that still deadlocks or releases ownership inconsistently.

Reproduction

The regression tests model two ranks without timing sleeps:

python -m pytest -q \
  test/registered/unit/mem_cache/test_unified_radix_abort_cleanup.py

The important cases are:

• local rank has no prefetch, peer has active I/O;
• local rank is pending, peer has active I/O;
• local rank has active I/O and must not release until after the collective barrier.

On the old implementation, the first two cases return before the barrier and fail the asserted collective contract. With this patch, every case joins exactly once and ownership is released only by the rank that owns it.

Experimental validation

The original integrated candidate was exercised in an isolated GLM 5.2 staging deployment on three H200 nodes in both 2-prefill/1-decode and 1-prefill/2-decode topologies. Real stream=true disconnects reached active KV transfer and produced KVTransferError: Aborted by AbortReq; bounded victim requests were 8/8 correct, with no foreign canary and no engine restart. This is integrated-stack evidence, not a claim of per-commit isolation.

For this current-main rewrite:

• python3 -m py_compile passed for the implementation and new test file.
• git diff --check passed.
• The deterministic unit cases were added to SGLang's registered CPU suite.
• A full local pytest import on macOS is blocked by SGLang's Linux/CUDA-only triton import path; repository CI should execute the registered CPU tests in the supported environment.

Relationship to other work

• Supersedes fork PR [v0.5.15 10/21] Apply sidecar coverage and abort cleanup to UnifiedRadix #10.
• Does not duplicate Kevin Flansburg's Mooncake transfer-quiescence work in upstream PR [Bugfix] Drain Mooncake transfers before releasing KV pages sgl-project/sglang#31357: this patch is specifically the rank-collective cleanup contract inside UnifiedRadixCache.
• Can be reviewed and merged independently of the Mooncake protocol PRs.

Performance

The change adds one small attention-group reduction on the abort path only. The normal prefetch and decode paths are unchanged.

Checklist

• Rebased onto current SGLang main
• Removed already-upstreamed sidecar behavior
• Added deterministic registered CPU tests
• Python compilation and git diff --check
• Run repository CI in the supported Linux environment

---

CI States

Latest PR Test (Base): ❌ Run #31407663783
Latest PR Test (Extra): ❌ Run #31407663386

[KastanDay]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 👍

Reviewed commit: d828be3546

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[KastanDay]

Note, we're not using UnifiedRadixCache currently, so skip.