[Fix] hicache: keep the device-only-HiCache SWA match gate beside the per-request ring gate

[avifenesh]

Motivation

#38269 (822e73c) changed the SWA match validator in python/sglang/srt/mem_cache/unified_cache/components/swa.py::SWAComponent.create_match_validator:

-        # unified_kv never caches the SWA ring (per-request, not content-stable),
-        # so SWA bookkeeping must not gate the match here.
-        swa_device_only_hicache = (
-            not self.tree_core.has_swa_host_pool and self.tree_core.enable_hicache
-        )
+        # A per-request SWA ring is not stored in tree nodes, so its bookkeeping
+        # must not gate prefix matching.
+        swa_req_ring = is_swa_req_ring(self.cache.token_to_kv_pool_allocator)
 ...
-                if swa_device_only_hicache and (node.backuped or not node.evicted):
+                if swa_req_ring and (node.backuped or not node.evicted):

The ring gate is right for the unified-KV ring, but it is narrower than the clause it replaced. With HiCache on and no SWA host pool (a paged or request-window SWA pool: DSV4 SWA storage: worker=target, storage=paged, storage=request_window), SWA never leaves the device, so every host-backed prefix is a host-only Full-KV tombstone: cd.value is None, cd.host_value is None, node.backuped. Before #38269 that node was a valid match boundary and load_back restored the Full KV. After it is_swa_req_ring(allocator) is False on that layout (swa_req_ring = ring_size is not None, only set on the ring path), the validator returns False on every such node, host_hit_length stays 0, Req.needs_host_load_back() is never true and init_load_back never runs. The host tier fills and is never read.

Where I hit it: DeepSeek-V4.1-Flash, TP4, 4x RTX PRO 6000 (SM120), HiCache on, paged SWA. Same box, same launch, upstream dsv4.1 before and after the commit that brought #38269 in:

metric	before #38269	with #38269	with this fix
cached_tokens_total{cache_source="host"} after a 36 x 64k replay pass	2.35M	never created (0)	2.35M
load_back_tokens_total	9.40M	never created (0)	9.40M
host restores in the replay pass	36/36	0/36	36/36
replay TTFT	0.373 s	10.3 s (= a cold prefill)	0.366 s
hicache_backup_tokens_total / hicache_host_used_tokens	grow	grow (11.7M / 2.92M)	grow

The backup side kept working, which is why it looks like a cache that is full and never hits. Nothing in the server args or the HiCache defaults changed between the two boot logs; the bisect over the 353-commit range landed on this hunk, and restoring the clause as a one-hunk derivative image brought the numbers back.

Modifications

One hunk in create_match_validator: keep upstream's ring gate and put the device-only-HiCache clause back beside it.

swa_not_in_tree = is_swa_req_ring(self.cache.token_to_kv_pool_allocator) or (
    not self.tree_core.has_swa_host_pool and self.tree_core.enable_hicache
)

Both describe the same thing for the validator: the SWA rows are not in the tree, so a host-backed Full-KV prefix is matchable and load_back plus the scheduler's window rebuild take it from there. The ring path is unchanged. A layout with an SWA host pool is unchanged too: there a node with no SWA host value is a real miss and still reads False.

Not a duplicate: preflight against upstream/main (ddd4600) applies clean, none of the added lines is on HEAD, and the open PRs that touch swa.py (#36729 shared byte budget for hybrid-SWA memory, #39283 buffer-mode prefetch pipeline) share no added line with this one and do not touch the validator.

Accuracy Tests

test/registered/unit/mem_cache/test_swa_host_match_gate.py (CPU, 7 cases, base-a-test-cpu): drives the validator closure with stand-ins for the component, the tree core and the nodes, no pools built. test_paged_swa_with_hicache_and_no_swa_host_pool_accepts_tombstone fails on the ring-only gate and passes with this fix; the ring case (with and without an SWA host pool), the SWA-host-pool miss, HiCache off, the evicted-and-not-backed-up node, the device-only match and the window count are pinned so #38269's behaviour stays.

On the box: the 12c cache cell (device hits under HiCache) and the fresh-pool prefill / prodshape cells on the fixed image are at parity with the reference (8b 524k +1.6%, 9b c=1..8 -4%, 9c cached_ratio 0.993, 9d 2.627 tokens/verify), gsm8k unchanged within its draw.

Benchmarking and Profiling

Above: replay TTFT of a host-backed 64k prefix 10.3 s -> 0.366 s (reference 0.373 s), host restores 0/36 -> 36/36, load_back_tokens_total 0 -> 9.40M. No other cell moved.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests: test/registered/unit/mem_cache/test_swa_host_match_gate.py.
• Update documentation as needed: the comment above the gate names both layouts.
• Update benchmark and profiling as needed: numbers above.
• Add the run-ci label if you want the CI to run.

---

CI States

Latest PR Test (Base): ❌ Run #34933368381
Latest PR Test (Extra): ❌ Run #34933368250
Latest PR Test (AMD ROCm 10): ❌ Run #34933368332