[Core][DSV4] Skip caching SWA blocks that can never serve a prefix-cache hit

[ivanium]

Purpose

DeepSeek-V4 pairs full-attention layers with SWA layers with different block sizes and window sizes. The full attn layers have block size of 256, while SWA layers and compressors have block sizes of 64, 4, or 8.

• Within each 256-aligned segment, only the trailing tail = ceil((sliding_window - 1) / block_size) blocks are reachable by SWA's right-to-left scan.
• Hash-blocks past the last 256-aligned boundary are unreachable because hits are always lcm-aligned.

This PR drops the unreachable blocks at cache-time via a new alignment_tokens kwarg threaded through cache_blocks -> SlidingWindowManager._cache_block_mask -> BlockPool.cache_full_blocks. Non-hybrid coordinators pass alignment_tokens=None and hit a fast path identical to the existing behavior.

Test Plan

New tests in tests/v1/core/test_prefix_caching.py:

• test_hybrid_cache_blocks_swa_tail_window_only — full-attn block_size=32, SWA block_size=8, sliding_window=8 (lcm=32, tail=1, per_segment=4). After caching 8 SWA hash-blocks, asserts only hashes 3 and 7 are in the prefix-cache hash map; 0–2 and 4–6 are not.
• test_hybrid_cache_blocks_clamped_to_lcm — full-attn block_size=32, SWA block_size=16, sliding_window=32. After caching 7 SWA hash-blocks (112 tokens), asserts hashes 0–5 are cached and hash 6 (past the last lcm boundary) is not.

Test Result

Passed. Claude is used to generate test cases but I have reviewed them.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[gemini-code-assist]

Code Review

This pull request optimizes prefix caching for hybrid models by introducing a block masking mechanism, ensuring that only blocks capable of serving future cache hits (such as the tail window in Sliding Window Attention) are stored. Key changes include adding a block_mask to the BlockPool, implementing LCM-aligned caching in the KVCacheCoordinator, and updating cache managers to support sparse hit semantics. Review feedback highlighted potential assertion failures and memory leaks when handling shared physical blocks in models like DeepSeek-V4. Additionally, a bug was identified where token_ids in BlockStored events are not correctly filtered when blocks are masked, which could impact distributed caching systems.

[gemini-code-assist]

While extra_keys_list is correctly filtered by the block_mask, the token_ids slice passed to the BlockStored event (line 319) is not filtered. This results in a mismatch between the number of hashes and the number of tokens in the event when blocks are masked out (e.g., in SWA tail-window caching). This will likely break distributed caching consumers or offloading mechanisms that rely on these events. Please ensure token_ids is filtered to only include tokens for the blocks actually being cached.

[njhill]

Thanks @ivanium!

Looks like we should rebase now