[Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash

[ivanium]

Purpose

DeepSeek-V4.1 keeps a 128-token sliding-window (SWA) KV cache per layer beside its prefix-cacheable MLA/indexer caches. Storing that window for prefix caching and KV connectors costs more than it saves, so this implements DeepSeek's encoder-side "SWA bounded replay": the SWA groups opt out of prefix caching, and after a Main/Indexer hit ending at H the scheduler recomputes the hit's last window [H-128, H) to rebuild them. Replayed tokens keep their cached KV in every other group (their paged-KV writes are padded), their window attention ignores positions below H-128, and decode always sees a full window. Outputs after a hit are therefore not bit-identical to a cold run, by design.

• Spec contract: SlidingWindowMLASpec(bounded_replay=True) derives prefix_cacheable=False and prefix_replay_tokens=sliding_window; UniformTypeKVCacheSpecs forwards it so the worker reads it through packed groups.
• Scheduler: a hit records Request.replay_start = H-128 and rewinds the computed count by the window; async connector loads replay once the KV arrives; chunks ending inside the replayed range allocate nothing new; hits no longer than the window are not adopted (they would be recomputed in full).
• Worker (model runner V2): replay_start lives in RequestState, is gathered per batch by the existing pos/seq_lens kernel and reaches attention metadata as CommonAttentionMetadata.replay_start; the slot-mapping kernel pads the replayed window's writes in prefix-cacheable groups, and compressed slot mappings (MLA and indexer) inherit that padding inside their kernel.
• DeepSeek-V4.1 kernels clamp the window at replay_start: SWA index and gather-length kernels, the FlashInfer mixed index kernel and the FlashMLA combine kernel.
• Switch: --swa-bounded-replay (CacheConfig.swa_bounded_replay, default on), read by the V4.1 attention layer. Needs model runner V2; on V1 it logs and keeps the SWA cache in prefix caching.
• Separate commit: the ratio-2 compressor ring wrote into the shared null block 0 during CUDA graph capture (all-zero dummy block tables); ring slots of block 0 are now padded.

KV-connector load failures: with #50388 a hybrid request that lost blocks is recomputed from position 0. Async loads then go back through admission, which recomputes the replay boundary. A running request under kv_load_failure_policy=recompute is restarted in place and keeps its worker-side replay boundary, so it would recompute with a stale clamp and write mask; a follow-up will preempt such requests instead (in-place restart also writes sliding-window KV of positions below the old window into the shared null block, replay or not). No other open PR implements SWA replay; #53558 (KV cache config builder) is orthogonal.

Usage

Nothing to configure: DeepSeek-V4.1 replays by default on model runner V2 (the default runner). --no-swa-bounded-replay restores the previous behaviour (SWA KV stored in the prefix cache). With replay on, KV connectors transfer only the MLA/indexer caches; the SWA cache is rebuilt locally after every hit, and hits of at most 128 tokens are not adopted.

Test Plan

pytest tests/v1/core/test_prefix_replay.py tests/v1/core/test_scheduler.py tests/v1/core/test_contiguous_kv_packing.py \
  tests/v1/worker/test_gpu_block_table.py tests/v1/worker/test_gpu_input_batch_v2.py \
  tests/v1/attention/test_deepseek_v4_swa_visible.py tests/v1/attention/test_indexer_deepseek_v4_slot_mapping.py \
  tests/kernels/test_compressor_kv_cache.py tests/v1/kv_connector/unit/test_kv_load_failure_recovery.py

Accuracy: deepseek-ai/DeepSeek-V4.1-Flash on TP4 GB200 with vllm serve, gsm8k (5-shot) run twice against the same server; pass 2 re-sends every prompt, so each request is a full-prompt prefix hit that replays only its last window (92.7% prefix-cache hit rate).

Test Result

All listed tests pass; pre-commit and mypy are clean. A per-layer probe (cold prefill vs. prefix-hit replay of one prompt) confirms the replayed rows follow the clamped window and the cached Main/indexer KV is untouched.

gsm8k, n=1319 (pass 1 cold / pass 2 all hits)	replay on (this PR)	replay off (--no-swa-bounded-replay)
exact_match (flexible-extract)	0.9265 / 0.9265	0.9295 / 0.9295

The 0.3% gap is the same in pass 1, where almost nothing is replayed, so it is run-to-run noise rather than a replay effect. An earlier two-pass GPQA-diamond run on an internal pre-release checkpoint showed the same parity (0.6869 / 0.6919 with replay vs 0.7020 / 0.6869 without, n=198).

🤖 Developed with Claude Code; all changes reviewed and tested by the author.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ivanium.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ivanium.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ivanium.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork