[ivanium]

Overview

Stacked on #62278; review only the last commit. The MooncakeStore lookup now detects shared-prefix junctions, and the store keeps the SWA tails at those junctions under sparse retention. Without this, a fresh instance (or one that evicted the prefix locally) gets 0 hits from the store for siblings that share a prefix, because store retention keeps only each request's own replay tails.

Claims

• DeepSeek-V4-Flash + MTP + MooncakeStore, local prefix cache reset before every request: siblings get 3584 / 3840 hit tokens from the store (d = 8 / 160 past an aligned 3840-token prefix). On main they get 0 in both cases.
• Shared-prefix GSM8K (200 questions, local cache reset before each): 194 questions hit 3584 tokens and 4 hit 3840 from the store, with accuracy 0.975. On main every question gets 0 hits, with accuracy 0.97.
• No dense SWA retention in the store: the store keeps only the tails at junction positions, in addition to the existing replay tails.

Validation

End to end (GB200, 4 GPUs, base 83c9a54): the same serve command as #62278, plus --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_both"}'. Before every request after the first, the client calls /reset_prefix_cache, so all hits come from the store.

	main	this PR
junction probe C, D (d = 8)	0, 0	3584, 3584
junction probe C, D (d = 160)	0, 0	3840, 3840
A replayed (own prompt, store sanity)	4864 / 5120	4864 / 5120
GSM8K accuracy (200)	0.97	0.975
GSM8K store hit rate (external_prefix_cache_hits / queries)	0%	81.2%

Unit tests (CPU): the store coordinator, scheduler, worker and connector suites, plus the core suites from #62278: 1562 passed. The new tests cover a two-instance save → lookup → save → lookup flow on one store, junctions in store_mask (several positions, never on Mamba groups), the tracker hand-off when a loading sibling skips its first save, and the lookup-response round trip, including payloads from writers that predate junction entries.

Details

Source. MooncakeStoreWorker.lookup() already queries the store hits per group. The store coordinator now also returns the longest single-group hit, and computes the junctions with the same junction_boundaries() as the local coordinator. The lookup response carries them as 8-byte entries with a sentinel group id. Old readers ignore those entries, because the loader reads only real group ids.

Consumer. The scheduler merges the store junctions into request.shared_prefix_boundaries at lookup, before compute. So GPU retention, Mamba chunk stops and this request's own store saves all use them. RequestTracker and ReqMeta carry the junctions to store_mask, which adds them to the reachable boundaries.

Lifetime. Blocks are pinned when the save job is created, in the same schedule() pass. The worker skips null block ids, so a tail block that the GPU already freed is not saved with wrong data.

Conflict. #62255 moves the Mooncake metadata and wire structs to new modules; one of the two will need a rebase.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR. Claude Code wrote and verified the change and ran the GPU checks; a Kimi Code agent fleet reviewed the design and wrote test extensions; Codex reviewed the diff.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ivanium.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork