[0z5a]

Purpose

Mooncake recurrent lookup currently queries and leases every candidate state, including states beyond the KV-supported resume point. Add the default-off mooncake_kda_last_hit_only option for FullAttention + align-mode Mamba caches, using the selected-only LastHitOnly contract proposed in Mooncake RFC #4255.

One existing coordinator pass bounds legal checkpoints from KV evidence, including Eagle and final-token recomputation. The connector sends recurrent keys in checkpoint-major order across the actual group/rank namespaces and returns the exact selected checkpoint for subsequent loading. Partial-hash KV tails retain their existing physical-key handling. Invalid responses and probe failures return a cache miss without a lease-renewing recurrent existence fallback. Shared group leases and unsupported cache types are rejected on opt-in; the default path and lookup wire format remain unchanged.

This draft depends on matching client/Master implementations from Mooncake PR #4314, tested at 7943c052be6aab81c71b60fea9483952dec21141. The selected-only response interface is still under review. No overlapping open vLLM PR was found for 4255 in:body or mooncake LastHitOnly on 2026-10-01. The change is based on vLLM 2e52a558f04307b3f6f842e937e35aac079121c1 and contains 889 changed lines across code, tests, documentation and the benchmark.

Test Plan

Run the six Mooncake Store suites in the existing Thor Python 3.12 / PyTorch 2.13.0+cu130 runtime, with the separately built CPU/TCP Mooncake binding exposed through PYTHONPATH and LD_LIBRARY_PATH:

MOONCAKE_LAST_HIT_MASTER=127.0.0.1:23351 "$PYTHON" -m pytest -q \
  tests/v1/kv_connector/unit/test_mooncake_store_{worker,coordinator,connector,layout,scheduler,hma_e2e}.py

for divisor in 1 8; do
  "$PYTHON" benchmarks/benchmark_mooncake_last_hit_only.py \
    --checkpoints 128 --iterations 1000 --kv-divisor "$divisor"
  "$PYTHON" benchmarks/benchmark_mooncake_last_hit_only.py \
    --checkpoints 128 --iterations 1000 --kv-divisor "$divisor" --master 127.0.0.1:23351
done

Run each benchmark scenario in three fresh processes, using --selective-first in the second. Also run the dependency's test_batch_probe_policy.py against the same Master and master_service_probe_policy_test --gtest_filter='BatchProbePolicyTest.*' to verify lease deadlines, soft pinning and eviction behavior directly.

Test Result

Validation	Baseline	Modified
Worker / coordinator / connector	261 passed	305 passed
Layout / scheduler / HMA save-load	62 passed	62 passed
Six-suite total	323 passed across two runs	367 passed, zero skipped in one run
Real TCP lookup → metadata → receiver buffers	—	4 passed, included above
Upstream Python probe contract	—	3 passed
Upstream Master lease / eviction checks	—	9 passed

The real TCP cases verify bytes loaded into both recurrent rank buffers for complete, rank-hole, KV-cap and Eagle scenarios. Unit tests cover 4096 physical existence patterns, unequal namespace counts, duplicate keys, partial-hash tails, recurrent-only lookup and invalid/RPC-error responses. Baseline and modified suites emit the same 14 Torch JIT deprecation warnings. Ruff check/format, Markdown lint, SPDX, forbidden-import, CUDA-call, boolean-context-manager, lazy-import and test-tethering checks passed.

Lookup latency in microseconds; values are medians of three per-process p50 measurements, with 20 warmups and 1000 measured lookups per path. The existing path is the flag-off path on the same worker and data.

Transport / KV checkpoints available	Before p50	After p50	Speedup	Individual-run speedup range
In-memory / 128	397.15	375.36	1.06×	1.05–1.06×
In-memory / 16	308.67	162.87	1.90×	1.89–1.90×
TCP / 128	2037.92	1235.93	1.65×	1.43–2.44×
TCP / 16	1694.88	678.66	2.50×	2.46–2.96×

The workload has 128 recurrent checkpoints, one KV namespace and four recurrent namespaces, with all recurrent states present. Selective lookup leases four recurrent physical keys instead of 512. TCP selective timing varies substantially between processes; these measurements cover connector lookup and Store control work, without model inference or GPU/token-throughput claims. The committed benchmarks/mooncake_last_hit_only_results.md includes p95 values, key/RPC counts and reproduction details.

---

Essential Elements of an Effective PR Description Checklist

• Concrete problem and resulting behavior, with related RFC/dependency.
• Test commands and reproduction details.
• Before/after results and real connector-to-Store E2E results.
• Usage documentation and dependency/supported-scope details.

[mergify]

Documentation preview: https://vllm--59522.org.readthedocs.build/en/59522/