fix(vllm): retain exact recurrent sources until store completion

[voipmonitor]

Purpose

An asynchronous hybrid-cache store must read the exact recurrent state that
the vLLM scheduler associated with its attention-cache prefix. Reconstructing
that state from incremental allocation deltas is unsafe because Mamba/GDN
align-mode block tables are sparse, mutable, and may release an interior state
before all tensor-parallel workers finish the store.

This pull request makes each store consume the scheduler-authoritative block
table and retains its source blocks until every worker reports terminal
completion.

Resulting behavior

• Exact recurrent boundary handoffs and per-group block tables replace
  connector-side positional reconstruction for the scheduled request.
• Every asynchronous store receives a durable job identifier. Source blocks are
  touched once and released only after completion reports from all vLLM ranks.
• Concurrent stores for one request have independent identities, futures, and
  completion counts.
• Null block placeholders are excluded and repeated source block identifiers
  receive one balanced retain/release pair.
• Recurrent cache groups reject lazy offload because deferred job selection
  cannot retain an exact boundary snapshot before its source is reusable.
• Scheduler heartbeats continue while retained stores still have pending worker
  completions.

Stack and attribution

This is a stacked pull request. Its branch contains the shareable cuMem runtime
from #33 unchanged, followed by one source-lifetime commit. After #33 merges,
GitHub will reduce this pull request to the source-lifetime commit.

The #33 commits retain their original authors and co-authors. The follow-up
commit is co-authored by Yifan Qiao, whose scheduler-authoritative recurrent
boundary work defines the vLLM interface consumed here.

The corresponding vLLM scheduler interface is reviewed in
local-inference-lab/vllm#557.

Compatibility

• Dense and attention-only object groups keep their existing store layout.
• Non-lazy stores gain per-job completion metadata; older workers that do not
  report that metadata are incompatible with source retention and must not be
  mixed in one engine.
• Lazy offload remains supported for cache layouts without recurrent groups.
• lmcache_driven cuMem transfer requires feat(cuda): add shareable cuMem IPC lifecycle #33 and the allocator compatibility
  changes in fix(kv): allow engine-driven expandable segments vllm#553 through [Question] Compatible lmcache version for vllm 0.7.3 LMCache/LMCache#555.

Validation

• 117 passed: LMCache MP connector, metadata, hybrid-cache layout,
  exact-boundary, multi-store, and worker-adapter tests.
• Ruff formatting and lint checks passed.
• TP4 byte-level qualification on four NVIDIA RTX PRO 6000 Blackwell
  Workstation Edition GPUs restored 8,192 external tokens into disjoint block
  allocations and matched every live recurrent and attention-cache byte on all
  four ranks for no-speculative, MTP:3, and DFlash2 serving.
• DRAM L1 and native-filesystem L2 qualification each restored 12,288 external
  tokens. A complete LMCache sidecar and vLLM restart restored the same prefix
  from native-filesystem L2.
• DFlash2 DCP4 full-CKV qualification matched four recurrent groups, rank-local
  target attention blocks, and the active 2,048-token draft sliding window on
  every rank.

[voipmonitor]

@coderabbitai review

[coderabbitai]

Action performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• dev/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 08c7e74d-b974-49c4-bb27-c062edef769c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/glm53-exact-recurrent-store-sources-20260901

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}