[maithilijoshi20]

Overview

Fix cache reset on P2P prefill nodes when store jobs are waiting for a decoder to fetch their KV blocks.

Fixes #63948.

Claims

• Queued stores report failure and release their primary-tier pins during reset.
• Late fetches fail promptly without reading released slots or leaving stale session bindings.
• Active transfers keep their pins until completion, and fresh symmetric lookup rounds remain usable after reset.

Validation

Tested on WSL Ubuntu 24.04 with Python 3.12, PyTorch 2.14.0+cu132, NIXL 1.5.1/UCX, and an RTX 3050 8 GB.

.venv/bin/python -m pytest \
  tests/v1/kv_offload/tiering \
  tests/v1/kv_offload/cpu/test_shared_offload_region.py \
  tests/v1/kv_offload/test_factory.py \
  tests/v1/kv_offload/test_file_mapper.py -q --tb=short
# 808 passed, 1 skipped

export VLLM_USE_FLASHINFER_SAMPLER=0 VLLM_PLUGINS=
export CUDA_HOME=/home/maithijoshi/.venvs/vllm-py312/lib/python3.12/site-packages/nvidia/cu13
VLLM_USE_V2_MODEL_RUNNER=1 .venv/bin/python -m pytest \
  tests/v1/kv_connector/unit/test_offloading_connector.py::test_p2p_cache_reset_with_absent_decoder \
  -q --tb=short
# 1 passed

The OPT-125m test waits for an active cascade with an absent decoder, calls LLM.reset_prefix_cache(reset_connector=True), and checks that the next request has zero cached tokens and identical greedy output. Unit tests cover queued stores, mid-drain fetches, symmetric lookup rounds, active transfer pins, and abort acknowledgements. Pre-commit checks pass for the changed files.

Without the fix, the issue's producer-reset reproducer fails at assert not self._jobs in TieringOffloadingManager.reset_cache(); with the fix, both unbound and session-bound stores drain successfully, as confirmed in the issue author's PR feedback.

Details

A prefill node can finish storing KV before its decoder sends a fetch. These jobs hold primary-tier slots but are not counted as active transfers, so waiting only for transfers leaves jobs behind when the parent resets its cache.

Drain fails unbound stores through the existing expiry path and withdraws queued session supply once. Fetches received during polling are rejected without binding their request IDs. In-flight writes retain their pins until the transport reports completion.

Withdrawn symmetric lookup rounds retain enough state to reject a late fetch without invalidating a fresh round under the same request ID. Aborted rounds use the existing acknowledgement path to avoid duplicate terminal messages.

This addresses producer stores waiting for decoder demand. Related #61667 covers consumer write quarantine. No duplicate fix was found when opening this PR. AI assistance was used for review, and validation.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill.
• AI assistance was used during the creation of this PR.
• Design Fit: Reuses existing expiry and transfer-finalization paths.
• Testing and Validation: Includes targeted regression tests and a real-engine reset test.
• Code Quality and Style: Keeps the implementation focused and passes pre-commit checks.
• Pull Request Contents: Explains the failure, fix, and validation.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[Vedant-Agarwal]

Thanks for picking this up. Ran the repros from #63948 on this branch: both pass now and fail on main.

Small thing: test_reset_cache_releases_parked_p2p_slots hardcodes 4096 for the page size, so it fails on hosts with bigger pages (16 KiB on macOS arm64). mmap.PAGESIZE or _mock_mmap_region(8) like the other tests would fix it.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @maithilijoshi20.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[liranschour]

Thanks for fixing #63948. The direction looks right, but I think there are a few correctness issues in the drain path before this can go in, mainly the interaction with fetches that arrive mid-drain and with symmetric (lookup-supplied) rounds. Details are inline. The last three comments are pre-existing or nits.

[liranschour]

LGTM

[orozery]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99191 for commit 69610c6f81bb.

ℹ️ This PR is 181 commits behind upstream main. Merged upstream main into this branch as 69610c6f81bb before starting CI. Pull the branch before pushing more changes.

[github-actions]

✅ @maithilijoshi20, CI is now available for this PR.

• /ci run starts upstream CI; /amd-ci run starts AMD CI only.
• Run commands test your branch with every commit currently on its upstream target branch. If it is behind, the bot merges the target branch into it first (pull before pushing again). Append --allow-stale to a run command to test an outdated branch as is, at your own risk.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
• /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.