Repository navigation
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries - #60533
Merged
wzhao18 merged 3 commits intoOct 8, 2026
Merged
Conversation
…boundaries Extract the core prefix-cache changes from vllm-project#59826. Use a resend-safe checkpoint boundary, resolve checkpoint granularity during worker initialization, and inspect EAGLE attention proof before applying the returned hit limit. Preserve the local cache registration and retention fixes with their core regression tests. Mooncake connector, connector tests, and Mooncake CI setup are excluded. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
wzhao18
requested review from
AndreasKaratzas,
ApostaC,
DarkLight1337,
WoosukKwon,
alexm-redhat,
heheda12345,
ivanium,
njhill,
orozery,
robertgshaw2-redhat,
tdoublep,
tomeras91,
yewentao256 and
ywang96
as code owners
October 8, 2026 03:34
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
mergify
Bot
added
kimi
k3
mrv2
6 tasks
Dao007forever
approved these changes
Oct 8, 2026
Ship the hash block size and cache-hit alignment to workers through KVCacheConfig instead of injecting them from the model runners, and share the fine-grained lookup veto with the coordinator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Member
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #93531 for commit
|
Collaborator
CI selector (shadow): 143 test steps (190 jobs) instead of 95 (120 jobs)Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works. Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated.
Selector would run (143)
Would skip (today's rules run them) (16)
Would add (today's rules do not run them) (64)
AMD mirrors: would skip (10)
AMD mirrors: would add (41)
17 changed files · base |
Contributor
Author
|
/ci retry |
|
✅ Queued 2 failed job(s) for retry in Buildkite CI #93531. |
njhill
approved these changes
Oct 8, 2026
Contributor
Author
|
/ci run |
|
❌ This PR is 4 commits behind upstream |
Contributor
Author
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #93635 for commit
|
Contributor
Author
|
/ci retry |
|
✅ No failed, timed-out, or expired jobs need retrying: https://buildkite.com/vllm/ci/builds/93635 |
Contributor
Author
|
/ci retry |
|
✅ Queued 2 failed job(s) for retry in Buildkite CI #93635. |
wzhao18
enabled auto-merge (squash)
October 8, 2026 18:30
5 of 6 tasks
Dao007forever
added a commit
to Dao007forever/vllm
that referenced
this pull request
Oct 9, 2026
…artial tails Mooncake part of vllm-project#59826 by Wei Zhao, rebased onto main after the core fixes landed in vllm-project#60533. Save the full-attention KV through the Mamba checkpoint plus the EAGLE proof margin, retain the source blocks until the asynchronous save completes, and let the external lookup inspect the proof beyond the hit limit. partial_tail_block_ranges computes the tail blocks for both the scheduler (pinning) and the worker (puts). Also provisions a Mooncake master for the Kimi K3 prefix-cache CI and adds a Mooncake case to its resend test. Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TwEq3d2Nyvagw8ujXUmfci Signed-off-by: Dao Le <Dao007forever@gmail.com> Signed-off-by: Dao007forever <daole@inferact.ai>
6 tasks done
ZJY0516
added a commit
to ZJY0516/vllm
that referenced
this pull request
Oct 9, 2026
…#60533 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
6 tasks done
Open
6 tasks done
6 tasks done
michael-gpu
pushed a commit
to practical-inference/vllm-ampere-extended
that referenced
this pull request
Oct 20, 2026
…oundaries (vllm-project#60533) Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR extracts the core prefix-caching fixes from #59826, excluding the Mooncake-specific changes.
P - 1cap and EAGLE’s block drop.