Repository navigation
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment Once the PR is approved or has the If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
Adds CPU-only assertions for the event payloads of the cache layouts listed under P1 in vllm-project#63149 but not asserted today: MLA, GDN and dense partial hits. Signed-off-by: ker <zhaokemail@foxmail.com> Co-authored-by: Claude Code <noreply@anthropic.com>
907a55c to
afba7e5
Compare
Overview
Refs #63149 (P1 "Establish coverage across cache layouts"), claimed there on
2026-10-23 as the CPU-level part. Test-only: adds
tests/v1/core/test_kv_cache_events_layouts.pywith CPU assertions for thecache layouts that have no event assertions today — MLA, GDN/recurrent, and
dense partial hits where
hash_block_size < block_size.Claims
kv_cache_spec_kind == "mla_attention"with a hash chain,token span and parent that match the reusable prefix — on store, on reuse, and
on a
kv_cache_report_mode="full"republish.kv_cache_spec_kind == "mamba"with onecheckpoint hash covering the reusable span; the event shape is identical for
both
mamba_typevalues.hash_block_size < block_size, a prompt ending inside a cache blockemits a tail
BlockStoredat hash granularity whose hash, parent and tokenspan are exactly what a sibling resumes from. The companion case shows an
attention tail past the last recurrent checkpoint is not a reusable hybrid
prefix.
The assertions are phrased semantically so they can serve as the expected-event
spec for the GPU matrix that @qwert123q proposed in the tracker.
Validation
No model/GPU evaluation applies: this changes no production code and no output.
CI already covers it —
.buildkite/test_areas/misc.yamlrunspytest -m 'cpu_test' v1/coreoncpu-small, andtests/v1is in that step'ssource_file_dependencies, so no Buildkite change is needed.Details
Why these cases. Event tests in
test_prefix_caching.pycoverFullAttentionSpecand sliding window; partial-block events are covered only atthe
BlockPool.cache_partial_blockprimitive level, not through the manager.Nothing asserts the emitted kind or hash semantics for MLA, a recurrent group,
or a sub-block hit.
Why this is not a duplicate.
the zero-hash chunk a recurrent group emits is left unasserted and only the
dense group's chunk event is checked.
report_all_hash_boundaries(opt-in, defaultFalse). Thesetests never pass it and assert the tail via
block_hashes[-1], so they holdeither way.
the parent asserted in the dense partial case is a published full-block
boundary.
BlockRemovedevents; nothing here assertsremoval counts or total event counts.
xfail; different feature axis,and there is no EAGLE in this change.
Not covered (stated in the module docstring): zero-hash recurrent chunk
events (#44488), off-grid single-chunk prefills, and DCP > 1.
AI assistance. AI assistance was used: the change was drafted with an AI
coding agent (Claude Code) and reviewed line by line by the submitter, who ran
the commands above.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.
🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.Need help on this PR? Tag
@codesmith-botwith what you need.