[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning - #53558
Open
wangxiyuan wants to merge 2 commits into
Open
wangxiyuan wants to merge 2 commits into
wangxiyuan wants to merge 2 commits into
Conversation
wangxiyuan
requested review from
AndreasKaratzas,
ApostaC,
DarkLight1337,
ProExpertProg,
WoosukKwon,
alexm-redhat,
heheda12345,
houseroad,
ivanium,
mgoin,
njhill,
orozery,
robertgshaw2-redhat,
tlrmchlsmth,
yewentao256,
youkaichao and
ywang96
as code owners
August 24, 2026 09:34
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
wangxiyuan
changed the title
wangxiyuan
changed the title
3 tasks
wangxiyuan
force-pushed
the
kv_config_builder
branch
from
August 25, 2026 08:30
531d4e9 to
0cd95bd
Compare
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
wangxiyuan
force-pushed
the
kv_config_builder
branch
from
August 31, 2026 08:38
0cd95bd to
a408951
Compare
|
✅ @wangxiyuan, CI is now available for this PR.
|
yzeyu71
added a commit
to yzeyu71/vllm-ascend
that referenced
this pull request
Sep 10, 2026
The branch is merged after vllm-project/vllm#53558 lands, so the try/except import fallbacks in worker.py and the dynamic patch target in patch_kv_cache_utils.py are no longer needed: the KV cache planning entry points are always in vllm.v1.core.kv_cache_planning. Signed-off-by: yangzeyu <532183776@qq.com>
yzeyu71
added a commit
to yzeyu71/vllm-ascend
that referenced
this pull request
Sep 10, 2026
The branch merges after vllm-project/vllm#53558, so the superseded DeepSeekV4 planning monkey-patches and the pre-#53558 kv_cache_utils targets are dropped: - patch/platform/patch_kv_cache_utils.py: keep only resolve_kv_cache_block_sizes, the Kimi K3 DSpark grouping, KVCacheConfig.has_mamba_layers and the GLM5-Next cache layout (ported from main), all wired to vllm.v1.core.kv_cache_planning. DeepSeekV4 planning lives in AscendKVCacheConfigBuilder. - tests/ut: restore the builder-based assertions for test_prefix_cache_cp_patches.py and test_attn_utils_v2.py. Signed-off-by: yangzeyu <532183776@qq.com>
yzeyu71
added a commit
to yzeyu71/vllm-ascend
that referenced
this pull request
Sep 10, 2026
The branch merges after vllm-project/vllm#53558, so the superseded DeepSeekV4 planning monkey-patches and the pre-#53558 kv_cache_utils targets are dropped: - patch/platform/patch_kv_cache_utils.py: keep only resolve_kv_cache_block_sizes, the Kimi K3 DSpark grouping, KVCacheConfig.has_mamba_layers and the GLM5-Next cache layout (ported from main), all wired to vllm.v1.core.kv_cache_planning. DeepSeekV4 planning lives in AscendKVCacheConfigBuilder. - tests/ut: restore the builder-based assertions for test_prefix_cache_cp_patches.py and test_attn_utils_v2.py. Signed-off-by: yangzeyu <532183776@qq.com>
wangxiyuan
force-pushed
the
kv_config_builder
branch
2 times, most recently
from
September 10, 2026 07:26
35a8b5a to
604e7c4
Compare
Contributor
Author
|
/ci run |
|
✅ Triggered Buildkite CI #88075 for commit |
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
4 tasks
wangxiyuan
force-pushed
the
kv_config_builder
branch
2 times, most recently
from
September 14, 2026 02:32
c264025 to
70c1f9f
Compare
Contributor
Author
|
/ci run |
|
✅ Triggered Buildkite CI #88712 for commit |
wangxiyuan
force-pushed
the
kv_config_builder
branch
from
September 14, 2026 06:59
70c1f9f to
d921fa3
Compare
Contributor
Author
|
/ci run |
|
✅ Triggered Buildkite CI #88755 for commit |
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
…ic KV cache planning Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
wangxiyuan
force-pushed
the
kv_config_builder
branch
from
September 15, 2026 01:14
d921fa3 to
ee06d03
Compare
Contributor
Author
|
/ci run |
|
❌ This PR is 11 commits behind upstream |
Contributor
Author
|
/ci run |
|
✅ Triggered Buildkite CI #89032 for commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Pick from #42309 to follow up kv cache config builder implementation.
Add a pluggable KVCacheConfigBuilder so platforms or models can override KV cache planning end-to-end. Resolution order: platform override > model declaration > default. Default builder forwards to existing get_kv_cache_configs, so behavior is unchanged today.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.Co-Authored-By: Yifan Qiao yifanqiao@inferact.ai
Co-Authored-By: Mengqing Cao cmq0113@163.com