Skip to content

[Bugfix] Restore KV cache metadata GET method for external event consumer - #56925

Merged
pavanimajety merged 5 commits into
vllm-project:mainfrom
wzhao18:wzhao/fix-dcp-kv-event-metadata
Sep 17, 2026
Merged

[Bugfix] Restore KV cache metadata GET method for external event consumer#56925
pavanimajety merged 5 commits into
vllm-project:mainfrom
wzhao18:wzhao/fix-dcp-kv-event-metadata

Conversation

@wzhao18

@wzhao18 wzhao18 commented Sep 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Purpose

get_kv_cache_group_metadata is deleted by #53941 as dead code but is actually used by external event consumers like Dynamo. The PR brings the function call back.

In addition, this PR fixes Dynamo KV-aware routing for DCP by reporting the effective scheduler KV block size used by event emission. Kimi K3 DCP8 emitted 12,288-token MLA blocks while metadata advertised 1,536, so Dynamo rejected those events. Testing shows it changed routing from 0/4 cache-hit predictions to 4/4 at 75% expected prefix match, with all four requests routed to the seeded prefill worker.

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

@wzhao18
wzhao18 requested a review from njhill as a code owner September 15, 2026 01:30
claude[bot]
claude Bot reviewed Sep 15, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the bug Something isn't working label Sep 15, 2026
@wzhao18 wzhao18 changed the title [Bugfix] Restore KV cache metadata retrieval for external event consumer [Bugfix] Restore KV cache metadata GET method for external event consumer Sep 15, 2026
@wzhao18
wzhao18 force-pushed the wzhao/fix-dcp-kv-event-metadata branch from 82f292c to 814ea79 Compare September 16, 2026 18:06
@njhill njhill added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 16, 2026
@njhill

njhill commented Sep 16, 2026

Copy link
Copy Markdown
Member

/ci run

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown

❌ This PR is 5 commits behind upstream main. Your branch must contain every commit currently on upstream main. No new CI build was started. Merge or rebase onto the latest main, then rerun /ci run. To test this branch at your own risk, use /ci run --allow-stale.

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown

✅ @wzhao18, CI is now available for this PR.

  • /ci run starts upstream CI; /amd-ci run starts AMD CI only.
  • Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
  • /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
  • /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
  • /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

@wzhao18
[Bugfix] Report DCP-adjusted KV event block sizes
c0004be 
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
@wzhao18
wzhao18 force-pushed the wzhao/fix-dcp-kv-event-metadata branch from 814ea79 to c0004be Compare September 16, 2026 21:50
@wzhao18

wzhao18 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

/ci run

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #89454 for commit c0004beb2675.

@pavanimajety pavanimajety mentioned this pull request Sep 16, 2026
Closed
4 tasks
@pavanimajety
pavanimajety enabled auto-merge (squash) September 16, 2026 22:19
@wzhao18

wzhao18 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

/ci run

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown

❌ This PR is 1 commit behind upstream main. Your branch must contain every commit currently on upstream main. No new CI build was started. Merge or rebase onto the latest main, then rerun /ci run. To test this branch at your own risk, use /ci run --allow-stale.

@wzhao18

wzhao18 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

/ci run

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #89465 for commit 3b6044e2eef9.

@wzhao18

wzhao18 commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

@njhill @pavanimajety ready for merge.

@wzhao18

wzhao18 commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

/ci run

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #89719 for commit 42285dcee716.

@pavanimajety
pavanimajety merged commit ccde9c6 into vllm-project:main Sep 17, 2026
137 checks passed
@hickeyma hickeyma mentioned this pull request Oct 5, 2026
Open
@hickeyma hickeyma mentioned this pull request Oct 5, 2026
Merged
@JulienDarve JulienDarve mentioned this pull request Oct 5, 2026
Merged
michael-gpu pushed a commit to practical-inference/vllm-ampere-extended that referenced this pull request Oct 20, 2026
@wzhao18
[Bugfix] Restore KV cache metadata GET method for external event cons…
e315138 
…umer (vllm-project#56925)

Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@njhill njhill njhill approved these changes
@pavanimajety pavanimajety pavanimajety approved these changes

Assignees

No one assigned

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wzhao18 @njhill @pavanimajety