[KVConnector] Handle KV events from multiple connectors

[hickeyma]

Purpose

This is a follow on PR to #28309. It was raised by @NickLucche in #28309 (review) to request support for KV events for multiple connectors.

Test Plan

pytest tests/v1/kv_connector/unit/test_multi_connector.py -v

Test using vLLM and LMCache from CLI:

PYTHONHASHSEED=0 LMCACHE_CONFIG_FILE=lmcache_config.yaml vllm serve Qwen/Qwen3-14B --kv-transfer-config '{"kv_connector":"MultiConnector", "kv_role":"kv_both", "kv_connector_extra_config":{"connectors":[{ "kv_connector": "LMCacheConnectorV1", "kv_role": "kv_both" }, { "kv_connector": "ExampleConnector", "kv_role": "kv_both" }]}}' --disable-log-requests --no-enable-prefix-caching --kv-events-config '{"enable_kv_cache_events": "True", "publisher": "zmq", "topic": "kv-events"}' --tensor-parallel-size 2

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

---

Note

Adds per-connector KV event handling and aggregation for MultiConnector and across workers.

• Introduces MultiConnectorKVEvents to hold KV events keyed by connector type
• Implements MultiConnector.get_kv_connector_kv_cache_events() to group events by connector; update_connector_output() now dispatches per-connector events downstream
• Updates KVOutputAggregator to merge MultiConnectorKVEvents across workers while preserving per-connector boundaries
• Adds comprehensive unit tests for event grouping/merging and edge cases in test_multi_connector.py

^{Written by Cursor Bugbot for commit b34f6a8a3760a496e0181ad4a5c399e4bf5ceb8a. This will update automatically on new commits. Configure here.}

---

Note

^{Cursor Bugbot is generating a summary for commit 6041228d3729cc70f0c4d92534f26913729453e1. Configure here.}

---

Note

Introduces per-connector KV cache event plumbing for multi-connector setups and aggregates events across workers while preserving connector boundaries.

• New MultiConnectorKVEvents to hold per-connector KVConnectorKVEvents
• MultiConnector.get_kv_connector_kv_cache_events() groups events by connector type; update_connector_output() dispatches per-connector events to each connector
• KVOutputAggregator in kv_connector/utils.py merges MultiConnectorKVEvents across workers (including mixed single/multi cases)
• Unit tests added in tests/v1/kv_connector/unit/test_multi_connector.py for event grouping, merging, and edge cases

^{Written by Cursor Bugbot for commit 6041228d3729cc70f0c4d92534f26913729453e1. This will update automatically on new commits. Configure here.}

---

Note

Adds per-connector KV cache event plumbing to multi-connector setups and preserves connector boundaries during aggregation.

• New MultiConnectorKVEvents container to hold per-connector KVConnectorKVEvents
• MultiConnector.get_kv_connector_kv_cache_events() groups events by connector type; update_connector_output() dispatches per-connector events to each connector
• KVOutputAggregator in kv_connector/utils.py merges MultiConnectorKVEvents across workers (also handles mixed single/multi cases)
• Unit tests in test_multi_connector.py for event grouping, merging, and edge cases

^{Written by Cursor Bugbot for commit 43b8ec71f51294bc09afb37f73df1a6ab1b165ba. This will update automatically on new commits. Configure here.}

[gemini-code-assist]

Code Review

This pull request adds support for handling KV cache events from multiple connectors, which is a valuable enhancement. The implementation introduces MultiConnectorKVEvents and updates the aggregation logic accordingly. While the overall approach is sound, I've identified several areas for improvement. My review focuses on simplifying the new tests, which are currently overly complex, removing redundant code in the event aggregation logic, and addressing a potential performance bottleneck by replacing a deepcopy operation with a more efficient alternative. These changes will improve the code's readability, maintainability, and performance.

[mergify]

Hi @hickeyma, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hickeyma.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[NickLucche]

Thanks for the followup @hickeyma .

It appears current impl is devaiating somewhat from my expected model (eg MultiKVConnectorStats). What's preventing us from implementing the abstract methods in MultiConnectorKVEvents and let the system program to that interface as it was no different from an LMCacheKVEvents instance?

[mergify]

Hi @hickeyma, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[hickeyma]

It appears current impl is devaiating somewhat from my expected model (eg MultiKVConnectorStats). What's preventing us from implementing the abstract methods in MultiConnectorKVEvents and let the system program to that interface as it was no different from an LMCacheKVEvents instance?

This is to handle Multiple connectors (different types) that generate KV events. Each connector type may process thier events differently before making them available to scheduler for publishing and therefore the specific connector KVConnectorKVEvents implementation needs to be called. MultiConnectorKVEvents is essentially only a contaner to pass the events.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hickeyma.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[hickeyma]

Superseded by #34522