[Core] Coalesce concurrent external-prefix KV loads

[samar1tan]

Summary

• add a core registry for coalescing concurrent asynchronous KV loads of the same exact external prefix
• park followers without allocating GPU blocks or issuing another connector load
• publish the owner's blocks through the ordinary GPU prefix cache before releasing followers
• opt the built-in global CPU offloading spec into the mechanism while keeping Tiering, custom specs, NIXL, hybrid/EAGLE layouts, and full-prompt hits fail-closed
• handle follower/owner aborts, load failure policies, replacement owners, and connector reset safety

Why

Concurrent requests with the same external-prefix hit currently allocate and materialize independent GPU KV copies. Connector-private in-flight gating can avoid duplicate IO, but it serializes request progress and cannot attach a follower to the owner's eventual block table because allocation has already happened.

This change moves exact-prefix coalescing into the scheduler, before follower allocation. One owner performs the external materialization; followers wait without owning blocks and then reuse the published APC blocks with normal refcounts.

Closes #44882.

Scope and behavior

The initial production opt-in is deliberately narrow: one full-attention KV group, prefix caching enabled, no EAGLE, exact contiguous block-aligned partial-prefix hits, and the built-in CPUOffloadingSpec. Unsupported configurations retain their existing behavior.

On successful completion, the owner publishes first and followers re-enter ordinary local APC lookup. On owner abort or recompute recovery, followers bypass both local and external cache lookup once and recompute locally. The fail policy applies the owner's terminal outcome to its followers. Connector cache reset is rejected while a shared load is pending or draining.

Objective value re-evaluation

This is substantive core/serving work rather than mechanical cleanup. It implements the first-scope design requested by the open issue: for N concurrent requests with the same eligible external prefix, external materializations and GPU prefix copies are reduced from N to 1. The production opt-in covers the built-in global CPU offloading path, and unsupported paths remain fail-closed.

The behavioral integration test observes the two-request case directly: one load job instead of two, no follower-owned blocks while the load is pending, and three shared GPU prefix blocks with refcount 2 after publication. No other open PR implements issue #44882 or matches the same area keywords.

Serving and model evaluation

• model-output/accuracy evaluation: not applicable; this changes scheduler bookkeeping and KV residency, not model math, logits, or token selection
• serving behavior evaluation: the integration tests verify the deterministic 2 -> 1 materialization reduction and GPU block sharing described above
• real-hardware performance evaluation: not run because CUDA/XPU hardware was unavailable; no latency or throughput claim is made

Duplicate-work check

• issue [Feature]: Avoid duplicate KV block allocation for shared external-prefix hits #44882 is still open, unassigned, and has no comments
• no open PR references 44882 in its body
• no open PR matched external prefix load coalescing
• no PR already exists for this fork branch

Validation

The commands below were run by Codex in the local workspace. The human submitter must independently review the diff and rerun or confirm the relevant tests before marking the PR ready.

• HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 .venv/bin/python -m pytest tests/v1/core/test_shared_external_prefix.py tests/v1/kv_connector/unit/test_remote_prefill_lifecycle.py tests/v1/kv_connector/unit/test_kv_load_failure_recovery.py -q --confcutdir=tests/v1 — 32 passed
• HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 .venv/bin/python -m pytest tests/v1/kv_connector/unit/offloading_connector/test_scheduler.py -k 'core_coalesces_concurrent_partial_prefix_loads' -vv — 2 passed
• full tests/v1/kv_connector/unit/offloading_connector/test_scheduler.py with a test-only HMA capability shim on macOS CPU/MPS — 89 passed
• pre-commit run --files <all changed files> — all hooks passed, including ruff, formatting, typos, mypy 3.10, SPDX, forbidden imports, and configuration checks
• git diff --check — passed

Real CUDA/XPU transfer benchmarking was not available in this environment.

AI assistance and human review

This implementation and PR description were created with OpenAI Codex assistance. The commit includes a Co-authored-by trailer.

• The human submitter has reviewed every changed line.
• The human submitter has rerun or independently confirmed the relevant tests.
• The human submitter understands and can defend the design and tests end-to-end.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀