[njhill]

Overview

Fixes #65204. Supersedes #65210 (carries over its connector gating, with Vegetog as co-author).

With the extensible KV cache (#56492), any KV connector made the engine round the KV block count down to a multiple of granule / gcd(granule, block_stride). For awkward strides that multiple is huge, so engines with --kv-offloading-size could fail to start. This PR pads segment placement instead, and applies it (and the defragmenting commit) only for connectors that register the KV cache for RDMA.

Claims

• --kv-offloading-size (and any connector that doesn't register the device KV cache) keeps its measured block count: no rounding, no padding, no defragmenting commit.
• NIXL / Mooncake / MoRIIO / UMBP keep one physical chunk per committed segment for RDMA registration at any block count, instead of losing up to one alignment unit (2048 blocks, ~11.6 GB, for the stride in [Bug]: Extensible KV cache + KV offload connector fails to start: granule alignment rounds reserved blocks to 0 #65204) or failing to start below it.
• UMBPStoreConnector is added to registers_kv_cache_memory(): it registers GPU KV storage through MORI. This also turns the extensible cache off for it on ROCm, as for MoRIIO.

Validation

GB200, all on this branch unless noted.

Reproducer: HuggingFaceTB/SmolLM2-360M-Instruct (20 KiB page = 2¹²·5, so main needs a multiple of 512 blocks), kv_offloading_size=1, block_size=16, max_model_len=1024, enforce_eager=True. Baseline = origin/main versions of the changed files over the same build.

gpu_memory_utilization	main	this PR
0.014	fails at startup: "No available memory for the cache blocks … decrease max_model_len" (102 blocks rounded to 0)	starts, 102 blocks, correct greedy output
0.018	1024 blocks	1331 blocks, same output

NIXL P/D: tests/v1/kv_connector/nixl_integration/run_accuracy_test.sh with PREFILL_BLOCK_SIZE=16 DECODE_BLOCK_SIZE=16 (Qwen3-0.6B, 1P1D): passes for the default layer-outermost layout (padded) and with --enable-cross-layers (block-outermost, single segment), 0 failed transfers. A separate DEBUG run with NixlConnector confirmed the padded path ran: capacity 19254 blocks (main would have rounded to 19232), NIXL registered the padded segments, and the final 18624 blocks committed exactly the nominal 31.83 GiB.

Unit tests: tests/v1/worker/test_extensible_kv_cache.py, tests/utils_/test_extensible_tensor.py, tests/v1/core/test_kv_cache_planning.py, tests/v1/engine/test_engine_args.py, tests/v1/worker/test_gpu_worker.py, tests/v1/worker/test_gpu_warmup_blocks.py all pass. New/changed coverage:

• test_pad_kv_cache_segments (CPU): uniform and mixed block strides, block-outermost no-op.
• test_defragmenting_commit_backs_each_segment_with_one_allocation[padded]: unpadded segments still share chunks after a defragmenting commit; padded ones get one chunk each at an unaligned count.
• test_padded_config_allocates_and_defragments (LBNHC, BLNHC, mixed strides): views, committed views and connector placements agree on a padded reservation.
• test_extensible_extend_asks_connector_for_its_memory_pool: a registering connector gets defragment=True.

Details

Root cause. A connector that registers each committed segment for RDMA needs that segment backed by one physical chunk. Segments were placed at multiples of capacity × block_stride, so they only started on commit granules when the block count was a granule multiple; the engine forced that by rounding the reserved capacity and the post-warmup count for every connector. CPU offloading only copies blocks and needs none of it.

Fix.

• When registers_kv_cache_memory(), each worker pads its own copy of the KV cache config in initialize_from_config (pad_kv_cache_segments): each layer region of a block-compact layer-outermost layout starts on that device's commit granule. This costs address space only; commits already map whole granules per segment (physical_bytes_for), so a defragmenting commit of any count backs each segment with exactly one chunk. Block-outermost layouts have one segment at offset 0 and are untouched. KV connectors with the extensible cache are already restricted to block-compact layouts.
• Padding is worker-local so the engine/scheduler configs keep unpadded placements: shrink_kv_cache_configs scales size by block count and scheduler-side connectors read size // num_blocks.
• ExtensibleKVCache takes the layout to derive one segment per layer region (gaps allowed) rather than inferring segments from a single stride, which can't tell padding from LHBNC head groups for small caches.
• Removed: engine-side align_extensible_kv_cache_capacity / granule_aligned_kv_cache_blocks, the kv_cache_commit_granule RPC, granule_aligned_blocks / granule_block_alignment, blocks_within(aligned=).
• The defragmenting commit is gated on registers_kv_cache_memory() ([Bugfix] Align extensible KV cache only for RDMA-registering connectors #65210).

Tradeoff/limitation. registers_kv_cache_memory() is a name-based allowlist, so out-of-tree connectors loaded via kv_connector_module_path that register device memory no longer get one-chunk segments by default (previously any connector did). Unpadded configs (no registering connector) behave as before.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Xgy3B2ybgXQggEmtMSLHgG

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[njhill]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99375 for commit b343e1db7cda.

ℹ️ This PR is 16 commits behind upstream main. Merged upstream main into this branch as b343e1db7cda before starting CI. Pull the branch before pushing more changes.