[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py by andakai · Pull Request #61183 · vllm-project/vllm

andakai · 2026-10-12T09:06:03Z

Overview

Move the KV cache grouping, sizing, and capacity-planning logic from vllm/v1/core/kv_cache_utils.py into a new module, vllm/v1/core/kv_cache_planning.py. This is a pure code move with no behavior change.

Split out from #53558 so the large move can be reviewed separately from the pluggable KVCacheConfigBuilder change.

Claims

kv_cache_utils.py goes from about 2.8k to about 1.2k lines. It keeps the block, hash, and free-queue utilities, plus the helpers that consume a finished KVCacheConfig: get_max_concurrency_for_kv_cache_config, generate_scheduler_kv_cache_config, get_kv_cache_capacity, and update_kv_cache_capacity.
kv_cache_planning.py holds the planning path: get_kv_cache_configs, get_kv_cache_groups, get_kv_cache_config_from_groups, page-size unification, packed/hybrid grouping, memory and max-model-len estimation, auto-fit, and TP-replica helpers (38 functions).
Function names and bodies are unchanged. No compatibility re-exports are left in kv_cache_utils.py; all in-tree callers are updated.

Validation

Every moved function is byte-identical to origin/main. This was checked by comparing each top-level definition with the AST. Reviewers can confirm with:
```
git diff --color-moved=dimmed-zebra origin/main...HEAD -- vllm/v1/core/
```
pre-commit passes (ruff, mypy, typos, and the test-tethering checks).

Ran the affected unit tests on CPU, on both this branch and origin/main. Both give the same result: 294 passed, 19 skipped.

pytest tests/v1/core/test_kv_cache_utils.py \
  tests/v1/core/test_contiguous_kv_packing.py \
  tests/v1/engine/test_init_error_messaging.py \
  tests/v1/kv_connector/unit/offloading_connector/test_config.py \
  tests/v1/simple_kv_offload/test_worker.py \
  tests/v1/kv_connector/unit/test_hidden_states_connector.py \
  tests/v1/attention/test_sparse_mla_kv_cache_layout.py \
  tests/v1/worker/test_dsv4_packed_zeroer_geometry.py

tests/models/test_initialization.py, tests/models/test_dspark_mla.py, and tests/v1/worker/test_gpu_model_runner.py import and collect cleanly. They need a GPU, so they are left to CI.

Details

Besides the move itself, these updates keep behavior and CI coverage the same:

Imports: updated in v1/engine/core.py, v1/hisparse/layout.py, v1/worker/gpu_model_runner.py, v1/worker/gpu/cudagraph_utils.py, and the affected tests.
Monkeypatch targets: the get_hisparse_host_pool_bytes patches and the max_memory_usage_bytes patch string now point at kv_cache_planning. Patching the old module would silently stop taking effect.
Subprocess test: the import embedded in the script string of test_is_kv_cache_spec_uniform now uses the new module.
Buildkite: wherever .buildkite/test-amd.yaml and .buildkite/test_areas/lm_eval.yaml list vllm/v1/core/kv_cache_utils.py as a source dependency, vllm/v1/core/kv_cache_planning.py is now listed too. Changes to the moved code still trigger the same jobs.

Out-of-tree code that imports planning functions from vllm.v1.core.kv_cache_utils needs to switch to vllm.v1.core.kv_cache_planning.

Follow-up: #53558 can rebase on top of this, so its diff only contains the builder changes.

Pull Request Checklist

I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).
AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

Split the KV cache grouping, sizing, and capacity-planning helpers out of vllm/v1/core/kv_cache_utils.py into a new vllm/v1/core/kv_cache_planning.py. kv_cache_utils.py keeps the block / hashing / free-queue utilities plus the capacity helpers that consume a finished KVCacheConfig (get_max_concurrency_for_kv_cache_config, generate_scheduler_kv_cache_config, get_kv_cache_capacity, update_kv_cache_capacity). This is a pure code move: function names and bodies are unchanged. Callers, tests (including monkeypatch targets), and Buildkite source dependencies are updated to the new module. Split out from vllm-project#53558 so the move can be reviewed separately from the pluggable KVCacheConfigBuilder change. Signed-off-by: Dakai An <dakaian108@gmail.com>

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

wangxiyuan

The change is very clean

ivanium · 2026-10-13T01:28:43Z

/ci run

github-actions · 2026-10-13T01:29:03Z

✅ Triggered Buildkite CI #94346 for commit 88e98bd2b937.

…bytes-profiling Both conflicts come from vllm-project#61183, which moved the KV cache planning helpers from kv_cache_utils to kv_cache_planning. The V1 and V2 inline minimal-config blocks that this branch replaces with build_minimal_kv_cache_config() picked up the new import upstream, so the resolution keeps the helper call. The helper and its test now import and patch kv_cache_planning, since kv_cache_utils no longer provides get_kv_cache_groups or get_kv_cache_config_from_groups. Signed-off-by: lesj0610 <lesj0610@gmail.com>

Resolve tests/v1/core/test_kv_cache_utils.py: keep the new BlockHashListWithBlockSize slice test; take upstream's removal of the null-block planning tests, which vllm-project#61183 moved to test_kv_cache_planning.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fangchen Li <fangchen.li@outlook.com>

…_planning Upstream vllm-project#61183 moved the KV cache planning logic from vllm/v1/core/kv_cache_utils.py to vllm/v1/core/kv_cache_planning.py, so the survivor KV-budget test's import of _pool_bytes_per_block, check_enough_kv_cache_memory and get_kv_cache_groups failed with ImportError. The three functions moved unchanged; only the import changes. No other import in the PR's files names a symbol the tree lacks. Assisted-by: Claude <noreply@anthropic.com> Signed-off-by: MJ <121681560+mjbrn@users.noreply.github.com>

Integrate 510 upstream commits (upstream/main @ ed3050a). Resolved 4 conflicted files: - v1/core/kv_cache_utils.py: upstream moved the KV planning logic to kv_cache_planning.py (vllm-project#61183), so this file takes upstream wholesale and the radiance KV-group-size logic (patch_kv_group_size.py) is re-applied to the new home: _radiance_pick_group_size + vllm_config plumbing in _get_kv_cache_groups_uniform_page_size and its get_kv_cache_groups call site. - tests/kernels/quantization/test_rdna3_moe_w4a16.py: keep upstream's new _make_ct_weights helper + compressed-tensors-layout test and gfx1100_only marker, while keeping the fork's rdna_only/wmma_only markers and gfx12 WMMA tests. - model_executor/layers/utils.py: keep upstream's gfx11 L2 cache-aliasing pad helpers and batch-invariant path, plus the radiance skinny-GEMM path. - v1/worker/gpu/attn_utils.py: adopt upstream's attn_group.build_metadata call (kernel block-table handling) while keeping the radiance _radiance_shared GDN shared-build injection. NOT built/tested on ROCm yet.

Upstream vllm-project#61183 moved _get_kv_cache_groups_uniform_page_size out of kv_cache_utils.py into the new kv_cache_planning.py, so the overlay patch's anchor no longer resolved (apply() is fatal on a missing anchor). Retarget the patch at the new file; it then idempotently no-ops on this source build and still applies against a pristine upstream install.

…vllm-project#61183) Signed-off-by: Dakai An <dakaian108@gmail.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>

…lanning get_kv_cache_groups moved from kv_cache_utils.py to kv_cache_planning.py upstream (vllm-project#61183). Re-apply the per-layer split there and move the regression test into tests/v1/core/test_kv_cache_planning.py, adapting it to the new _pool_bytes_per_block / _max_memory_usage_bytes_from_groups signatures. Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com>

…vllm-project#61183) Signed-off-by: Dakai An <dakaian108@gmail.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

… KV cache planning helpers from kv_cache_planning Root cause: vllm#61183 moved estimate_max_model_len and get_kv_cache_configs from kv_cache_utils to kv_cache_planning Upstream: vllm-project/vllm#61183 Fix: test_hpu_model_runner imports estimate_max_model_len/get_kv_cache_configs from vllm.v1.core.kv_cache_planning Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>

… KV cache, Accept FLy kwargs, Build NIXL 1.5.0 (+1 more) (#1900) This PR consolidates 4 hourly-CI fixes against vllm@`852989f580820849113d67d13fd8053e279d0444`. ## Bug 1: Import KV cache planning helpers from kv_cache_planning - **State machine id**: import_error_estimate_max_model_len_kv_cache_utils - **Commit**: 20b5875 ### Root cause vllm#61183 moved estimate_max_model_len and get_kv_cache_configs from kv_cache_utils to kv_cache_planning ### Culprit Probable culprit: [PR #61183](vllm-project/vllm#61183) - pinned by symbol archaeology, bisect not run. Candidate range: [compare](vllm-project/vllm@8d7b93f...852989f). ### Fix test_hpu_model_runner imports estimate_max_model_len/get_kv_cache_configs from vllm.v1.core.kv_cache_planning ## Bug 2: Accept FLy kwargs in the HPU rejection_sample override - **State machine id**: spec_decode_parallel_device_acquire_failed - **Commit**: 6b6326f ### Root cause vllm#53987 (FLy) passes fly_window_size/fly_entropy_* to rejection_sample; HPU override lacks them -> TypeError in every spec-decode run ### Culprit Probable culprit: vllm-project/vllm@c1dc2e9 - pinned by symbol archaeology, bisect not run. Candidate range: [compare](vllm-project/vllm@8d7b93f...852989f). ### Fix add fly_window_size/fly_entropy_threshold/fly_entropy_top_k for signature parity; assert FLy unused on HPU ## Bug 3: Build NIXL 1.5.0 in install_nixl.py - **State machine id**: nixl_min_version_1_5_0_required_runtime_error - **Commit**: c25d022 ### Root cause vllm#61032 prepares KV transfer descriptors as NIXL 1.5 strided lists and refuses to start below NIXL 1.5.0, but install_nixl.py builds 0.7.0 ### Culprit Probable culprit: [PR #61032](vllm-project/vllm#61032) - pinned by symbol archaeology, bisect not run. Candidate range: [compare](vllm-project/vllm@8d7b93f...852989f). ### Fix build tag v1.5.0 (bindings now ship as nixl_cu12) and install it with the matching nixl meta package; use system liburing and exclude libcore/libpull so auditwheel can repair the wheel; reuse only a repaired cached wheel and rebuild over an older installed NIXL ## Bug 4: Disable the NIXL libfabric PD e2e test until an OFI NIXL 1.5 exists - **State machine id**: e2e_run_pd_disaggregate_nixl_libfabric_test - **Commit**: 5f3bda5 ### Root cause vllm#61032 requires NIXL >= 1.5.0, but the libfabric test builds the intel-staging/nixl v0.6.0_OFI fork, the newest branch with Gaudi libfabric support ### Culprit Probable culprit: [PR #61032](vllm-project/vllm#61032) - pinned by symbol archaeology, bisect not run. Candidate range: [compare](vllm-project/vllm@8d7b93f...852989f). ### Fix comment out run_pd_disaggregate_nixl_libfabric_test so CI no longer discovers it; re-enable once the fork has an OFI branch >= 1.5.0 --------- Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>

### What this PR does / why we need it? Port `vllm-ascend` to vLLM main `c0b6f74` (bumped from `a98247a`) while keeping the `v0.31.0` release lane green. The upstream window moved the KV-cache planning helpers into a new module, made EAGLE draft groups explicit, typed DeepSeek-V4 configs through upstream Transformers, removed Medusa/MLP speculator support, and changed several spec-decode/KV-cache/EPLB signatures. Every gate below is either a `vllm_version_is("0.31.0")` split or an attribute/argument probe where two main dev builds share one version string. | Files | Upstream vLLM change (why it breaks vllm-ascend) | vllm-ascend adaptation | |-------|--------------------------------------------------|------------------------| | `.github/vllm-main-verified.commit` | — (main-lane pin only) | Verified main commit bumped `a98247a` -> `c0b6f74` | | `tests/e2e/pull_request/four_card/test_minimax_m3.py` | The main-lane pin re-JITs the static kernels, so the first generate on a cold cache shifts the near-uniform dummy logprobs (~3e-3) enough to flip a greedy token | On main run a throwaway `generate_greedy_logprobs` before the baseline/replay pair so both measure the warmed steady state and share the cached prefix; v0.31.0 keeps the single-shot comparison | | `tests/ut/core/test_circular_buffer.py` | [vllm#61126](vllm-project/vllm#61126) (cascade-attention removal) moved the prefix-block query up to `KVCacheManager` and dropped it from single-type managers | On main assert only the manager's own `get_num_skipped_tokens(65536)`; keep the combined `get_num_common_prefix_blocks(...)` assert on v0.31.0 | | `tests/ut/core/test_kv_cache_interface.py` | [vllm#61183](vllm-project/vllm#61183) moved `is_kv_cache_spec_uniform` into `vllm.v1.core.kv_cache_planning` | Version-aware import (`kv_cache_utils` on v0.31.0 / `kv_cache_planning` on main) | | `tests/ut/distributed/ascend_store/_mock_deps.py` | [vllm#61183](vllm-project/vllm#61183) added the `vllm.v1.core.kv_cache_planning` module | Add it to the mocked-vLLM module list | | `tests/ut/distributed/ascend_store/test_request_lifecycle.py` | [vllm#58668](vllm-project/vllm#58668) trims `num_computed_tokens` to the tokens actually kept when a request stops mid token-list | On main subtract `(accepted + 1) - <newly appended output tokens>` from the expected computed count; keep the old expectation on v0.31.0 | | `tests/ut/distributed/kv_transfer/kv_p2p/test_mooncake_connector.py` | [vllm#61183](vllm-project/vllm#61183) moved `get_kv_cache_config_from_groups` / `is_kv_cache_spec_uniform` | Version-aware import | | `tests/ut/models/test_deepseek_v41_cache.py` | [vllm#61183](vllm-project/vllm#61183) relocated the planning helpers; [vllm#53558](vllm-project/vllm#53558) gave `_max_memory_usage_bytes_from_groups` a leading `builder` argument | Import from the live planning module and probe the upstream signature to call the 2-arg (v0.31.0) vs 3-arg (main) form, feeding an Ascend `_get_pool_bytes_per_block` builder | | `tests/ut/ops/test_gdn_attn_builder.py` | [vllm#58737](vllm-project/vllm#58737) derives `num_reqs` from the block table and asserts its row count equals `seq_lens`; Ascend's `unpadded()` keeps the graph-padded rows | New UT: a padded 4-row block table with 2 real requests must still yield a 2-row `cache_indices` | | `tests/ut/patch/platform/test_packed_kv_cache_compat.py` | [vllm#61183](vllm-project/vllm#61183) relocated the patched module; [vllm#61455](vllm-project/vllm#61455) removed `_annotate_eagle_groups` from main | Patch `cache_patch._kv_cache_planning`; new UT asserts the packed hook tolerates an upstream module without the EAGLE-annotation API | | `tests/ut/patch/platform/test_patch_eplb.py` | [vllm#57549](vllm-project/vllm#57549) emits the async EPLB cycle-committed log from the async worker wrapper on main | Expect the workspace-move log only on v0.31.0 | | `tests/ut/patch/platform/test_patch_kv_cache_coordinator.py` | [vllm#61183](vllm-project/vllm#61183) relocated the patch entry point | Assert against `kcu_patch._kv_cache_planning` instead of `vllm.v1.core.kv_cache_utils` | | `tests/ut/patch/platform/test_patch_pp_mtp.py` | [vllm#56322](vllm-project/vllm#56322) makes `ModelConfig.verify_with_parallel_config` read `parallel_config.enable_hc_sp` | Add `enable_hc_sp=False` to the constructed parallel config (ignored on v0.31.0) | | `tests/ut/patch/platform/test_prefix_cache_cp_patches.py` | [vllm#60043](vllm-project/vllm#60043) removed the `mamba_cache_mode="all"` leftover and requires `"align"` with prefix caching; [vllm#53558](vllm-project/vllm#53558) added the `builder` argument | Version-select the mamba cache mode; probe the upstream admission signature for the `builder` argument | | `tests/ut/spec_decode/test_mla_dspark.py` | [vllm#60692](vllm-project/vllm#60692) renamed the capture hook `precompute_context_kv(num_reqs)` -> `prepare_forward(num_reqs, attn_metadata)` | Version-split the expected capture args/kwargs | | `tests/ut/test_compressed_prefix_cache.py` | [vllm#61183](vllm-project/vllm#61183) moved `is_kv_cache_spec_uniform` | Version-aware import | | `tests/ut/worker/test_attn_utils_v2.py` | [vllm#61455](vllm-project/vllm#61455) annotates EAGLE groups from `spec.is_draft`; [vllm#60766](vllm-project/vllm#60766) resolves pin memory lazily via `is_pin_memory_available()` | New UT that draft-layer specs get tagged; on main monkeypatch `is_pin_memory_available` to `False` | | `tests/ut/worker/test_model_runner_v2.py` | [vllm#56716](vllm-project/vllm#56716) suppresses EPLB even for `skip_eplb` dummy runs; [vllm#60946](vllm-project/vllm#60946) renamed the PCP staging hook | Add a `runner.eplb` stub; call the version-appropriate staging method | | `tests/ut/worker/test_mtp_pcp_speculator_v2.py` | [vllm#60885](vllm-project/vllm#60885) rejects a mismatched explicit float16/bfloat16 KV-cache dtype | Build the model config with `bfloat16` instead of `float32` | | `tests/ut/worker/test_pcp_manager_v2.py` | [vllm#60946](vllm-project/vllm#60946) renamed `prepare_inputs_to_capture` -> `prepare_inputs_for_capture_or_profile` | Patch and call the live method name per lane | | `tests/ut/worker/test_v2_dummy_padding.py` | [vllm#61767](vllm-project/vllm#61767) collects DP votes through `get_dp_group().cpu_all_reduce_()` instead of `dist.all_reduce(..., group=cpu_group)` | Version-split the mocked collective | | `tests/ut/worker/v2/test_eplb_controller.py` | [vllm#57549](vllm-project/vllm#57549) gives `add_model` an optional `model_name` and names the draft entry | Expect the 3-arg call (`model_name="draft (draft)"`) on main, 2-arg on v0.31.0 | | `vllm_ascend/_310p/worker_310p.py` | [vllm#61183](vllm-project/vllm#61183) moved `get_kv_cache_groups` | Version-aware import | | `vllm_ascend/distributed/ec_transfer/ec_memcache_connector.py` | [vllm#57904](vllm-project/vllm#57904) dropped `_get_encoder_cache_hidden_dim` from `cpu.common` (encoder widths are measured at startup) | On main size the receive buffer as `out_hidden_size * (1 + len(deepstack_visual_indexes))` (Qwen3-VL DeepStack) else `get_inputs_embeds_size()`; keep the helper on v0.31.0 | | `vllm_ascend/models/deepseek_v4/compressor.py` | [vllm#61392](vllm-project/vllm#61392) switched `DeepseekV4Config` to the upstream Transformers class | Import from `transformers` on main, from `vllm.transformers_utils.configs.deepseek_v4` on v0.31.0 | | `vllm_ascend/models/deepseek_v4/indexer.py` | Same [vllm#61392](vllm-project/vllm#61392) | Same version-aware import | | `vllm_ascend/models/deepseek_v4/mm_preprocess.py` | Same [vllm#61392](vllm-project/vllm#61392) | Same version-aware import | | `vllm_ascend/models/deepseek_v4/model.py` | [vllm#61392](vllm-project/vllm#61392) moves the config to Transformers, folds `compress_ratios`/`num_hash_layers` into `layer_types`/`mlp_layer_types`, and keys YaRN under the compressor (`compress`) group | Restore the legacy fields on main from the new lists, read the `compress`-scoped rope params with safe defaults, and keep the legacy path on v0.31.0 | | `vllm_ascend/models/deepseek_v41/cache_config.py` | [vllm#61183](vllm-project/vllm#61183) moved `may_override_num_blocks` | Version-aware import | | `vllm_ascend/models/glm5next/cache_config.py` | [vllm#61183](vllm-project/vllm#61183) moved `create_kv_cache_group_specs` / `may_override_num_blocks` | Version-aware import | | `vllm_ascend/ops/gdn_attn_builder.py` | [vllm#58737](vllm-project/vllm#58737) derives `num_reqs` from the block table and asserts row parity with `seq_lens` | On main trim the padded block table to `seq_lens.shape[0]` before `mamba_get_block_table_tensor` when the sizes differ | | `vllm_ascend/patch/__init__.py` | [vllm#61183](vllm-project/vllm#61183) moved `get_kv_cache_config_from_groups` to `kv_cache_planning` | Update the patch-registry doc | | `vllm_ascend/patch/platform/__init__.py` | [vllm#61183](vllm-project/vllm#61183) moved the KV-cache planning/patcher module | Update the patch-registry doc | | `vllm_ascend/patch/platform/patch_eplb.py` | [vllm#57549](vllm-project/vllm#57549) moves the async EPLB cycle-committed log onto the async worker path on main | Report completion from the wrapped async worker (rank 0) on main; keep the workspace-move log on v0.31.0 | | `vllm_ascend/patch/platform/patch_kv_cache_utils.py` | [vllm#61183](vllm-project/vllm#61183) moved the planning helpers; [vllm#53558](vllm-project/vllm#53558) added the `builder` argument; [vllm#61455](vllm-project/vllm#61455) dropped `_annotate_eagle_groups` | Resolve the live planning module by version, patch its symbols, gate the EAGLE annotation on attribute presence, and dispatch the admission helper by argument count | | `vllm_ascend/quantization/methods/kv_cache/turboquant/cache.py` | [vllm#61183](vllm-project/vllm#61183) moved `may_override_num_blocks` | Version-aware import | | `vllm_ascend/spec_decode/__init__.py` | [vllm#61701](vllm-project/vllm#61701) removed Medusa/MLP speculator support | Import `AscendMedusaProposer` only on v0.31.0; raise a clear error for `method="medusa"` on main | | `vllm_ascend/spec_decode/medusa_proposer.py` | [vllm#61701](vllm-project/vllm#61701) removed `vllm.v1.spec_decode.medusa` | Add `# type: ignore[import-not-found]`; the module only loads on the lane that still ships it | | `vllm_ascend/worker/v1/model_runner.py` | Same [vllm#61701](vllm-project/vllm#61701) | Gate the Medusa import and the `isinstance(self.drafter, AscendMedusaProposer)` branch on v0.31.0 | | `vllm_ascend/worker/v2/attn_utils.py` | [vllm#61455](vllm-project/vllm#61455) annotates EAGLE groups from `spec.is_draft` | On main tag the spec with `is_draft` from `attn_module.is_draft_layer` | | `vllm_ascend/worker/v2/pcp_manager.py` | [vllm#60946](vllm-project/vllm#60946) added the `moe_gathers_tokens` argument and renamed `prepare_inputs_to_capture` -> `prepare_inputs_for_capture_or_profile` | Forward the new argument; version-split the staging hook | | `vllm_ascend/worker/v2/sample/apply_top_k_top_p.py` | [vllm#61996](vllm-project/vllm#61996) added `has_topp_only_rows` to the split top-p pipeline | Accept and ignore the extra argument on main (the PyTorch fallback handles top-p-only rows) | | `vllm_ascend/worker/v2/spec_decode/dflash/aclgraph.py` | [vllm#60692](vllm-project/vllm#60692) renamed `precompute_context_kv` -> `prepare_forward` | Pass the capture hook the installed vLLM expects | | `vllm_ascend/worker/v2/spec_decode/lmhead_tp_utils.py` | [vllm#59554](vllm-project/vllm#59554) added `num_steps` to `Speculator.sample_draft` | Version-split the override signature and forward `num_steps` on main | | `vllm_ascend/worker/v2/spec_decode/rejection_sampler_utils.py` | [vllm#61451](vllm-project/vllm#61451) renamed the logits-stats kernel argument `num_speculative_steps` -> `num_logits` and moved bonus-token detection into `_is_bonus_logit` | Pass `num_logits` on main; keep `num_speculative_steps` on v0.31.0 | | `vllm_ascend/worker/worker.py` | [vllm#61183](vllm-project/vllm#61183) moved `get_kv_cache_groups`/`unify_hybrid_kv_cache_specs`; [vllm#57516](vllm-project/vllm#57516) reclaims cached memory via `torch.accelerator.empty_cache()` inside `load_model` | Version-aware imports; on main neutralize `torch.npu.empty_cache` while loading weights under the sleep-mode pluggable memory pool | ### Does this PR introduce _any_ user-facing change? No. This is an internal main2main compatibility update; on the `v0.31.0` release lane behavior is unchanged, and on main the only behavioral note is that Medusa speculative decoding now raises a clear error (upstream removed it in vllm#61701). ### How was this patch tested? - Unit tests: the changed UT files above (KV-cache planning/coordinator, EPLB, PCP/spec-decode, DeepSeek-V4/V4.1 cache, mamba align, `worker/worker.py`). - E2E: `tests/e2e/pull_request/four_card/test_minimax_m3.py`. - Run the main lane against the pinned `c0b6f74` and the `v0.31.0` lane via `vllm_version_is`. --- ### Checklist #### Testing - [ ] **Boundary / edge-case coverage**: I reviewed whether this change involves boundary conditions (e.g. empty input, single/zero element, min/max values, off-by-one, sequence-length limits, integer overflow). Where applicable, I added unit tests covering these edge cases in `tests/ut/` or `tests/e2e/`. - [ ] **Regression test**: If this PR fixes a bug, I added a regression test that would fail without the fix. - [ ] **Error / failure path coverage**: I covered abnormal paths (exceptions, timeouts, out-of-range, division by zero) in addition to the happy path. - [ ] **Meaningful & reliable tests**: New tests assert real behavior and are not written merely to satisfy CI. - [ ] **Performance impact**: If this PR touches a performance-sensitive path, I attached benchmark results and added performance regression tests. - vLLM main: vllm-project/vllm@a98247a Signed-off-by: zhangxinyuehfad <starmoon_zhang@163.com>

andakai requested review from AndreasKaratzas, ApostaC, DarkLight1337, Harry-Chen, LucasWilkinson, MatthewBonanni, WoosukKwon, alexm-redhat, heheda12345, ivanium, khluu, njhill, orozery, robertgshaw2-redhat, wzhao18, yewentao256 and ywang96 as code owners October 12, 2026 09:06

claude Bot reviewed Oct 12, 2026

View reviewed changes

Merge branch 'main' into refactor/kv-cache-planning-split

64dbee8

mergify Bot added ci/build nvidia dflash labels Oct 12, 2026

github-project-automation Bot added this to NVIDIA Oct 12, 2026

mergify Bot added the mrv2 Model Runner V2 specific label Oct 12, 2026

github-project-automation Bot added this to Sprint - DFlash Oct 12, 2026

github-project-automation Bot moved this to Backlog in Sprint - DFlash Oct 12, 2026

mergify Bot added kv-connector kv-cache-manager labels Oct 12, 2026

wangxiyuan approved these changes Oct 12, 2026

View reviewed changes

ivanium enabled auto-merge (squash) October 13, 2026 01:28

github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 13, 2026

mergify Bot removed the needs-rebase label Oct 13, 2026

ivanium merged commit df6c7ad into vllm-project:main Oct 13, 2026
218 of 219 checks passed

github-project-automation Bot moved this from Ready to Done in NVIDIA Oct 13, 2026

github-project-automation Bot moved this from Backlog to Done in Sprint - DFlash Oct 13, 2026

ivanium mentioned this pull request Oct 13, 2026

[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning #53558

Merged

sierraisland mentioned this pull request Oct 13, 2026

[Fix] Import get_kv_cache_groups from kv_cache_planning and update vLLM LKG vllm-project/tpu-inference#3813

Merged

3 tasks done

stu-cao mentioned this pull request Oct 14, 2026

[Bugfix][V1] Retain the EAGLE hit blocks below a sliding window across a KV connector import #61011

Open

gokay-ai mentioned this pull request Oct 19, 2026

[Bugfix] Bill hidden-state extraction at its own KV cache group width #56822

Open

pawel-olejniczak mentioned this pull request Oct 19, 2026

[FIX_FOR_VLLM_CUSTOM=852989f580820849113d67d13fd8053e279d0444] Import KV cache, Accept FLy kwargs, Build NIXL 1.5.0 (+1 more) vllm-project/vllm-gaudi#1900

Merged

zhangxinyuehfad mentioned this pull request Oct 21, 2026

[Misc] vllm main2main 1017 vllm-project/vllm-ascend#19302

Merged

5 tasks

basemam mentioned this pull request Oct 23, 2026

[Core][Bugfix] Drop bounded_replay when promoting SlidingWindowMLASpec #63411

Open

6 tasks done

Navigation Menu

Repository navigation

[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py - #61183

[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py#61183
ivanium merged 7 commits into
vllm-project:mainfrom
andakai:refactor/kv-cache-planning-split

andakai commented Oct 12, 2026

Uh oh!

claude Bot left a comment

Uh oh!

wangxiyuan left a comment

Uh oh!

ivanium commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

andakai commented Oct 12, 2026

Overview

Claims

Validation

Details

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

wangxiyuan left a comment

Choose a reason for hiding this comment

Uh oh!

ivanium commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants