[MRV2] Buffer util simplifications by njhill · Pull Request #56888 · vllm-project/vllm

njhill · 2026-09-14T20:11:23Z

Unify similar async_copy_to_gpu and async_tensor_h2d methods (remove the former)
Adjust CpuGpuBuffer for staged+pinned transfers
Ensure pinned buffers are used for h2d copy in a few more places

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

njhill · 2026-09-14T20:11:34Z

/ci run

github-actions · 2026-09-14T20:11:55Z

✅ Triggered Buildkite CI #88906 for commit b9898de6ffd3.

njhill · 2026-09-14T20:15:24Z

/ci run

github-actions · 2026-09-14T20:15:42Z

✅ Triggered Buildkite CI #88908 for commit 0f22e8533908.

njhill · 2026-09-14T20:18:23Z

/ci run

github-actions · 2026-09-14T20:18:41Z

✅ Triggered Buildkite CI #88909 for commit 6537cf1d497e.

njhill · 2026-09-14T21:21:57Z

/ci run

github-actions · 2026-09-14T21:23:00Z

✅ Triggered Buildkite CI #88923 for commit 54fc37235daa.

njhill · 2026-09-14T22:07:17Z

/ci run

github-actions · 2026-09-14T22:07:36Z

✅ Triggered Buildkite CI #88939 for commit c8b62299d597.

njhill · 2026-09-14T23:20:57Z

/ci run

github-actions · 2026-09-14T23:21:15Z

✅ Triggered Buildkite CI #88950 for commit 6b9d3437e7d2.

mergify · 2026-09-15T04:43:23Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @njhill.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

njhill · 2026-09-15T15:50:36Z

/ci run

github-actions · 2026-09-15T15:51:54Z

✅ Triggered Buildkite CI #89122 for commit 7ef5655f97e1.

- Unify similar async_copy_to_gpu and async_tensor_h2d methods - Adjust CpuGpuBuffer for staged+pinned transfers - Ensure pinned buffers are used for h2d copy in a few more places Signed-off-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

njhill · 2026-09-15T18:30:44Z

/ci retry

github-actions · 2026-09-15T18:32:01Z

✅ Triggered Buildkite CI #89150 for commit d8a233c8d026, running 4 failed step(s) from Buildkite CI #89122.

Signed-off-by: Nick Hill <nickhill123@gmail.com>

### What this PR does / why we need it? #### Change Summary The PR advances the main2main lane to vLLM v0.30.0 (commit `ced6857afa0ea7b2e3f0846a62e1394e90f15607`), adapting vllm-ascend to every upstream change in the `84030bbe` -> `4991f97` -> `ced6857` range. Because both CI lanes now install vLLM v0.30.0, all `vllm_version_is("0.29.0")` forks are permanently false and are collapsed to the main behavior. | Files | Upstream vLLM change | vllm-ascend adaptation | |-------|---------------------|------------------------| | `.github/vllm-main-verified.commit` | — | Updated verified main commit hash `84030bbe` -> `4991f97` -> `ced6857` (v0.30.0 tag commit) | | `.github/vllm-release-tag.commit` | — | Bumped the release boundary to `v0.30.0` | | `.github/workflows/pr_test.yaml` | — | Added a dual-version cpu-ut matrix (`vllm_versions`) for the main2main lane; temporarily commented out pre-commit/mypy and set `cpu-ut` to `if: false`; dropped the `needs.cpu-ut` requirement from the ready gate; added a main2main-specific cpu-ut failure hint | | `Dockerfile` | — | `VLLM_TAG` default -> `v0.30.0` | | `Dockerfile.310p` | — | `VLLM_TAG` default -> `v0.30.0` | | `Dockerfile.310p.openEuler` | — | `VLLM_TAG` default -> `v0.30.0` | | `Dockerfile.a3` | — | `VLLM_TAG` default -> `v0.30.0` | | `Dockerfile.a3.openEuler` | — | `VLLM_TAG` default -> `v0.30.0` | | `Dockerfile.a5` | — | `VLLM_TAG` default -> `v0.30.0` | | `Dockerfile.a5.openEuler` | — | `VLLM_TAG` default -> `v0.30.0` | | `Dockerfile.openEuler` | — | `VLLM_TAG` default -> `v0.30.0` | | `README.md` | — | CI notes updated to v0.30.0 | | `README.zh.md` | — | CI notes updated to v0.30.0 | | `tests/e2e/nightly/single_node/ops/singlecard_ops/triton/test_num_nans.py` | — | `VLLM_VERSION` default 0.29.0 -> 0.30.0 | | `tests/ut/_310p/test_model_runner_v2_310p.py` | v0.30.0 boundary | Dropped the `vllm_version_is` gate test; `_needs_kv_cache_zeroing_310p` always uses `spec_config.use_eagle_block_drop()` | | `tests/ut/core/test_dyntra_lb_scheduler.py` | — | Removed the 0.29 `KVConnectorBlockState.block_ids` assertions; assert the `req_ids` form only | | `tests/ut/core/test_scheduler_connector_block_state.py` | — | Removed the 0.29 `block_ids` snapshot branch | | `tests/ut/kv_offload/test_native_cpu_offload.py` | — | Block -> chunk API: assert `spec.num_chunks` | | `tests/ut/kv_offload/test_npu_offload_spec.py` | — | `num_blocks`/`kv_bytes_per_block` -> `num_chunks`/`kv_bytes_per_chunk` | | `tests/ut/models/test_deepseek_v41_registration.py` | [vllm#56741](vllm-project/vllm#56741) | Removed the V4.1 0.29 skip; registration is always active | | `tests/ut/patch/platform/test_patch_mamba_block_aligned_split.py` | — | Removed the 0.29 gate for `mamba_fine_grained_prefix_cache` | | `tests/ut/patch/platform/test_patch_speculative_config_dspark.py` | [vllm#55914](vllm-project/vllm#55914) / [vllm#56930](vllm-project/vllm#56930) | Removed the DeepSeek V4.1 0.29 `pytest.skip` | | `tests/ut/patch/platform/test_patch_use_v2_model_runner.py` | [vllm#54523](vllm-project/vllm#54523) | Deleted the 0.28.0 PCP+DP validator workaround tests; reworked the version-routing fixtures | | `tests/ut/patch/worker/test_patch_dspark_pp.py` | — | Parametrize `legacy: bool` instead of version strings | | `tests/ut/quantization/configs/test_modelslim_config.py` | [vllm#56741](vllm-project/vllm#56741) | Removed the V4.1 0.29 skip | | `tests/ut/spec_decode/test_dspark_proposer.py` | [vllm#56741](vllm-project/vllm#56741) | Removed the V4.1 0.29 skip marker | | `tests/ut/spec_decode/test_eagle_proposer.py` | — | `uses_xdrope_dim` -> `mrope_num_dims` | | `tests/ut/test_compressed_prefix_cache.py` | — | `replay_boundaries` now unconditional | | `tests/ut/worker/test_encoder_acl_graph.py` | — | `axis_keys=()` now unconditional | | `tests/ut/worker/test_model_runner_v1.py` | [vllm#56741](vllm-project/vllm#56741) | Removed the V4.1 0.29 skips and the `vllm_version_is` patches | | `tests/ut/worker/test_model_runner_v2.py` | v0.30.0 boundary | Version-routing fixture rework | | `tests/ut/worker/test_pcp_manager_v2.py` | [vllm#56107](vllm-project/vllm#56107) / [vllm#53867](vllm-project/vllm#53867) | Dropped the 0.29 `req_states` coverage / added `padded_num_reqs` coverage; removed the 0.28/0.29 branches | | `tests/ut/worker/v2/test_pp_utils.py` | — | Replaced the version-routing matrix with `use_legacy_spec_pp() is False` | | `vllm_ascend/_310p/model_runner_310p.py` | — | Removed the 0.29 xdrope-position branch | | `vllm_ascend/_310p/spec_decode/llm_base_proposer_310.py` | — | Removed the 0.29 xdrope `target_positions[0]` squeeze | | `vllm_ascend/_310p/worker/v2/model_runner.py` | [vllm#57270](vllm-project/vllm#57270) | Removed the 0.29 `max_seq_len_np` kwarg and the 0.29 eagle-block-drop branch | | `vllm_ascend/_310p/worker/v2/rope.py` | — | mrope `num_dims = model_config.mrope_num_dims` (dropped the 0.29 constant 3) | | `vllm_ascend/_310p/worker/v2/states.py` | [vllm#56908](vllm-project/vllm#56908) | `UvaBuffer.uva` property -> method; final v0.30.0 form | | `vllm_ascend/attention/mla_v1.py` | [vllm#56181](vllm-project/vllm#56181) | Draft TND_NTD layout forced via `_EXTRA_CTX.is_draft_model`; dropped the 0.29 gate | | `vllm_ascend/attention/utils.py` | [vllm#55353](vllm-project/vllm#55353) / [vllm#56157](vllm-project/vllm#56157) | Ascend-owned `_seq_lens_cpu`/`_num_computed_tokens_cpu`/`dcp_local_seq_lens_cpu` are now unconditional | | `vllm_ascend/batch_invariant.py` | — | `reduce_sum` accepts NumPy-style `axis` + `dtype`, rejects `dim`+`axis` together, forwards `dtype` to the native fallback | | `vllm_ascend/core/dyntra_lb_scheduler.py` | — | `KVConnectorBlockState` always uses `req_ids`+`resolve_block_ids` (dropped the 0.29 `block_ids` snapshot) | | `vllm_ascend/core/kv_cache_interface.py` | [vllm#53906](vllm-project/vllm#53906) | MLA `get_storage_block_size` override unconditional; dropped the 0.29 `storage_block_size` property | | `vllm_ascend/core/recompute_scheduler.py` | — | Dropped the xdrope kwarg and the 0.29 `block_ids` snapshot | | `vllm_ascend/core/scheduler_profiling_chunk.py` | — | Dropped the 0.29 `block_ids` snapshot | | `vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/npu.py` | — | Block -> chunk API (`num_chunks`, `kv_bytes_per_chunk`) unconditional | | `vllm_ascend/lora/punica_npu.py` | [vllm#53555](vllm-project/vllm#53555) + boundary | `add_lora_logits` per-adapter matmul fallback for heads smaller than the rank; final state drops the `apply_lora_full_linear` binding (both supported targets predate #53555) | | `vllm_ascend/models/__init__.py` | [vllm#56741](vllm-project/vllm#56741) | V4.1 registration (`DeepseekV41ForCausalLM`/`DSparkModel`) unconditional | | `vllm_ascend/models/deepseek_v41/engram/embedding.py` | [vllm#56741](vllm-project/vllm#56741) | `deepseek_v4_1` -> `deepseek_v41` import unconditional | | `vllm_ascend/models/deepseek_v41/engram/hash_state.py` | [vllm#56741](vllm-project/vllm#56741) | `deepseek_v4_1` -> `deepseek_v41` import unconditional | | `vllm_ascend/models/deepseek_v41/engram/parallel.py` | [vllm#56741](vllm-project/vllm#56741) | `deepseek_v4_1` -> `deepseek_v41` import unconditional | | `vllm_ascend/models/deepseek_v41/model.py` | [vllm#56741](vllm-project/vllm#56741) | `deepseek_v4_1` -> `deepseek_v41` import unconditional | | `vllm_ascend/models/deepseek_v41/vl_model.py` | [vllm#56741](vllm-project/vllm#56741) / [vllm#56554](vllm-project/vllm#56554) | `deepseek_v41` imports; drop `IMAGE_PAD_ID`/alignment-pad handling on main | | `vllm_ascend/ops/mla.py` | [vllm#56157](vllm-project/vllm#56157) | `MLAAttention.supports_pcp_dcp = True` set on the class, unconditional | | `vllm_ascend/ops/rotary_embedding.py` | [vllm#56446](vllm-project/vllm#56446) | YaRN mscale signature adaptation; v0.29 branch collapsed | | `vllm_ascend/patch/__init__.py` | [vllm#55914](vllm-project/vllm#55914) / [vllm#56930](vllm-project/vllm#56930) | Registry entry for the new draft-EP patch; dropped version gates | | `vllm_ascend/patch/platform/__init__.py` | [vllm#56741](vllm-project/vllm#56741 Engram | `patch_engram_config` imported unconditionally | | `vllm_ascend/patch/platform/patch_balance_schedule.py` | — | Dropped the 0.29 `block_ids` snapshot | | `vllm_ascend/patch/platform/patch_kv_cache_coordinator.py` | [vllm#54736](vllm-project/vllm#54736) + boundary | Accept/forward `allow_partial_hash_hits`; collapsed the 0.29 gate | | `vllm_ascend/patch/platform/patch_parallel_config.py` | [vllm#54523](vllm-project/vllm#54523) | Removed the 0.29 `_validate_parallel_config` PCP+DP workaround; keeps `use_sequence_parallel_moe` | | `vllm_ascend/patch/platform/patch_speculative_config.py` | [vllm#55914](vllm-project/vllm#55914) without [vllm#56930](vllm-project/vllm#56930) | Skip `_verify_with_expert_parallelism` for non-MoE draft (`runner_type=="draft"`) | | `vllm_ascend/patch/platform/patch_use_v2_model_runner.py` | [vllm#54523](vllm-project/vllm#54523) | Removed the 0.28.0 PCP+DP validation workaround | | `vllm_ascend/patch/worker/patch_bind_kv_cache.py` | [vllm#53781](vllm-project/vllm#53781) | Ascend `bind_kv_cache_to_layers` (assign the raw allocation); collapsed the 0.29 gate | | `vllm_ascend/patch/worker/patch_deepseek_v2.py` | [vllm#53781](vllm-project/vllm#53781) | Accept/ignore `index_group_builder`; `SparseMLAIndexGroupBuilder` import collapse | | `vllm_ascend/patch/worker/patch_mamba_utils.py` | [vllm#56898](vllm-project/vllm#56898) | `GPUInputBatch` import source version-gated, then collapsed to `gpu_input_batch.InputBatch` | | `vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py` | [vllm#53781](vllm-project/vllm#53781) | Register Ascend `bind_kv_cache_to_layers`; expose tuple element 0 for the device filter; collapsed the 0.29 gate | | `vllm_ascend/patch/worker/patch_v2/patch_dspark.py` | — | Legacy Spec+PP bypass comment (inactive on v0.30.0) | | `vllm_ascend/patch/worker/patch_v2/patch_spec_pp.py` | [vllm#56888](vllm-project/vllm#56888) | Alias `async_tensor_h2d as async_copy_to_gpu`; collapsed the 0.29 gate | | `vllm_ascend/patch/worker/patch_v2/patch_uva.py` | [vllm#56908](vllm-project/vllm#56908) | `uva` property vs method; final v0.30.0 method form | | `vllm_ascend/spec_decode/llm_base_proposer.py` | [vllm#56254](vllm-project/vllm#56254) + [vllm#55914](vllm-project/vllm#55914) / [vllm#56930](vllm-project/vllm#56930) + boundary | Gate the V4.1 DSpark import by `HAS_TRITON`; static `_draft_embed_accepts_mm` check instead of the runtime `embed_input_ids` probe; collapsed version gates | | `vllm_ascend/utils.py` | — | `vllm_version_is` docstring 0.29 -> 0.30; Kimi MLA custom-op registration unconditional | | `vllm_ascend/worker/model_runner_v1.py` | [vllm#56741](vllm-project/vllm#56741) | V4.1 dsa metadata/cache imports unconditional; removed xdrope position handling | | `vllm_ascend/worker/v2/aclgraph_utils.py` | [vllm#51700](vllm-project/vllm#51700) | `ModelAclGraphManager.__init__` accepts/forwards `ubatch_runner`; `UBatchRunner` import collapse | | `vllm_ascend/worker/v2/model_runner.py` | [vllm#56888](vllm-project/vllm#56888) / [vllm#53867](vllm-project/vllm#53867) / [vllm#51700](vllm-project/vllm#51700) / [vllm#57270](vllm-project/vllm#57270) + [vllm#56181](vllm-project/vllm#56181) + boundary | Async-copy alias; pass `BatchExecutionDescriptor` to `maybe_partition_pcp_batch`; `ubatch_runner`; `make_dummy(is_padding)`; keep the replicated PCP draft on the global batch; collapsed version gates. Also restores the `_check_oproj_tp_graph_step` guard and the PD decode-recompute `gather_batch_req_state` override accidentally removed by the v0.28.0-boundary cleanup (`e30adae80`) | | `vllm_ascend/worker/v2/pcp_manager.py` | [vllm#56888](vllm-project/vllm#56888) / [vllm#56107](vllm-project/vllm#56107) / [vllm#53867](vllm-project/vllm#53867) + [vllm#56181](vllm-project/vllm#56181) + boundary | Async alias; drop `req_states`; forward `padded_num_reqs`; `prepare_draft_prefill` no-op / `restore_for_sampling` skip; collapsed version gates | | `vllm_ascend/worker/v2/pp_utils.py` | — | `use_legacy_spec_pp()` returns `False` | | `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py` | [vllm#56181](vllm-project/vllm#56181) | Route through `_build_uniform_attn_metadata`/`_build_attn_metadata`; re-hook the Ascend rotary-positions injection onto the new methods; dropped the 0.29 gate | | `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py` | [vllm#56181](vllm-project/vllm#56181) | Same `BatchExecutionDescriptor` routing; dropped the 0.29 gate | | `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py` | [vllm#56181](vllm-project/vllm#56181) | Same routing; dropped the 0.29 gate | ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? - vLLM main: vllm-project/vllm@84030bb --------- Signed-off-by: hfadzxy <starmoon_zhang@163.com> Co-authored-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>

njhill requested review from MatthewBonanni, WoosukKwon, benchislett, luccafong, mgoin, pavanimajety, vadiklyutiy and yewentao256 as code owners September 14, 2026 20:11

claude Bot reviewed Sep 14, 2026

View reviewed changes

mergify Bot added nvidia speculative-decoding labels Sep 14, 2026

github-project-automation Bot added this to NVIDIA Sep 14, 2026

mergify Bot added the mrv2 Model Runner V2 specific label Sep 14, 2026

njhill force-pushed the mrv2-buf-simplify branch from b9898de to 0f22e85 Compare September 14, 2026 20:14

njhill force-pushed the mrv2-buf-simplify branch from 0f22e85 to 6537cf1 Compare September 14, 2026 20:18

njhill force-pushed the mrv2-buf-simplify branch from 6537cf1 to da8e1c7 Compare September 14, 2026 20:47

njhill requested review from 22quinn, LucasWilkinson, ZJY0516 and houseroad as code owners September 14, 2026 20:47

njhill force-pushed the mrv2-buf-simplify branch from da8e1c7 to 54fc372 Compare September 14, 2026 21:11

njhill force-pushed the mrv2-buf-simplify branch from 54fc372 to c8b6229 Compare September 14, 2026 22:05

njhill requested review from bigPYJ1151 and xuechendi as code owners September 14, 2026 22:05

mergify Bot added the cpu Related to CPU backends label Sep 14, 2026

njhill force-pushed the mrv2-buf-simplify branch from c8b6229 to 6b9d343 Compare September 14, 2026 23:19

njhill added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 14, 2026

WoosukKwon approved these changes Sep 15, 2026

View reviewed changes

github-project-automation Bot moved this to Ready in NVIDIA Sep 15, 2026

mergify Bot added the needs-rebase label Sep 15, 2026

njhill force-pushed the mrv2-buf-simplify branch from 6b9d343 to 7ef5655 Compare September 15, 2026 15:35

njhill enabled auto-merge (squash) September 15, 2026 15:52

njhill added 3 commits September 15, 2026 11:30

[MRV2] Buffer util simplifications

5d85e70

- Unify similar async_copy_to_gpu and async_tensor_h2d methods - Adjust CpuGpuBuffer for staged+pinned transfers - Ensure pinned buffers are used for h2d copy in a few more places Signed-off-by: Nick Hill <nickhill123@gmail.com>

fix cpu

445ae04

Signed-off-by: Nick Hill <nickhill123@gmail.com>

fix test

d8a233c

Signed-off-by: Nick Hill <nickhill123@gmail.com>

njhill force-pushed the mrv2-buf-simplify branch from 7ef5655 to d8a233c Compare September 15, 2026 18:30

njhill merged commit f2aad6a into vllm-project:main Sep 15, 2026
19 checks passed

github-project-automation Bot moved this from Ready to Done in NVIDIA Sep 15, 2026

njhill deleted the mrv2-buf-simplify branch September 15, 2026 20:58

keneoneth pushed a commit to keneoneth/vllm that referenced this pull request Sep 16, 2026

[MRV2] Buffer util simplifications (vllm-project#56888)

e3ffb7d

Signed-off-by: Nick Hill <nickhill123@gmail.com>

thegoldenflow mentioned this pull request Sep 18, 2026

[Bugfix][Attention] Size FlashInfer paged-KV buffers for local attention #50022

Open

zhangxinyuehfad mentioned this pull request Sep 23, 2026

[CI][Misc] main2main vllm v0.30.0 vllm-project/vllm-ascend#16527

Merged

Uh oh!

Conversation

njhill commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

njhill commented Sep 14, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

njhill commented Sep 14, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

njhill commented Sep 14, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

njhill commented Sep 14, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

njhill commented Sep 14, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

njhill commented Sep 14, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

mergify Bot commented Sep 15, 2026

Uh oh!

njhill commented Sep 15, 2026

Uh oh!

github-actions Bot commented Sep 15, 2026

Uh oh!

njhill commented Sep 15, 2026

Uh oh!

github-actions Bot commented Sep 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

njhill commented Sep 14, 2026 •

edited

Loading