[CI][Main2Main] Adapt vLLM through September 3 after #14872 - #15627
zhao-stack wants to merge 8 commits into
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request performs a compatibility update for the vLLM main CI lane, advancing the vLLM pin to 5d09eb2c. It addresses several breaking changes introduced in recent vLLM commits, specifically regarding KV cache grouping, slot mapping, and multimodal configuration access. The changes ensure that the Ascend-specific NPU model runners and KV cache management logic remain functional on both the v0.27.1 legacy lane and the current vLLM main branch. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Misc][Feature] Update compatibility with upstream vLLM changesSuggested PR Summary:
### What this PR does / why we need it?
This PR updates the vllm-ascend integration to maintain compatibility with recent upstream vLLM changes (including #51718, #53515, #53869, #53694, and #54282). It adapts the KV cache allocation, tensor layout planning, and speculator interfaces to handle renamed fields (such as `shared_by` to `layers` and `compress_ratio` to `tokens_per_state`) and the new standardized shared KV backing allocation.
Feedback on the current implementation:
- In `vllm_ascend/worker/model_runner_v1.py` and `vllm_ascend/worker/worker.py`, the `layout` object returned by `get_resolved_kv_cache_layout()` can be `None`. Accessing its attributes directly will raise an `AttributeError` at runtime.
- In `vllm_ascend/patch/worker/patch_mamba_utils.py`, `UniformTypeKVCacheSpecs` and `MambaSpec` are used in `isinstance` checks but are not imported, which will raise a `NameError` at runtime.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The changes are covered by updated and newly added unit tests across various modules.| layout = self.vllm_config.cache_config.get_resolved_kv_cache_layout() | ||
| tensor_sizes = {tensor.size for tensor in kv_cache_config.kv_cache_tensors} | ||
| regions: list[tuple[str, int, int]] = [] | ||
| if len(tensor_sizes) == 1 and layout.is_layer_compact and layout.is_block_compact: |
There was a problem hiding this comment.
The layout object returned by get_resolved_kv_cache_layout() can be None in certain configurations or tests. Accessing layout.is_layer_compact directly without a None check will raise an AttributeError at runtime. We should add a check to ensure layout is not None before accessing its attributes.
layout = self.vllm_config.cache_config.get_resolved_kv_cache_layout()\n tensor_sizes = {tensor.size for tensor in kv_cache_config.kv_cache_tensors}\n regions: list[tuple[str, int, int]] = []\n if layout is not None and len(tensor_sizes) == 1 and layout.is_layer_compact and layout.is_block_compact:| layout = self.vllm_config.cache_config.get_resolved_kv_cache_layout() | ||
| if ( | ||
| has_attention | ||
| and has_mamba | ||
| and layout.is_layer_compact | ||
| and layout.is_block_compact |
There was a problem hiding this comment.
The layout object returned by get_resolved_kv_cache_layout() can be None in certain configurations or tests. Accessing layout.is_layer_compact directly without a None check will raise an AttributeError at runtime. We should add a check to ensure layout is not None before accessing its attributes.
layout = self.vllm_config.cache_config.get_resolved_kv_cache_layout()\n if (\n has_attention\n and has_mamba\n and layout is not None\n and layout.is_layer_compact\n and layout.is_block_compact| mamba_spec = kv_cache_group.kv_cache_spec | ||
| if isinstance(mamba_spec, UniformTypeKVCacheSpecs): | ||
| mamba_spec = mamba_spec.kv_cache_specs[layer_name] | ||
| assert isinstance(mamba_spec, MambaSpec) |
There was a problem hiding this comment.
The classes UniformTypeKVCacheSpecs and MambaSpec are used in isinstance checks but they are not imported in this file. This will raise a NameError at runtime. We should import them locally to ensure they are available.
from vllm.v1.kv_cache_interface import MambaSpec, UniformTypeKVCacheSpecs\n mamba_spec = kv_cache_group.kv_cache_spec\n if isinstance(mamba_spec, UniformTypeKVCacheSpecs):\n mamba_spec = mamba_spec.kv_cache_specs[layer_name]\n assert isinstance(mamba_spec, MambaSpec)
1c4686c to
74417f3
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
74417f3 to
72b3e90
Compare
zhao-stack
changed the title
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
e4cf657 to
3200a31
Compare
|
/rerun Rerun (failed jobs only):
|
zhao-stack
changed the title
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Build on PR vllm-project#14872 at d9d162d. Adapt the exact post-e6bfe03 upstream interval, including #53896, #52506, #51358, #54079, #54282, #54436, #53576, #53388 and the #53906 storage-field collision. Preserve v0.27.1 contracts and leave its pin unchanged. Signed-off-by: shenzhao <shenzhao9@huawei.com>
Complete the e126687a (#53896) main-only contract: inherited V2 setup disables slot mappings for CircularBufferSpec groups, but the Ascend kernel previously ignored the flag. Emit PAD before indexing state rows and retain the explicit v0.27.1 no-flag path. Add CPU forwarding and NPU CP/padding regressions. Signed-off-by: shenzhao <shenzhao9@huawei.com>
Update target Gumbel test calls for the #54282 required drafting argument and main API while retaining every existing assertion and the legacy call path. Add draft-noise salt coverage. Move the new #53896 slot-mapping regression into the existing PR one-card lane so ready-all actually runs it without CI configuration changes. Signed-off-by: shenzhao <shenzhao9@huawei.com>
Register the PR #53896 circular-buffer slot-mapping regression in the required estimated_times map. Use the existing 600s default as an initial scheduling estimate, not a measured duration or test timeout. This fixes the missing registration introduced when the new in-range regression was moved into the PR suite; no runtime, routing, skip, or assertion changes. Signed-off-by: shenzhao <shenzhao9@huawei.com>
Gate the vLLM #53896 CircularBufferSpec adaptation explicitly on the main lane instead of probing symbol availability. Keep the v0.27.1 slot-mapping behavior and cover both lane selections. Signed-off-by: shenzhao <shenzhao9@huawei.com>
Signed-off-by: shenzhao <shenzhao9@huawei.com>
Adapt upstream vllm PR #51358 (6b110badbb22d3f66c7218b71138f13b7a6b3419): drain boundary offers, snapshot current block tables before CoW retention release, pass scheduler-local state to connectors, and clear it before worker dispatch. AscendOffloadingConnector inherits the upstream consumer. Add boundary snapshot and no-connector regression tests. Signed-off-by: shenzhao <shenzhao9@huawei.com>
2e9d108 to
a96f178
Compare
Restore the #54436 InputBatch max_seq_len_np removal lost during rebase. Match #53896's newly evaluated Mamba page-size contract by using torch.float32 in the existing uniform-group fixture. Both failures were observed in run 34098781247; retain assertions and the CPU gate. Signed-off-by: shenzhao <shenzhao9@huawei.com>
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Keep the working set that made the A2 unit tests pass on the Sep-4 kv_config_builder vLLM (get_max_layers_per_page_size / may_override / _annotate_eagle_groups(use_deepseek_v4_fallback) / storage_block_size setter / bind_kv_cache kv_cache_groups / is_kv_cache_spec_uniform). Temporary: superseded by rebasing onto vllm-ascend main2main vllm-project#15627. Signed-off-by: yangzeyu <532183776@qq.com>
…e_num_blocks The vllm-project#15627 clean migration calls kv_cache_planning.may_override_num_blocks (no aliasing shim), so the monkeypatch must target the public name. The old private name only existed in the b2f-based shimmed branch. Signed-off-by: yangzeyu <532183776@qq.com>
…53558 vLLM The vllm-project#15627 base targets vLLM main where get_kv_cache_groups lives in kv_cache_utils; the Sep-4 #53558 fork moved the group-planning entry to kv_cache_planning. worker.py's top-level import therefore failed on the A2 #53558 box, breaking every vllm_ascend.worker.worker.* patch (module never registers as a package attr). Also retarget test_attn_utils_v2 to the public planning may_override_num_blocks (the shim-era private name is gone). Signed-off-by: yangzeyu <532183776@qq.com>
Under the #53906 storage_block_size optional-override semantics the field on AscendMLAAttentionSpec stays None (matching the vllm-project#15627-only storage tests that assert is None); the derived physical size must be read via get_storage_block_size(). These three base tests asserted the field directly (== 32), which only held under the pre-#53906 property semantics. Signed-off-by: yangzeyu <532183776@qq.com>
1 participant
What this PR does / why we need it?
以已合入 PR14872 的最新 main
fd815467c221ee600137f6bdd53fe354d5e7c999为基线，仅升级 PR14872 之后的区间：e6bfe03ad73a3330cb427885aa90d97a12e1c704（8 月 28 日节点）560ef78bfe734ea894f8a866e50406574021b9f5（9 月 4 日节点）Per-change upstream evidence
下表代码路径除明确 tests/ 外均相对于 vllm_ascend/；所有上游原因均已验证在
(old, new]内。全部适配仅针对 main commit lane。patch/platform/patch_kv_cache_utils.py：分组补丁入口、Uniform 分组统计vllm/v1/core/kv_cache_utils.pypatch/worker/patch_mamba_utils.py：分组返回值与三处状态复制vllm/v1/worker/mamba_utils.pyworker/model_runner_v1.py：Mamba 复制函数生产端vllm/v1/worker/gpu_model_runner.pyworker/v2/block_table.py：slot enablement 透传vllm/v1/worker/gpu/block_table.py_310p/worker/v2/model_runner.py与block_table.py：310P recurrent group 标记vllm/v1/worker/gpu/model_runner.pyops/triton/v2/block_table/compute_slot_mappings.py：设备 kernelvllm/v1/worker/gpu/block_table.pypatch/worker/patch_bind_kv_cache.py：group metadata 与 ring trackervllm/v1/worker/utils.pyworker/v2/model_runner.py、_310p/worker/v2/model_runner.py：dummy state flagvllm/v1/worker/gpu/model_runner.pycore/kv_cache_interface.py：MLA storage 字段冲突vllm/v1/kv_cache_interface.pyattention/dsa_v1.py、attention/context_parallel/dsa_cp.py：metadata 物理行数vllm/v1/kv_cache_interface.pyworker/model_runner_v1.py、_310p/worker/v2/model_runner.py：cache viewsvllm/v1/kv_cache_interface.pycore/recompute_scheduler.py：边界状态交接vllm/v1/core/sched/scheduler.pydistributed/device_communicators/npu_communicator.py：graph-capture 字段vllm/distributed/parallel_state.pyworker/v2/sample/gumbel.py：draft 参数与随机盐vllm/v1/worker/gpu/sample/gumbel.pytests/ut/models/minimax_m3/test_minimax_m3_vit.py：多模态 mockvllm/model_executor/models/interfaces.pytests/ut/patch/platform/test_patch_mamba_block_aligned_split.py：scheduler mockvllm/v1/core/sched/scheduler.pyworker/v2/model_runner.py、_310p/worker/v2/model_runner.py：InputBatch 构造及对应回归vllm/v1/worker/gpu/input_batch.pytests/ut/distributed/ascend_store/test_pool_worker.py：Mamba dtype fixturevllm/v1/kv_cache_interface.pyExact-contract scan
新基线为
fd815467c221ee600137f6bdd53fe354d5e7c999，上游 old/new 如上；engine7bd156ac14fa5d289c942ce88652d7de85307817，scenario main2main、profile exact-contracts、无 external roots。因基线改变，已重新完成 validate 和 predict（退出码 0）。指纹e9b767e728e0fac32f4a8ed3fdf022ff61e55c5a7f44a4ecc09d7ee991500f67；engine package SHA256966906dea108013cb4f163ae04543b40d8880b654efab3eab476293778bf6663。predict 耗时 1372.33 秒，全部计划能力 analyzed/included。结果为 6 个 introduced 根因 / 8 条关系，逐条核实四项 gates 全真；两处旧 KV grouping hook、bind 参数、Mamba 返回协议、普通/310P BlockTables 参数、普通/310P execute_model 参数均由本 PR 适配覆盖。另有 35 warnings、7 preexisting、314 unresolved、1 fixed，未批量当作升级修复。#53906、#51358 语义交接、Gumbel draft delta 和两项 CPU 失败另以直接源码证据记录。旧 c72 扫描不作为新基线完成依据；后续仅修改适配代码不重复扫描。
Does this PR introduce any user-facing change?
将 verified-main 节点移至 9 月 4 日，适配其接口和行为变化；本 PR 暂不支持双版本。
How was this patch tested?
2b04dd8522c4c1cb7e5962bff9d128fa3b4f2313，attempt 1，最终 SUCCESS。pre-commit、CPU、三种设备构建、全部 12 个设备任务及 ci-gate 均通过。CPU 3600 passed / 37 skipped，两项原失败均 PASSED。vllm@560ef78：A2 单卡 5 个分片，A3 双卡 2 个分片、四卡 2 个分片、八卡 1 个任务，310P 单卡与四卡各 1 个任务，全部 SUCCESS。test_slot_mapping_enabled.py4 passed；Gumbel 所在 A2 分片 中test_gumbel_sampling.py32 passed，包含 draft salt 回归。以下为 workflow 自动维护的版本尾注，按其当前内容保留；本 PR 目标及实际 CPU checkout 均为上文 560ef78：