[CI] main2main vllm 0828 - #15410
[CI] main2main vllm 0828#15410LQDLove wants to merge 87 commits into
Conversation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request updates the vllm-ascend codebase to track the latest vLLM main branch while preserving support for the v0.27.1 release lane. It introduces conditional logic to handle breaking upstream contract changes, ensuring compatibility across both development paths without impacting user-facing APIs. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Misc] Adapt KV cache and model runner for vLLM main compatibilitySuggested PR Summary:
### What this PR does / why we need it?
This PR introduces compatibility updates to support both vLLM v0.27.1 and newer vLLM main snapshots. It addresses upstream changes such as the renaming of `compress_ratio` to `tokens_per_state`, `shared_by` to `layers`, and the introduction of standardized shared KV backing allocations (vLLM #51718). It also adapts speculators, model runners, and the PCP manager to handle updated keyword arguments and graph capture contracts.
Feedback on current changes:
- In `vllm_ascend/worker/v2/pcp_manager.py`, `self.input_buffers` must be explicitly assigned to `self._input_buffers` to prevent runtime errors where sequence length views are missing.
- In `vllm_ascend/worker/v2/attn_utils.py`, `AttentionLayerBase` and `UniformTypeKVCacheSpecs` are referenced in `allocate_kv_cache_main` but are not imported, which will raise a `NameError` at runtime.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tested with updated unit tests covering both v0.27.1 and newer vLLM main branches.| if max_num_reqs is not None and max_num_tokens is not None: | ||
| self._input_buffers = AscendInputBuffers( | ||
| max_num_reqs=2 * max_num_reqs, | ||
| max_num_tokens=max_num_tokens, | ||
| device=device, | ||
| ) |
There was a problem hiding this comment.
The base class PCPManager initializes and uses self.input_buffers for persistent local buffers. Since AscendPCPManager overrides this with self._input_buffers (an instance of AscendInputBuffers), you must explicitly assign self.input_buffers = self._input_buffers. Otherwise, self.input_buffers remains the base InputBuffers instance, which lacks the Ascend-specific sequence length views and metadata, leading to runtime errors and test failures (e.g., in test_pcp_manager_uses_persistent_ascend_input_buffers).
| if max_num_reqs is not None and max_num_tokens is not None: | |
| self._input_buffers = AscendInputBuffers( | |
| max_num_reqs=2 * max_num_reqs, | |
| max_num_tokens=max_num_tokens, | |
| device=device, | |
| ) | |
| if max_num_reqs is not None and max_num_tokens is not None: | |
| self._input_buffers = AscendInputBuffers( | |
| max_num_reqs=2 * max_num_reqs, | |
| max_num_tokens=max_num_tokens, | |
| device=device, | |
| ) | |
| self.input_buffers = self._input_buffers |
| vllm_config = get_current_vllm_config() | ||
| attn_layers = get_layers_from_vllm_config(vllm_config, AttentionLayerBase) |
There was a problem hiding this comment.
AttentionLayerBase and UniformTypeKVCacheSpecs are referenced in allocate_kv_cache_main but are not imported in this file, which will raise a NameError at runtime when this function is called. Please import both classes (importing them inline inside the function is safe and avoids circular dependencies).
vllm_config = get_current_vllm_config()
from vllm.v1.attention.layer import AttentionLayerBase
from vllm.v1.kv_cache_interface import UniformTypeKVCacheSpecs
attn_layers = get_layers_from_vllm_config(vllm_config, AttentionLayerBase)
1c28e1c to
32a1b66
Compare
32a1b66 to
808333f
Compare
|
/cancel https://github.com/vllm-project/vllm-ascend/actions/runs/33385103557 |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
808333f to
435a40f
Compare
|
/e2e tests/e2e/pull_request/four_card/_310p/test_model_runner_v2_310p.py tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py |
435a40f to
ea8c600
Compare
|
/e2e tests/e2e/pull_request/four_card/_310p/test_model_runner_v2_310p.py tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py |
|
/e2e tests/e2e/pull_request/four_card/_310p/test_model_runner_v2_310p.py tests/e2e/pull_request/four_card/spec_decode/test_dspark_deepseekv4.py tests/e2e/pull_request/four_card/context_parallel/test_accuracy.py |
Signed-off-by: shenzhao <shenzhao9@huawei.com>
Use the PCP-local block table backing during full graph capture so replay reads the same storage that runtime prepare_attn updates. Add a regression test for the captured pointer contract. Signed-off-by: shenzhao <shenzhao9@huawei.com>
Signed-off-by: shenzhao <shenzhao9@huawei.com>
Signed-off-by: shenzhao <shenzhao9@huawei.com>
Recover aligned KV backing regions from tensor storage and register each physical allocation once across aliased KVCacheTensor descriptors. Keep support for private per-layer storage by registering the actual strided address span, and cover both Mooncake connector paths with regression tests. Signed-off-by: shenzhao <shenzhao9@huawei.com>
Drop the shared Mooncake registration refactor and limit the PR 14872 fix to the legacy hybrid connector. Register each descriptor's actual tensor byte range and merge only overlapping ranges before handing them to Mooncake. Signed-off-by: shenzhao <shenzhao9@huawei.com>
vLLM #51718 removed AttentionSpec.indexes_kv_by_block_stride on main. AscendMLAAttentionSpec.merge(), the kpool MLA spec construction, and the GLM-5.3 kpool indexer spec opt-in still read or wrote that field unconditionally, raising AttributeError on the main lane. Gate all three usages behind vllm_version_is(0.27.1). Signed-off-by: liaoqidan <1107297340@qq.com>
Signed-off-by: liaoqidan <1107297340@qq.com>
- Gate mamba_has_prefill_checkpoint_blocks on the mock scheduler (added in vLLM main, absent in v0.27.1) for the mamba block-aligned split patch tests - Gate dp_sync in the v2 ExecuteModelState construction (added in vLLM main) for the PCP manager sample_tokens tests Signed-off-by: liaoqidan <1107297340@qq.com>
Signed-off-by: liaoqidan <1107297340@qq.com>
Provide _hidden_restore_idx and a real compilation_config so the merged is_full_decode_graph padding path works under UT mocks. Signed-off-by: liaoqidan <1107297340@qq.com>
…PC regions Hybrid KV cache tensors shared across layer groups only registered the minimum address of each descriptor, leaving overlay groups (full attention, SWA, state cache) with zero registered addresses so their block-id mappings were never transferred and decode read misaligned KV. Collect every unique address with the full set of referencing groups. Also align IPC regions down/up to 2MB pages so devmm IPC export no longer rejects unaligned registrations with Invalid para (-22). Signed-off-by: liaoqidan <1107297340@qq.com>
8a65699 to
bdd31fa
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
github-actions
Bot
added
documentation
Extend the run-selected-tests, run-selected-tests-upstream, and run-selected-tests-a5 matrices to include both the vLLM main verified commit and the v0.27.1 release tag, so a single PR exercises both pipelines. main_commit stays for cpu-ut and the mypy checkout. Signed-off-by: liaoqidan <1107297340@qq.com>
bdd31fa to
fc00e1c
Compare
|
/cancel https://github.com/vllm-project/vllm-ascend/actions/runs/33960594456 |
github-actions
Bot
added
merge-conflicts
and removed
merge-conflicts
documentation
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
2 participants
What this PR does / why we need it?
This PR upgrades the verified vLLM main anchor to
e6bfe03ad73a3330cb427885aa90d97a12e1c704while preserving the vLLMv0.27.1release lane. It adapts vllm-ascend to the upstream contract changes introduced across that range.All adaptations are gated with
vllm_version_is("0.27.1")so the release-lane behavior is unchanged, and each logical change point is a separate commit.Upstream contract changes covered
skip_gatherin vocab parallel embedding / samplingKVCacheTensor.shared_by→layers/layer_stride; MLAcompress_ratio→tokens_per_state; standardized KV cache layoutdp_syncThe branch is rebased onto current
origin/mainand includes the 310P Mamba KV fixture exposing bothshared_byandlayersso the same test passes on both lanes.Does this PR introduce any user-facing change?
No. This is an internal compatibility update; it does not add an Ascend-specific public API.
How was this patch tested?
Unit tests adapted/added for both the v0.27.1 and main lanes (KV cache layout, MLA specs, spec-decode, PCP capture, DFlash2 factories, routed experts).
CI on the PR (pending).
vLLM main: vllm-project/vllm@ba07e4a