[Feature] [MRV2] Supports the combination of SFA PCP, DCP, and MTP - #16597
Conversation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. Tip 💡 Consider Linking a Related Issue or RFCYour PR title contains the [Feature] tag, indicating a bug fix or new feature. Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:
🙏 Thanks for helping us keep the project well-organized! |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces support for the concurrent use of SFA (Sequence-Parallel Attention), PCP (Prefill Context Parallelism), DCP (Decode Context Parallelism), and MTP (Multi-Token Prediction) within the Ascend backend. By centralizing sequence length calculations and enhancing metadata builders to account for global block table state, the changes ensure consistent KV cache management and synchronization across parallel ranks. These improvements facilitate more robust handling of complex parallel execution modes in speculative decoding scenarios. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Attention][Feature] Support PCP and DCP integration for SFA and speculative decodingSuggested PR Summary:
### What this PR does / why we need it?
This PR refactors context parallel (CP) and speculative decoding integration for Ascend. It imports `get_dcp_local_seq_lens` from vLLM core, updates metadata builders to support PCP+DCP compact KV views, and adds broadcast logic for replicated draft tokens in speculative decoding. It also bypasses parallel config validation for separate draft models.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tested with new and updated unit tests in `tests/ut/attention/` and `tests/ut/worker/`.Feedback on Review Comments:
All review comments are valid and point out important robustness improvements, such as using getattr to avoid AttributeErrors, avoiding direct mutation of potentially frozen config objects, and ensuring tensors are contiguous before broadcasting.
pisceskkk
left a comment
There was a problem hiding this comment.
lgtm, leave some nits.
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
…ency Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
|
/rerun Rerun (failed jobs only):
|
…t mapping (vllm-project#16597) Partial backport of vllm-project#16597 to releases/v0.26.0rc: only the SFA DCP replicated slot-mapping padding fix and its regression test are included. _build_slot_mapping_replicated_view sized its repeat_interleave output by num_input_tokens, which can include graph padding, while the repeated query lengths cover only real tokens; this raises a RuntimeError whenever num_input_tokens > sum(query_lens). The fix sizes it by the actual query token count from query_start_loc_cpu. The SFA-PCP / MTP-PCP feature work in the original commit requires MRV2 PCP groundwork (vllm-project#15809, vllm-project#14960) that is not on this branch and is intentionally not included. (cherry picked from commit 0301ec6) Signed-off-by: weiguihua2 <984323595@qq.com> Signed-off-by: lllyys <221728326+lllyys@users.noreply.github.com>
What this PR does / why we need it?
This PR enables MRV2 speculative decoding with the combined SFA, PCP, DCP, and MTP configuration by keeping KV-cache metadata and draft execution consistent across parallel ranks.
The main changes include:
get_dcp_local_seq_lensimplementation for interleave-aware, per-rank DCP sequence-length calculation.Does this PR introduce any user-facing change?
Yes. MRV2 now supports using SFA with PCP, DCP, and MTP together. No new configuration options are introduced.
How was this patch tested?
Added and updated unit tests covering:
DCP local sequence lengths for different DCP sizes, interleave sizes, input dtypes, ranks, and empty batches.
PCP+DCP compact KV metadata with global request block tables and allocated block counts.
Stale and padded block-table entries, including valid block ID 0.
Logical DCP collective rank ordering.
PCP attention-context block-count snapshots.
Replicated SFA indexer metadata and slot-mapping behavior.
Existing SFA PCP/DCP metadata-builder paths.
vLLM main: vllm-project/vllm@84030bb