[Feature][MRV2] sfa support dcp +pcp - #15809
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces support for running Prefill Context Parallelism (PCP) and Data Context Parallelism (DCP) simultaneously within the Sparse Flash Attention (SFA) backend for Ascend hardware. By creating a unified metadata builder and implementation class, the changes enable efficient coordination between the two parallel strategies, ensuring correct KV cache indexing and token layout management. This integration includes necessary updates to the resolution logic, platform validation, and context management to support the combined execution path. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. Tip 💡 Consider Linking a Related Issue or RFCYour PR title contains the [Feature] tag, indicating a bug fix or new feature. Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:
🙏 Thanks for helping us keep the project well-organized! |
There was a problem hiding this comment.
Code Review
Suggested PR Title:\n\nmarkdown\n[Attention][Feature] Support PCP and DCP simultaneously for SFA\n\n\nSuggested PR Summary:\n\nmarkdown\n### What this PR does / why we need it?\nThis PR enables simultaneous support for Prefill Context Parallel (PCP) and Decode Context Parallel (DCP) with Static Fine-grained Attention (SFA) on Ascend hardware. It introduces `AscendSFAPCPDCPMetadataBuilder` and `AscendSFAPCPDCPImpl` to handle the combined PCP and DCP execution path, manages PCP gathered-token layouts in DCP metadata, and updates parallel configuration validation to allow DCP overlaid on PCP.\n\nAdditionally, a critical bug was identified in `vllm_ascend/attention/context_parallel/sfa_cp.py` where using `elif` instead of `if` prevents `pcp_slot_mapping` from being set when `num_prefills == 0` in mixed batches, which can lead to assertion failures.\n\n### Does this PR introduce _any_ user-facing change?\nNo, this is an internal enablement of PCP and DCP simultaneous execution for SFA on Ascend.\n\n### How was this patch tested?\nTested with new and updated unit tests in `tests/ut/attention/test_sfa_cp.py`, `tests/ut/test_platform.py`, `tests/ut/worker/test_attn_utils_v2.py`, and `tests/ut/worker/test_pcp_manager_v2.py`.\n
ec5df49 to
8ebe59f
Compare
ab388a1 to
e29732d
Compare
b78958f to
cfe332e
Compare
fc085df to
8b7f5ee
Compare
af2e2dd to
4eddcf5
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Signed-off-by: weiguihua2 <984323595@qq.com>
Reuse the fused DCP combine operator and re-enable MRV2 accuracy CI. Validate the TP2 PCP2 DCP4 graph e2e against existing goldens. Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
Signed-off-by: weiguihua2 <984323595@qq.com>
4eddcf5 to
10a5fdb
Compare
Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
4096edc to
19ce722
Compare
|
/rerun Rerun (failed jobs only):
|
|
platform.py now advertises both dcp_size == pcp_size and dcp_size == tp_size * pcp_size, but the new PCP+DCP accuracy E2E only covers TP2/PCP2/DCP4. |
### What this PR does / why we need it? Model Runner v2 SFA DCP adapted for PCP. This PR enables combined PCP and DCP support for SFA in Model Runner V2. The main changes include: - Add AscendSFAPCPDCPMetadataBuilder and AscendSFAPCPDCPImpl for combined PCP and DCP execution. - Adapt slot mappings and block tables for KV cache indexing under combined parallelism. - Add PCPDCPContext and update AscendPCPManager to track gathered-token layouts. - Update SFA metadata builder and implementation selection to dispatch to the combined execution path. - Update parallel configuration validation to support PCP and DCP stacking. - Extend unit tests for SFA context parallelism, configuration validation, backend selection, and PCP metadata management. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - vLLM main: vllm-project/vllm@b2f6858 --------- Signed-off-by: weiguihua2 <984323595@qq.com> Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
### What this PR does / why we need it? Model Runner v2 SFA DCP adapted for PCP. This PR enables combined PCP and DCP support for SFA in Model Runner V2. The main changes include: - Add AscendSFAPCPDCPMetadataBuilder and AscendSFAPCPDCPImpl for combined PCP and DCP execution. - Adapt slot mappings and block tables for KV cache indexing under combined parallelism. - Add PCPDCPContext and update AscendPCPManager to track gathered-token layouts. - Update SFA metadata builder and implementation selection to dispatch to the combined execution path. - Update parallel configuration validation to support PCP and DCP stacking. - Extend unit tests for SFA context parallelism, configuration validation, backend selection, and PCP metadata management. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - vLLM main: vllm-project/vllm@b2f6858 --------- Signed-off-by: weiguihua2 <984323595@qq.com> Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
### What this PR does / why we need it? Model Runner v2 SFA DCP adapted for PCP. This PR enables combined PCP and DCP support for SFA in Model Runner V2. The main changes include: - Add AscendSFAPCPDCPMetadataBuilder and AscendSFAPCPDCPImpl for combined PCP and DCP execution. - Adapt slot mappings and block tables for KV cache indexing under combined parallelism. - Add PCPDCPContext and update AscendPCPManager to track gathered-token layouts. - Update SFA metadata builder and implementation selection to dispatch to the combined execution path. - Update parallel configuration validation to support PCP and DCP stacking. - Extend unit tests for SFA context parallelism, configuration validation, backend selection, and PCP metadata management. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - vLLM main: vllm-project/vllm@b2f6858 --------- Signed-off-by: weiguihua2 <984323595@qq.com> Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
### What this PR does / why we need it? Model Runner v2 SFA DCP adapted for PCP. This PR enables combined PCP and DCP support for SFA in Model Runner V2. The main changes include: - Add AscendSFAPCPDCPMetadataBuilder and AscendSFAPCPDCPImpl for combined PCP and DCP execution. - Adapt slot mappings and block tables for KV cache indexing under combined parallelism. - Add PCPDCPContext and update AscendPCPManager to track gathered-token layouts. - Update SFA metadata builder and implementation selection to dispatch to the combined execution path. - Update parallel configuration validation to support PCP and DCP stacking. - Extend unit tests for SFA context parallelism, configuration validation, backend selection, and PCP metadata management. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - vLLM main: vllm-project/vllm@b2f6858 --------- Signed-off-by: weiguihua2 <984323595@qq.com> Signed-off-by: weiguihua2 <weiguihua2@huawei.com> Signed-off-by: like-0517 <ithwlike@126.com>
…t mapping (vllm-project#16597) Partial backport of vllm-project#16597 to releases/v0.26.0rc: only the SFA DCP replicated slot-mapping padding fix and its regression test are included. _build_slot_mapping_replicated_view sized its repeat_interleave output by num_input_tokens, which can include graph padding, while the repeated query lengths cover only real tokens; this raises a RuntimeError whenever num_input_tokens > sum(query_lens). The fix sizes it by the actual query token count from query_start_loc_cpu. The SFA-PCP / MTP-PCP feature work in the original commit requires MRV2 PCP groundwork (vllm-project#15809, vllm-project#14960) that is not on this branch and is intentionally not included. (cherry picked from commit 0301ec6) Signed-off-by: weiguihua2 <984323595@qq.com> Signed-off-by: lllyys <221728326+lllyys@users.noreply.github.com>
What this PR does / why we need it?
Model Runner v2 SFA DCP adapted for PCP.
This PR enables combined PCP and DCP support for SFA in Model Runner V2.
The main changes include:
Does this PR introduce any user-facing change?
No
How was this patch tested?