[Refactor]refactor p2p connector#6551
Merged
wangxiyuan merged 2 commits intovllm-project:mainfrom Feb 7, 2026
Merged
Conversation
Signed-off-by: lty <linhebiwen@gmail.com>
Contributor
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Contributor
Summary of ChangesHello @luoxiaolin712, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request focuses on refactoring the peer-to-peer (P2P) connector components, specifically within the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the p2p connector logic in mooncake_connector.py and mooncake_layerwise_connector.py. The changes simplify the code by generalizing the calculation of block_len and region_len, removing duplicated if/elif/else blocks. This improves code readability and maintainability. However, I've found a potential issue in the refactored logic for calculating block_len that seems to be carried over from the previous implementation. It incorrectly uses the element size of the first tensor for all tensors in a group, which could lead to miscalculations if the tensors have different data types. I've added comments with suggestions to fix this.
| for i in range(len(first_kv_cache_tuple)): | ||
| block_shape = first_kv_cache_tuple[i].shape[-block_rank:] | ||
| logger.info("block_shape: %s", block_shape) | ||
| self.block_len.append(first_kv_cache[i].element_size() * math.prod(block_shape)) |
Contributor
There was a problem hiding this comment.
There seems to be a potential bug here. first_kv_cache is assigned first_kv_cache_tuple[0], so it's the first tensor in the tuple. When calculating block_len, you are using first_kv_cache[i].element_size(), which is equivalent to first_kv_cache.element_size() for any valid i. This means you are using the element size of the first tensor (first_kv_cache_tuple[0]) for all tensors in first_kv_cache_tuple. If the tensors in first_kv_cache_tuple have different dtypes, this will lead to an incorrect block_len calculation. It should probably be first_kv_cache_tuple[i].element_size() to get the element size of the correct tensor in the tuple.
Suggested change
| self.block_len.append(first_kv_cache[i].element_size() * math.prod(block_shape)) | |
| self.block_len.append(first_kv_cache_tuple[i].element_size() * math.prod(block_shape)) |
| for i in range(len(first_kv_cache_tuple)): | ||
| block_shape = first_kv_cache_tuple[i].shape[-block_rank:] | ||
| logger.info("block_shape: %s", block_shape) | ||
| self.block_len.append(first_kv_cache[i].element_size() * math.prod(block_shape)) |
Contributor
There was a problem hiding this comment.
Similar to the other file, there's a potential bug here. first_kv_cache is first_kv_cache_tuple[0]. The code first_kv_cache[i].element_size() uses the element size of the first tensor for all calculations within the loop. If tensors in first_kv_cache_tuple can have different dtypes, this will be incorrect. You should use first_kv_cache_tuple[i].element_size() to ensure you're using the element size of the correct tensor.
Suggested change
| self.block_len.append(first_kv_cache[i].element_size() * math.prod(block_shape)) | |
| self.block_len.append(first_kv_cache_tuple[i].element_size() * math.prod(block_shape)) |
Signed-off-by: lty <linhebiwen@gmail.com>
845473182
pushed a commit
to 845473182/vllm-ascend
that referenced
this pull request
Feb 9, 2026
…to qwen3next_rebase * 'main' of https://github.com/vllm-project/vllm-ascend: [Patch] Remove the patch of MiniCPM (vllm-project#5975) [P/D] layerwise connector support recompute scheduler (vllm-project#5900) [CI] Add workflow support for lint image build (vllm-project#6489) [Bugfix] Fix problematic dummy_run & improper input_batch_size in eagle (vllm-project#6517) [Refactor]310p_e2e test case update (vllm-project#6539) [Refactor]refactor p2p connector (vllm-project#6551) [Refactor]refactor 310p attention impl and add ut (vllm-project#6579) [Refactor]refactor 310p ops and add ut (vllm-project#6591) [Ops][Refactor] Remove custom rotary_embedding operator (vllm-project#6523) [Lint]Style: Convert `vllm-ascend/` to ruff format(new Batch vllm-project#8) (vllm-project#6604) [Test] Add initial multi modal cases of Qwen2.5-VL-7B-Instruct for disaggregated encoder (vllm-project#5301) [CI] Fix broken CI (vllm-project#6599) [Lint]Style: Convert `vllm-ascend/` to ruff format(Batch vllm-project#10) (vllm-project#6173) [Lint]Style: Convert `vllm-ascend/` to ruff format(Batch vllm-project#11) (vllm-project#6176) [Lint]Style: Convert `vllm-ascend/` to ruff format(Batch vllm-project#8) (vllm-project#6129) [Lint]Style: Convert `vllm-ascend/` to ruff format(Batch vllm-project#7) (vllm-project#6023) [CI][Misc] Some improvement for github action (vllm-project#6587) [Image] Bump mooncake version to v0.3.8.post1 (vllm-project#6428)
chenchuw886
pushed a commit
to chenchuw886/vllm-ascend
that referenced
this pull request
Feb 12, 2026
### What this PR does / why we need it?
Redundant code is removed, and repeated logic is combined through the
p2p connector refactor, making the code easy to extend.
### Does this PR introduce _any_ user-facing change?
NA
### How was this patch tested?
P节点：
```
vllm serve /mnt/weight/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8002 \
--data-parallel-size 2 \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_producer",
"kv_port": "30000",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},
{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"0"
}
}
]
}
}'
```
D节点：
```
vllm serve /mnt/share/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8003 \
--data-parallel-size 4 \
--tensor-parallel-size 4 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_consumer",
"kv_port": "30100",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"1"
}
}
]
}
}'
```
- vLLM version: v0.15.0
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0
---------
Signed-off-by: lty <linhebiwen@gmail.com>
Signed-off-by: momochenchuw <chenchuw@huawei.com>
ZRJ026
pushed a commit
to ZRJ026/vllm-ascend
that referenced
this pull request
Feb 28, 2026
### What this PR does / why we need it?
Redundant code is removed, and repeated logic is combined through the
p2p connector refactor, making the code easy to extend.
### Does this PR introduce _any_ user-facing change?
NA
### How was this patch tested?
P节点：
```
vllm serve /mnt/weight/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8002 \
--data-parallel-size 2 \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_producer",
"kv_port": "30000",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},
{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"0"
}
}
]
}
}'
```
D节点：
```
vllm serve /mnt/share/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8003 \
--data-parallel-size 4 \
--tensor-parallel-size 4 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_consumer",
"kv_port": "30100",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"1"
}
}
]
}
}'
```
- vLLM version: v0.15.0
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0
---------
Signed-off-by: lty <linhebiwen@gmail.com>
Signed-off-by: zrj026 <zhangrunjiang026@gmail.com>
maoxx241
pushed a commit
to maoxx241/vllm-ascend
that referenced
this pull request
Mar 2, 2026
### What this PR does / why we need it?
Redundant code is removed, and repeated logic is combined through the
p2p connector refactor, making the code easy to extend.
### Does this PR introduce _any_ user-facing change?
NA
### How was this patch tested?
P节点：
```
vllm serve /mnt/weight/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8002 \
--data-parallel-size 2 \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_producer",
"kv_port": "30000",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},
{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"0"
}
}
]
}
}'
```
D节点：
```
vllm serve /mnt/share/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8003 \
--data-parallel-size 4 \
--tensor-parallel-size 4 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_consumer",
"kv_port": "30100",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"1"
}
}
]
}
}'
```
- vLLM version: v0.15.0
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0
---------
Signed-off-by: lty <linhebiwen@gmail.com>
ZRJ026
pushed a commit
to ZRJ026/vllm-ascend
that referenced
this pull request
Mar 4, 2026
### What this PR does / why we need it?
Redundant code is removed, and repeated logic is combined through the
p2p connector refactor, making the code easy to extend.
### Does this PR introduce _any_ user-facing change?
NA
### How was this patch tested?
P节点：
```
vllm serve /mnt/weight/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8002 \
--data-parallel-size 2 \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_producer",
"kv_port": "30000",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},
{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"0"
}
}
]
}
}'
```
D节点：
```
vllm serve /mnt/share/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8003 \
--data-parallel-size 4 \
--tensor-parallel-size 4 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_consumer",
"kv_port": "30100",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"1"
}
}
]
}
}'
```
- vLLM version: v0.15.0
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0
---------
Signed-off-by: lty <linhebiwen@gmail.com>
Signed-off-by: zrj026 <zhangrunjiang026@gmail.com>
LCAIZJ
pushed a commit
to LCAIZJ/vllm-ascend
that referenced
this pull request
Mar 7, 2026
### What this PR does / why we need it?
Redundant code is removed, and repeated logic is combined through the
p2p connector refactor, making the code easy to extend.
### Does this PR introduce _any_ user-facing change?
NA
### How was this patch tested?
P节点：
```
vllm serve /mnt/weight/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8002 \
--data-parallel-size 2 \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_producer",
"kv_port": "30000",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},
{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"0"
}
}
]
}
}'
```
D节点：
```
vllm serve /mnt/share/DeepSeek-V3.2-Exp-W8A8 \
--host 0.0.0.0 \
--port 8003 \
--data-parallel-size 4 \
--tensor-parallel-size 4 \
--enable-expert-parallel \
--seed 1024 \
--served-model-name model \
--max-model-len 8192 \
--max-num-batched-tokens 8192 \
--max-num-seqs 16 \
--enforce-eager \
--trust-remote-code \
--gpu-memory-utilization 0.92 \
--quantization ascend \
--async-scheduling \
--additional-config '{"ascend_scheduler_config":{"enabled":true}}' \
--kv-transfer-config \
'{
"kv_connector": "MultiConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"use_layerwise": false,
"connectors": [
{
"kv_connector": "MooncakeConnectorV1",
"kv_role": "kv_consumer",
"kv_port": "30100",
"kv_connector_extra_config": {
"use_ascend_direct": true,
"prefill": {
"dp_size": 2,
"tp_size": 8
},
"decode": {
"dp_size": 4,
"tp_size": 4
}
}
},{
"kv_connector": "AscendStoreConnector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"backend": "mooncake",
"mooncake_rpc_port":"1"
}
}
]
}
}'
```
- vLLM version: v0.15.0
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0
---------
Signed-off-by: lty <linhebiwen@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
Redundant code is removed, and repeated logic is combined through the p2p connector refactor, making the code easy to extend.
Does this PR introduce any user-facing change?
NA
How was this patch tested?
P节点：
D节点：