[refactor] Refactoring forward_context and model_runner_v1#1422
Merged
ganyi1996ppo merged 2 commits intovllm-project:v0.9.1-devfrom Jun 25, 2025
Merged
[refactor] Refactoring forward_context and model_runner_v1#1422ganyi1996ppo merged 2 commits intovllm-project:v0.9.1-devfrom
ganyi1996ppo merged 2 commits intovllm-project:v0.9.1-devfrom
Conversation
Signed-off-by: zzzzwwjj <1183291235@qq.com>
github-actions
Bot
added
documentation
zzzzwwjj
force-pushed
the
v0.9.1-dev
branch
2 times, most recently
from
cfd63c5 to
48fd2a1
Compare
…raph_mode Signed-off-by: zzzzwwjj <1183291235@qq.com>
whx-sjtu
reviewed
Jun 25, 2025
Collaborator
There was a problem hiding this comment.
Do we need to add another version here corresponding to 310P?
Collaborator
Author
There was a problem hiding this comment.
We can do it in the future.
MengqingCao
reviewed
Jun 25, 2025
Collaborator
There was a problem hiding this comment.
Don't we maintain etp any more?
Collaborator
There was a problem hiding this comment.
Given the absence of relevant scenarios, employing EP or full TP is sufficient, for now. We may subsequently advocate implementing expert tensor parallelism in vLLM to support scenarios where the number of nodes exceeds the number of experts.
There was a problem hiding this comment.
However, we do have customer scenarios that require such configurations. While DeepSeek models might not need this, there are use cases involving large-scale MoE (Mixture of Experts) models that require splitting across both Tensor Parallelism (TP) and Expert Parallelism (EP), or sometimes just TP alone. This is exactly the case with the current Jieyue Xingchen models
wangxiyuan
approved these changes
Jun 25, 2025
Collaborator
|
LGTM |
1 similar comment
Collaborator
|
LGTM |
Collaborator
|
This solution is not fully aligned with the current ETP solution. For example, EP and ETP cannot be supported at the same time. |
wangxiyuan
pushed a commit
that referenced
this pull request
Jul 21, 2025
### What this PR does / why we need it? Remove ETP/EP maintained in branch main. We drop this as there is no relevant scenarios to use ETP now, and we may subsequently advocate implementing expert tensor parallelism in vLLM to support scenarios where the expert is needed to be sliced This is a part of #1422 backport. Fixes #1396 #1154 ### Does this PR introduce _any_ user-facing change? We'll not maintain etp/ep in vllm-ascend anymore, and use the tp/ep in vllm instead. ### How was this patch tested? CI passed with new added and existing test. - vLLM version: v0.9.2 - vLLM main: vllm-project/vllm@fe8a2c5 Signed-off-by: MengqingCao <cmq0113@163.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Sep 26, 2025
Remove ETP/EP maintained in branch main. We drop this as there is no relevant scenarios to use ETP now, and we may subsequently advocate implementing expert tensor parallelism in vLLM to support scenarios where the expert is needed to be sliced This is a part of vllm-project#1422 backport. Fixes vllm-project#1396 vllm-project#1154 We'll not maintain etp/ep in vllm-ascend anymore, and use the tp/ep in vllm instead. CI passed with new added and existing test. - vLLM version: v0.9.2 - vLLM main: vllm-project/vllm@fe8a2c5 Signed-off-by: MengqingCao <cmq0113@163.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
### What this PR does / why we need it? Remove ETP/EP maintained in branch main. We drop this as there is no relevant scenarios to use ETP now, and we may subsequently advocate implementing expert tensor parallelism in vLLM to support scenarios where the expert is needed to be sliced This is a part of vllm-project#1422 backport. Fixes vllm-project#1396 vllm-project#1154 ### Does this PR introduce _any_ user-facing change? We'll not maintain etp/ep in vllm-ascend anymore, and use the tp/ep in vllm instead. ### How was this patch tested? CI passed with new added and existing test. - vLLM version: v0.9.2 - vLLM main: vllm-project/vllm@fe8a2c5 Signed-off-by: MengqingCao <cmq0113@163.com>
Clorist33
pushed a commit
to Clorist33/vllm-ascend
that referenced
this pull request
Dec 9, 2025
### What this PR does / why we need it? Remove ETP/EP maintained in branch main. We drop this as there is no relevant scenarios to use ETP now, and we may subsequently advocate implementing expert tensor parallelism in vLLM to support scenarios where the expert is needed to be sliced This is a part of vllm-project#1422 backport. Fixes vllm-project#1396 vllm-project#1154 ### Does this PR introduce _any_ user-facing change? We'll not maintain etp/ep in vllm-ascend anymore, and use the tp/ep in vllm instead. ### How was this patch tested? CI passed with new added and existing test. - vLLM version: v0.9.2 - vLLM main: vllm-project/vllm@fe8a2c5 Signed-off-by: MengqingCao <cmq0113@163.com>
yangzhe-2026
pushed a commit
to yangzhe-2026/vllm-ascend
that referenced
this pull request
May 6, 2026
### What this PR does / why we need it? Remove ETP/EP maintained in branch main. We drop this as there is no relevant scenarios to use ETP now, and we may subsequently advocate implementing expert tensor parallelism in vLLM to support scenarios where the expert is needed to be sliced This is a part of vllm-project#1422 backport. Fixes vllm-project#1396 vllm-project#1154 ### Does this PR introduce _any_ user-facing change? We'll not maintain etp/ep in vllm-ascend anymore, and use the tp/ep in vllm instead. ### How was this patch tested? CI passed with new added and existing test. - vLLM version: v0.9.2 - vLLM main: vllm-project/vllm@fe8a2c5 Signed-off-by: MengqingCao <cmq0113@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
A refactoring of
forward_contextandmodel_runner_v1, add some context which is necessary in model inference intoforward_context, and refactordummy_runlogic, make it more reasonable.Some details for this PR:
examplesdir;Does this PR introduce any user-facing change?
This PR remove
expert_tensor_parallel_sizeinadditional_config, we will useenable_expert_parallelto control whether expert_parallel is enable, which is consistent with vLLM.How was this patch tested?