[BugFix] Fix torchair+mtp bug after deleting deepseek_mtp.#3590
[BugFix] Fix torchair+mtp bug after deleting deepseek_mtp.#3590wangxiyuan merged 2 commits intovllm-project:mainfrom
Conversation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request renames the previous_hidden_states parameter to hidden_states in TorchairDeepSeekMTP.forward. While this change seems correct for interface alignment, it introduces a critical inconsistency with vllm_ascend/spec_decode/mtp_proposer.py. The calls to the compiled model within the torchair graph path in that file still use the old previous_hidden_states keyword argument, which will result in a TypeError at runtime. It appears that the corresponding updates in vllm_ascend/spec_decode/mtp_proposer.py are missing from this pull request.
| hidden_states: Optional[torch.Tensor] = None, | ||
| intermediate_tensors: Optional[IntermediateTensors] = None, | ||
| inputs_embeds: Optional[torch.Tensor] = None, | ||
| spec_step_idx: int = 0, | ||
| ) -> torch.Tensor: | ||
| hidden_states = self.model(input_ids, positions, kv_caches, | ||
| attn_metadata, previous_hidden_states, | ||
| attn_metadata, hidden_states, |
There was a problem hiding this comment.
Renaming previous_hidden_states to hidden_states in the forward method signature introduces a breaking change for its callers that use keyword arguments. The torchair graph execution path in vllm_ascend/spec_decode/mtp_proposer.py calls the compiled version of this model using previous_hidden_states as a keyword argument. This will lead to a TypeError at runtime because the parameter name has changed.
Specifically, in vllm_ascend/spec_decode/mtp_proposer.py, there are calls like:
# line 178
previous_hidden_states=previous_hidden_states,
# line 463
previous_hidden_states=self.hidden_states[:num_input_tokens],These calls will fail with the new signature. It seems the corresponding changes in vllm_ascend/spec_decode/mtp_proposer.py to use the hidden_states keyword are missing from this pull request.
whx-sjtu
force-pushed
the
rm_ds_mtp
branch
2 times, most recently
from
02c3a6c to
c596866
Compare
Signed-off-by: whx-sjtu <2952154980@qq.com>
…ect#3590) This is a missing bug fix introduced by PR vllm-project#3561 - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: whx-sjtu <2952154980@qq.com>
…ect#3590) This is a missing bug fix introduced by PR vllm-project#3561 - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: whx-sjtu <2952154980@qq.com> Signed-off-by: luolun <luolun1995@cmbchina.com>
…ect#3590) This is a missing bug fix introduced by PR vllm-project#3561 - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: whx-sjtu <2952154980@qq.com> Signed-off-by: hwhaokun <haokun0405@163.com>
…ect#3590) This is a missing bug fix introduced by PR vllm-project#3561 - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: whx-sjtu <2952154980@qq.com> Signed-off-by: nsdie <yeyifan@huawei.com>
…ect#3590) This is a missing bug fix introduced by PR vllm-project#3561 - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: whx-sjtu <2952154980@qq.com>
…ect#3590) This is a missing bug fix introduced by PR vllm-project#3561 - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: whx-sjtu <2952154980@qq.com>
2 participants
This is a missing bug fix introduced by PR #3561