Update vllm pin to 12.24#5307
Conversation
Signed-off-by: leo-pony <nengjunma@outlook.com>
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
There was a problem hiding this comment.
Code Review
This pull request updates the vLLM pin and introduces a v_head_size parameter to AscendQKVParallelLinear. However, the new parameter is not used in the layer's output size calculations, which is a critical bug. I've left a comment with details on how to fix it. The documentation change looks fine.
|
@shenchuxiaofugui Could you help take a look? It's not a function issue. It's my local env issue. |
|
export HCCL_BUFFSIZE=1024 vllm serve Qwen3-30B-A3B-m4 |
### What this PR does / why we need it? Fix vllm break in the pr: 1. [Add MiMo-V2-Flash support] (vllm-project/vllm#30836) ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Co-authored-by: zxwang [1476209578@qq.com](mailto:1476209578@qq.com) - vLLM version: release/v0.13.0 - vLLM main: vllm-project/vllm@5fbfa8d --------- Signed-off-by: leo-pony <nengjunma@outlook.com> Signed-off-by: zxwang <1476209578@qq.com> Co-authored-by: zxwang <1476209578@qq.com> Signed-off-by: zrj026 <zhangrunjiang026@gmail.com>
### What this PR does / why we need it? Fix vllm break in the pr: 1. [Add MiMo-V2-Flash support] (vllm-project/vllm#30836) ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Co-authored-by: zxwang [1476209578@qq.com](mailto:1476209578@qq.com) - vLLM version: release/v0.13.0 - vLLM main: vllm-project/vllm@5fbfa8d --------- Signed-off-by: leo-pony <nengjunma@outlook.com> Signed-off-by: zxwang <1476209578@qq.com> Co-authored-by: zxwang <1476209578@qq.com> Signed-off-by: zrj026 <zhangrunjiang026@gmail.com>
What this PR does / why we need it?
Fix vllm break in the pr:
Does this PR introduce any user-facing change?
How was this patch tested?
Co-authored-by: zxwang 1476209578@qq.com