[Bugfix] Strengthen the check of X-data-parallel-rank in Hybrid LB mode - #32314
Conversation
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a bugfix to properly validate the X-data-parallel-rank in hybrid load balancing mode. The core of the fix is in vllm/v1/engine/input_processor.py, where the validation now correctly uses the local data parallel size as the upper bound for the rank. This prevents an out-of-bounds error and ensures a proper error message is returned to the user. Additionally, a new local_engines_only property has been added to ParallelConfig, which is a good refactoring that simplifies and clarifies the code in multiple files where this condition is checked. The changes are correct and well-implemented.
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
Head branch was pushed to by a user without write access
|
@njhill Hi, would you please merge it again? The previous commit missed adaptation of a test case. |
…de (vllm-project#32314) Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
…de (vllm-project#32314) Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
…de (vllm-project#32314) Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
…de (vllm-project#32314) Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
…de (vllm-project#32314) Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
…de (vllm-project#32314) Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
Purpose
When running vllm with hybrid LB mode, e.g.,
and then
We will receive
500 Internal Server Errorbecause index 2 is out of range about local dp size. So that the index ofself.core_enginesinDPLBAsyncMPClientis out of bound. Limit the X-data-parallel-rank of the request to local dp size if using hybrid LB mode to fix this issue.After this patch, we will receive "data_parallel_rank 2 is out of range [0, 2)." as expected.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.