[Bugfix] Account for local DP workers in startup thread allocation - #52385
Conversation
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: real-cpu <zhaochenrui757@gmail.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment Once the PR is approved or has the If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
/ci run |
|
✅ @cr-zhao, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #83996 for commit |
|
/ci retry |
|
✅ Queued 1 failed job(s) for retry in Buildkite CI #83996. |
|
/ci retry |
|
✅ Queued 1 failed job(s) for retry in Buildkite CI #83996. |
|
✅ Triggered Buildkite CI #84104 for commit |
|
/ci run |
|
✅ Triggered Buildkite CI #84105 for commit |
|
/ci run |
|
✅ Triggered Buildkite CI #84109 for commit |
|
/ci retry |
|
✅ Queued 1 failed job(s) for retry in Buildkite CI #84109. |
|
Hi @njhill, thanks for the review! All ci checks have passed, and the PR is now ready to merge. |
…llm-project#52385) Signed-off-by: real-cpu <zhaochenrui757@gmail.com> Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
2 participants
Purpose
Fixes #52330.
MultiprocExecutorsized each worker's startup thread pool using only theworkers within one engine. With multiple local data-parallel engines, every
engine therefore claimed the node's full CPU budget, oversubscribing CPU
threads and severely slowing weight loading.
This change:
data_parallel_size_localwhen counting workers that share thenode's startup CPU budget;
headless configurations where the local DP size is zero;
This is not a duplicate of an existing PR. Before submitting, I checked open
PRs referencing #52330 and searched open PRs for the affected data-parallel
startup/OMP thread-management area; no matching implementation was open.
Test Plan
issue's DP=4 block-quantized checkpoint reproduction on B200 hardware.
Test Result
.venv/bin/python -m compileall -q vllm/v1/executor/multiproc_executor.py tests/distributed/test_multiproc_executor.py: passed.git diff --check: passed.arm64 and its existing
.venvdoes not contain PyTorch or pytest. The testavoids creating GPU workers, but importing the vLLM executor still requires
PyTorch. Linux CI and hardware validation are required.
AI assistance was used to investigate the issue, implement the change, and
draft the regression test and PR description. The human submitter reviewed the
changes and is responsible for the contribution.
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.