Repository navigation
[Bugfix] Avoid TCPStore port collision for co-located non-DP Ray engines - #53666
Merged
njhill merged 1 commit intoAug 27, 2026
Merged
Conversation
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
Author
|
Ray's integration tests https://buildkite.com/ray-project/release/builds/105307/table passed. |
Contributor
Author
|
/ci run |
|
❌ @jeffreywang88, A reviewer with write access must run |
Contributor
Author
|
Hey @njhill could you please review? thank you! |
Member
|
/ci run |
njhill
enabled auto-merge (squash)
August 27, 2026 18:35
|
✅ @jeffreywang88, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #85860 for commit |
aoshen02
added a commit
to aoshen02/vllm
that referenced
this pull request
Aug 28, 2026
vllm/v1/executor/ray_executor_v2.py: keep binder-side TCPStore port selection on the rank-0 actor; drop main's driver-side _select_tcpstore_port call (vllm-project#53666), which the bind-and-hold approach subsumes. main's sharded_rdt actor options (vllm-project#43375) auto-merged. tests/distributed/test_ray_v2_executor.py: keep test_dist_init_store_owns_port; drop main's _select_tcpstore_port tests including test_colocated_tcpstore_ports_differ, since that helper no longer exists. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
am-cohere
pushed a commit
to am-cohere/vllm
that referenced
this pull request
Sep 1, 2026
…nes (vllm-project#53666) Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
mikeshawcode
pushed a commit
to mikeshawcode/vllm
that referenced
this pull request
Sep 1, 2026
…nes (vllm-project#53666) Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com> Signed-off-by: mikeshawcode <michaelwshaw2@gmail.com>
mylibrar
pushed a commit
to tanyuqian/vllm
that referenced
this pull request
Sep 3, 2026
…nes (vllm-project#53666) Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
4 tasks done
yfw
added a commit
to NVIDIA-NeMo/RL
that referenced
this pull request
Sep 14, 2026
vLLM 0.29 ships vllm-project/vllm#53666 and #50969: RayWorkerV2 now binds the torch.distributed TCPStore itself on a kernel-assigned port and holds the socket (`self._dist_init_store = store`) until init_process_group reuses it, so the probe/bind window that let the MessageQueue take the same port is gone, and `_select_tcpstore_port` no longer exists. `_patch_vllm_ray_executor_v2_tcpstore_port` therefore found no anchor and logged "may fail with EADDRINUSE" on every worker start. Detect the upstream marker, log at info level and leave the file alone. The port-arithmetic tests skip when the installed vLLM carries the upstream fix; a new test pins the no-warn/no-edit behaviour against a synthetic 0.29-style source. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
_select_tcpstore_porttreatslocal_dp_rank is Noneas "no DP", but a non-DP config sets it to 0.https://github.com/vllm-project/vllm/blob/342b8ebd8b/vllm/v1/executor/ray_executor_v2.py#L272
https://github.com/vllm-project/vllm/blob/342b8ebd8b/vllm/config/parallel.py#L920.
Every co-located engine then picks port 100; two replicas on one host race and one dies with
EADDRINUSE. This PR fixs this by passingNonewhen there's no DP group.Ray Serve LLM tests do run 2 replicas × TP=2 per node, but Serve restarts the crashed replica, which then finds 100 taken and uses 101, and therefore the app goes healthy and the test passes.
Test Plan
Two TP=2 servers started together on one host (4 GPUs):
Also added unit tests:
pytest tests/distributed/test_ray_v2_executor.py.Test Result
tcp://<ip>:100; second died withEADDRINUSE.AI Assistance
This PR is written with Claude Code assistance, and I've reviewed every line.
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.