Repository navigation
fix(sglang): Port collision when colocating 2 workers on same node with DP - #584
Merged
Merged
Conversation
trevor-m
requested review from
alec-flowers,
csahithi,
ishandhanani and
nlevin-ui
as code owners
October 7, 2026 19:49
|
|
SGLang binds each KV-event publisher at the configured port plus its DP rank (the attention-DP rank, or the replica rank without attention DP). Two DP workers on one node got consecutive base ports, so worker 0's rank 1 bound worker 1's base and failed with "Address already in use". NVIDIA#473 reserved a dp-size block, but only in Dynamo sidecar mode with KV events, and it read only dp-size. Reserve the block for every SGLang process, as the vLLM backend does for its local DP ranks, and size it with SGLangBackend.num_dp_ranks (dp-size x attn-dp-size). sgl-project/sglang#41818 added --attn-dp-size and left dp-size counting replicas; the deprecated dp-size + enable-dp-attention spelling resolves to the same layout. The sidecar follower-publisher check, the replica NCCL-port rule and the AIPerf follower scrape now read the same resolver, so --attn-dp-size works there too. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
trevor-m
force-pushed
the
colocate-dp-ports
branch
from
October 13, 2026 17:13
2336c3a to
13452fa
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #584 +/- ##
=======================================
Coverage ? 85.38%
=======================================
Files ? 162
Lines ? 23470
Branches ? 0
=======================================
Hits ? 20041
Misses ? 3429
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merged
7 of 11 tasks
nvpohanh
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Oct 16, 2026
* chore: bump srt-slurm to v2.59.1 Move the srt-slurm submodule from v2.51.1 (545233a0) to v2.59.1 (ae3d6877). v2.59.1 adds NVIDIA/srt-slurm#584, which reserves one KV-event port per DP rank for SGLang workers, so two attention-DP workers on one node no longer collide on base port + DP rank. The pin now includes NVIDIA/srt-slurm#573 and #602, so drop their backport patches. The #602 patch no longer applies to v2.59.1 and would fail every srt-slurm job at the launcher's git apply. Upstream #573 merged a different design (idle GPUs on worker nodes are excused only after a worker GPU UUID probe) than the backported filter. Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep the participating-GPU filter on srt-slurm v2.59.1 v2.59.1 includes NVIDIA/srt-slurm#573, which keeps every exporter row and excuses idle worker-node GPUs only after a DCGM worker GPU UUID probe. The InferenceX power validator (infx/results/power/multinode.py) recomputes device validity without that excusal. An NVIDIA point that uses part of an exclusive node would store publication_valid=true, recompute unexpected_device, and lose its power metrics. Keep 573-participating-gpus.patch, which still applies to v2.59.1, until the validator follows the upstream rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When colocating multiple workers on a single node, srt-slurm assigns each worker's kv event publisher a port sequentially:
Worker 0 gets port 5200
Worker 1 gets port 5201
However, if we are using DP then sglang will apply dp worker rank as an offset to the base kv event publisher port to determine that DP worker's port. This will lead to a collision since worker 0 rank 1 will get port 5200+1 which collides with worker 1 rank 0 port.
To fix this, we spread out the base kv event publisher port for each worker based on the dp size, so worker 0 gets 5200 and worker 1 gets 5202 for example.