Launch sglang engines gated and release them once they are running - #2096
Open
fzyzcjy wants to merge 1 commit into
Open
Launch sglang engines gated and release them once they are running#2096fzyzcjy wants to merge 1 commit into
fzyzcjy wants to merge 1 commit into
Conversation
fzyzcjy
requested review from
Shi-Dong,
Zhichenzzz,
guapisolo,
jybsuper,
maocheng23,
yueming-yuan and
yushengsu-thu
as code owners
August 3, 2026 15:01
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 3, 2026 23:43
9a663a9 to
f3701c7
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 4, 2026 00:38
8423dcd to
0fdb385
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 4, 2026 00:38
f3701c7 to
0cfae7d
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 4, 2026 02:36
0fdb385 to
e0a4dce
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 4, 2026 02:36
0cfae7d to
2a9824b
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 4, 2026 06:53
e0a4dce to
ee99103
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 4, 2026 06:53
2a9824b to
13811cb
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 4, 2026 07:24
ee99103 to
109bed1
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 4, 2026 07:24
13811cb to
2589578
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 4, 2026 07:31
109bed1 to
b243dc4
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 4, 2026 07:31
2589578 to
206df79
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 6, 2026 03:46
b243dc4 to
ce501c4
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
2 times, most recently
from
August 6, 2026 07:04
bfc30f2 to
e024b0e
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 9, 2026 23:31
165008b to
0e6df86
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
2 times, most recently
from
August 10, 2026 00:44
14427e3 to
9d64207
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 10, 2026 06:00
4aef8a7 to
fd52c7c
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 10, 2026 06:00
9d64207 to
37be5f8
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 10, 2026 16:06
fd52c7c to
536beb7
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 10, 2026 16:06
37be5f8 to
5877b7a
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 10, 2026 16:33
536beb7 to
c472209
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
2 times, most recently
from
August 12, 2026 02:48
22b54d0 to
2cd2419
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 12, 2026 02:48
c472209 to
2441d03
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
2 times, most recently
from
August 12, 2026 14:31
97b950a to
cdc6b8b
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 12, 2026 14:36
71b8062 to
1b1e4c2
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
3 times, most recently
from
August 13, 2026 01:13
de3bffa to
cca083c
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 13, 2026 03:11
503f4b5 to
9442cbe
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
4 times, most recently
from
August 13, 2026 06:39
8ff168d to
b2fa901
Compare
Pass the allocated gate port to every engine so sglang idles at its out-of-band control port instead of initializing straight away, and have RayWorkerManager release each cell after its workers were started. The release retries until the port answers, because it only starts listening once the engine reaches its gate.
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-118
branch
from
August 14, 2026 04:45
b2fa901 to
fd2f4ec
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op13-117
branch
from
August 14, 2026 04:45
360867b to
fd391b2
Compare
This was referenced Aug 26, 2026
guapisolo
approved these changes
Aug 27, 2026
guapisolo
left a comment
guapisolo
left a comment
Collaborator
There was a problem hiding this comment.
Approved in stack context after reviewing PR #2096 changes. The upstream gate dependency is tracked in #1837, while #2098 removes the test hang before #2176 and the repair is retained through checked tip #2649; I left one non-blocking P2 about keeping the gate port allocator-owned.
| @@ -193,8 +197,6 @@ def _compute_server_args( | |||
| if sglang_overrides: | |||
| kwargs.update(sglang_overrides) | |||
Collaborator
There was a problem hiding this comment.
[P2] Please treat gated_launch_port as allocator-owned, like host/port. Under an explicit override, SGLang listens on that port while ServerCell activates the allocator port, so startup waits on the wrong endpoint; this is non-blocking for the intended configuration, but we should reject the override in ServerGroupConfig.resolve() and omit the corresponding CLI field.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1837