Register the engines with the router from the rollout process instead of ray actors - #1862
Open
fzyzcjy wants to merge 1 commit into
Open
Register the engines with the router from the rollout process instead of ray actors#1862fzyzcjy wants to merge 1 commit into
fzyzcjy wants to merge 1 commit into
Conversation
fzyzcjy
requested review from
Shi-Dong,
Zhichenzzz,
maocheng23,
yueming-yuan and
yushengsu-thu
as code owners
July 28, 2026 12:56
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
from
August 3, 2026 23:44
1ddfff5 to
b86f9cd
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-3
branch
from
August 4, 2026 00:39
28437a3 to
51e1316
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
2 times, most recently
from
August 4, 2026 06:57
32288a2 to
80e518c
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-3
branch
2 times, most recently
from
August 6, 2026 07:03
5c849ef to
cefbc28
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
2 times, most recently
from
August 10, 2026 06:07
ad04c0f to
690860c
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-3
branch
from
August 10, 2026 06:07
cefbc28 to
30e603f
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
from
August 10, 2026 16:07
690860c to
41fde51
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-3
branch
from
August 10, 2026 16:34
282b8b1 to
7db556d
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
2 times, most recently
from
August 12, 2026 02:48
2c55eca to
868e4bf
Compare
fzyzcjy
changed the base branch from
tom/refactor-miles/op3-3
to
tom/refactor-miles/op3-19
August 12, 2026 02:50
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
from
August 12, 2026 14:31
868e4bf to
c9c00ee
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-19
branch
2 times, most recently
from
August 12, 2026 23:56
8b9a268 to
aebc0f6
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
from
August 12, 2026 23:56
c9c00ee to
1c53279
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-19
branch
from
August 13, 2026 01:13
aebc0f6 to
c20738b
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
from
August 13, 2026 01:13
1c53279 to
bc4ef91
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-19
branch
from
August 13, 2026 03:25
c20738b to
f40e568
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
2 times, most recently
from
August 13, 2026 06:01
aed73d2 to
e37f7fd
Compare
… of ray actors SGLangEngine registered itself with the router inside init and unregistered inside shutdown, which is the last thing on it that is not launching a server: it needed the router address, the legacy-api decision and its own node rank just to publish a url the rollout process already derives from the allocator. ServerGroup now publishes its node-0 engines once their servers answer, and withdraws them in stop_engines before the actor is killed. The engine keeps nothing router-related, and use_legacy_router_api moves next to the client it decides for. Two behaviour changes worth naming: registration now happens after every engine in a group has initialized rather than per engine inside init, and simulate_crash no longer withdraws the worker from the router first -- which is what a real crash does anyway.
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-19
branch
from
August 14, 2026 04:46
afa0642 to
3b12cee
Compare
fzyzcjy
force-pushed
the
tom/refactor-miles/op3-18
branch
from
August 14, 2026 04:46
e37f7fd to
0b0f59d
Compare
guapisolo
approved these changes
Aug 24, 2026
guapisolo
left a comment
guapisolo
left a comment
Collaborator
There was a problem hiding this comment.
Approving this PR in the context of the #2176 safe-merge checkpoint. Moving router-membership ownership into the rollout process is the right direction, but the original ordering could mark an engine alive before router registration succeeded. #2090 commits serving state only after successful router publication; that fix lands before #2176 and remains present through the checked chain tip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1837