Launch the sgl and miles routers through RayWorkerManager - #2051
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
61d00c7 to
2066269
Compare
50341b4 to
0101c50
Compare
0135573 to
c4c712b
Compare
0101c50 to
fe10660
Compare
c4c712b to
ca8f68f
Compare
fe10660 to
7a3ef67
Compare
ca8f68f to
fea6d20
Compare
7a3ef67 to
04a0a2f
Compare
fea6d20 to
d0d8c55
Compare
04a0a2f to
eef744c
Compare
d0d8c55 to
cd6e01f
Compare
eef744c to
4e46159
Compare
1cb441f to
6e3b6b1
Compare
3fe1f05 to
69207e8
Compare
6e3b6b1 to
5ce1a4a
Compare
69207e8 to
e0a35b6
Compare
80e27d9 to
9c43ef8
Compare
8b9877b to
d0b2809
Compare
9c43ef8 to
3f30a23
Compare
d0b2809 to
c3f297b
Compare
3f30a23 to
1769125
Compare
| @@ -5,7 +5,9 @@ | |||
| def compute_specs(args) -> list[BaseWorkerSpec]: | |||
| return [ | |||
| *inference.specs_router(args), | |||
There was a problem hiding this comment.
[P3] Keep debug_train_only free of rollout services
Codex review: launch_worker_manager() runs before the later InferenceController.init() guard, so compute_specs() still emits router workers in --debug-train-only mode. This does not affect training correctness, but it unnecessarily starts a router actor and binds ports despite the documented train-only contract; could we return no router specs when args.debug_train_only is set?
| async def alloc_ports(self) -> None: | ||
| self.self_addrs = {} | ||
|
|
||
| node_ip = await self.actor_handle._get_node_ip.remote() |
There was a problem hiding this comment.
[P2] Preserve the MILES_HOST_IP router override
Codex review: this now derives the router host only from CommandActor._get_node_ip(), whereas the previous path used get_host_info() and honored MILES_HOST_IP. This is non-blocking when Ray’s node IP is already usable, but deployments that set the override for a container/pod address will bind and publish the Ray-detected IP instead. The behavior remains present through #2176 and #2649; could the router spec carry an explicit bind/advertised-host override?
c3f297b to
9bea8bb
Compare
1769125 to
9109b90
Compare
9bea8bb to
9e4f595
Compare
9109b90 to
7104b97
Compare
9e4f595 to
e89d381
Compare
7104b97 to
b9cf9af
Compare
e89d381 to
e6769b0
Compare
729ca7f to
7f47dbe
Compare
9980dea to
fddd65e
Compare
7f47dbe to
814eabd
Compare
fddd65e to
c7a042d
Compare
814eabd to
3692a27
Compare
c7a042d to
8de5de6
Compare
3692a27 to
83c7bcd
Compare
8de5de6 to
f462e48
Compare
Part of #1837