Skip to content

[3/n] [Serve] Defer rank assignment after replica is allocated#58477

Merged
abrarsheikh merged 17 commits into
masterfrom
LLM-2497-abrar-rank-p3
Nov 19, 2025
Merged

[3/n] [Serve] Defer rank assignment after replica is allocated#58477
abrarsheikh merged 17 commits into
masterfrom
LLM-2497-abrar-rank-p3

fix function types

29ce266
Select commit
Loading
Failed to load commit list.
Cursor / Cursor Bugbot completed Nov 14, 2025 in 2m 7s

Bugbot Review

Bugbot Analysis Progress (2m 9s elapsed)

✅ Gathered PR context (2s)
✅ Analyzed code changes (1s)
✅ Completed bug detection — 1 potential bug found (2m 1s)
✅ Validation and filtering completed (0s)
✅ Posted analysis results — 1 bug reported (5s)
✅ Analysis completed successfully (0s)

Final Result: Bugbot completed review and found 1 potential issue

Request ID: serverGenReqId_e4711ed6-5de2-4207-926d-e6ccc2e7b40f

Details

Bug: Broken Rank Assignment in Replica Recovery

During replica recovery after controller restart, initialize_and_get_metadata.remote() is called without passing a rank parameter, but the method signature now requires it. The _assign_rank_callback is never set during recovery since start() isn't called, and the rank assignment logic in check_ready() is skipped because _ready_obj_ref is already set by recover(). This causes recovered replicas to initialize without a rank, breaking the deferred rank assignment system.

python/ray/serve/_private/deployment_state.py#L669-L672

else:
self._ready_obj_ref = (
self._actor_handle.initialize_and_get_metadata.remote()
)

Fix in Cursor Fix in Web