Skip to content

Wire up InferenceController and RolloutExecutor and make the controller API async - #1843

Open
fzyzcjy wants to merge 1 commit into
tom/refactor-miles/op2-1from
tom/refactor-miles/op2-5
Open

Wire up InferenceController and RolloutExecutor and make the controller API async#1843
fzyzcjy wants to merge 1 commit into
tom/refactor-miles/op2-1from
tom/refactor-miles/op2-5

Conversation

@fzyzcjy

@fzyzcjy fzyzcjy commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Part of #1837

@fzyzcjy
fzyzcjy requested a review from yueming-yuan as a code owner July 28, 2026 12:55
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from b140598 to 0887196 Compare August 3, 2026 23:39
@fzyzcjy
fzyzcjy requested a review from Rockdu as a code owner August 3, 2026 23:39
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch 2 times, most recently from 8957d29 to 3b71f64 Compare August 4, 2026 00:39
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 0887196 to 5f089d6 Compare August 4, 2026 06:56
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 3b71f64 to cd62224 Compare August 4, 2026 06:56
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 5f089d6 to 1a71cad Compare August 10, 2026 06:06
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from cd62224 to 44bfe59 Compare August 10, 2026 06:06
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 1a71cad to cc79ac3 Compare August 10, 2026 16:07
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 44bfe59 to 23c4648 Compare August 10, 2026 16:07
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from cc79ac3 to 8dc50ea Compare August 10, 2026 16:34
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch 2 times, most recently from 788a07a to 2811f05 Compare August 12, 2026 02:48
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 8dc50ea to fb28f27 Compare August 12, 2026 14:31
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 2811f05 to 20fa50d Compare August 12, 2026 14:31
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from fb28f27 to ee9ca13 Compare August 13, 2026 06:01
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 20fa50d to aed6aba Compare August 13, 2026 06:01
…er API async

Squashed from:
- Run the inference controller in the driver and the executor as an actor
- Await the now-async rollout preparation
- Make the whole inference controller API async
- Make the async controller surface hold in the migration guide and its test
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from aed6aba to 7848ec1 Compare August 14, 2026 04:46
@guapisolo

Copy link
Copy Markdown
Collaborator

Could we pull two small, dependency-light fixes forward and place them immediately after #1843, so this prefix can merge without waiting for the full refactor stack?

  1. Extract the minimal process-seam fix from Cut the eval fleet along the process seam #2614: keep the controller-side and executor-side EvalFleet state separate, and pass only data such as EvalFleetInfo / EvalFleetPin across the process boundary. This closes the serialized-copy/state-divergence bug introduced here. This should be described as a process-boundary correctness fix only: Delete the rollout health monitor ahead of its rewrite #1847 and Temporarily remove vanilla fault tolerance for the eval fleet #2497 intentionally remove the old eval-fleet FT path, so it must not claim that dedicated eval recovery is restored.
  2. Preserve replay compatibility for persisted audit JSONL with component="rollout_manager", either through a legacy alias or an explicit migration to rollout_executor. Let every miles process name itself as the component it is #2503 still accepts only rollout_executor, so older records remain unreadable or get dropped.

Ideally these are minimal follow-up commits/PRs directly on top of #1843, and we merge that smaller safe prefix before the remaining #1837 stack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants