Repository navigation
Conversation
…host --custom-agent-function-mode worker sends each agent call to an agent worker (python -m miles.rollout.agentic.worker) at --custom-agent-function-worker-url, which runs it through the existing subproc path there, so a harness's CPU work leaves the GPU box. The protocol is a submit plus long-polls of at most 25 s and a DELETE to cancel. The run's agent env vars travel with each call, never credentials by value. A lost or unreachable worker is an InfraAbort, and a rollout abort cancels only the calls of its own rollout (its router), so the dedicated eval engines' calls survive training's oversampling abort. TrainStepOutcome moves to a module of its own so the worker imports without torch, sglang or ray.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a078777516
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| except asyncio.CancelledError: | ||
| _finish(call, {"kind": "cancelled"}) |
There was a problem hiding this comment.
When a rollout abort cancels a worker call, call_agent_function sends SIGTERM but leaves its shielded result reader running so the child's cancellation cleanup can still report. Marking the call done immediately here makes the remote caller stop polling before those reports arrive. For agents using report_sandbox(..., status="finished") during teardown, the final status is consequently lost and the dashboard continues showing an active sandbox after it has been destroyed. This breaks the behavior explicitly covered for subproc mode by test_a_hung_subproc_call_reports_before_it_returns_and_its_cancel_teardown. Keep a report-draining path alive through child cleanup, including when the caller task itself is cancelled.
Useful? React with 👍 / 👎.
Layer P22 of the CUA upstreaming stack. It is built and reviewed in celve, and goes upstream with the whole stack once every layer is finished. Base: P9's branch
LIN-2516/discard-infra-aborts(#34), because a lost or unreachable worker reportsInfraAbort. P23 (sandbox mode) stacks on this layer. Issue: LIN-2710, hub LIN-2504.Problem. Every agent call runs on the rollout host:
subprocgives each one a child process there. A host-process harness (Harborcomputer-1,terminus-2) and its verifier burn that host's CPUs, about 3.2 cores of a GPU box in LIN-2399's Harbor recipe. Miles has no way to run them elsewhere.Fix.
--custom-agent-function-mode worker. Each call goes to an agent worker (python -m miles.rollout.agentic.worker, aiohttp) at--custom-agent-function-worker-url, which runs it through the existingsubprocpath on its own host. Harbor is imported in-process on the worker, as undersubproc, and Harbor itself is unchanged.miles/rollout/agentic/remote.py). Every worker response comes within 25 s, so a proxy that cuts slow responses can sit in between.POST /callstakes{call_id, fn "module:qualname", JSON kwargs, env, scope}and is idempotent percall_id.GET /calls/{id}?after&wait≤25returns the child'sreport_to_calleritems, then the outcome: an ok value, anInfraAbortwith itsexit_status, or an error with its traceback.DELETE /calls/{id}cancels the call; its child gets SIGTERM, as a cancelledsubproccall's does.--keep-envkeeps the host's own value for named vars, such as paths that differ there.--orphan-timeout(120 s) is cancelled.--custom-agent-function-worker-envnames the env vars sent with each call: by defaultHARBOR_*,AGENT_*,LITE_*and each provider's key-file and address vars fromPROVIDER_CREDENTIALS. Names ending inSECRET,SECRET_KEY,TOKEN,PASSWORDorAPI_KEY, and every providerkey_env_varsname, are never sent by value; a*_FILEpath is.InfraAbort("WorkerUnreachable"), and a call the worker no longer has isInfraAbort("WorkerLost"). P9 discards both as infrastructure failures.generatebefore a session opens, not as a per-sample agent error.call_agent_abort_hook, which the oversampling abort and the hot-restart takeover both call, first cancels this process's worker calls. Every case ends the trial: an admitted call is cancelled by its DELETE; a submit the DELETE overtakes is refused by the tombstone; a call whose DELETE is lost stops being polled, so the worker's orphan timer ends it.sglang_router_ip:port) asabort_scope, and the hook cancels only that scope's calls. The dedicated eval engines (--eval-num-gpus) generate from the same process through their own router, so training's oversampling abort no longer cancels eval's calls. The hot-restart takeover passesevery_scope=True. Without this, a Harbor smoke lost 14–21 of 64 eval episodes asWorkerCancelled. The hook reads the router only when worker calls are in flight, sosubprocandinlineruns are unchanged.TrainStepOutcomemoves tomiles/backends/training_utils/outcome.py(re-exported fromtypes.py), andInfraAbortand the traceback wrapper move tomiles/rollout/agentic/errors.py(re-exported fromagent_function.py). The worker then imports without torch, sglang or ray, andremote.pyimports them without a cycle.--agent-worker-urlonexamples/experimental/harbor/run.pypasses the worker flags.harbor_env_vars(..., agent_worker=True)callsprovision_provider(remote=True), which skips the local key supply and SDK check but still forwards the provider's endpoint vars. The README gains a section on running the trials on a CPU box._Inflight,AgentWorkerEndpoint,_Call,WorkerConfig) are module dicts, plain dicts and keyword arguments here.Tests (EC2 CPU VM, py3.12, the CI CPU requirements, SGLang
sglang-milesb0ed761650):tests/fast/rollout/agentic/test_worker.py(22 tests, ~19 s) runs a real worker on a local port: child process, env forwarding and--keep-env, scope, ordered reports,InfraAbortand error passthrough, cancel, rollout abort through the hook and throughinference_rollout_train.abort, abort scoping, the tombstone, an abort during a held submit, a lost DELETE reaped by the orphan timer, secrets, resubmit, lost and unreachable workers, and the light import._reachesforced toTrue,test_a_rollout_abort_leaves_another_rollouts_calls_runningfails onassert still_running.test_a_rollout_abort_cancels_the_calls_running_on_workers,test_a_rollout_abort_leaves_another_rollouts_calls_runningandtest_the_rollout_abort_entrypoint_cancels_a_trial_waiting_on_its_environmentfail withTimeoutError.test_agentic_v2.pygains two tests: worker mode passes the URL, the forwarded env and the router scope to the call, and worker mode without a URL raises before a session opens.test_launch_common.pygains two for the launcher.test_hot_restart.py's fake hooks take the new keyword.pytest -n 6 tests/fast/rollout/agentic tests/fast/examples/experimental/harbor tests/fast/utils/test_hot_restart.py tests/fast/utils/test_agent_abort_hook.py tests/fast/utils/test_typed_agent_abort.py tests/fast/utils/test_misc.py tests/fast/utils/event_logger tests/fast/rollout/generate_hub tests/fast/rollout/inference_rollout tests/fast/rollout/test_sglang_rollout.py tests/fast/test_train.py tests/fast/test_train_async.py: head 594 passed, 17 skipped, 6 failed; base 569 passed, 17 skipped, 5 failed. The 5 shared failures aregenerate_hub/test_multi_turn.py'sagentic_tool_callcases, the same on the base. The sixth,inference_rollout/integration/test_multi_turn.py::test_rollout[eval-agentic_tool_call], lost a port race under-n 6(address already in use) and passes serially at head.pre-commit runon the changed files: passed.Limits.
--keep-envnames them.Related upstream work. None found for running agent functions on another host. This route supersedes the agent-server drafts #30 and celve/harbor#1.
Origin: #40 (merge
2782de698d), plus theabort_scopepart of #43 (merge6e77a81cb3), both onLIN-1985/investigate-existing-recipes-for(LIN-2510). #43's recipe defaults, per-router recorder and legacy-eval refusal are left out, and so are #40's Qwen3-VL CUA launcher changes, since that launcher is not upstream. Live 8×8 of LIN-2399's Harbor recipe at #40's head, worker on an EC2 m7i.8xlarge: turn 4.12 s → 4.09 s, GPU-box harness 3.17 → 0 cores, 17.5 CPU-s of worker per episode, 0 InfraAbort.