[celve]

Layer P9 of the CUA upstreaming stack. It is built and reviewed in celve, and goes upstream with the whole stack once every layer is finished. Base: main (radixark/miles 6114c3a10f). Issue: LIN-2516, hub LIN-2504.

Problem. An agentic episode that failed because the sandbox or the model service broke trains as a policy failure. An agent function can only return or raise, and either way the session it recorded is scored, usually 0, so GRPO pushes down whatever the policy did in an episode the infrastructure broke. When aborted groups are dropped, nothing says why. And evaluation crashes when every attempt of a dataset aborted.

Fix.

• InfraAbort(exit_status, message) in miles/rollout/agentic/agent_function.py. An agent function raises it to mark its sample ABORTED, with the cause in metadata["exit_status"]. The cause survives the subproc boundary. Miles's own session-path aborts now record a cause too: NoModelCalls, CollectFailed, AllTruncated. If collecting the samples also fails after an InfraAbort, the agent's cause wins. If collection succeeded, the collected samples' output-store refs are handed to record_dropped_samples.
• No new filter or metric family. The existing apply_aborted_filter already drops a group with an aborted sample before any dynamic filter runs. It now names the cause in its reason, group_has_aborted_<exit_status>, taken from the first aborted sample, or unknown. The existing rollout/dynamic_filter/drop_<reason> metric is now the one count of aborted groups, by cause. The fully-async buffer's aborted-group drop reports through the same on_dynamic_filter_drop(reason), and its separate aborted_groups_filtered counter is removed, so sync and fully async count aborts the same way.
• Harbor example. Rule: discard only what the policy can't cause. _discard_cause discards a trial that failed before its agent started (SandboxUnavailable or AgentSetupFailed), and one whose in-process agent lost the model service (litellm's APIConnectionError / ServiceUnavailableError → ServerUnreachable). Multi-step trials are checked too. Timeouts, turn and length limits, and verifier crashes stay scored 0. Installed agents' Api* errors also stay scored 0, because Harbor gets them by regex over the agent's own output, which the policy writes.
• Evaluation treats an aborted attempt as upstream treats any attempt with reward=None. The reward model skips ABORTED samples, so their reward stays None, and the existing code already scores them 0, counts them in eval/<ds>-none_reward_ratio and logs its warning ("likely errored/aborted trials"). Nothing is excluded from the score or from pass@k, and no eval metric is added. Two fixes remain in metrics.py:
  • The crash. On main, a dataset whose every attempt aborted raises TypeError: type NoneType doesn't define __round__ method in _compute_zero_std_metrics: all-None rewards look like a zero-std group, and their None gets rounded. _is_zero_std now returns False for a group with a None reward. This is the same one-line fix as @dzorlu's Eval metrics: a group with an aborted episode no longer crashes the zero-std count radixark/miles#4518.
  • pass@k grouping, a separate bug on main: pass@k is grouped by the global --n-samples-per-eval-prompt even when the dataset sets its own n_samples_per_eval_prompt. With 4 per dataset and 3 globally, compute_pass_rate asserts (len(flat_rewards)=8 num_groups=2 group_size=3). With 2 globally, it silently reports pass@1/pass@2 over the wrong groups. _eval_samples_per_prompt takes the dataset's own count.
• Docs: agentic-rollout.md documents InfraAbort and gains a "Which failures to discard" table, and fully-async.md documents the per-cause drop reason.

Metric rename, for chart users.

• Fully async: rollout/fully_async/aborted_groups_filtered → the sum of rollout/dynamic_filter/drop_group_has_aborted_*.
• Sync: rollout/dynamic_filter/drop_group_has_aborted → rollout/dynamic_filter/drop_group_has_aborted_<cause>, which is ..._unknown when the sample recorded no cause.

Removed by Linyu's call: the --max-consecutive-aborted-groups streak guard from #7, its separate rollout/aborted/drop_<cause> metric, and its evaluation exclusion (aborted_ratio, aborted/<cause>, attempt-based pass@k, no score for an all-aborted dataset).

Tests (CPU devbox, SGLang ~/sglang, Megatron-LM, pytest threads capped at 2):

• Regression, with the test changes applied to main 6114c3a10f and no fix: pytest tests/fast/ray/rollout/test_metrics.py tests/fast/rollout/test_filters.py tests/fast/rollout/test_fully_async_data_buffer.py tests/fast/rollout/test_fully_async_rollout.py tests/fast/rollout/inference_rollout/test_sample_completion_backfill.py gives 17 failed, 280 passed. The failures: the filter's reason carries no cause (test_filters.py ×5); the sync loop reports only drop_group_has_aborted (test_sample_completion_backfill.py ×2); the fully-async buffer counts aborts in aborted_groups_filtered rather than as a drop reason (test_fully_async_data_buffer.py ×3, test_fully_async_rollout.py ×5); a dataset whose every attempt aborted crashes evaluation with round(None); and pass@k is grouped by the global sample count (test_metrics.py ×2). All pass on this head.
• On this head (f04ff77a6b): pytest tests/fast/ray/rollout/test_metrics.py tests/fast/rollout/{test_filters.py,test_fully_async_data_buffer.py,test_fully_async_rollout.py,inference_rollout,generate_hub,agentic} tests/fast/utils/test_arguments.py tests/fast/examples/experimental/{harbor,hud} gives 1,122 passed, 23 skipped, 5 failed. The 5 failures are test_multi_turn.py[agentic_tool_call], which fail the same way on main 6114c3a10f (a session_id metadata mismatch).
• pre-commit run on the changed files: passed, with no files changed.

Limits.

• An outage still draws new prompts for as long as it lasts. The drop counts show it, but nothing stops the rollout.
• The HUD example is unchanged. Upstream's HUD code moved to GenerateFnOutput and gained contract tests, which require every early return to be a FAILED write-off with remove_sample. The source's HUD hunks would contradict them, so they were dropped (Discard infrastructure failures in agentic training and evaluation #7 marked HUD runs that never launched, or whose first model request failed, as ABORTED). Under remove_sample, such a sample is left out of the loss but still counts at 0 in its group's baseline.
• Which Harbor failures count as infrastructure is an allow-list of exception names. A new provider error type stays scored 0 until someone adds it.

Related upstream work.

• Tao Lin's (@nblintao) rollout: let agent functions discard infra-aborted samples radixark/miles#2801 and agent functions: one failure contract for the OpenEnv, NeMo Gym and Harbor legs radixark/miles#2802 (open, last updated 2026-10-02) define the same InfraAbort(exit_status) interface and failure vocabulary. This PR keeps that interface by design so the two can converge, and credits him for it.
• @nanjiangwill's fix(harbor): discard infrastructure-only failures radixark/miles#3617 maps Harbor's typed Api* errors for installed agents to ServerUnreachable. This PR does not, because those errors come from a regex over output the policy writes. It discards only litellm's exceptions from in-process agents, plus failures before the agent started.
• @dzorlu's Eval metrics: a group with an aborted episode no longer crashes the zero-std count radixark/miles#4518 makes the same _is_zero_std change for the same round(None) crash. Whichever lands first, the other drops that hunk. The regression test here drives log_eval_rollout_data with a dataset whose every attempt aborted.

Origin: #7 (merge e47e96e027), commits 64e3f21d6b, 28479341e7, 0be4b3cdb7, 4e82cfa0d4, 5f53498151, f255defc33, squashed and ported onto main, then reworked to reuse the existing aborted filter.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T10:48:05.644536Z	faf94db	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.