Skip to content

agent functions: one failure contract for the OpenEnv, NeMo Gym and Harbor legs - #2802

Draft
nblintao wants to merge 57 commits into
tao/agent-fn-infra-abortfrom
tao/agent-fn-failure-semantics
Draft

agent functions: one failure contract for the OpenEnv, NeMo Gym and Harbor legs#2802
nblintao wants to merge 57 commits into
tao/agent-fn-infra-abortfrom
tao/agent-fn-failure-semantics

Conversation

@nblintao

@nblintao nblintao commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Stacked on #2801.

Purpose

The OpenEnv, NeMo Gym and Harbor agent functions ended failed episodes three different ways that all trained as reward 0 without saying why — and OpenEnv's comment claimed returning None discards the sample, which it never did. Give the three legs one failure contract.

What

One rule, applied in place in each leg (no shared module yet — this makes the copies identical so a later extraction is mechanical):

  • Discard (InfraAbort) only what the policy cannot have caused: sandbox create failed past the retry budget (SandboxUnavailable), agent/env server unreachable (ServerUnreachable), a tbench2 server without the canonical scoring contract (NonCanonicalVerifier).
  • Score 0 with a named exit_status for everything else, wall-clock timeouts included (TimeLimitExceeded) — a discarded timeout is an exit the policy can learn to take.
  • One exit_status vocabulary across the legs (Harbor's set plus the three discard causes), documented next to the who-can-cause-what table.

Behaviour change

  • New discards: an unreachable agent/env server after the policy started, and OpenEnv episodes scored by a non-canonical server — both previously trained as false-negative 0s.
  • Samples that were already reward 0 stay reward 0, now with an exit_status the dashboards can split on.
  • OpenEnv's exit_status strings change (timeoutTimeLimitExceeded, completedSubmitted).

Xinyu-Kang and others added 30 commits August 20, 2026 14:05
Co-authored-by: Minhao Li <edison@Minhaos-MacBook-Air.local>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
…irectives (#2674)

Co-authored-by: guapisolo <guapisolo@gmail.com>
… backend (#1313)

Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: maocheng23 <35615230+maocheng23@users.noreply.github.com>
Co-authored-by: guapisolo <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Zhichen Zeng <zhichen.zeng@radixark.ai>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
Shi-Dong and others added 17 commits August 29, 2026 00:02
Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
@nblintao
nblintao force-pushed the tao/agent-fn-failure-semantics branch from 12e4c99 to 7a9a107 Compare August 31, 2026 21:43
nblintao and others added 2 commits August 31, 2026 17:44
An agentic episode can end without a verdict for a reason the policy had
nothing to do with (the sandbox platform refused to create a sandbox, the
environment host died). Today it is trained on as reward 0: returning None
from the agent function means "no metadata to merge", and any exception is
swallowed, so the recorded session becomes a sample either way.

Add miles.rollout.agent_function.InfraAbort(exit_status): raising it from an
agent function marks the sample ABORTED with metadata["exit_status"]. It is
reserved for failures the policy cannot cause; anything it can cause
(timeouts, a sandbox it broke) must stay reward 0, or the policy learns to
trigger it to escape the penalty. None and other exceptions keep today's
meaning.

For the discard to hold, the sync rollout loops now drop a group containing
an ABORTED sample regardless of the dynamic filter, as the fully-async
buffer already did. rollout/aborted/drop_<exit_status> counts the drops by
cause on all three paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arbor legs

The three legs ended failed episodes three different ways that meant the
same thing (reward 0 without saying why), and OpenEnv's "drop the sample"
comment described a discard that never happened (None is not a discard).

Apply one rule in place in each leg: discard (InfraAbort) only what the
policy cannot have caused -- a sandbox that could not be created, an
unreachable agent/env server, a tbench2 server without the canonical
scoring contract; score 0 with a named exit_status for everything else,
including wall-clock timeouts, so no outcome the policy can trigger becomes
an exit from the penalty. The exit_status vocabulary is Harbor's plus the
three discard causes and is documented in the agentic-rollout guide.

No shared module yet: this makes the three copies say the same thing so
extracting one later is mechanical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.