agent functions: one failure contract for the OpenEnv, NeMo Gym and Harbor legs - #2802
Draft
nblintao wants to merge 57 commits into
Draft
agent functions: one failure contract for the OpenEnv, NeMo Gym and Harbor legs#2802nblintao wants to merge 57 commits into
nblintao wants to merge 57 commits into
Conversation
Co-authored-by: Minhao Li <edison@Minhaos-MacBook-Air.local>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
…2675) Co-authored-by: guapisolo <guapisolo@gmail.com>
…irectives (#2674) Co-authored-by: guapisolo <guapisolo@gmail.com>
… backend (#1313) Co-authored-by: yueming-yuan <yym022502@gmail.com> Co-authored-by: maocheng23 <35615230+maocheng23@users.noreply.github.com> Co-authored-by: guapisolo <guapisolo@gmail.com>
…n-test under label approval (#2718)
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com> Co-authored-by: Zhichenzzz <zczeng@uw.edu>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Zhichen Zeng <zhichen.zeng@radixark.ai> Co-authored-by: Zhichenzzz <zczeng@uw.edu>
#2768) Co-authored-by: Cursor Agent <cursoragent@cursor.com>
nblintao
force-pushed
the
tao/agent-fn-failure-semantics
branch
from
August 28, 2026 22:47
36f96e1 to
12e4c99
Compare
This was referenced Aug 28, 2026
…#2142) Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
nblintao
force-pushed
the
tao/agent-fn-failure-semantics
branch
from
August 31, 2026 21:43
12e4c99 to
7a9a107
Compare
An agentic episode can end without a verdict for a reason the policy had nothing to do with (the sandbox platform refused to create a sandbox, the environment host died). Today it is trained on as reward 0: returning None from the agent function means "no metadata to merge", and any exception is swallowed, so the recorded session becomes a sample either way. Add miles.rollout.agent_function.InfraAbort(exit_status): raising it from an agent function marks the sample ABORTED with metadata["exit_status"]. It is reserved for failures the policy cannot cause; anything it can cause (timeouts, a sandbox it broke) must stay reward 0, or the policy learns to trigger it to escape the penalty. None and other exceptions keep today's meaning. For the discard to hold, the sync rollout loops now drop a group containing an ABORTED sample regardless of the dynamic filter, as the fully-async buffer already did. rollout/aborted/drop_<exit_status> counts the drops by cause on all three paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arbor legs The three legs ended failed episodes three different ways that meant the same thing (reward 0 without saying why), and OpenEnv's "drop the sample" comment described a discard that never happened (None is not a discard). Apply one rule in place in each leg: discard (InfraAbort) only what the policy cannot have caused -- a sandbox that could not be created, an unreachable agent/env server, a tbench2 server without the canonical scoring contract; score 0 with a named exit_status for everything else, including wall-clock timeouts, so no outcome the policy can trigger becomes an exit from the penalty. The exit_status vocabulary is Harbor's plus the three discard causes and is documented in the agentic-rollout guide. No shared module yet: this makes the three copies say the same thing so extracting one later is mechanical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
nblintao
force-pushed
the
tao/agent-fn-failure-semantics
branch
from
August 31, 2026 21:46
7a9a107 to
2462c4c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2801.
Purpose
The OpenEnv, NeMo Gym and Harbor agent functions ended failed episodes three different ways that all trained as
reward 0without saying why — and OpenEnv's comment claimed returningNonediscards the sample, which it never did. Give the three legs one failure contract.What
One rule, applied in place in each leg (no shared module yet — this makes the copies identical so a later extraction is mechanical):
InfraAbort) only what the policy cannot have caused: sandbox create failed past the retry budget (SandboxUnavailable), agent/env server unreachable (ServerUnreachable), a tbench2 server without the canonical scoring contract (NonCanonicalVerifier).exit_statusfor everything else, wall-clock timeouts included (TimeLimitExceeded) — a discarded timeout is an exit the policy can learn to take.exit_statusvocabulary across the legs (Harbor's set plus the three discard causes), documented next to the who-can-cause-what table.Behaviour change
reward 0stayreward 0, now with anexit_statusthe dashboards can split on.exit_statusstrings change (timeout→TimeLimitExceeded,completed→Submitted).