feat(agents): sandbox-bound CLI agents (Codex, Claude Code, custom) with capture + in-box SWE-bench grading

[Glorf]

Summary

A reusable layer for running CLI coding agents inside a Gym sandbox and collecting RL-ready trajectories, on the sandbox-provider + adapter framework. Benchmark-agnostic — only the task metadata and the grader change.

• SandboxCliAgent owns the per-rollout lifecycle: start a sandbox -> per-rollout capture proxy -> install + run the CLI in-box -> collect the git patch -> assemble the trajectory -> grade. A new agent is a ~30-line subclass or a YAML manifest.
• Agents: codex_swe_agent (Responses wire), claude_code_swe_agent (Messages wire via the translate_anthropic interceptor -> any OpenAI-compatible backend), custom_agent (manifest-driven).
• Adapters are sandbox-bound (per rollout): each rollout spins its own capture proxy keyed to a session_id; the in-box *_BASE_URL points at it; torn down on exit. CaptureStore writes durable per-session JSONL (token-ids on a Gym model server). choose_trajectory prefers a well-formed (paired tool calls) trajectory, so both codex and Claude Code produce healthy trajectories.
• In-box grading (the only benchmark-specific seam): SWE-bench via the official swebench harness; Terminal-Bench / Harbor by staging the task's tests, running test.sh, and reading the verifier reward file; lightweight pytest membership fallback.

How to run

Per agent — start the agent server, then drive a dataset (swap codex_swe_agent for claude_code_swe_agent / custom_agent):

ng_run "+config_paths=[responses_api_agents/codex_swe_agent/configs/codex_swe_agent.yaml]" \
  +policy_base_url=https://<endpoint> ++codex_swe_agent.responses_api_agents.codex_swe_agent.model=<model-id>
# NEMO_GYM_MODEL_API_KEY = the real upstream key; the in-box agent only ever sees a dummy
# (the per-rollout capture proxy injects the real key on the way upstream).

ng_collect_rollouts +agent_name=codex_swe_agent \
  +input_jsonl_fpath=<dataset>.jsonl +output_jsonl_fpath=rollouts.jsonl +limit=N

Per task — only responses_create_params.metadata differs:

• SWE-bench — metadata.instance_dict (repo, version, base_commit, test_patch, FAIL_TO_PASS, PASS_TO_PASS); image resolves __->_1776_; graded by the swebench harness. Example: responses_api_agents/codex_swe_agent/data/example.jsonl.
• Terminal-Bench / Harbor — metadata.docker_image (public, auto-mirrored to ECR) + metadata.harbor_tests ({"/tests/test.sh": "...", "/tests/test_outputs.py": "..."}); graded by running test.sh and reading the verifier reward file. Example: responses_api_agents/codex_swe_agent/data/terminal_bench_example.jsonl.

ng_collect_rollouts +agent_name=codex_swe_agent \
  +input_jsonl_fpath=responses_api_agents/codex_swe_agent/data/terminal_bench_example.jsonl \
  +output_jsonl_fpath=tb.jsonl +limit=1

Build full datasets: SWE-bench via HF princeton-nlp/SWE-bench_Verified; Terminal-Bench via harbor datasets download terminal-bench@2.0 (each task's instruction.md + task.toml docker_image + tests/). See the agent READMEs for the full per-agent/per-task matrix.

Validated on real ECS Fargate

• SWE-bench Verified: golden patch -> 1.0, no patch -> 0.0; Codex + Claude Code each solve real instances; 10 diverse repos via ng_run + ng_collect_rollouts.
• Terminal-Bench 2.0 (Harbor): 5 tasks x Codex + Claude Code — multiple solves (reward 1.0 via the verifier reward file), the grader discriminates, and trajectories are healthy (paired tool calls) for both agents.

Dependencies (stacked PRs)

Stacked on the open base PRs below — its diff includes their changes until they land on main, so review/merge after them. Opened as a draft for that reason.

Sandbox

• Sandbox provider framework — NVIDIA-NeMo/Gym#1377
• ECS Fargate sandbox provider — hemildesai/Gym#2 (Glorf:feat/ecs-fargate-sandbox)

Adapters / interceptors

• Adapter middleware framework (base) — NVIDIA-NeMo/Gym#1518 (Glorf:feat/adapter-base)

Related interceptor families on the same feat/adapter-base (siblings, not required here): caching Glorf/Gym#2, observability Glorf/Gym#3, request-rewriting Glorf/Gym#4.

Test plan

• uv run pytest tests/unit_tests/test_sandbox_cli_agent.py tests/unit_tests/test_adapter_capture.py tests/unit_tests/test_adapter_translate_anthropic.py responses_api_agents/codex_swe_agent/tests responses_api_agents/claude_code_swe_agent/tests responses_api_agents/custom_agent/tests
• (needs AWS + model creds) graded SWE-bench + Terminal-Bench rollouts on ECS Fargate per the run commands above

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-feat-sandbox-cli-agents.docs.buildwithfern.com/nemo/gym

Here are the markdown pages you've updated:

• nemo/gym/model-server/adapters
• nemo/gym/model-server

[Glorf]

Superseded by an upstream-head PR so it can join the GitHub stack (fork-head PRs can't be stacked).