Skip to content

SWE Agents Opencode update - #2817

Merged
bxyu-nvidia merged 6 commits into
mainfrom
sdd/swe-agent-opencode-super35
Aug 27, 2026
Merged

SWE Agents Opencode update#2817
bxyu-nvidia merged 6 commits into
mainfrom
sdd/swe-agent-opencode-super35

Conversation

@sdevare-nv

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds two capabilities to the SWE agent harness (responses_api_agents/swe_agents/), plus fixes to
the DeepSWE verifier path found while baselining that dataset.

1. opencode patch mode. New opencode_patch_mode config field (worktree | committed,
default worktree), exported to the harness as PATCH_MODE and forwarded to the bench CLI as
--patch-mode. worktree is the existing capture — git diff of the working tree with untracked
files marked intent-to-add — and is unchanged. committed instead diffs the pre-run HEAD against
the most advanced commit the agent left behind, searched across HEAD and every local branch. That
mode is required by task families whose problem statement tells the agent to commit its solution:
DeepSWE's statements end with "work on this in a new branch from main and commit everything when
you are done"
, so those rollouts finish with a clean tree and git diff records every patch as
0 bytes regardless of whether the model solved the task. Both modes emit a plain
base -> final tree unified diff, so the eval side is unchanged.

2. opencode trajectory replay. New opencode_replay.py module plus wiring, so a rollout can
resume a partially-completed trajectory on a fresh container instead of restarting the task. When a
request's input carries a prior trajectory (function_call / function_call_output items beyond
the seed messages), it is converted to chat-completion format and surfaced as
problem_info["replay_messages"]; the processors materialize it (plus the recorded system prompt
and, for opencode, a subagent manifest) and forward the paths to run_infer.sh as positional args
(opencode #13/#14, openhands #18). Recorded subagent sessions are linked to the exact parent
task call that spawned them rather than by metadata order, so parallel siblings and nested agents
can't consume each other's turns, and live continuations are merged back onto the recorded root.
Per-session trajectory records now also carry replay linkage and global ordering
(recorded_session_id, spawn_call_id, spawn_index, global_turn, …), and
SWEBenchVerifyResponse.subagent_trajectories is populated so returned create-params are directly
replay-ready.

3. DeepSWE verifier fixes.

  • Carry optional tests/grader.py / tests/config.json from Harbor bundles (DeepSWE v1.1 delegates
    preparation and scoring to them) and mount them into the eval container. Optional, so older
    bundles still convert and grade.
  • Read the reward from /logs/verifier/reward.json (new tasks) as well as reward.txt (older tasks
    and crash sentinels).
  • Fix mkdir -p /logs/vserifier/logs/verifier; the typo meant the directory the verifier writes
    its reward into was never created, so the reward read back empty and the task graded 0.
  • Skip injecting the online Maven mirror for verifiers that deliberately run offline (mvn -o /
    --offline) against their image's build-time JVM cache — Maven records which repository supplied
    each cached artifact, so a differently-named mirror makes cached plugins look unavailable.

4. Reverts #2120 (SWE agent rollout observations), removing observability.py,
tests/test_observability.py and their call sites.

Tests: responses_api_agents/swe_agents/tests/test_app.py gains 26 tests covering the replay path —
gym-side message conversion (openhands and opencode), replay file materialization and system-prompt
pinning, positional-arg ordering, subagent manifest mounting with ENABLE_SUBAGENTS=1,
parent-task-call linkage for parallel and nested children, completed-invocation truncation, legacy
payload parsing, live-continuation merging, and preservation of the new trajectory record fields.

Note: end-to-end opencode replay also needs --replay-messages-file / --replay-subagents-file in
the opencode fork. The commit currently pinned by configs/swebench_opencode.yaml
(nv-opencode@sdd/dev) does not have them — its run_infer.sh stops at positional #12 — so until
that lands and the pin moves, an opencode replay request is a no-op and the agent starts the task
from scratch. openhands replay is unaffected. Bumping the pin also requires clearing
cache/swe_agents/swe_opencode_setup, since OpenCodeHarnessProcessor.setup() returns early once
the tree exists and never re-checks-out the configured commit.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format/copyright pass).
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
@bxyu-nvidia
bxyu-nvidia merged commit 11a9391 into main Aug 27, 2026
36 checks passed
@bxyu-nvidia
bxyu-nvidia deleted the sdd/swe-agent-opencode-super35 branch August 27, 2026 16:00
yfw pushed a commit that referenced this pull request Sep 1, 2026
Replace responses_api_agents/swe_agents with the upstream NVIDIA-NeMo/Gym
tree at 11a9391 ("SWE Agents Opencode update", #2817), fast-forwarding the
directory from the pinned submodule base 2251ef7.

Brings in the two upstream commits that touch this directory:
- 9a87438 [rollout-observability][7/7] Add SWE agent rollout observations (#2120)
- 11a9391 SWE Agents Opencode update (#2817)

New capabilities from #2817:
- opencode_patch_mode (worktree|committed, default worktree). The committed
  mode diffs the pre-run HEAD against the most-advanced commit the agent left
  behind, which DeepSWE-style prompts require -- those tell the agent to commit,
  so the tree is clean at the end and worktree mode records 0-byte patches.
- opencode trajectory replay via the new opencode_replay.py, letting a rollout
  resume a partially-completed trajectory on a fresh container.

Notes:
- The directory had no local modifications at 2251ef7, so this is a clean
  fast-forward of upstream code, not a merge.
- #2817 deletes observability.py and test_observability.py that #2120 added;
  the resulting app.py has no ng_agent_observations references, so the change
  is self-contained and does not depend on the resources_servers/swebench
  changes #2120 also made.
- configs/, prompts/, setup_scripts/ and swe_bench_ext/ are unchanged upstream
  and opencode_patch_mode defaults to today's behavior, so existing NeMo-RL
  configs referencing this agent are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants