SWE Agents Opencode update - #2817
Merged
Merged
Conversation
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
Signed-off-by: Sugam Devare <sdevare@nvidia.com>
bxyu-nvidia
approved these changes
Aug 27, 2026
yfw
pushed a commit
that referenced
this pull request
Sep 1, 2026
Replace responses_api_agents/swe_agents with the upstream NVIDIA-NeMo/Gym tree at 11a9391 ("SWE Agents Opencode update", #2817), fast-forwarding the directory from the pinned submodule base 2251ef7. Brings in the two upstream commits that touch this directory: - 9a87438 [rollout-observability][7/7] Add SWE agent rollout observations (#2120) - 11a9391 SWE Agents Opencode update (#2817) New capabilities from #2817: - opencode_patch_mode (worktree|committed, default worktree). The committed mode diffs the pre-run HEAD against the most-advanced commit the agent left behind, which DeepSWE-style prompts require -- those tell the agent to commit, so the tree is clean at the end and worktree mode records 0-byte patches. - opencode trajectory replay via the new opencode_replay.py, letting a rollout resume a partially-completed trajectory on a fresh container. Notes: - The directory had no local modifications at 2251ef7, so this is a clean fast-forward of upstream code, not a merge. - #2817 deletes observability.py and test_observability.py that #2120 added; the resulting app.py has no ng_agent_observations references, so the change is self-contained and does not depend on the resources_servers/swebench changes #2120 also made. - configs/, prompts/, setup_scripts/ and swe_bench_ext/ are unchanged upstream and opencode_patch_mode defaults to today's behavior, so existing NeMo-RL configs referencing this agent are unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Sugam Devare <sdevare@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds two capabilities to the SWE agent harness (
responses_api_agents/swe_agents/), plus fixes tothe DeepSWE verifier path found while baselining that dataset.
1. opencode patch mode. New
opencode_patch_modeconfig field (worktree|committed,default
worktree), exported to the harness asPATCH_MODEand forwarded to the bench CLI as--patch-mode.worktreeis the existing capture —git diffof the working tree with untrackedfiles marked intent-to-add — and is unchanged.
committedinstead diffs the pre-run HEAD againstthe most advanced commit the agent left behind, searched across HEAD and every local branch. That
mode is required by task families whose problem statement tells the agent to commit its solution:
DeepSWE's statements end with "work on this in a new branch from main and commit everything when
you are done", so those rollouts finish with a clean tree and
git diffrecords every patch as0 bytes regardless of whether the model solved the task. Both modes emit a plain
base -> final treeunified diff, so the eval side is unchanged.2. opencode trajectory replay. New
opencode_replay.pymodule plus wiring, so a rollout canresume a partially-completed trajectory on a fresh container instead of restarting the task. When a
request's
inputcarries a prior trajectory (function_call/function_call_outputitems beyondthe seed messages), it is converted to chat-completion format and surfaced as
problem_info["replay_messages"]; the processors materialize it (plus the recorded system promptand, for opencode, a subagent manifest) and forward the paths to
run_infer.shas positional args(opencode
#13/#14, openhands#18). Recorded subagent sessions are linked to the exact parenttask call that spawned them rather than by metadata order, so parallel siblings and nested agents
can't consume each other's turns, and live continuations are merged back onto the recorded root.
Per-session trajectory records now also carry replay linkage and global ordering
(
recorded_session_id,spawn_call_id,spawn_index,global_turn, …), andSWEBenchVerifyResponse.subagent_trajectoriesis populated so returned create-params are directlyreplay-ready.
3. DeepSWE verifier fixes.
tests/grader.py/tests/config.jsonfrom Harbor bundles (DeepSWE v1.1 delegatespreparation and scoring to them) and mount them into the eval container. Optional, so older
bundles still convert and grade.
/logs/verifier/reward.json(new tasks) as well asreward.txt(older tasksand crash sentinels).
mkdir -p /logs/vserifier→/logs/verifier; the typo meant the directory the verifier writesits reward into was never created, so the reward read back empty and the task graded 0.
mvn -o/--offline) against their image's build-time JVM cache — Maven records which repository suppliedeach cached artifact, so a differently-named mirror makes cached plugins look unavailable.
4. Reverts #2120 (SWE agent rollout observations), removing
observability.py,tests/test_observability.pyand their call sites.Tests:
responses_api_agents/swe_agents/tests/test_app.pygains 26 tests covering the replay path —gym-side message conversion (openhands and opencode), replay file materialization and system-prompt
pinning, positional-arg ordering, subagent manifest mounting with
ENABLE_SUBAGENTS=1,parent-task-call linkage for parallel and nested children, completed-invocation truncation, legacy
payload parsing, live-continuation merging, and preservation of the new trajectory record fields.
Note: end-to-end opencode replay also needs
--replay-messages-file/--replay-subagents-fileinthe opencode fork. The commit currently pinned by
configs/swebench_opencode.yaml(
nv-opencode@sdd/dev) does not have them — itsrun_infer.shstops at positional#12— so untilthat lands and the pin moves, an opencode replay request is a no-op and the agent starts the task
from scratch. openhands replay is unaffected. Bumping the pin also requires clearing
cache/swe_agents/swe_opencode_setup, sinceOpenCodeHarnessProcessor.setup()returns early oncethe tree exists and never re-checks-out the configured commit.
Checklist
pre-commit run --all-files) (so CI lint/format/copyright pass).git commit -s) (so the DCO check passes).