VLLMModel docs in main Readme - #13
Merged
Merged
Conversation
Signed-off-by: Brian Yu <bxyu@nvidia.com>
soares-f
pushed a commit
that referenced
this pull request
Sep 8, 2025
Signed-off-by: Brian Yu <bxyu@nvidia.com> Signed-off-by: soares-f <soarescmsa@gmail.com>
soares-f
pushed a commit
that referenced
this pull request
Sep 15, 2025
Signed-off-by: Brian Yu <bxyu@nvidia.com> Signed-off-by: soares-f <soarescmsa@gmail.com>
abhibha-nvidia
pushed a commit
that referenced
this pull request
Sep 28, 2025
Signed-off-by: Brian Yu <bxyu@nvidia.com> Signed-off-by: Abhibha Gupta <abhibhag@nvidia.com>
abhibha-nvidia
pushed a commit
that referenced
this pull request
Sep 29, 2025
Signed-off-by: Brian Yu <bxyu@nvidia.com> Signed-off-by: Abhibha Gupta <abhibhag@nvidia.com>
abubakaria56
pushed a commit
to abubakaria56/Gym
that referenced
this pull request
Mar 2, 2026
Signed-off-by: Brian Yu <bxyu@nvidia.com>
abubakaria56
pushed a commit
to abubakaria56/Gym
that referenced
this pull request
Mar 2, 2026
Signed-off-by: Brian Yu <bxyu@nvidia.com>
bxyu-nvidia
pushed a commit
that referenced
this pull request
Aug 27, 2026
## What does this PR do? Adds two capabilities to the SWE agent harness (`responses_api_agents/swe_agents/`), plus fixes to the DeepSWE verifier path found while baselining that dataset. **1. opencode patch mode.** New `opencode_patch_mode` config field (`worktree` | `committed`, default `worktree`), exported to the harness as `PATCH_MODE` and forwarded to the bench CLI as `--patch-mode`. `worktree` is the existing capture — `git diff` of the working tree with untracked files marked intent-to-add — and is unchanged. `committed` instead diffs the pre-run HEAD against the most advanced commit the agent left behind, searched across HEAD and every local branch. That mode is required by task families whose problem statement tells the agent to commit its solution: DeepSWE's statements end with *"work on this in a new branch from main and commit everything when you are done"*, so those rollouts finish with a clean tree and `git diff` records every patch as 0 bytes regardless of whether the model solved the task. Both modes emit a plain `base -> final tree` unified diff, so the eval side is unchanged. **2. opencode trajectory replay.** New `opencode_replay.py` module plus wiring, so a rollout can resume a partially-completed trajectory on a fresh container instead of restarting the task. When a request's `input` carries a prior trajectory (`function_call` / `function_call_output` items beyond the seed messages), it is converted to chat-completion format and surfaced as `problem_info["replay_messages"]`; the processors materialize it (plus the recorded system prompt and, for opencode, a subagent manifest) and forward the paths to `run_infer.sh` as positional args (opencode `#13`/`#14`, openhands `#18`). Recorded subagent sessions are linked to the exact parent task call that spawned them rather than by metadata order, so parallel siblings and nested agents can't consume each other's turns, and live continuations are merged back onto the recorded root. Per-session trajectory records now also carry replay linkage and global ordering (`recorded_session_id`, `spawn_call_id`, `spawn_index`, `global_turn`, …), and `SWEBenchVerifyResponse.subagent_trajectories` is populated so returned create-params are directly replay-ready. **3. DeepSWE verifier fixes.** - Carry optional `tests/grader.py` / `tests/config.json` from Harbor bundles (DeepSWE v1.1 delegates preparation and scoring to them) and mount them into the eval container. Optional, so older bundles still convert and grade. - Read the reward from `/logs/verifier/reward.json` (new tasks) as well as `reward.txt` (older tasks and crash sentinels). - Fix `mkdir -p /logs/vserifier` → `/logs/verifier`; the typo meant the directory the verifier writes its reward into was never created, so the reward read back empty and the task graded 0. - Skip injecting the online Maven mirror for verifiers that deliberately run offline (`mvn -o` / `--offline`) against their image's build-time JVM cache — Maven records which repository supplied each cached artifact, so a differently-named mirror makes cached plugins look unavailable. **4. Reverts #2120** (SWE agent rollout observations), removing `observability.py`, `tests/test_observability.py` and their call sites. <!-- TODO(author): state why the revert is included, and whether it will be re-landed in another form. Reviewers will ask, since #2120 is part of the merged #2114–#2120 series. --> Tests: `responses_api_agents/swe_agents/tests/test_app.py` gains 26 tests covering the replay path — gym-side message conversion (openhands and opencode), replay file materialization and system-prompt pinning, positional-arg ordering, subagent manifest mounting with `ENABLE_SUBAGENTS=1`, parent-task-call linkage for parallel and nested children, completed-invocation truncation, legacy payload parsing, live-continuation merging, and preservation of the new trajectory record fields. Note: end-to-end opencode replay also needs `--replay-messages-file` / `--replay-subagents-file` in the opencode fork. The commit currently pinned by `configs/swebench_opencode.yaml` (`nv-opencode@sdd/dev`) does not have them — its `run_infer.sh` stops at positional `#12` — so until that lands and the pin moves, an opencode replay request is a no-op and the agent starts the task from scratch. openhands replay is unaffected. Bumping the pin also requires clearing `cache/swe_agents/swe_opencode_setup`, since `OpenCodeHarnessProcessor.setup()` returns early once the tree exists and never re-checks-out the configured commit. ## Checklist - [x] I have read the [contributing guidelines](https://docs.nvidia.com/nemo/gym/latest/contribute/development-setup). - [x] The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs. - [x] Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable). - [x] Pre-commit checks pass locally (`pre-commit run --all-files`) (so CI lint/format/copyright pass). - [x] All commits have DCO sign-off (`git commit -s`) (so the DCO check passes). --------- Signed-off-by: Sugam Devare <sdevare@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.