feat(swe_agents): drive OpenHands through the decoupled env + verifier (#1249) [2/3]

[adil-a]

Drive OpenHands through the decoupled env + verifier (NVIDIA-NeMo#1249)

Stacked PR 2 of 3 · base: stack/1-swe-env-and-verifier · next: stack/3-mini-swe-agent-reuse

Builds on PR 1 (the swe_env library + verifier). This is the cutover: the OpenHands swe_agents harness now provisions its sandbox through the shared library and scores through the verifier over HTTP, instead of doing all of it inline.

What changes

• app.py shrinks (−3.8k lines net across this PR). The legacy two-container eval path — the second concurrently-launched container, the /trajectories_mount bind mount, and the sleep-until-predictions-file handshake — is deleted. The agent now: provisions one working sandbox via swe_env, self-drives OpenHands inside it, extracts the unified-diff patch, and POSTs it to the verifier for grading. The agent remains the rollout-row owner (it still emits the frozen SWEBenchVerifyResponse with its agent-side timing/error metrics, merging in the verifier's eval-side fields).
• swe_env_adapter.py — a thin adapter wrapping AsyncSweEnvironment for the OpenHands self-driving consumption mode (one long exec; OpenHands owns its in-sandbox runtime).
• Duplicate parser removed. swe_agents/swe_bench_ext/{parsing,frameworks,utils}.py are deleted; the single copy now lives in swe_env/parsing/ (PR 1). Nothing imports the old path.
• Config dedup. Shared env keys move into configs/swe_env_base.yaml; the agent YAMLs reference it while keeping the legacy dotted paths addressable so pinned overrides and num_workers>1 reconstruction are unchanged.
• Reference end-to-end driver + a documented SWE-bench Verified example (see below).

Tests

responses_api_agents/swe_agents/tests/   82 passed

PR 1's suites stay green on this branch (swe_env 168, verifier 14). ruff clean.

End-to-end: SWE-bench Verified with OpenHands

Serve a tool-calling model (the agent sends tool_choice=auto, so the vLLM tool-call flags are required):

docker run -d --name swe-vllm --gpus '"device=0"' \
  -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 --ipc=host \
  vllm/vllm-openai:latest \
  --model Qwen/Qwen2.5-Coder-32B-Instruct-AWQ \
  --enable-auto-tool-choice --tool-call-parser hermes \
  --max-model-len 32768 --gpu-memory-utilization 0.92

Run one instance through the full decoupled chain (provision → self-drive → extract patch → verify):

python responses_api_agents/swe_agents/scripts/openhands_decoupled_rollout.py \
  --instance astropy__astropy-13453 \
  --model Qwen/Qwen2.5-Coder-32B-Instruct-AWQ \
  --model-host 127.0.0.1 --model-port 8000 --max-iter 30

The chain was validated end-to-end on this box: provision → drive → extract → verify_task returns a reward. Grading determinism was confirmed by verifying the gold patch for a pure-compute instance (astropy__astropy-13453) → resolved=True, reward=1.0.

Review notes

• The bulk of the deletions are the legacy eval path; the additions are the adapter + the e2e driver + the example doc.
• Best reviewed after PR 1, since it consumes the library and verifier introduced there.

🤖 Generated with Claude Code

[adil-a]

Superseded by the stacked PRs opened directly on NVIDIA-NeMo/Gym (NVIDIA-NeMo#1677, NVIDIA-NeMo#1678, NVIDIA-NeMo#1679).