feat(swe_agents): drive OpenHands through the decoupled env + verifier (#1249) [2/3]

[adil-a]

Drive OpenHands through the decoupled env + verifier (#1249)

Stacked PR 2 of 3 · base: stack/1-swe-env-and-verifier · next: stack/3-mini-swe-agent-reuse

Builds on PR 1 (the swe_env library + verifier). This is the cutover: the OpenHands swe_agents harness now provisions its sandbox through the shared library and scores through the verifier over HTTP, instead of doing all of it inline.

What changes

• app.py shrinks (−3.8k lines net across this PR). The legacy two-container eval path — the second concurrently-launched container, the /trajectories_mount bind mount, and the sleep-until-predictions-file handshake — is deleted. The agent now: provisions one working sandbox via swe_env, self-drives OpenHands inside it, extracts the unified-diff patch, and POSTs it to the verifier for grading. The agent remains the rollout-row owner (it still emits the frozen SWEBenchVerifyResponse with its agent-side timing/error metrics, merging in the verifier's eval-side fields).
• swe_env_adapter.py — a thin adapter wrapping AsyncSweEnvironment for the OpenHands self-driving consumption mode (one long exec; OpenHands owns its in-sandbox runtime).
• Duplicate parser removed. swe_agents/swe_bench_ext/{parsing,frameworks,utils}.py are deleted; the single copy now lives in swe_env/parsing/ (PR 1). Nothing imports the old path.
• Config dedup. Shared env keys move into configs/swe_env_base.yaml; the agent YAMLs reference it while keeping the legacy dotted paths addressable so pinned overrides and num_workers>1 reconstruction are unchanged.
• Reference end-to-end driver + a documented SWE-bench Verified example (see below).

Tests

responses_api_agents/swe_agents/tests/   82 passed

PR 1's suites stay green on this branch (swe_env 168, verifier 14). ruff clean.

End-to-end: SWE-bench Verified with OpenHands

Serve a tool-calling model (the agent sends tool_choice=auto, so the vLLM tool-call flags are required):

docker run -d --name swe-vllm --gpus '"device=0"' \
  -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 --ipc=host \
  vllm/vllm-openai:latest \
  --model Qwen/Qwen2.5-Coder-32B-Instruct-AWQ \
  --enable-auto-tool-choice --tool-call-parser hermes \
  --max-model-len 32768 --gpu-memory-utilization 0.92

Run one instance through the full decoupled chain (provision → self-drive → extract patch → verify):

python responses_api_agents/swe_agents/scripts/openhands_decoupled_rollout.py \
  --instance astropy__astropy-13453 \
  --model Qwen/Qwen2.5-Coder-32B-Instruct-AWQ \
  --model-host 127.0.0.1 --model-port 8000 --max-iter 30

The chain was validated end-to-end on this box: provision → drive → extract → verify_task returns a reward. Grading determinism was confirmed by verifying the gold patch for a pure-compute instance (astropy__astropy-13453) → resolved=True, reward=1.0.

Review notes

• The bulk of the deletions are the legacy eval path; the additions are the adapter + the e2e driver + the example doc.
• Best reviewed after PR 1, since it consumes the library and verifier introduced there.

🤖 Generated with Claude Code

---

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[adil-a]

Superseded — content converged into #1738 on top of #1572.