[Sync][D] coding_agent_rl: agent-in-sandbox RL subsystem (slime #1923/#1956/#1960/#1954/#1957/#1958/#1963/#1979/#1981/#1982/#1961)

[aoshen02]

PR-D — agent-in-sandbox RL subsystem (RFC #107). Stacked on PR-A (#145) (base sync/slime-mega-A; needs A's #1984 group_id rename + Sample.group_id).

Adds the agent-in-sandbox RL feature vime lacked. Copied slime's final state (all 11 D PRs applied) + translated sglang→vLLM.

New

• vime/agent/ — trajectory.py, sandbox.py, parsing.py, adapters/{common,openai,anthropic}.py (+ __init__).
• examples/coding_agent_rl/ — README, generate.py, sandbox.py, aiohttp_threaded.py, run_qwen36_35b_a3b_swe_8nodes.sh.

3 sglang→vLLM seams

1. adapters/common.py: call_sglang_generate→call_engine_generate; POST /generate→/inference/v1/generate; body input_ids→token_ids, max_new_tokens→max_tokens, drop return_logprob, add logprobs:1+model=; response output_token_logprobs→choices[0].token_ids+logprobs.content[].logprob (canonical field map); per-rid /abort_request→task.cancel() + connection teardown on cancel; header X-SMG-Routing-Key→x-session-id.
2. coding_agent_rl/generate.py: sglang_router_ip/port→vllm_router_ip/port.
3. parsing.py: no change (ReasoningParser/FunctionCallParser lazy; coding_agent_rl defaults to XML fallback).

Validation

• py_compile PASS on all 11 new .py.
• Live-imported the whole layer against vime (adapters, generate, sandbox import clean; Sample.group_id/SingletonMeta/load_tokenizer resolve); field-map helpers smoke-tested (canonical→vLLM body, token_ids+logprobs.content parse).

Notes

• #1984 agent-half rename (rollout_id→group_id in trajectory.py) already present (copied slime HEAD; consistent with PR-A).
• Run-script sglang-only flags dropped (documented inline); EAGLE spec left commented (needs draft-model path).
• Does not satisfy PR-G's retool tool_sandbox.py (that's a separate in-process tool registry; PR-G must port examples/retool/tool_sandbox.py itself).

🤖 Generated with Claude Code

---

Fidelity ledger (audited 2026-06-04 vs slime 8ef1fb47..7a7aba4)

op	detail
Ported	coding-agent-in-sandbox subsystem (vime/agent/ adapters + trajectory + sandbox)
Translated (sglang→vllm)	entire agent adapter layer: SGLANG_URL_KEY→ENGINE_URL_KEY, OpenAIAdapter(sglang_url=)→(engine_url=), slime.agent→vime.agent
Fixed in this audit (b86ac85)	ported test_agent_trajectory + test_agent_sdk_adapters (wired into new agent-adapter cpu job, green); added extra_pip_deps template support
Deferred in this audit	test_agent_adapters.py ported but NOT wired — 3/14 cases mock sglang's /generate meta_info.output_token_logprobs response schema; need rewriting to vime's vLLM response format. tools/replay_openai_jsonl.py to verify.
Verified	trajectory + sdk_adapters PASS (gb200 job 8919)

[gemini-code-assist]

Code Review

This pull request introduces an end-to-end Software Engineering (SWE) coding-agent reinforcement learning (RL) example, adding custom generation scripts, sandbox management helpers, and Anthropic/OpenAI-compatible adapters. The code review identifies several critical improvements: resolving a task cancellation issue on transient network errors in the common adapter, preventing race conditions during concurrent Node tarball decompression, ensuring clean repository states before patch application attempts, and gracefully handling missing swepro assets or malformed JSON results in the sandbox. Additionally, the review highlights the need to include missing fields like tool_call_id, id, and type in translated tool results and calls within the Anthropic adapter to ensure compatibility with standard Hugging Face chat templates.

[gemini-code-assist]

The except block catches aiohttp.ClientError and asyncio.TimeoutError and calls task.cancel(), which cancels the entire current task (the rollout/generate task) on any transient network error or timeout. This masks standard network exceptions as CancelledError, making debugging extremely difficult. Since async with automatically cleans up the connection on error or cancellation, we should only catch asyncio.CancelledError if we want to log or re-raise it, and avoid calling task.cancel() on the current task.

Suggested change

	except (asyncio.CancelledError, aiohttp.ClientError, asyncio.TimeoutError):
	# vLLM ``/inference/v1/generate`` has no per-request HTTP abort endpoint (unlike
	# sglang ``/abort_request``). Cancelling the in-flight task tears down the aiohttp
	# request, which drops the streaming connection so the engine stops generating.
	if task is not None:
	task.cancel()
	raise
	except asyncio.CancelledError:
	# Cancelling the in-flight task tears down the aiohttp request,
	# which drops the connection so the engine stops generating.
	raise

[gemini-code-assist]

When running multiple Ray workers concurrently on the same node, they will all attempt to decompress the Node tarball to the same shared path in /tmp at the same time. This creates a race condition that can corrupt the tarball or cause file access errors. We should use a process-unique partial file name and safely rename/cleanup to avoid collisions.

    if host_tarball.suffix == ".xz":
        plain = Path(tempfile.gettempdir()) / f"coding_agent_rl.{host_tarball.stem}.tar"
        if not plain.exists():
            tmp = plain.with_suffix(f".{os.getpid()}.tar.partial")
            try:
                with lzma.open(host_tarball, "rb") as src, open(tmp, "wb") as dst:
                    shutil.copyfileobj(src, dst)
                if not plain.exists():
                    os.replace(tmp, plain)
                else:
                    tmp.unlink()
            except Exception:
                if tmp.exists():
                    tmp.unlink()
                raise
        host_tarball = plain

[gemini-code-assist]

If git apply --3way fails, it can leave the repository in a dirty or conflicted state. Subsequent patch application attempts (like git apply or patch) will then run on top of these dirty files and fail, even if they would have succeeded on a clean repository. We should reset the repository to a clean state before each attempt.

Suggested change

	async def _apply_diff(ev: Sandbox, workdir: str, diff_text: str) -> bool:
	if not diff_text.strip():
	return True
	await ev.write_file(_PATCH, diff_text, user="agent")
	for cmd in [
	f"cd {workdir} && git apply --3way --whitespace=nowarn {_PATCH}",
	f"cd {workdir} && git apply --whitespace=nowarn {_PATCH}",
	f"cd {workdir} && patch -p1 --no-backup-if-mismatch < {_PATCH}",
	]:
	ec, _, _ = await ev.exec(cmd, user="agent", check=False, timeout=120)
	if ec == 0:
	return True
	return False
	async def _apply_diff(ev: Sandbox, workdir: str, diff_text: str) -> bool:
	if not diff_text.strip():
	return True
	await ev.write_file(_PATCH, diff_text, user="agent")
	for cmd in [
	f"cd {workdir} && git apply --3way --whitespace=nowarn {_PATCH}",
	f"cd {workdir} && git apply --whitespace=nowarn {_PATCH}",
	f"cd {workdir} && patch -p1 --no-backup-if-mismatch < {_PATCH}",
	]:
	await ev.exec(f"cd {workdir} && git reset --hard HEAD && git clean -fd", user="agent", check=False, timeout=60)
	ec, _, _ = await ev.exec(cmd, user="agent", check=False, timeout=120)
	if ec == 0:
	return True
	return False

[gemini-code-assist]

The translated tool result message is missing the tool_call_id field. Many Hugging Face chat templates (like Qwen's or Llama's) require tool_call_id to match tool results with tool calls, and omitting it can cause rendering errors or template failures. We should map tool_use_id to tool_call_id in the translated message.

Suggested change

	translated.append({"role": "tool", "content": _flatten(b.get("content"))})
	elif isinstance(b, dict) and b.get("type") == "text":
	if isinstance(b, dict) and b.get("type") == "tool_result":
	tool_msg = {"role": "tool", "content": _flatten(b.get("content"))}
	if b.get("tool_use_id"):
	tool_msg["tool_call_id"] = b["tool_use_id"]
	translated.append(tool_msg)

[gemini-code-assist]

The translated tool call is missing id and type: "function" fields. Standard Hugging Face chat templates expect these fields to be present in the tool_calls list of the assistant message. Omitting them can break template rendering.

Suggested change

	elif b.get("type") == "tool_use":
	tcs.append({"function": {"name": b.get("name", "tool"), "arguments": b.get("input") or {}}})
	elif isinstance(b, dict) and b.get("type") == "tool_use":
	tcs.append({
	"id": b.get("id") or f"toolu_{secrets.token_hex(8)}",
	"type": "function",
	"function": {"name": b.get("name", "tool"), "arguments": b.get("input") or {}}
	})

[gemini-code-assist]

If both run_script_path and parser_script_path are missing or empty, no files are written to _SWEPRO_DIR. In this case, the shell expansion _SWEPRO_DIR/* will fail, causing chmod to return a non-zero exit code and crashing the evaluation. Using chmod -R 755 on the directory itself is safe and avoids shell expansion failures.

Suggested change

	await ev.exec(f"chmod 755 {_SWEPRO_DIR}/* && chown -R agent:agent {_SWEPRO_DIR}", user="root", check=True)
	await ev.exec(f"chmod -R 755 {_SWEPRO_DIR} && chown -R agent:agent {_SWEPRO_DIR}", user="root", check=True)

[gemini-code-assist]

If the swepro result file is corrupted or contains invalid JSON, json.loads will raise a JSONDecodeError, crashing the entire rollout. We should wrap it in a try-except block to handle malformed files gracefully.

Suggested change

	raw = await ev.read_file(result_f, user="agent")
	parsed = json.loads(raw) if raw else {"tests": []}
	raw = await ev.read_file(result_f, user="agent")
	try:
	parsed = json.loads(raw) if raw else {"tests": []}
	except json.JSONDecodeError:
	parsed = {"tests": []}

[CalvinXKY]

LGTM