Repository navigation
feat: separate persistent tool execution from the agent loop - #218
Merged
Merged
Conversation
hallerite
force-pushed
the
feat/split-execution
branch
from
October 8, 2026 02:19
c95b708 to
fa06a0d
Compare
hallerite
force-pushed
the
feat/split-execution
branch
from
October 29, 2026 23:27
80461a8 to
df8fe7a
Compare
hallerite
marked this pull request as ready for review
October 29, 2026 23:30
Services started through the supervisor would run beside the agent loop instead of in the task workspace. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the loop Shell commands start sessions of their own so timeouts and output floods can kill their whole process group, which also took them out of the kernel's group: a kernel restart or shutdown left a running bash skill command alive. The REPL now kills the process groups of the kernel's descendants first. RemoteREPL sent interrupt, finish_interrupt and close as blocking RPCs from the supervisor's event loop, where a slow worker could stall every agent for up to 30 seconds. They now run in order on a control thread; a tool call waits for pending control calls so it cannot overtake an interrupt's release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Walking /proc from the kernel's PID missed the commands of a kernel that had already died, whose children are reparented, and a provisioner PID of 0 would have walked the whole process tree. Each kernel now carries a random RLM_KERNEL_TOKEN in its environment, which every process it starts inherits; restart and shutdown kill the process groups of processes that carry it, including after a crash. Interrupts and closes after shutdown are no-ops instead of failing on the closed control thread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With --port 0 the kernel picks a free port, and the worker prints it once it accepts connections, so an orchestrator needs no port selection of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hallerite
force-pushed
the
feat/split-execution
branch
from
October 31, 2026 01:03
5825fb2 to
ff8767d
Compare
The bash tool starts each command in a session of its own, which the kernel's cleanup didn't reach: a cancelled local turn waited for the command's timeout, and a background job it left behind outlived the session. Native shell commands now carry the kernel's token, so restart and shutdown kill them like the kernel's own commands, and a local turn's cancellation interrupts them. Tools run one at a time on the REPL's thread, so a restart never meets a running shell command. The compaction notice named the harness's messages.jsonl, which in split execution is on another machine; it now names the worker's copy, as the system prompt does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every tool call sent the whole append-only messages.jsonl to the worker and got back the whole programmatic tool log, so a long split session failed once the ledger outgrew one 16 MiB frame. The harness now sends only the ledger bytes the worker hasn't seen, in chunks that fit a frame, and the worker writes each at its offset; the worker returns only the tool calls logged since the last call, which the harness appends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shell cleanup, history sync and compaction notice checks now run as end-to-end engine sessions with a scripted model instead of calling helpers on stand-in objects: a cancelled turn stops its bash command and closing the engine stops a job it left behind; a prompt larger than one frame reaches the split worker's history whole; and the compaction test runs local and split, checking the notice names the history the kernel reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he workspace Under split execution the supervisor stays with the agent loop, but rlm.shell, rlm.services and rlm.watch.path started processes and watched files on the supervisor's machine, so the worker refused them and the system prompt was swapped for a reduced one. The execution worker now hosts them in the workspace with the same job, service and watch code (WorkspaceHost), the supervisor reaches it through the root agent's worker (RemoteHost) and keeps its records and a copy of each job's output, and split execution uses the same runtime prompt as local execution. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fast job can finish, and its output and completion reach the supervisor, before the jobs.start reply that registers it. On a one-CPU workspace VM this happens every time: those events were dropped, so the job came back without its output and run() waited out yield_after. They are now held and applied once the job is registered. The split e2e test pins its worker to one CPU, which reproduces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calls overlap on the execution connection (the event long-poll, kernel callbacks, tool calls), so one end can send a small frame while its previous one is still unacknowledged; Nagle then holds it for the peer's delayed ACK. Measured on Prime, that added 40 ms to the slowest tenth of rlm.shell.run calls with output in every run; with TCP_NODELAY on both ends it is gone and medians are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hallerite
force-pushed
the
feat/split-execution
branch
from
October 31, 2026 12:26
5fd21f8 to
9184e4a
Compare
…tion The worker answered the supervisor's events long-poll as soon as a job printed, before it exited, so learning that the job finished took another round trip: rlm.shell.run of a command with output cost three round trips instead of two. An events call now holds a running job's new output up to 50 ms for the job to finish. A running job's output reaches the supervisor at most that much later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bash command or cell that printed more than 1 MiB was killed or interrupted, though the conversation only ever shows a result's first and last 10 KB. Output is now kept to its first and last 512 KiB while it is produced (bash splits that between stdout and stderr), so the command runs to its end in bounded memory and a tool result still fits one execution frame. The result carries how much more the whole output measured, so the conversation shows the same start, end, token count and line count as for the whole output. The session log keeps the start and end, with a note where the middle was dropped. The bash tool again translates \r\n and \r to \n, as subprocess text mode did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With a tool output budget larger than what a long output kept, the head and tail slices overlapped, so the conversation showed part of the output twice and miscounted the truncated bytes. When an output's middle was dropped, each end shown is now at most what every tool keeps of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A kernel restart or shutdown killed every process that inherited the kernel's environment marker, which includes daemons the agent started and background jobs a bash command left behind, though a daemon is meant to outlive the terminal that started it and the grader may need it running. Restart and shutdown now kill what is still in the kernel's session, as a closing terminal does: the kernel leads a session of its own, and a bash command leads a process group in its caller's session, so a timeout or a cancel still stops the command and what it started. A process in a session of its own outlives the kernel and the agent, and the bash tool's commands, which the kernel does not run, are not stopped with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 023b3e9. Configure here.
A command inherited its caller's stdin: from a terminal, a command reading it in its own background process group stopped on SIGTTIN and waited out its timeout, and under ACP it read the harness's own protocol stream. A command now reads end-of-file instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A job a command put in the background without redirecting its output holds the call open until the timeout, which then killed the command's process group and the job with it. Once bash itself has exited, the call now stops waiting without killing anything, so the job outlives the call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
child.history() went through the supervisor as one reply, which broker and execution frames cap at 16 MiB, so a long child conversation failed to read, locally too, where main read the child's ledger file whole. The supervisor now returns the ledger in pages of at most 8 MiB of JSON from a byte offset, and the handle reads them until the end. The history test reads a child's history past the frame limit, local and split, and checks a printed count: a traceback shows the cell's source, so a marker printed at the end of the cell also appeared when it failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two figures, light and dark: where the loop, credentials and tools run in colocated and split execution, and what one agent step sends over the wire. The README's split section said the supervisor's shell jobs, services and watches return errors in this mode; they run beside the kernels. It also gives the per-call cost and what a closing session stops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mikasenghaas
approved these changes
Nov 1, 2026
rasdani
added a commit
to PrimeIntellect-ai/verifiers
that referenced
this pull request
Nov 2, 2026
Update the RLM harness's pinned nano-rlm commit from `15513ae8` to `48da1d9c43bdcf2ef20b7cff9e4c6df800810b30`. `15513ae8` was the head of PrimeIntellect-ai/nano-rlm#240 when #3011 pinned it. #240 has since been squash-merged as `48da1d9c`, so `15513ae8` is on no branch and will be garbage collected. `48da1d9c` has the same tree as #240's final head. [Upstream changes](PrimeIntellect-ai/nano-rlm@15513ae...48da1d9) also bring in nano-rlm main since #240's original base: - PrimeIntellect-ai/nano-rlm#218: separates persistent tool execution from the agent loop. Local mode is affected too: bash background jobs outlive the kernel, and output keeps the first and last 512 KiB. - PrimeIntellect-ai/nano-rlm#246: spent limits are reported as ACP stop reasons. - PrimeIntellect-ai/nano-rlm#244 and PrimeIntellect-ai/nano-rlm#245: tests and docs. `tests/unit/harnesses/test_rlm.py` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Default dependency bump changes runtime agent/tool behavior for harnesses using the pinned ref, though unit tests for the harness still pass. > > **Overview** > Updates the **RLM harness** default `version` git ref for **nano-rlm** from `15513ae8…` to **`48da1d9c…`**, so installs track the squash-merged commit from upstream PR #240 instead of a ref that is no longer on any branch. > > Rollouts that rely on the default pin will pull the newer nano-rlm tree, which (per upstream) includes **persistent tool execution outside the agent loop** (e.g. bash background jobs surviving the kernel with head/tail output truncation), **ACP stop reasons when spend limits hit**, and related test/doc updates. Custom `version` overrides are unchanged. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit cea2226. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Bump default nano-rlm ref in `RLMHarnessConfig` to `48da1d9c` > Updates the default `version` for nano-rlm from `15513ae8` to `48da1d9c` in [harness.py](https://github.com/PrimeIntellect-ai/verifiers/pull/3044/files#diff-3955da24b5fd88f62482297bbda7ba0936f633927f15576db27fed6e58f668ec). Applies only when no version is explicitly configured. Behavioral Change: runs without an explicit version now use nano-rlm at ref `48da1d9c43bdcf2ef20b7cff9e4c6df800810b30`. > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized cea2226.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Why
Today the harness lives inside the task's machine. The agent loop, the model credentials, the supervisor and the conversation ledger share a filesystem and a process tree with every command the agent runs. For RL that is the wrong boundary: a command can read the API key, kill or starve the loop, or rewrite the ledger; the harness's install and processes become part of the environment being graded; and every task image has to be able to host the harness.
This PR moves the boundary to the tool call. The harness keeps the loop, the credentials, the supervisor and the authoritative conversation, and a small worker in the workspace runs the tools. The model sees the same tools, prompt and results as before.
How it works
Tool calls cross the boundary (following Damian's RFC comment).
ipython,bash,editandfetchtravel as a name and arguments topython -m rlm.executionin the workspace, which hosts the persistent IPython kernel, the builtin skills, interruption and kernel recovery. Recursive agents use the same worker, each with a kernel of its own.rlm.*calls come back. A cell's calls into the supervisor (rlm.agent,rlm.inbox,rlm.shell, …) return over the same connection, scoped to the agent that made them. The supervisor's shell jobs, services and path watches run in the worker beside the kernels, while the supervisor keeps their records and inbox events. Nothing is refused in split mode, and the runtime prompt is the one local execution uses.The ledger follows the cell. Before each cell the worker receives the conversation's new records, in chunks that fit a frame, so
rlm.history()reads locally in the workspace. A child's history comes from the supervisor in pages.Transport. Length-prefixed JSON frames of up to 16 MiB, over a trusted command's stdio or over TCP with a token on a private network. With
--port 0the worker announceslistening on HOST:PORT. TCP does not encrypt, so use a private network or a tunnel. The orchestrator provisions the workspace, starts the worker, and owns teardown and the episode's outcome. It configuresexecution_commandandexecution_cwd, orexecution_tcp, in the ACP runtime metadata, and initialization advertisesai.prime.rlm/execution-v1.What it costs. A tool call adds one round trip between the machines and each
rlm.*call one more;rlm.shell.runtakes two. The worker holds a running job's new output for up to 50 ms, so a short command's output and exit arrive together, and both ends setTCP_NODELAY. On Prime's private network a round trip takes 26–200 ms, depending on the pair of VMs.Local and split behave the same
A wire needs bounded messages and explicit process ownership. These changes therefore apply to local execution too, and what the agent sees stays as on main:
Validation
Suite (396 passed). Real stdio and TCP workers cover:
Most of these run in both local and split.
Docker. In a network-isolated workspace container, all tools work, the harness's files and credentials are out of reach, and killing the container fails the session promptly.
Prime, with verifiers#2803. The harness and the workspace ran on separate VMs on the rollout's private network.
rlm.*API scored 1.0, graded in the workspace. A workspace powered off mid-turn ended the rollout as lost.Terminal-Bench 2.1. On 78 tasks, one attempt each, split mode solved 62 where main solved 63, with no infrastructure errors.
Limits
Companion: verifiers#2803 runs this worker on a Prime workspace VM, and the workspace-loss outcome policy is in verifiers#2831.
🤖 Generated with Claude Code
Note
Medium Risk
Large new execution path spanning RPC, remote supervisor operations, and process lifecycle; credentials stay in the harness but misconfiguration or worker disconnect fails sessions.
Overview
Adds split execution: orchestrators can set
execution_commandorexecution_tcpon the ACP runtime contract so the harness keeps the loop, model credentials, supervisor, and ledger whilepython -m rlm.executionrunsipython,bash,edit, andfetchin the task workspace over stdio or token-authenticated TCP. Initialize now advertisesai.prime.rlm/execution-v1.The engine swaps in
RemoteREPL, streams ledger chunks and tool RPCs on 16 MiB frames, and proxiesrlm.*via a worker-side broker socket. The root supervisorattach_hostso shell jobs, services, and path watches execute in the workspace throughWorkspaceHost/RemoteShellJobswhile records and inbox events stay on the harness.Tool output handling changes apply locally too: bounded head/tail capture (512 KiB per command/cell),
omittedsizes for accurate conversation truncation, paginatedchild.history(), bash streaming with process-group lifecycle and empty stdin, and kernel session cleanup on shutdown/interrupt. README and diagrams document colocated vs split layout and round-trip costs.Reviewed by Cursor Bugbot for commit 220db7d. Bugbot is set up for automated code reviews on this repo. Configure here.