Skip to content

feat: separate persistent tool execution from the agent loop - #218

Merged
hallerite merged 21 commits into
mainfrom
feat/split-execution
Nov 1, 2026
Merged

hallerite merged 21 commits into
mainfrom
feat/split-execution

Conversation

@hallerite

@hallerite hallerite commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Why

Today the harness lives inside the task's machine. The agent loop, the model credentials, the supervisor and the conversation ledger share a filesystem and a process tree with every command the agent runs. For RL that is the wrong boundary: a command can read the API key, kill or starve the loop, or rewrite the ledger; the harness's install and processes become part of the environment being graded; and every task image has to be able to host the harness.

This PR moves the boundary to the tool call. The harness keeps the loop, the credentials, the supervisor and the authoritative conversation, and a small worker in the workspace runs the tools. The model sees the same tools, prompt and results as before.

Colocated, the agent loop, model credentials and the agent's tools share one machine. Split, the harness VM keeps the loop, credentials, supervisor and ledger, and the workspace VM runs the rlm.execution worker with the tools, over one authenticated connection.

How it works

Tool calls cross the boundary (following Damian's RFC comment). ipython, bash, edit and fetch travel as a name and arguments to python -m rlm.execution in the workspace, which hosts the persistent IPython kernel, the builtin skills, interruption and kernel recovery. Recursive agents use the same worker, each with a kernel of its own.

rlm.* calls come back. A cell's calls into the supervisor (rlm.agent, rlm.inbox, rlm.shell, …) return over the same connection, scoped to the agent that made them. The supervisor's shell jobs, services and path watches run in the worker beside the kernels, while the supervisor keeps their records and inbox events. Nothing is refused in split mode, and the runtime prompt is the one local execution uses.

The ledger follows the cell. Before each cell the worker receives the conversation's new records, in chunks that fit a frame, so rlm.history() reads locally in the workspace. A child's history comes from the supervisor in pages.

Transport. Length-prefixed JSON frames of up to 16 MiB, over a trusted command's stdio or over TCP with a token on a private network. With --port 0 the worker announces listening on HOST:PORT. TCP does not encrypt, so use a private network or a tunnel. The orchestrator provisions the workspace, starts the worker, and owns teardown and the episode's outcome. It configures execution_command and execution_cwd, or execution_tcp, in the ACP runtime metadata, and initialization advertises ai.prime.rlm/execution-v1.

One agent step in split mode: the harness sends the tool call with the new ledger records, the cell runs in the workspace and calls back for rlm.* operations, and the result returns to the harness. A tool call costs one round trip, each rlm.* call one more.

What it costs. A tool call adds one round trip between the machines and each rlm.* call one more; rlm.shell.run takes two. The worker holds a running job's new output for up to 50 ms, so a short command's output and exit arrive together, and both ends set TCP_NODELAY. On Prime's private network a round trip takes 26–200 ms, depending on the pair of VMs.

Local and split behave the same

A wire needs bounded messages and explicit process ownership. These changes therefore apply to local execution too, and what the agent sees stays as on main:

  • Long output. A command or cell runs to its end and keeps the first and last 512 KiB of its output. The result carries the whole output's size, so the model sees the same truncation, token count and line count as before.
  • Processes live as on a Linux box. A kernel restart or a closing session stops what is attached to the kernel, as a closing terminal does. Each bash command leads its own process group: a timeout or a cancelled turn stops it and returns its partial output. Bash background jobs and daemons outlive the call, the kernel and the agent, so an agent that leaves a server running gets graded with it.
  • Bash reads an empty stdin, never the harness's terminal or protocol stream.
kernel restart timeout or cancelled turn session end
started from a cell, still attached stopped cell interrupted stopped
bash command unaffected stopped, partial output kept stopped
bash background job runs on runs on runs on
daemon (a session of its own) runs on runs on runs on

Validation

  • Suite (396 passed). Real stdio and TCP workers cover:

    • persistent state and all four tools, token rejection and concurrent sessions;
    • cancellation, endless and long output, and the process lifetimes above;
    • recursive agents with scoped callbacks and supervisor jobs, services and watches;
    • histories larger than a frame.

    Most of these run in both local and split.

  • Docker. In a network-isolated workspace container, all tools work, the harness's files and credentials are out of reach, and killing the container fails the session promptly.

  • Prime, with verifiers#2803. The harness and the workspace ran on separate VMs on the rollout's private network.

    • In split mode, file tasks and the rlm.* API scored 1.0, graded in the workspace. A workspace powered off mid-turn ended the rollout as lost.
    • In both split and local execution:
      • the model's view of a 300,000-line output matched the truncation of the whole output;
      • an agent that never finishes left a daemon or a held bash job that completed the task after the harness was gone;
      • in a latency run, every fast job came back complete with its output.
  • Terminal-Bench 2.1. On 78 tasks, one attempt each, split mode solved 62 where main solved 63, with no infrastructure errors.

Limits

  • Both sides need the same nano-rlm revision.
  • A worker process alone does not isolate the filesystem; the command or VM that runs it defines the boundary.
  • Uploaded skill packages are not yet supported by the verifiers adapter.

Companion: verifiers#2803 runs this worker on a Prime workspace VM, and the workspace-loss outcome policy is in verifiers#2831.

🤖 Generated with Claude Code


Note

Medium Risk
Large new execution path spanning RPC, remote supervisor operations, and process lifecycle; credentials stay in the harness but misconfiguration or worker disconnect fails sessions.

Overview
Adds split execution: orchestrators can set execution_command or execution_tcp on the ACP runtime contract so the harness keeps the loop, model credentials, supervisor, and ledger while python -m rlm.execution runs ipython, bash, edit, and fetch in the task workspace over stdio or token-authenticated TCP. Initialize now advertises ai.prime.rlm/execution-v1.

The engine swaps in RemoteREPL, streams ledger chunks and tool RPCs on 16 MiB frames, and proxies rlm.* via a worker-side broker socket. The root supervisor attach_host so shell jobs, services, and path watches execute in the workspace through WorkspaceHost / RemoteShellJobs while records and inbox events stay on the harness.

Tool output handling changes apply locally too: bounded head/tail capture (512 KiB per command/cell), omitted sizes for accurate conversation truncation, paginated child.history(), bash streaming with process-group lifecycle and empty stdin, and kernel session cleanup on shutdown/interrupt. README and diagrams document colocated vs split layout and round-trip costs.

Reviewed by Cursor Bugbot for commit 220db7d. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/rlm/tools/bash.py Resolved
Comment thread src/rlm/execution.py Resolved

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/rlm/tools/ipython.py Resolved
Comment thread src/rlm/tools/ipython.py Resolved
Comment thread src/rlm/execution.py Resolved
hallerite and others added 7 commits October 31, 2026 01:00
Services started through the supervisor would run beside the agent loop
instead of in the task workspace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the loop

Shell commands start sessions of their own so timeouts and output floods can
kill their whole process group, which also took them out of the kernel's group:
a kernel restart or shutdown left a running bash skill command alive. The REPL
now kills the process groups of the kernel's descendants first.

RemoteREPL sent interrupt, finish_interrupt and close as blocking RPCs from the
supervisor's event loop, where a slow worker could stall every agent for up to
30 seconds. They now run in order on a control thread; a tool call waits for
pending control calls so it cannot overtake an interrupt's release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Walking /proc from the kernel's PID missed the commands of a kernel that had
already died, whose children are reparented, and a provisioner PID of 0 would
have walked the whole process tree. Each kernel now carries a random
RLM_KERNEL_TOKEN in its environment, which every process it starts inherits;
restart and shutdown kill the process groups of processes that carry it,
including after a crash. Interrupts and closes after shutdown are no-ops
instead of failing on the closed control thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With --port 0 the kernel picks a free port, and the worker prints it once it
accepts connections, so an orchestrator needs no port selection of its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hallerite
hallerite force-pushed the feat/split-execution branch from 5825fb2 to ff8767d Compare October 31, 2026 01:03

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/rlm/tools/bash.py Resolved
Comment thread src/rlm/engine.py Resolved
hallerite and others added 4 commits October 31, 2026 01:22
The bash tool starts each command in a session of its own, which the
kernel's cleanup didn't reach: a cancelled local turn waited for the
command's timeout, and a background job it left behind outlived the
session. Native shell commands now carry the kernel's token, so restart
and shutdown kill them like the kernel's own commands, and a local turn's
cancellation interrupts them. Tools run one at a time on the REPL's
thread, so a restart never meets a running shell command.

The compaction notice named the harness's messages.jsonl, which in split
execution is on another machine; it now names the worker's copy, as the
system prompt does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every tool call sent the whole append-only messages.jsonl to the worker and
got back the whole programmatic tool log, so a long split session failed once
the ledger outgrew one 16 MiB frame. The harness now sends only the ledger
bytes the worker hasn't seen, in chunks that fit a frame, and the worker writes
each at its offset; the worker returns only the tool calls logged since the
last call, which the harness appends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shell cleanup, history sync and compaction notice checks now run as
end-to-end engine sessions with a scripted model instead of calling helpers
on stand-in objects: a cancelled turn stops its bash command and closing the
engine stops a job it left behind; a prompt larger than one frame reaches the
split worker's history whole; and the compaction test runs local and split,
checking the notice names the history the kernel reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he workspace

Under split execution the supervisor stays with the agent loop, but rlm.shell,
rlm.services and rlm.watch.path started processes and watched files on the
supervisor's machine, so the worker refused them and the system prompt was
swapped for a reduced one. The execution worker now hosts them in the
workspace with the same job, service and watch code (WorkspaceHost), the
supervisor reaches it through the root agent's worker (RemoteHost) and keeps
its records and a copy of each job's output, and split execution uses the
same runtime prompt as local execution.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/rlm/workspace_host.py Resolved
hallerite and others added 2 commits October 31, 2026 12:13
A fast job can finish, and its output and completion reach the supervisor,
before the jobs.start reply that registers it. On a one-CPU workspace VM this
happens every time: those events were dropped, so the job came back without
its output and run() waited out yield_after. They are now held and applied
once the job is registered.

The split e2e test pins its worker to one CPU, which reproduces it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calls overlap on the execution connection (the event long-poll, kernel
callbacks, tool calls), so one end can send a small frame while its previous
one is still unacknowledged; Nagle then holds it for the peer's delayed ACK.
Measured on Prime, that added 40 ms to the slowest tenth of rlm.shell.run
calls with output in every run; with TCP_NODELAY on both ends it is gone and
medians are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hallerite
hallerite force-pushed the feat/split-execution branch from 5fd21f8 to 9184e4a Compare October 31, 2026 12:26

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/rlm/supervisor.py Resolved
hallerite and others added 2 commits October 31, 2026 12:35
…tion

The worker answered the supervisor's events long-poll as soon as a job printed,
before it exited, so learning that the job finished took another round trip:
rlm.shell.run of a command with output cost three round trips instead of two.
An events call now holds a running job's new output up to 50 ms for the job to
finish. A running job's output reaches the supervisor at most that much later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bash command or cell that printed more than 1 MiB was killed or interrupted,
though the conversation only ever shows a result's first and last 10 KB. Output
is now kept to its first and last 512 KiB while it is produced (bash splits that
between stdout and stderr), so the command runs to its end in bounded memory and
a tool result still fits one execution frame.

The result carries how much more the whole output measured, so the conversation
shows the same start, end, token count and line count as for the whole output.
The session log keeps the start and end, with a note where the middle was
dropped. The bash tool again translates \r\n and \r to \n, as subprocess text
mode did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/rlm/compaction.py Resolved
hallerite and others added 2 commits November 1, 2026 01:44
With a tool output budget larger than what a long output kept, the head and
tail slices overlapped, so the conversation showed part of the output twice
and miscounted the truncated bytes. When an output's middle was dropped, each
end shown is now at most what every tool keeps of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A kernel restart or shutdown killed every process that inherited the kernel's
environment marker, which includes daemons the agent started and background
jobs a bash command left behind, though a daemon is meant to outlive the
terminal that started it and the grader may need it running.

Restart and shutdown now kill what is still in the kernel's session, as a
closing terminal does: the kernel leads a session of its own, and a bash
command leads a process group in its caller's session, so a timeout or a
cancel still stops the command and what it started. A process in a session of
its own outlives the kernel and the agent, and the bash tool's commands, which
the kernel does not run, are not stopped with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 023b3e9. Configure here.

Comment thread src/rlm/tools/bash.py Resolved
hallerite and others added 4 commits November 1, 2026 03:16
A command inherited its caller's stdin: from a terminal, a command reading it
in its own background process group stopped on SIGTTIN and waited out its
timeout, and under ACP it read the harness's own protocol stream. A command
now reads end-of-file instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A job a command put in the background without redirecting its output holds
the call open until the timeout, which then killed the command's process
group and the job with it. Once bash itself has exited, the call now stops
waiting without killing anything, so the job outlives the call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
child.history() went through the supervisor as one reply, which broker and
execution frames cap at 16 MiB, so a long child conversation failed to read,
locally too, where main read the child's ledger file whole. The supervisor
now returns the ledger in pages of at most 8 MiB of JSON from a byte offset,
and the handle reads them until the end.

The history test reads a child's history past the frame limit, local and
split, and checks a printed count: a traceback shows the cell's source, so
a marker printed at the end of the cell also appeared when it failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two figures, light and dark: where the loop, credentials and tools run in
colocated and split execution, and what one agent step sends over the wire.
The README's split section said the supervisor's shell jobs, services and
watches return errors in this mode; they run beside the kernels. It also
gives the per-call cost and what a closing session stops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hallerite
hallerite merged commit 208aea9 into main Nov 1, 2026
9 checks passed
@hallerite
hallerite deleted the feat/split-execution branch November 1, 2026 16:05
rasdani added a commit to PrimeIntellect-ai/verifiers that referenced this pull request Nov 2, 2026
Update the RLM harness's pinned nano-rlm commit from `15513ae8` to
`48da1d9c43bdcf2ef20b7cff9e4c6df800810b30`.

`15513ae8` was the head of
PrimeIntellect-ai/nano-rlm#240 when
#3011 pinned it. #240
has since been squash-merged as `48da1d9c`, so `15513ae8` is on no
branch and will be garbage collected. `48da1d9c` has the same tree as
#240's final head.

[Upstream
changes](PrimeIntellect-ai/nano-rlm@15513ae...48da1d9)
also bring in nano-rlm main since #240's original base:
- PrimeIntellect-ai/nano-rlm#218: separates
persistent tool execution from the agent loop. Local mode is affected
too: bash background jobs outlive the kernel, and output keeps the first
and last 512 KiB.
- PrimeIntellect-ai/nano-rlm#246: spent limits
are reported as ACP stop reasons.
- PrimeIntellect-ai/nano-rlm#244 and
PrimeIntellect-ai/nano-rlm#245: tests and docs.

`tests/unit/harnesses/test_rlm.py` passes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Default dependency bump changes runtime agent/tool behavior for
harnesses using the pinned ref, though unit tests for the harness still
pass.
> 
> **Overview**
> Updates the **RLM harness** default `version` git ref for **nano-rlm**
from `15513ae8…` to **`48da1d9c…`**, so installs track the squash-merged
commit from upstream PR #240 instead of a ref that is no longer on any
branch.
> 
> Rollouts that rely on the default pin will pull the newer nano-rlm
tree, which (per upstream) includes **persistent tool execution outside
the agent loop** (e.g. bash background jobs surviving the kernel with
head/tail output truncation), **ACP stop reasons when spend limits
hit**, and related test/doc updates. Custom `version` overrides are
unchanged.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
cea2226. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- Macroscope's pull request summary starts here -->
<!-- Macroscope will only edit the content between these invisible
markers, and the markers themselves will not be visible in the GitHub
rendered markdown. -->
<!-- If you delete either of the start / end markers from your PR's
description, Macroscope will append its summary at the bottom of the
description. -->
> [!NOTE]
> ### Bump default nano-rlm ref in `RLMHarnessConfig` to `48da1d9c`
> Updates the default `version` for nano-rlm from `15513ae8` to
`48da1d9c` in
[harness.py](https://github.com/PrimeIntellect-ai/verifiers/pull/3044/files#diff-3955da24b5fd88f62482297bbda7ba0936f633927f15576db27fed6e58f668ec).
Applies only when no version is explicitly configured. Behavioral
Change: runs without an explicit version now use nano-rlm at ref
`48da1d9c43bdcf2ef20b7cff9e4c6df800810b30`.
>
> <!-- Macroscope's review summary starts here -->
>
> <sup><a href="https://app.macroscope.com">Macroscope</a> summarized
cea2226.</sup>
> <!-- Macroscope's review summary ends here -->
>
<!-- Macroscope's pull request summary ends here -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants