[mikasenghaas]

Summary

Companion to PrimeIntellect-ai/verifiers#3042 and PrimeIntellect-ai/prime#1140 (released in prime-runs 0.1.6). The file monitor and the Prime monitor encode each episode in short steps, so a large episode no longer stalls the orchestrator's event loop for as long as encoding it takes.

• Pins.
  • The verifiers submodule moves to verifiers main, which includes verifiers#3042: to_json encodes one node at a time and releases the GIL every millisecond, and record_run records the run on every trace.
  • prime-runs 0.1.6, already on main, contains prime#1140: the SDK uploads episode.to_json() as it is when the episode and its traces name the upload's run. The dispatcher's record_run makes that true for RL episodes.
  • Verifiers main also includes feat: remove the eval's platform push and the prime-runs dependency verifiers#3046, which removes vf-eval's platform push and verifiers' prime-runs dependency. In uv.lock (uv 0.12.0), only the verifiers metadata changes.
• The file monitor writes episode.to_json().
  • The stream line is byte for byte what orjson.dumps(episode.to_record()) gave before.
  • The index row comes from index_record(episode). It dumps everything but the nodes' payloads, one trace at a time, and gives the same row.
  • ChunkedJsonl.append takes a line in parts, so the newline is written without copying a long line.
• The load test's lag limits drop from p99 100 ms and max 250 ms to 25 ms and 100 ms, so a regression shows. Across runs at production sizes, the highest values were a p99 of 12.5 ms and a max of 22 ms.
• A hosted orchestrator stamps the platform's run id. A hosted launch sets only $RUN_ID, not $PRL_RUN_ID, so the orchestrator stamped episodes with a random id, and the SDK re-stamped every upload. It now falls back to $RUN_ID, the Prime run id.

Verification

Headline: event-loop lag at 1,000 production-sized episodes (~8 GB of JSON), monitor load test, dev box:

	Main (before)	This PR
prime: lag p99 / max	14 / 17 ms	3.1 / 3.5 ms
file: lag p99 / max	47 / 93 ms	4.8 / 15 ms
file: log() max	177 ms	195 ms

• A second run: inside the full unit suite, with more load on the box, it measured prime 11 / 18 ms, file 5.2 / 22 ms, and file log() max 224 ms.
• Why the GIL release matters: encoding node by node without it left the file monitor at 15–18 / 31–66 ms, varying from run to run. A busy encoder thread keeps the GIL until CPython's 5 ms switch interval forces a handoff, and the event loop waits that out several times per wakeup.

The stall no longer grows with the episode. One thread encodes episodes of 10 to 10,000 turns (1,000 tokens per turn) while a 5 ms ticker on the event loop records the worst stall. Medians of 5 runs, under the orchestrator's GC policy:

Turns	Episode	Encode CPU, before / after	Worst stall, before	Worst stall, after
10	0.2 MB	0.4 / 0.5 ms	5.5 ms	5.2 ms
100	2.2 MB	4.0 / 4.5 ms	8.0 ms	1.6 ms
1,000	22 MB	50 / 51 ms	40 ms	3.1 ms
3,000	65 MB	152 / 167 ms	100 ms	5.2 ms
10,000	217 MB	508 / 571 ms	326 ms	15 ms

• Before: the stall grows linearly with the episode.
• After: it stays around 2–5 ms up to 3,000 turns, about the GIL switch interval. The 10-turn case is that interval alone, since each episode encodes in under a millisecond.
• The rest at 10,000 turns: the trace header's pydantic dump, which holds every model call (one per turn). Alone it stalls 4.4 / 8.4 / 15 ms at 1,000 / 3,000 / 10,000 turns. Agentic episodes in production have a few hundred turns, so this part stays small. Encoding the calls one at a time, like the nodes, would remove it.
• CPU: encoding costs up to ~12% more, from the extra calls and GIL releases.

Encoding: test_received_episode_encodes_as_its_record_did sends a two-trace episode over the real delta wire into a WireEpisode. Then:

• the file monitor's stream line equals orjson.dumps(to_record()), byte for byte;
• its index row equals the row from the full record;
• after record_run, the SDK takes the episode's own to_json(), which carries the same record as the old stamped line.

Tests: pytest tests/unit -m "not gpu": 1,077 passed, 7 failed. The same 7 fail on main on this dev box: an attention kernel without sm_120 support, and no ninja for the CPU Adam extension.

CI: on the CI runner, the file monitor at 1,000 production-sized episodes measured event-loop lag p99 6.0 ms and max 12 ms, against 63 / 152 ms on main. Its slowest log() was 382 ms, under the 1 s limit.

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes affect how every episode is persisted and indexed and how run identity is chosen on hosted launches; behavior is contract-tested but touches high-volume I/O on the training path.

Overview
Large episodes no longer block the orchestrator event loop during trace logging. The file monitor writes stream lines with episode.to_json() (incremental, GIL-friendly encoding from verifiers) instead of a single orjson.dumps(to_record()), and builds index rows via new index_record() so summarization skips heavy node payloads without materializing the full record first. ChunkedJsonl.append accepts byte parts so the newline is appended without copying megabyte-scale lines.

The orchestrator run_id now falls back to $RUN_ID when $PRL_RUN_ID is unset, so hosted runs align with the platform id and Prime uploads are not re-stamped with a random id.

Monitor load tests use stricter event-loop lag thresholds to catch regressions. A new unit test asserts on-disk bytes and SDK upload JSON still match the legacy to_record() encoding.

^{Reviewed by Cursor Bugbot for commit f83e849. Bugbot is set up for automated code reviews on this repo. Configure here.}