Persist large V1 traces off the event loop

[xeophon]

Overview

This moves V1 trace persistence off the event loop while retaining one durable JSONL append per completed trace.

• Make completion persistence awaitable across native, env-server, and legacy eval paths.
• Serialize and write in a worker thread behind one async lock per eval run.
• Use Pydantic's public TypeAdapter.dump_json bytes API and append binary JSONL, avoiding the text decode/re-encode path.
• Shield lock acquisition and the active worker so finalized traces still persist when their caller is cancelled.

Why

Large traces previously called model_dump_json() and performed a text-mode append synchronously from completion callbacks. Three traces completing together could monopolize the event loop for the entire combined serialization window. The text path also converted Pydantic's UTF-8 output to a Python string and encoded it again for the file, adding roughly one payload-sized transient allocation.

A per-run lock keeps serialization and writes ordered without introducing a persistent writer lifecycle. Awaiting the shielded persistence task preserves queued traces, error propagation, and the existing per-trace open/write/close boundary.

Observed impact

The event-loop workload used three simultaneous 29,128,201-byte traces (87,384,603 bytes total). Five-run component-wise medians:

Metric	Before	After	Change
Three-trace wall time	206.081 ms	214.333 ms	8.252 ms slower (4.0%)
Maximum event-loop gap	206.775 ms	74.585 ms	132.190 ms saved (63.9%)

A single 29,256,201-byte Unicode trace measured the serialization allocation path:

Metric	Before	After	Saved
Serialization/write time	84.767 ms	72.586 ms	12.181 ms (14.4%)
Peak traced allocation	87,837,018 B	58,580,197 B	29,256,821 B (33.3%)
Process high-water RSS	334,413,824 B	276,283,392 B	58,130,432 B (17.4%)

The serialized JSON representation, subclass fields, ordering, write-error propagation, and per-trace file-close behavior remain unchanged.

---

Note

Low Risk
Changes are localized to eval output I/O and completion callbacks; JSONL format and per-trace durability semantics are preserved, with only concurrency/cancellation behavior around writes being new.

Overview
V1 eval trace persistence no longer blocks the asyncio event loop during large results.jsonl appends. Each completed trace still gets one durable JSONL line with the same ordering guarantees as before.

append_trace is now async: Pydantic serializes to UTF-8 bytes via TypeAdapter.dump_json, and the file append runs in a worker thread (asyncio.to_thread) behind a per-run asyncio.Lock so concurrent completions cannot interleave lines. asyncio.shield keeps an in-flight persist from being dropped on caller cancellation.

Episode.run takes an optional async on_complete hook (awaited when each trace is finalized). Native run_eval, env-server run_eval_server, and run_legacy_eval each create a shared write lock and await append_trace from their completion paths.

^{Reviewed by Cursor Bugbot for commit 35372c2. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Persist large V1 traces off the event loop using async writes and a shared lock

• Replaces synchronous append_trace calls with an async version in output.py that uses asyncio.to_thread to run file writes in a worker thread, keeping the event loop unblocked.
• Serializes concurrent trace writes via an injected asyncio.Lock shared across rollouts, preserving whole-line ordering in results.jsonl.
• Wraps each write in a shielded task so cancellation does not corrupt partially written traces.
• Updates Episode.run, run_eval, run_eval_server, and run_legacy_eval to pass the shared lock and await the async callback.
• Behavioral Change: the on_complete callback in Episode.run is now async; the default is None instead of a no-op lambda.

^{Macroscope summarized 35372c2.}

[macroscopeapp]

Approvability

Verdict: Approved

This is a focused performance improvement that moves blocking trace serialization off the event loop using standard async patterns. Changes are limited to trace persistence code with consistent updates to all call sites.

^{You can customize Macroscope's approvability policy. Learn more.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1ed6625dd6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Dismissing prior approval to re-evaluate d97c79d