Serialize intercepted completions directly to JSON bytes

[xeophon]

Overview

Serialize non-streaming intercepted model completions directly to JSON bytes before returning them to harnesses. This keeps the response contract unchanged while reducing synchronous work and transient allocations on the shared interception-server event loop.

Why

The previous completion-return paths called web.json_response(completion). That route first uses stdlib json.dumps to materialize a complete Python string, then asks aiohttp to UTF-8 encode that string into the final response bytes.

Large completions therefore held the JSON string, encoder working memory, and final byte body at the same time. Serialization is synchronous, so it also stalled every rollout multiplexed through that interception server.

Implementation

A focused _completion_response helper now:

• serializes the JSON-native completion with pydantic_core.to_json;
• passes the resulting bytes directly to web.Response;
• preserves application/json; charset=utf-8, status, null, and the existing NaN/Infinity policy;
• falls back to web.json_response when pydantic-core cannot encode an otherwise stdlib-compatible value, including escaped lone Unicode surrogates.

Only the three successful model-completion return sites use the helper: ordinary/tool returns, user-simulator stop returns, and context-length returns after a completed turn. Error, streaming, auxiliary, state, task, and upstream client codec paths are unchanged.

Performance

Measured with a 32 MiB completion over 10-loop medians:

Metric	Before	After	Saved
Serialization wall time	33.330 ms	8.955 ms	24.375 ms (73.1%)
Event-loop stall	33.386 ms	8.988 ms	24.398 ms (73.1%)
Peak traced allocation	72.001 MiB	32.002 MiB	39.999 MiB (55.6%)
Retained traced allocation	32.002 MiB	32.002 MiB	unchanged

The retained allocation is the final response body, which must remain alive until aiohttp writes it. The peak reduction comes from eliminating the intermediate JSON string and full-body string-to-bytes encoding pass. These figures isolate response serialization rather than model or network latency.

---

Note

Low Risk
Narrow change to response encoding on the interception hot path with a stdlib-compatible fallback; wire shape and status behavior for errors are unchanged.

Overview
Non-streaming successful model completion responses from the interception server now go through a new _completion_response helper instead of web.json_response(completion).

The helper encodes the completion dict with pydantic_core.to_json (inf_nan_mode="constants") into bytes and returns web.Response with application/json; charset=utf-8. If pydantic-core cannot serialize the payload, it falls back to the previous web.json_response path.

Only the three happy-path completion returns in handle_request use this helper: after a @stop/limit with a prior turn, after context_length truncation with a prior turn, and on the normal tool / no-user-simulator return. Error, streaming, aux, and state routes are unchanged.

^{Reviewed by Cursor Bugbot for commit cf1b400. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Serialize intercepted completions directly to JSON bytes using pydantic_core.to_json

Adds a _completion_response helper in server.py that serializes completion dicts to bytes via pydantic_core.to_json with inf_nan_mode="constants", returning a web.Response with explicit application/json content type. Falls back to aiohttp.web.json_response on PydanticSerializationError. All three completion-returning branches in handle_request (refusal, context-length exceeded, tool/no-user-simulator) now use this helper.

^{Macroscope summarized cf1b400.}

[macroscopeapp]

Approvability

Verdict: Approved

Performance optimization that serializes JSON responses more efficiently using pydantic_core's to_json(). The change includes a fallback to the original web.json_response() behavior, making it low-risk and self-contained.

^{You can customize Macroscope's approvability policy. Learn more.}