[xeophon]

An intercepted model call has one session-owned task for hooks, inference, response encoding, and trace commit. Retries share that task and reuse its completed result. Tool decisions and rewritten results live in one record per call occurrence.

Before — main

The server already coalesces running requests and replays completed responses, but maintains a separate in-flight future, response cache, and finalization callbacks. A normal socket disconnect can leave inference running; cancellation of the original handler can cancel its producer. Tool state is split across decision, result, and gate-tracking collections.

flowchart LR
    H[Original request] --> P[Handler-owned producer]
    P --> C[Client get_response or relay]
    C --> F[Finalize attempt]
    F --> I[In-flight future]
    F --> R[Completed-response cache]
    Q[Concurrent retry] --> I
    L[Later retry] --> R

Loading

After — this PR

One retained task represents both the running attempt and its result. HTTP readers wait through a cancellation shield; a disconnected or cancelled reader does not cancel the model call. Ending the rollout cancels the session's outstanding tasks. The client completes inference while the server owns HTTP delivery, keepalives, and error framing.

flowchart LR
    H[Original request or retry] --> D[HTTP delivery and keepalives]
    D -->|Shielded wait| T[Session-owned task]
    T --> C[Hooks and client completion]
    C --> E[Encode response before trace commit]
    E --> R[Task retains result for replay]
    R --> D
    X[Rollout ends] -. cancels .-> T

Loading

Failure handling follows the commit boundary:

• A retryable failure before trace commit releases the attempt so a subsequent retry can run again.
• Terminal failures and failures after commit remain replayable, preventing a retry from recording another answer for the same retained attempt.
• Errors after SSE headers carry status and retryability. The bundled harness retries errors marked retryable through its existing Tenacity loop.

ToolState keeps the blocked decision, gate consultation, and replacement result together. Repeated histories receive the same rewritten result. Request hooks and tool pre-checks share their evaluator. The retained turn task also acknowledges input delivery after previewing the rewritten request, keeping notification delivery tied to the actual model turn.

Matching explicit idempotency keys or marked SDK retries share work; fresh identical unmarked requests remain independent. Replay is bounded and local to the rollout session; it does not survive a process restart. Completion content remains buffered until the turn is ready, with keepalives during the wait. This PR owns lifecycle and tool state. #2675 owns dialects, sampling, and SDK transport; #2676 owns client unification and training records.

Live CI defaults unpinned agent roles to Docker; explicitly configured runtime placements remain intact.

Stack: main → #2674 → #2675 → #2676. The server/trainer companion is prime-rl #3660.

Validation on current main (e71c69c): all 163 unit tests passed on this branch. The full stack also passes 175 unit tests and contributor style/type checks.

---

Note

High Risk
Changes core interception idempotency, streaming error/retry semantics, and tool-gate enforcement at the model boundary; incorrect retry or coalescing could duplicate turns or let blocked tools run.

Overview
This PR unifies intercepted model turns around one session-owned asyncio task that handles hooks, client._complete, response encoding, and trace commit. Concurrent SDK retries and idempotent replays coalesce on that task (shielded from reader disconnect/cancel); retryable failures are dropped from replay storage so a later retry can run again, while completed or post-commit errors stay replayable.

Eval client now owns streaming completion: EvalClient._complete relays SSE, parses the stream, and returns parsed Response plus raw events; RelayReply no longer carries content-type. The interception server focuses on HTTP delivery, keepalives, and framing errors with retryable on ReplayResponse (including SSE after commit).

Harness chat() retries transport errors and APIError bodies with retryable: true; terminal API errors do not retry (new unit coverage).

Tool policy moves into per-occurrence ToolState (blocked, gate consulted, pinned result), replacing split decision/result/gate maps; request rewriting skips re-running hooks on already-pinned tool results and shares _run_request_hooks with pre-checks.

Tests/CI: integration eval config defaults each agent seat to runtime: docker when unset.

^{Reviewed by Cursor Bugbot for commit b0a3837. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Simplify interception turn lifecycle with shared replay tasks and ToolState

• Reworks InterceptionServer.handle_request to coalesce concurrent identical requests onto one shared ReplayResponse task, replay completed non-retryable results, and drop retryable failures from replay storage in server.py
• Moves stream collection and parsing from the server into EvalClient._complete, which returns the parsed response plus original data events and maps timeout/transport failures to defined error types in eval.py
• Adds per-prefix, per-call-id ToolState so blocked tool results cannot enter a model request before their gate is consulted, and request rewriting no longer re-runs hooks on already-pinned results in session.py
• Retries OpenAI API errors only when their error body marks them retryable; terminal API errors propagate without another request, in core.py
• Risk: RelayReply drops its content-type field and IdempotentRequest replaces separate response/future fields with a single task; the removed _collect_stream and response-capture helpers mean any out-of-tree consumers of these shapes must be updated

^{Macroscope summarized b0a3837.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-15T20:58:44.088904Z	b0a3837	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR substantially rewires production interception, streaming, retry/idempotency, trace-commit, and tool-gating behavior across core modules rather than making a contained refactor. An unresolved review comment also identifies a transport-cleanup failure path that may incorrectly suppress retries.

Notes:

• All code in this push has already been reviewed. Approvability was decided on eligibility alone.

No code changes detected at b0a3837. Prior analysis still applies.

^{You can add or adjust custom eligibility rules. Learn more.}

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1558d27247

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve retryability when stream cleanup fails

If reply.close() raises—for example, while a transport is closing a reset streamed connection—the exception originates in this finally block and therefore bypasses all of the preceding except mappings. It reaches InterceptionServer._fail() as a plain exception, which converts it to a non-retryable 400 InterceptionError; this prevents the harness from retrying what is still an upstream transport failure and can also replace the more useful exception from stream consumption. Map cleanup failures to ProviderError/model_error (without masking an existing error) before they leave _complete().

Useful? React with 👍 / 👎.