fix(v1): restore the unbounded model-call timeout

[mikasenghaas]

What broke

#2218 consolidated client construction into build_async_openai / DEFAULT_TIMEOUT. In doing so it replaced the relay client's explicit no-timeout:

-        # No timeout: agentic completions are slow and the rollout timeout is the real backstop.
-            timeout=None,
+DEFAULT_TIMEOUT = httpx.Timeout(connect=5.0, read=600.0, write=600.0, pool=600.0)  # Mirrors the OAI SDK defaults
+        self.client = httpx.AsyncClient(timeout=DEFAULT_TIMEOUT, limits=DEFAULT_LIMITS)

The comment explaining why there was no timeout was removed along with it.

The cap now applies to every model-call path, because DEFAULT_TIMEOUT feeds both the relay client in clients/eval.py and build_async_openai, which judge.py and clients/train.py use.

Why it matters for RL

600s is a reasonable default for chat-shaped evals and the wrong one for RL. A single agentic turn on a long SWE trajectory routinely runs past ten minutes. When it does:

1. httpx raises TimeoutException
2. clients/eval.py maps it to a 504 via model_error
3. the rollout dies as ProviderError: Request timed out

The rollout is lost even though the model was still producing. Worse, the loss is biased, not uniform — it selectively kills the longest trajectories, which are the ones RL cares most about. Observed on a 6-node SWE RL run against a local vLLM fleet.

The fix

Restore the pre-#2218 behaviour for read/write/pool, and put back the rationale that was deleted with it:

# No read timeout: agentic completions are slow and the rollout timeout is the real
# backstop. The connect bound stays so an unreachable endpoint still fails fast.
DEFAULT_TIMEOUT = httpx.Timeout(connect=5.0, read=None, write=None, pool=None)

This isn't leaving rollouts unbounded — verifiers already bounds them properly through RolloutTimeouts (setup / agent / episode / finalize). A transport-level read timeout is redundant with that real backstop and only truncates slow but healthy generations.

One deliberate difference from a pure revert: connect stays at 5.0 rather than reverting to a blanket timeout=None, so an unreachable endpoint still fails fast instead of hanging until the rollout deadline. Only the timeouts that bound generation are lifted.

🤖 Generated with Claude Code

Note

Restore unbounded read/write/pool timeouts in DEFAULT_TIMEOUT for model calls

Sets read, write, and pool timeouts to None in base.py, removing the previous 600s limits. The connect timeout remains bounded at 5s. Behavioral Change: any httpx client using DEFAULT_TIMEOUT will no longer time out on slow or long-running model responses.

^{Macroscope summarized 5167355.}

---

Note

Medium Risk
Shared HTTP client defaults affect every eval/train/judge model path; removing read timeouts can let hung provider connections run until rollout timeouts unless those layers fire reliably.

Overview
DEFAULT_TIMEOUT in verifiers/v1/clients/base.py no longer caps long model responses at 600s. read, write, and pool are set back to None; connect stays at 5.0s so dead endpoints still fail quickly.

That shared default is used by build_async_openai (train/judge paths) and the eval relay httpx.AsyncClient, so the change applies across all v1 model-call clients. Rollout limits via RolloutTimeouts remain the intended backstop; the restored comment documents that agentic generations should not be cut off by transport read timeouts.

^{Reviewed by Cursor Bugbot for commit 5167355. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Approved 5167355

Small configuration change removing bounded HTTP timeouts for model calls, with clear documentation that the rollout timeout serves as the actual backstop. The author is the original author of this file and provided appropriate rationale.

^{You can customize Macroscope's approvability policy. Learn more.}