Skip to content

language_models: Add stream idle timeout for ChatGPT subscription requests - #59123

Closed
rtfeldman wants to merge 1 commit into
mainfrom
richard/ai-317-agent-gets-stuck-loading-for-chatgpt-subscriptions
Closed

language_models: Add stream idle timeout for ChatGPT subscription requests#59123
rtfeldman wants to merge 1 commit into
mainfrom
richard/ai-317-agent-gets-stuck-loading-for-chatgpt-subscriptions

Conversation

@rtfeldman

Copy link
Copy Markdown
Contributor

ChatGPT subscription (Codex backend) requests can stall — the server accepts the connection but never sends response headers, or goes silent mid-stream — and Zed would wait forever, leaving the agent stuck on a loading spinner with no error and no retry, since Zed's HTTP client only has a connect timeout. This adds an activity-based idle timeout to those requests: a timer covers the window from sending the request until response headers arrive, and is then reset every time a stream event arrives. If nothing happens for 5 minutes, the request fails with a retryable HttpSend error so the agent's existing backoff/retry logic takes over. The OAuth token refresh request gets a similar (60s) bound, since it could hang a turn the same way before the completion request even started.

This has some history: #57891 attempted to fix the same stall with a 10-second response-header timeout, and was reverted in #58035 after reports of GPT models abruptly stopping mid-turn. The problem was that the Codex backend doesn't send response headers until the model starts producing output, so at high reasoning effort healthy requests routinely take longer than 10 seconds to first byte, and the timeout was killing them on every attempt. (OpenCode shipped the same 10s default around the same time and hit the same regression: anomalyco/opencode#29548.) An idle timeout sidesteps that failure mode because it only needs to bound silence rather than time-to-first-byte plus thinking time, so it can be generous. The 5-minute value matches the stream_idle_timeout_ms default that OpenAI's own Codex CLI uses against this same backend (codex-rs/core/src/model_provider_info.rs), and is also the default headersTimeout that Node's undici applies (300s) — i.e. the bound OpenCode effectively ran with before their regression (see anomalyco/opencode#15555). Notably, the complaints about the Codex CLI's 5-minute default run in the too slow direction (openai/codex#17003), not the false-positive direction, which is good evidence this won't cut off healthy long thinks.

The timeout is only wired up for the ChatGPT subscription provider for now, since that's where the stalls have been reported (#57636).

Closes AI-317

Release Notes:

  • Fixed the agent getting stuck loading indefinitely when a ChatGPT subscription request stalls; stalled requests now time out after 5 minutes of inactivity and retry automatically.

…uests

The Codex backend sometimes accepts a connection but never sends
response headers, or goes silent mid-stream, leaving the agent stuck
on a spinner forever since Zed has no read timeout. Add an
activity-based 5-minute idle timeout (matching the Codex CLI's
stream_idle_timeout default) that resets on every stream event and
surfaces as a retryable HttpSend error. Also bound the OAuth token
refresh request, which had the same unbounded-hang problem.
@rtfeldman rtfeldman self-assigned this Jun 11, 2026
@cla-bot cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Jun 11, 2026
@zed-community-bot zed-community-bot Bot added the staff Pull requests authored by a current member of Zed staff label Jun 11, 2026
@rtfeldman rtfeldman closed this Jun 11, 2026
@rtfeldman
rtfeldman deleted the richard/ai-317-agent-gets-stuck-loading-for-chatgpt-subscriptions branch June 11, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The user has signed the Contributor License Agreement staff Pull requests authored by a current member of Zed staff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant