Skip to content

fix(cli): prevent agent-loop freeze when a provider stalls after headers - #12588

Merged
marius-kilocode merged 1 commit into
mainfrom
fix/provider-first-byte-timeout
Jul 28, 2026
Merged

fix(cli): prevent agent-loop freeze when a provider stalls after headers#12588
marius-kilocode merged 1 commit into
mainfrom
fix/provider-first-byte-timeout

Conversation

@marius-kilocode

@marius-kilocode marius-kilocode commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

What this fixes

The agent turn no longer hangs indefinitely when a provider accepts a request, returns response headers, and then never sends a byte of body data. Previously step-finish:tool-calls was recorded and the next step-start never arrived, the session stayed busy with no error, and the HTTP server kept responding — the freeze reported in #8656.

That issue is not MCP-specific (the reporter reproduced it with plain bash tool calls and no MCP); it is a transport-level gap in Kilo's provider fetch wrapper.

Root cause

Kilo already had a default 5-minute request timeout (REQUEST_TIMEOUT_MS), but the fetch wrapper cleared it the moment response headers arrived. The post-header, pre-first-byte window was therefore unbounded by default: a response whose headers landed but whose body never produced data could sit forever, leaving the agent loop frozen between a finished tool call and the next LLM step.

Change

Extend the same configured timeout deadline to the wait for the response body's first byte.

  • The connection-phase timer still covers the fetch up to headers.
  • Once headers arrive, the remaining deadline is handed to wrapFirstByte, which aborts the request if no body data arrives in time.
  • After the first byte the guard becomes a passthrough, so idle gaps inside an already-streaming response (reasoning, buffering, slow token generation) are never touched. Those stay opt-in via chunkTimeout.
  • timeout: false still disables the bound entirely.

The signal is bytes on the wire before any content, not the absence of normalized AI SDK events, so it cannot fire on long prompt processing or reasoning the way the reverted stream watchdog did — see #12249 and its revert #12497. The implementation lives in the Kilo-owned provider mirror; the shared provider.ts change is a minimal marked hook.

requestTimeout now falls back to the default for invalid/unset values so the wait is always bounded; only timeout: false disables it.

Test

The regression test injects a simulated stalled socket as the provider's own fetch through the plugin config hook (the supported injection point: provider.ts loads plugins before reading cfg.provider), so the openai-compatible SDK, Kilo's fetch wrapper, SSE parsing, the processor and the agent loop all stay production code. Nothing global is patched and no process env is mutated.

The stalled request is transient, so the test asserts the outcome a user cares about: the turn recovers by retrying and completing instead of freezing. Timeline with the fix:

```
text | step-start | tool:bash:completed | step-finish:tool-calls | step-start | text | step-finish:stop
```

The second step-start is what #8656 reports as never arriving.

Red without the fix, green with it:

provider calls stalls recovered result
fix stashed 2 1 0 fails after 40s, frozen at `step-finish:tool-calls`
fix applied 3 1 1 passes, turn completes with text

A second test keeps the raw freeze reachable through the documented timeout: false opt-out and asserts the server stays responsive during the hang, with explicit abort-and-wait-for-idle teardown.

Scope

This bounds one class of agent-turn freeze: a provider response that sends headers and no body data. It does not cover stalls where the provider sends keepalives or SSE comments and then goes silent (those need chunkTimeout), nor non-transport causes.

#8656 is referenced as a matching symptom rather than closed by this change; confirming it as the reported cause needs either reporter validation on this build or transport logs showing the affected request received headers but no body data.

A provider can accept a request, return response headers, and then never
send a byte of body data. The connection-phase request timeout was cleared
as soon as headers arrived, so nothing bounded that wait and the agent turn
hung indefinitely after a tool call completed: step-finish:tool-calls was
recorded and the next step-start never arrived, with the HTTP server still
responsive.

Extend the same configured timeout deadline to the wait for the response
body's first byte. The connection-phase timer covers the fetch up to
headers; once headers arrive, the remaining deadline is handed to a
first-byte guard that aborts the request if no data arrives. After the
first byte the guard becomes a passthrough, so idle gaps inside an already
streaming response (reasoning, buffering, slow token generation) are never
touched and remain opt-in via chunkTimeout.

This is a transport-level signal (bytes on the wire, before any content)
rather than the absence of normalized AI SDK events, so it cannot fire on
long prompt processing or reasoning the way the reverted stream watchdog
did. timeout: false still disables the bound entirely.

Adds a hermetic regression test that injects a simulated stalled socket
through the provider's own fetch option via the plugin config hook, so the
SDK, Kilo's fetch wrapper, SSE parsing, the processor and the agent loop
all stay production code. The stalled request is transient, so the test
asserts the turn recovers by retrying and completing instead of freezing.
The test goes red without the fix (no retry, frozen at step-finish) and
green with it.

Refs #8656
@marius-kilocode marius-kilocode changed the title fix(cli): bound provider first response byte by the request timeout fix(cli): prevent agent-loop freeze when a provider stalls after headers Jul 28, 2026
@kilo-code-bot

kilo-code-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (8 files)
  • .changeset/stalled-provider-first-byte.md
  • packages/kilo-docs/pages/code-with-ai/agents/custom-models.md
  • packages/opencode/src/kilocode/provider/provider.ts
  • packages/opencode/src/provider/provider.ts
  • packages/opencode/test/kilocode/fixture/stall-plugin.ts
  • packages/opencode/test/kilocode/fixture/stall-transport.ts
  • packages/opencode/test/kilocode/issue-8656-stall.test.ts
  • packages/opencode/test/kilocode/provider/first-byte.test.ts

Reviewed by claude-sonnet-5 · Input: 38 · Output: 15K · Cached: 1.2M

Review guidance: REVIEW.md from base branch main

@marius-kilocode
marius-kilocode merged commit f336cfe into main Jul 28, 2026
32 checks passed
@marius-kilocode
marius-kilocode deleted the fix/provider-first-byte-timeout branch July 28, 2026 09:00
t7tran pushed a commit to t7tran/kilocode that referenced this pull request Aug 14, 2026
…te-timeout

fix(cli): prevent agent-loop freeze when a provider stalls after headers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants