fix(api-server): cancel orphaned agent task when SSE client disconnects - #3399
Closed
binhnt92 wants to merge 1 commit into
Closed
fix(api-server): cancel orphaned agent task when SSE client disconnects#3399binhnt92 wants to merge 1 commit into
binhnt92 wants to merge 1 commit into
Conversation
When a streaming /v1/chat/completions client disconnects mid-stream (network drop, browser tab close), response.write raises ConnectionResetError but the agent_task created via ensure_future is never cancelled. The orphaned agent continues consuming API tokens and memory with no reference held to it. Wrap the SSE write loop in try/except to catch disconnect errors and cancel the agent task in the except handler.
teknium1
added a commit
that referenced
this pull request
Mar 27, 2026
The original PR (#3399) caught disconnect errors and cancelled the asyncio task, but run_in_executor tasks can't be interrupted by asyncio cancellation — the agent thread keeps running and consuming API tokens. Wire agent.interrupt() via a mutable agent_ref container: - _run_agent() stores the AIAgent at agent_ref[0] before run_conversation - On SSE disconnect, the except block calls agent.interrupt() which sets _interrupt_requested and signals tools to abort - The agent stops at the next loop iteration boundary Added 2 tests: interrupt is called on disconnect, agent_ref=None still handles disconnect gracefully.
teknium1
added a commit
that referenced
this pull request
Mar 27, 2026
…nect (salvage #3399) (#3427) Salvage of #3399 by @binhnt92 with true agent interruption added on top. When a streaming /v1/chat/completions client disconnects mid-stream, the agent is now interrupted via agent.interrupt() so it stops making LLM API calls, and the asyncio task wrapper is cancelled. Closes #3399.
Contributor
angelburgosrosado
pushed a commit
to angelburgosrosado/hermes-agent
that referenced
this pull request
Apr 27, 2026
…nect (salvage NousResearch#3399) (NousResearch#3427) Salvage of NousResearch#3399 by @binhnt92 with true agent interruption added on top. When a streaming /v1/chat/completions client disconnects mid-stream, the agent is now interrupted via agent.interrupt() so it stops making LLM API calls, and the asyncio task wrapper is cancelled. Closes NousResearch#3399.
02356abc
pushed a commit
to 02356abc/hermes-agent
that referenced
this pull request
May 14, 2026
…nect (salvage NousResearch#3399) (NousResearch#3427) Salvage of NousResearch#3399 by @binhnt92 with true agent interruption added on top. When a streaming /v1/chat/completions client disconnects mid-stream, the agent is now interrupted via agent.interrupt() so it stops making LLM API calls, and the asyncio task wrapper is cancelled. Closes NousResearch#3399.
gweeteve
pushed a commit
to gweeteve/hermes-agent
that referenced
this pull request
Jun 2, 2026
…nect (salvage NousResearch#3399) (NousResearch#3427) Salvage of NousResearch#3399 by @binhnt92 with true agent interruption added on top. When a streaming /v1/chat/completions client disconnects mid-stream, the agent is now interrupted via agent.interrupt() so it stops making LLM API calls, and the asyncio task wrapper is cancelled. Closes NousResearch#3399.
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…nect (salvage NousResearch#3399) (NousResearch#3427) Salvage of NousResearch#3399 by @binhnt92 with true agent interruption added on top. When a streaming /v1/chat/completions client disconnects mid-stream, the agent is now interrupted via agent.interrupt() so it stops making LLM API calls, and the asyncio task wrapper is cancelled. Closes NousResearch#3399.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a streaming
/v1/chat/completionsclient disconnects mid-stream (network drop, browser tab close, Open WebUI navigation),response.write()raisesConnectionResetErrorbut theagent_taskcreated viaasyncio.ensure_futureis never cancelled. The orphaned agent keeps running — making LLM API calls, consuming tokens and memory — with no reference held to it. This repeats on every dropped streaming client.Changes Made
Wrapped the SSE write loop in
_write_sse_chat_completionwithtry/exceptto catch disconnect errors (ConnectionResetError,BrokenPipeError,ConnectionAbortedError,OSError). On disconnect, if the agent task is still running, it's cancelled and awaited. If the agent already finished before the disconnect, it's left alone.How to Test
4 tests covering:
ConnectionResetErrorBrokenPipeErrorChecklist