fix(gateway): prevent Discord disconnects from blocking event loop - #56212
Merged
Merged
Conversation
models_dev.py's fetch uses a synchronous requests.get(timeout=15). Called from the async gateway message handlers, it blocked the event loop for up to 15s, starving Discord heartbeats and causing ClientConnectionResetError disconnects. Adds get_model_context_length_async() which offloads the entire sync resolution chain to a worker thread via asyncio.to_thread(), and switches the two async gateway call sites (_prepare_inbound_message_text, _handle_message_with_agent) to await it. The loop stays responsive; the sync path remains the single source of truth for the cache. Salvaged from PR #22753 by @itenev. Follow-up: dropped the unused fetch_models_dev_async/lookup_models_dev_context_async aiohttp variants from the original PR (dead code with zero callers that had drifted from the sync cache logic) — the to_thread wrapper already runs the sync path off-loop, so they were redundant.
12 tasks
alt-glitch
added
type/bug
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Discord gateway no longer disconnects with
ClientConnectionResetErrorwhen the agent resolves model metadata during message handling.Root cause:
agent/models_dev.pyfetches the models.dev registry with a synchronousrequests.get(timeout=15). Called from the async gateway handlers, that call blocked the single-threaded asyncio event loop for up to 15s — starving Discord heartbeats and triggering "Cannot write to closing transport" disconnects.Changes
agent/model_metadata.py: addget_model_context_length_async()— offloads the entire existing sync resolution chain to a worker thread viaasyncio.to_thread(). The sync path stays the single source of truth for the models.dev cache.gateway/run.py: switch the two async call sites (_prepare_inbound_message_text,_handle_message_with_agent) toawait get_model_context_length_async(...). The third call site (_format_session_info) is a plain syncdefand is correctly left untouched.Salvage note
Salvaged from @itenev's PR #22753 with authorship preserved. Dropped the original PR's
fetch_models_dev_async/lookup_models_dev_context_asyncaiohttp variants (86 LOC): they had zero callers (dead code) and had drifted from the sync cache logic (missing the fresh-by-mtime disk-cache short-circuit). Theto_threadwrapper already runs the sync path off-loop, so the parallel aiohttp implementation was redundant. Net: 119/-4 → 34/-4.Validation
E2E: cold-cache resolution (0.49s real network fetch through the sync chain) while a 50ms heartbeat task ran concurrently.
to_thread)Tests:
tests/agent/test_models_dev.py+tests/agent/test_model_metadata.py— 137 passed.Infographic