fix(telegram): diagnose blocked-loop init hangs, unbind DoH discovery from system DNS - #64370
Merged
Merged
Conversation
…em DNS The NousResearch#63309 hang class — gateway stuck at 'Connecting to Telegram (attempt 1/8)' with no retry, no timeout, for minutes — can only occur when the event loop thread itself is blocked in a synchronous call: _await_with_thread_deadline's timer fires off-loop, but its expiry hand-off (call_soon_threadsafe) still needs the loop to run, and the gateway's outer wait_for is a pure loop timer. When the loop is pinned, every layer goes silent simultaneously and the process wedges with no evidence of where. Two changes: 1. Loop-blocked watchdog in _await_with_thread_deadline: a second daemon timer fires one grace period (5s) after the deadline; if the loop still hasn't processed the expiry, it logs a WARNING from the timer thread and faulthandler-dumps all thread stacks to stderr — converting the silent hang into a trace that names the exact blocking frame. A threading.Event set by the expiry callback (and on normal exit) keeps completed awaits from ever being misreported. 2. discover_fallback_ips: the system-resolver leg runs socket.getaddrinfo in a worker thread with no timeout, and asyncio.gather waited on it unboundedly — a wedged OS resolver stalled discovery for minutes between the two startup log lines. Its result only feeds a log message, so it no longer gates discovery: DoH legs (already client-bounded) are gathered alone and the system leg is awaited with a _DOH_TIMEOUT cap, best-effort. Refs NousResearch#63309 Tests: 3 watchdog regressions (blocked-loop dump fires; responsive-loop timeout does not; completed await does not) + 2 hung-resolver regressions (DoH results returned promptly; worst-case seed fallback stays bounded).
kshitijk4poor
enabled auto-merge (rebase)
July 14, 2026 11:39
alt-glitch
added
type/bug
1 task
This was referenced Aug 3, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Salvage of #63345 (CI blocked for first-time contributor). Addresses #63309 — gateway stuck at "Connecting to Telegram (attempt 1/8)..." for 9+ minutes with no retry line, no timeout message, process alive.
Changes
Attribution
Cherry-picked from @jbbottoms's #63345 with authorship preserved.
Validation
58 passed (telegram init deadline + telegram network). ruff + py_compile clean.