Skip to content

fix(gateway): add per-platform connect timeout to prevent blocking (#17242) - #17383

Closed
vominh1919 wants to merge 1 commit into
NousResearch:mainfrom
vominh1919:fix/17242-platform-init-timeout
Closed

fix(gateway): add per-platform connect timeout to prevent blocking (#17242)#17383
vominh1919 wants to merge 1 commit into
NousResearch:mainfrom
vominh1919:fix/17242-platform-init-timeout

Conversation

@vominh1919

Copy link
Copy Markdown
Contributor

Fixes #17242: Platform initialization failure blocks other platforms

Problem

When multiple messaging platforms are configured (e.g., Telegram + Feishu/Lark), if one platform's connect() call hangs or takes very long, it blocks initialization of all subsequent platforms.

Real-world scenario: Telegram's retry loop (8 attempts × 15s exponential backoff = up to 120s) can prevent Feishu from ever starting — common in regions where Telegram is network-restricted.

Root Cause

The startup loop in GatewayRunner.start() iterates platforms sequentially and awaits each adapter.connect() without a timeout:

for platform, platform_config in self.config.platforms.items():
    success = await adapter.connect()  # can hang indefinitely

Fix

Wrap each platform's connect() call in asyncio.wait_for() with a 90-second timeout:

_platform_connect_timeout = 90  # seconds
success = await asyncio.wait_for(adapter.connect(), timeout=_platform_connect_timeout)

On timeout:

  • The platform is logged as timed out
  • Resources are cleaned up via _safe_adapter_disconnect()
  • The platform is queued for background reconnection (just like other transient failures)
  • The next platform starts immediately

Why 90 seconds?

  • Telegram's 8-retry loop can take up to ~120s in the worst case
  • 90s is generous enough for slow networks but prevents indefinite blocking
  • The timeout is applied per-platform, so total startup time = min(90s, connect_time) × num_platforms

Testing

  • Syntax verified with ast.parse()
  • Timeout handler is consistent with existing error handling (same cleanup, same reconnection queue)

Discussed in: #17242

When multiple platforms are configured (e.g., Telegram + Feishu), a
slow/hanging platform connect() blocks initialization of all others.
Telegram's retry loop (8 attempts × 15s backoff = 120s) can prevent
Feishu from ever starting.

Wrap each adapter.connect() call in asyncio.wait_for() with a 90s
timeout. On timeout, the platform is queued for background reconnection
and the next platform starts immediately.

Fixes NousResearch#17242
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery labels Apr 29, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #13602 — both add per-platform connect timeout via asyncio.wait_for() to prevent one platform blocking others. #13602 uses asyncio.gather() for concurrency while this uses sequential + timeout.

@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #13602 — both add per-platform connect timeout.

@teknium1

Copy link
Copy Markdown
Contributor

Closed as superseded by #17429 (salvage of #17270). @tmimmanuel's PR was submitted ~5 hours earlier the same day, included tests, and covered the reconnect watcher too — so we went with that one. Thanks for the independent fix; the analysis in your PR body matched the root cause exactly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Platform initialization failure blocks other platforms (Telegram blocks Feishu)

3 participants