Skip to content

fix(telegram): fall back to reachable IP when proxy can't reach primary anycast IP - #59090

Open
Djaler wants to merge 1 commit into
NousResearch:mainfrom
Djaler:fix/telegram-proxy-fallback-ip
Open

fix(telegram): fall back to reachable IP when proxy can't reach primary anycast IP#59090
Djaler wants to merge 1 commit into
NousResearch:mainfrom
Djaler:fix/telegram-proxy-fallback-ip

Conversation

@Djaler

@Djaler Djaler commented Jul 5, 2026

Copy link
Copy Markdown

Summary

TELEGRAM_PROXY and the fallback-IP transport were mutually exclusive, so a proxy whose upstream egress can only reach part of the api.telegram.org anycast block (e.g. a DPI-bypass router) could not connect at all — even when a reachable Telegram IP existed.

Two root causes:

  1. _is_retryable_connect_error() excluded httpx.ProxyError. httpx.ProxyError is not a subclass of httpx.ConnectError, so when the primary DNS path resolved to an IP the proxy couldn't reach, the resulting ProxyError: Host unreachable was treated as fatal and the transport never fell through to the fallback IPs.

  2. The adapter engaged TelegramFallbackTransport only when no proxy was set (if fallback_ips and not proxy_url). With a proxy configured, plain httpx DNS picked an unreachable IP and there was no IP-pinning fallback.

Fix

  • Treat httpx.ProxyError as retryable so the transport falls through to the configured/discovered fallback IPs.
  • Engage TelegramFallbackTransport whenever fallback IPs are present (it already threads TELEGRAM_PROXY through internally), so proxy + pinned IPs work together. The log line now distinguishes the proxied path.

Net diff: +35 / -7 across two files, no behavior change for the default (no-proxy, no-fallback) path.

Repro

With TELEGRAM_PROXY=socks5://<proxy> where the proxy's egress reaches only a subset of Telegram's anycast IPs, the gateway logged repeated Proxy Server could not connect: Host unreachable. and never connected. After this change it falls through to the reachable IP and polling succeeds.

Test Plan

Ran the relevant Telegram suites locally — 104 passed:

  • tests/gateway/test_telegram_network.py
  • tests/gateway/test_telegram_network_reconnect.py
  • tests/gateway/test_telegram_send_path_health.py
  • tests/gateway/test_telegram_conflict.py
  • tests/gateway/test_telegram_init_deadline.py
  • tests/gateway/test_telegram_closewait_limits_31599.py
  • tests/gateway/test_telegram_send_draft_format.py

No new env vars, no config surface, no core-tool changes — a self-contained bug fix in the Telegram adapter's connection path.

The fallback-IP transport and TELEGRAM_PROXY were mutually exclusive: the
adapter only engaged TelegramFallbackTransport when no proxy was set, and
_is_retryable_connect_error() excluded httpx.ProxyError (which is not a
subclass of httpx.ConnectError).

When TELEGRAM_PROXY points at a proxy whose upstream egress can only reach a
subset of the api.telegram.org anycast block (e.g. a DPI-bypass router), the
primary DNS path resolves to an IP the proxy cannot reach and raises
ProxyError 'Host unreachable'. Because that error was non-retryable and the
fallback branch was skipped whenever a proxy was configured, the whole request
failed instead of falling through to a reachable fallback IP.

Fixes:
- Treat httpx.ProxyError as retryable so the transport falls through to the
  configured/discovered fallback IPs.
- Engage TelegramFallbackTransport whenever fallback IPs are present (it
  threads TELEGRAM_PROXY through internally), so proxy + pinned-IP work
  together. Log line distinguishes the proxied path.

Verified against tests/gateway/test_telegram_network*.py and related Telegram
suites (104 passed).
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Jul 5, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: #58790 (parent CLOSE_WAIT fd-leak issue, now closed) and its fixes #58804/#58803 (keepalive limits in the fallback transport), and #47172 (open — proxy-exhaustion diagnostic hint). This PR fixes a different mechanism in the same TelegramFallbackTransport path: httpx.ProxyError was excluded from retryable errors and the fallback transport only engaged when no proxy was set, so TELEGRAM_PROXY + pinned fallback IPs could not work together. Not a duplicate — competing/complementary fix in the same file, different bug.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating this to the existing Telegram fallback transport. The premise is present on current main: plugins/platforms/telegram/adapter.py:3385 excludes proxy configurations from the fallback branch, while plugins/platforms/telegram/telegram_network.py:268 does not classify httpx.ProxyError as retryable. The fallback transport already forwards TELEGRAM_PROXY to its inner transports at telegram_network.py:61-68, so the proposed direction fits the existing design.

Problems

  • The PR diff contains no tests. Existing coverage exercises ConnectTimeout and ConnectError fallback behavior (tests/gateway/test_telegram_network.py:149-181) and proxy construction (:320-336), but not the new ProxyError retry path or proxy-plus-fallback adapter selection.

Suggested changes

  • Add a transport regression test for primary httpx.ProxyError followed by a successful fallback IP.
  • Add an adapter-path test proving non-empty fallback IPs plus a proxy instantiate TelegramFallbackTransport for both request pools.

This is an automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants