Skip to content

fix(telegram): recover when polling updater stops while process stays alive (#55769) - #55921

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-7647a357
Jun 30, 2026
Merged

fix(telegram): recover when polling updater stops while process stays alive (#55769)#55921
teknium1 merged 1 commit into
mainfrom
hermes/hermes-7647a357

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

The Telegram gateway can silently stop receiving messages while the process stays alive (systemd green, threads healthy, send path fine) — no logs, no errors — until manually restarted (#55769). This makes the gateway self-heal from that state.

Root cause: the polling heartbeat's _probe_pending_updates already handled a wedged-but-running long-poll consumer, but treated a fully stopped updater (running == False, no reconnect in flight) as "someone else's job" — it reset its counter and returned. Because get_me() on the general request path stays healthy, neither PTB's error_callback nor the connectivity heartbeat ever fires. Result: process alive, send path fine, polling dead, indefinitely.

Changes

  • plugins/platforms/telegram/adapter.py_probe_pending_updates now detects updater.running == False and feeds it into the existing _handle_polling_network_error recovery ladder (stop → drain pool → start_polling). Debounced over two consecutive probes via a new _polling_not_running_count. The in-flight-reconnect guard is moved ahead of the updater check so the reconnect's own transient stop()start_polling() window (where running is briefly False) can't false-trip. No new restart machinery, no new config keys, no new env vars.
  • tests/gateway/test_telegram_pending_update_probe.py — replaced the test that encoded the old buggy "stopped updater = no-op" assumption with coverage for: single stopped probe does not escalate, two consecutive stopped probes trigger recovery, a recovered (running) updater resets the counter, and an in-flight reconnect suppresses escalation.

Validation

Before After
Updater stops (running=False) watchdog blind, silent forever detected, routed into recovery ladder
Reconnect-in-flight stop/start window (n/a — never reached) guarded first, no false trip
Targeted tests 9/9 pass
Sibling reconnect/polling tests 62/62 pass

Salvaged from @PRATHAMESH75's PR #55789, cherry-picked onto current main with authorship preserved.

Infographic

Telegram polling self-heal

Nous Research

… alive

The polling heartbeat's pending-update probe treated a stopped updater
(running=False) as "someone else's job" and silently reset its counter,
so a long-poll task that disappears with no reconnect in flight was never
recovered. get_me() on the general request path stays healthy, so neither
PTB's error_callback nor the connectivity probe ever fires — the gateway
keeps running but stops receiving messages indefinitely (#55769).

Detect the stopped-updater case directly in _probe_pending_updates and feed
it into the existing _handle_polling_network_error ladder, debounced over two
consecutive probes so a just-starting updater or the brief stop()->start_polling()
window of an in-flight reconnect never trips it.
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages P3 Low — cosmetic, nice to have duplicate This issue or pull request already exists labels Jun 30, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #55905, which is the same salvage of #55789 and was merged (2026-06-30) -- the stopped-updater recovery branch (_polling_not_running_count, updater.running == False -> escalate after two probes) is already in main:plugins/platforms/telegram/adapter.py. This PR is redundant; closing/superseded is the maintainer's call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have platform/telegram Telegram bot adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants