Skip to content

fix(telegram): recover when a polling-conflict retry fails - #27099

Closed
randiri100 wants to merge 1 commit into
NousResearch:mainfrom
randiri100:fix/telegram-conflict-retry-recovery
Closed

fix(telegram): recover when a polling-conflict retry fails#27099
randiri100 wants to merge 1 commit into
NousResearch:mainfrom
randiri100:fix/telegram-conflict-retry-recovery

Conversation

@randiri100

Copy link
Copy Markdown

Problem

When Telegram returns a 409 Conflict: terminated by other getUpdates request, _handle_polling_conflict stops the updater, waits, and retries start_polling(). If that retry itself fails (e.g. a TimedOut/NetworkError because the host's connectivity is also flaky at that moment), the except block only logs the failure and returns — relying on a comment that says "wait for the next conflict to trigger another retry attempt".

But once start_polling() fails, the long-poll is dead and no error callback can ever fire again. There is no "next conflict". The bot is left permanently wedged with polling stopped, until the gateway process is manually restarted. Outbound sends keep working, which masks the failure — cron-driven messages still go out while the bot silently stops receiving anything.

Observed in production: a 409 conflict at the same time as a brief network blip → one retry → start_polling() raised TimedOut → handler returned → ~2h of dropped inbound messages until a manual restart.

Fix

The network-error handler (_handle_polling_network_error) already guards against this exact dead-end: when its retry's start_polling() fails, it self-schedules the next attempt instead of relying on a callback that can't fire.

This applies the same pattern to the conflict handler. When a conflict retry's start_polling() fails:

  • a transient network failure routes into _handle_polling_network_error (exponential-backoff ladder, which can escalate to a retryable-fatal gateway restart);
  • anything else re-enters the conflict ladder.

Either way the recovery is self-scheduled as a background task, so the poller is never left wedged with nothing to wake it.

Test

Adds test_conflict_retry_network_failure_schedules_recovery to tests/gateway/test_telegram_conflict.py, asserting a conflict retry that fails with a network error routes into the network reconnect ladder. Full suite (test_telegram_conflict.py + test_telegram_network_reconnect.py) passes — 22 tests.

🤖 Generated with Claude Code

When a 409 "terminated by other getUpdates request" conflict triggers a
retry and that retry's start_polling() itself fails, the conflict handler
only logged the failure and returned, on the assumption that a "next
conflict" callback would drive another retry attempt.

But once start_polling() fails the long-poll is dead, so no error callback
will ever fire again. There is no "next conflict" — the bot stays wedged
with polling stopped until the gateway is manually restarted. Outbound
sends keep working, which masks the failure.

This is the same dead-end the network-error handler already guards
against by self-scheduling its next retry. Apply the same pattern to the
conflict handler: when a retry's start_polling() fails, self-schedule
recovery — route transient network failures into the network reconnect
ladder (which can escalate to a retryable-fatal gateway restart) and
re-enter the conflict ladder otherwise.

Add a regression test covering a conflict retry that fails with a
network error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cardtest15-coder

This comment was marked as spam.

@cardtest15-coder

This comment was marked as spam.

@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround platform/telegram Telegram bot adapter comp/gateway Gateway runner, session dispatch, delivery labels May 16, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Competes with #25630 (same polling-conflict recovery pattern). Both fix the dead-poller scenario where _handle_polling_conflict's retry start_polling() fails and nothing re-triggers recovery. Also related to #25232.

@teknium1

Copy link
Copy Markdown
Contributor

Closing in favor of #25630 (broader scope on the same bug — #23783 / silent-limbo after failed polling-conflict retry).

Both PRs apply the same insight: _handle_polling_conflict's except branch silently returns, leaving the bot wedged. Both PRs apply the self-rescheduling pattern from _handle_polling_network_error. #25630 (@CryptoByz) was submitted first and additionally addresses Stage 1 (hermes update / watcher respawn race against Telegram's ~30s server-side session grace).

Thanks for the parallel work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P1 High — major feature broken, no workaround platform/telegram Telegram bot adapter type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants