Skip to content

fix(telegram): cancel delayed deliveries on disconnect - #55971

Merged
teknium1 merged 2 commits into
mainfrom
hermes/hermes-eee44b9d
Jul 1, 2026
Merged

fix(telegram): cancel delayed deliveries on disconnect#55971
teknium1 merged 2 commits into
mainfrom
hermes/hermes-eee44b9d

Conversation

@teknium1

@teknium1 teknium1 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Telegram no longer dispatches buffered messages into a torn-down session after disconnect. Salvage of #17082 (@CRWuTJ), re-targeted onto current main where the adapter now lives at plugins/platforms/telegram/adapter.py.

Root cause: text/photo/media-group flushes and the polling-error recovery task sit behind an asyncio.sleep(). disconnect() only cancelled the media-group and photo-batch tasks — the text-batch tasks and polling-error task leaked, and no flush had a drop guard, so any flush already past its sleep() still called handle_message() after teardown, spawning an agent on a dead session (stale / duplicate deliveries).

Changes

  • plugins/platforms/telegram/adapter.py:
    • _drop_delayed_deliveries flag set by _mark_disconnected / _set_fatal_error, cleared by _mark_connected; checked in all 3 enqueue + 3 flush paths so a flush that loses the race to teardown drops instead of dispatching, and late update handlers don't schedule new delayed tasks during teardown.
    • New _cancel_pending_delivery_tasks() cancels + clears all four task maps (media-group, photo, text, polling-error), skipping the current task; awaits only real awaitables.
    • disconnect() calls _mark_disconnected() first, then the shared cancel helper (removes the two ad-hoc cancellation blocks).
    • Media-group flush finally block guarded so a cancelled stale flush can't erase a replacement task handle.
  • tests/gateway/test_telegram_text_batching.py: 9 regression tests (disconnect drops pending text/photo/media flushes, late enqueue dropped, stale media flush doesn't clear newer task, cancel helper skips current task, full disconnect clears all maps).
  • scripts/release.py: AUTHOR_MAP entry for @CRWuTJ.

Validation

Check Result
Telegram gateway (-k telegram) 1078 passed
test_telegram_text_batching.py 14 passed
E2E (real asyncio tasks): disconnect mid-flush, late enqueue, reconnect clears flag, normal delivery all 4 pass

Infographic

infographic

Nous Research

@alt-glitch alt-glitch added type/bug Something isn't working platform/telegram Telegram bot adapter comp/gateway Gateway runner, session dispatch, delivery sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages P3 Low — cosmetic, nice to have labels Jul 1, 2026
CRWuTJ and others added 2 commits June 30, 2026 17:29
Buffered text/photo/media-group flushes and the polling-error recovery
task sit behind an asyncio.sleep(). On disconnect they kept running and
dispatched handle_message() into a torn-down session, producing stale or
duplicate deliveries. disconnect() only cancelled media-group and photo
batch tasks — text batches and the polling-error task leaked.

Set a _drop_delayed_deliveries flag from _mark_disconnected/_set_fatal_error
(cleared by _mark_connected) and check it in all enqueue+flush paths so a
flush that wins the race against teardown drops instead of dispatching.
_cancel_pending_delivery_tasks() now cancels+clears all four task maps,
skipping the current task. Media-group flush finally-block guarded so a
cancelled stale flush cannot erase a replacement task handle.
@teknium1
teknium1 force-pushed the hermes/hermes-eee44b9d branch from e868a78 to b2a6f5f Compare July 1, 2026 00:30
@teknium1
teknium1 merged commit 698c287 into main Jul 1, 2026
31 checks passed
@teknium1
teknium1 deleted the hermes/hermes-eee44b9d branch July 1, 2026 00:39
dvbaecker added a commit to dvbaecker/hermes-agent that referenced this pull request Aug 13, 2026
…troying them

The disconnect drop-guard (NousResearch#55971) correctly prevents dispatch into a
torn-down session. Destroying the event was wrong: by enqueue/flush time
python-telegram-bot has already acked the update and advanced the polling
offset, so Telegram never redelivers. Result: silent permanent loss, no
log, no error.

Hold inbound events (text/photo/media-group) when the drop-guard fires,
salvage pending batch maps on teardown, cancel+await the redispatch task
in the delivery cancel map (lifecycle-tracked), and redispatch from
_mark_connected after reconnect. Cap the hold queue (default 64), dedupe
by object identity, discard on non-retryable fatal. Cancel-after-pop in
flush paths also holds.

Distinct from NousResearch#72037 (cancel-after-pop during follow-up supersession) and
NousResearch#81528 (boundary discard). Tests use delay=0 and entered/release Events —
no wall-clock races; includes production terminal-step coverage.
kshitijk4poor pushed a commit that referenced this pull request Aug 15, 2026
…troying them

The disconnect drop-guard (#55971) correctly prevents dispatch into a
torn-down session. Destroying the event was wrong: by enqueue/flush time
python-telegram-bot has already acked the update and advanced the polling
offset, so Telegram never redelivers. Result: silent permanent loss, no
log, no error.

Hold inbound events (text/photo/media-group) when the drop-guard fires,
salvage pending batch maps on teardown, cancel+await the redispatch task
in the delivery cancel map (lifecycle-tracked), and redispatch from
_mark_connected after reconnect. Cap the hold queue (default 64), dedupe
by object identity, discard on non-retryable fatal. Cancel-after-pop in
flush paths also holds.

Distinct from #72037 (cancel-after-pop during follow-up supersession) and
#81528 (boundary discard). Tests use delay=0 and entered/release Events —
no wall-clock races; includes production terminal-step coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have platform/telegram Telegram bot adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants