fix(gateway): retry restart notification on transient send-path failures - #68569
Closed
occam-7 wants to merge 3 commits into
Closed
fix(gateway): retry restart notification on transient send-path failures#68569occam-7 wants to merge 3 commits into
occam-7 wants to merge 3 commits into
Conversation
_send_restart_notification() fires during gateway startup, but platform adapters may still be settling after reconnect. The Telegram adapter's _send_path_degraded flag can still be True from the disconnect/reconnect cycle, causing the one-shot notification to fail silently. The notification file is deleted in the finally block regardless of outcome, so a failed notification is permanently lost. Add a retry loop (5 attempts, 1s delay) for transient/retryable errors like send_path_degraded, giving the adapter time to complete its first successful polling cycle and clear the degraded flag. Non-retryable errors still fail immediately.
alt-glitch
added
type/bug
…nd-path failures Same root cause as _send_restart_notification: during gateway startup, platform adapters may not be fully ready. The Telegram adapter's _send_path_degraded flag can still be True, causing the home-channel startup notification to fail silently. Add the same retry loop (5 attempts, 1s delay) for transient errors like send_path_degraded.
occam-7
force-pushed
the
fix/restart-notify-retry
branch
from
July 21, 2026 12:16
7c33ac8 to
0496c2f
Compare
_begin_polling_generation() sets _send_path_degraded=True proactively, blocking all sends through the live adapter until the first getUpdates completes. Since getUpdates is a long-poll with up to ~30s timeout, this blocks lifecycle notifications (restart, startup) for up to 30s after gateway boot. The flag should only reflect actual errors. Remove the proactive set — let real network errors set it, and _record_polling_progress clear it when getUpdates succeeds.
teknium1
reviewed
Jul 30, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for targeting a real restart-delivery gap. Current main still makes one attempt, returns on a false result (gateway/run.py:17892-17899), then unconditionally removes the marker (gateway/run.py:17911).
Problems
- Removing the
_send_path_degradedassignment conflicts with the current polling-health contract: each new generation is deliberately gated until a dedicatedgetUpdatesresponse succeeds (plugins/platforms/telegram/adapter.py:782-787,2076-2093), andsend()refuses delivery while the path is degraded (4332-4333). - The proposed five retries still fall through to the existing unconditional marker deletion after exhaustion (
gateway/run.py:17911), so a longer transient outage still loses the notification. - The PR changes no tests, despite existing integration coverage of the polling-progress invariant in
tests/test_telegram_polling_progress_ptb.py:131-148and233-254.
Suggested changes
- Preserve the Telegram safety gate and rework retry at the current
DeliveryTransportboundary (gateway/run.py:17852,17882-17887), including relay routing. - Specify and test marker ownership and consumption for retryable failure, success, exhaustion, and shutdown.
This is an automated hermes-sweeper review.
| @@ -2043,7 +2043,14 @@ def _begin_polling_generation(self) -> tuple[int, asyncio.Event]: | |||
| self._polling_generation = getattr(self, "_polling_generation", 0) + 1 | |||
Contributor
There was a problem hiding this comment.
Do not remove this gate without replacing its safety proof: current main deliberately keeps every new polling generation degraded until its dedicated getUpdates request succeeds (plugins/platforms/telegram/adapter.py:782-787, 2076-2093), and send() refuses delivery while that flag is set (4332-4333).
teknium1
added
sweeper:risk-compatibility
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two related issues cause gateway restart/startup notifications to silently fail on Telegram:
1. Notifications fire before adapter is ready
After gateway restart,
_send_restart_notification()and_send_home_channel_startup_notifications()fire during startup, but the Telegram adapter may not be fully ready. The notification file is deleted on failure, permanently losing the message.2.
_send_path_degradedis set prematurely_begin_polling_generation()sets_send_path_degraded = Trueproactively, blocking ALL sends through the live adapter until the firstgetUpdatescompletes. SincegetUpdatesis a long-poll with up to ~30s timeout, this blocks lifecycle notifications for up to 30 seconds. The flag should only reflect actual errors, not the "still initializing" state.Fix
Retry on transient failures: Both
_send_restart_notification()and_send_home_channel_startup_notifications()now retry up to 5 times (1s delay) on transient errors likesend_path_degraded.Remove proactive degraded flag:
_begin_polling_generation()no longer sets_send_path_degraded = True. The flag is only set on actual network errors and cleared after successfulgetUpdates. The Bot API'ssendMessageuses a separate connection fromgetUpdates, so sends are safe during polling startup.