fix(telegram): fail fast when user images cannot be accessed - #22413
Open
HiddenPuppy wants to merge 2 commits into
Open
HiddenPuppy wants to merge 2 commits into
HiddenPuppy wants to merge 2 commits into
Conversation
When a user sends an image via Telegram and the vision analysis pipeline fails (e.g. no Gemini key configured), the agent previously received either an empty user message or a hint telling it to retry with vision_analyze — causing up to 30+ minutes of futile retries. Changes: - Normalize MIME type 'image/jpg' → 'image/jpeg' in the Telegram photo handler to avoid non-standard MIME types - In _enrich_message_with_vision: track all_failed across all images. When ALL analyses fail AND the user provided no caption, return a clear fail-fast message instead of prompting the agent to retry - In _build_media_placeholder: return a neutral fallback string instead of empty string when there are no media_urls at all, preventing the agent from receiving empty user input Fixes NousResearch#22385
alt-glitch
added
type/bug
teknium1
reviewed
Jul 13, 2026
teknium1
left a comment
teknium1
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for isolating the post-cache failure path. The core issue remains on current main: gateway/run.py:15054-15065 still tells the model to retry vision_analyze after a failed automatic analysis.
Problems
- The Telegram hunk is stale: the live adapter moved from
gateway/platforms/telegram.pytoplugins/platforms/telegram/adapter.py; the active MIME assignment is now atplugins/platforms/telegram/adapter.py:7801. GitHub currently reports this PR asCONFLICTING. - The diff adds no regression tests. Existing
_enrich_message_with_visioncoverage intests/gateway/test_vision_memory_leak.pycovers successful analyses, not the all-failed and exception paths this change relies on.
Suggested changes
- Transplant the MIME fix to the bundled Telegram adapter, using its existing extension-to-MIME map at
plugins/platforms/telegram/adapter.py:236-242. - Add focused tests for all-failed/no-caption, exception/no-caption, and mixed success/failure image inputs, plus the empty-media placeholder if retained.
Automated hermes-sweeper review.
| cached_path = cache_image_from_bytes(bytes(image_bytes), ext=ext) | ||
| event.media_urls = [cached_path] | ||
| event.media_types = [f"image/{ext.lstrip('.')}" ] | ||
| # Normalize MIME: "image/jpg" → "image/jpeg" (standard MIME) |
Collaborator
There was a problem hiding this comment.
This handler has moved on current main to plugins/platforms/telegram/adapter.py:7737-7813, where the live assignment still emits image/jpg at line 7801. Please transplant this normalization there (the plugin already defines _TELEGRAM_IMAGE_EXT_TO_MIME at lines 236-242) and cover it with a regression test.
| @@ -12171,6 +12180,7 @@ async def _enrich_message_with_vision( | |||
| ) | |||
Collaborator
There was a problem hiding this comment.
Please add regression coverage for this new all-failed path: success: false and thrown exceptions with no caption should return the fail-fast text, while a mixed success/failure batch must preserve the successful description and the original caption behavior.
teknium1
added
sweeper:risk-message-delivery
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a user sends an image via Telegram and the vision analysis pipeline fails (e.g. no Gemini key configured, transient API error), the agent previously received either an empty user message or a hint telling it to retry with
vision_analyze. This caused the agent to spend 30+ minutes in futile retry loops (issue #22385).Root Cause
Two gaps conspired to create the loop:
_enrich_message_with_visionpipeline subsequently failed for all images, the agent received the user's caption text (or an empty string if there was no caption), plus a hint telling it to tryvision_analyze— which would fail again with the same reason, causing another retry.media_urlsat all,_build_media_placeholderreturned an empty string, giving the agent nothing to work with.Changes
gateway/platforms/telegram.pyimage/jpg→image/jpegto use standard MIME typesgateway/run.py—_enrich_message_with_visionall_failedacross all images. When ALL analyses fail AND the user provided no caption, return a clear fail-fast message ([The user sent an image but I'm unable to see it...]) instead of prompting the agent to retrygateway/run.py—_build_media_placeholderTesting
pytest tests/gateway/— 751 passed, 1 skipped, 1 pre-existing failure (unrelated DingTalk test)Fixes #22385