Skip to content

fix(gateway): route Telegram image-MIME documents through the photo cache path - #20700

Closed
jackControls wants to merge 1 commit into
NousResearch:mainfrom
jackControls:fix/telegram-image-as-document
Closed

fix(gateway): route Telegram image-MIME documents through the photo cache path#20700
jackControls wants to merge 1 commit into
NousResearch:mainfrom
jackControls:fix/telegram-image-as-document

Conversation

@jackControls

@jackControls jackControls commented May 6, 2026

Copy link
Copy Markdown

What & Why

When a Telegram user sends an image with the "Send as file" option (or, on mobile, sends without ticking "Compress images"), python-telegram-bot delivers it as msg.document with an image/* MIME type — not as msg.photo.

The current document handler in gateway/platforms/telegram.py has no entry for image MIMEs in SUPPORTED_DOCUMENT_TYPES. So:

  1. ext is set to "" (no image/png.png mapping)
  2. if ext not in SUPPORTED_DOCUMENT_TYPES: fires
  3. The user sees "Unsupported document type 'unknown'"
  4. The vision pipeline (native attach / vision_analyze) never sees the bytes

This is surprising for users who do tick "Compress images" — they do see the image — but breaks the moment they uncompress.

Fix: add a small early branch in the existing elif msg.document: handler that detects image/* MIME types and routes them through the same cache_image_from_bytes + photo-batch enqueue path as msg.photo. ~30 lines, lives in the existing document branch (no new top-level handlers), and reuses the existing _photo_batch_key / _enqueue_photo_event plumbing. Photos and image-as-files behave identically end-to-end after this lands.

Recognized image MIMEs / extensions: image/jpeg, image/jpg, image/png, image/webp, image/gif, image/heic, image/heif, image/bmp.

How to test

With agent.image_input_mode: "native" + a multimodal main model (Qwen-VL, etc.):

  1. Photo via "Compress images" — works (existing behaviour)
  2. Same PNG via "Send as file" — model now describes it correctly (was hallucinating before)
  3. .heic / .gif / .webp document — same behaviour
  4. application/pdf document — still hits the existing PDF handling path (no regression)
  5. video/mp4 document — still hits the existing video handling path (no regression)

Tested on

  • End-to-end (production): Linux x86_64 (Debian 13 / "trixie") in a Docker container, python-telegram-bot 21.x, vLLM serving a Qwen-VL multimodal model. Mobile Telegram (iOS) sending both compressed photos and "Send as file" PNG/JPG, plus PDF / MP4 documents to verify no regressions.
  • Unit tests (locally on macOS 15, Python 3.11, against this branch):
    pytest tests/gateway/test_telegram_documents.py \
           tests/gateway/test_telegram_photo_interrupts.py \
           tests/gateway/test_telegram_caption_merge.py -v
    ============================== 53 passed in 5.17s ==============================
    

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter comp/gateway Gateway runner, session dispatch, delivery labels May 6, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #16710 (and related: #20147, #19446, #19023) — all route Telegram image documents (sent as files) through the photo cache path instead of rejecting as unsupported. See also issue #20128.

…ache path

When a Telegram user picks "Send as file" instead of "Compress images",
python-telegram-bot delivers the PNG/JPG as msg.document with an
image/* MIME type, not as msg.photo. The current document handler has
no entry for image MIMEs in SUPPORTED_DOCUMENT_TYPES, so the file
falls through to the "Unsupported document type 'unknown'" branch and
the vision pipeline (native attach / vision_analyze) never sees the
bytes. Models receive only the caption text and tend to hallucinate
explanations like "couldn't read PNG as text" since they have no
image to describe.

Add an early branch in the existing elif msg.document: handler that
detects image/* MIME types and routes them through the same
cache_image_from_bytes + photo-batch enqueue path as msg.photo. After
the patch, photos and image-as-files behave identically end-to-end.

Recognized: image/jpeg, image/jpg, image/png, image/webp, image/gif,
image/heic, image/heif, image/bmp.
@jackControls
jackControls force-pushed the fix/telegram-image-as-document branch from f59e206 to 42b93ba Compare May 6, 2026 12:15
@jackControls jackControls changed the title gateway/telegram: route image-MIME documents through the photo cache path fix(gateway): route Telegram image-MIME documents through the photo cache path May 6, 2026
@jackControls

Copy link
Copy Markdown
Author

Thanks — closing as a duplicate of #16710 / #20147 / #19446 / #19023 (per maintainer feedback). Will track those for the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants