fix(telegram): truncate outbound captions by UTF-16 length, not codepoints - #49324
briandevans wants to merge 2 commits into
Conversation
…oints Telegram measures caption length in UTF-16 code units (1024 cap), the same way it measures the 4096-unit message cap. Astral-plane characters (emoji, CJK Extension B, musical symbols, …) encode as surrogate pairs and consume two UTF-16 code units each, while Python's str slice counts codepoints. So the nine outbound caption sites that truncated with a naive `caption[:1024]` (and `alt_text[:1024]`) codepoint slice let a caption of <=1024 codepoints containing such characters exceed 1024 UTF-16 units, and Telegram then rejects the entire send with `Bad Request: message caption is too long`. The repo already solved this exact discrepancy for message *text* via `utf16_len()` and `_prefix_within_utf16_limit()` in gateway/platforms/base.py, but the helper had zero call sites — it was written and never wired in. Wire it into the nine caption sites (send_voice, send_audio, send_multiple_images alt_text, send_image_file, send_document, send_video, send_image / animation paths), preserving the existing `if caption else None` guard. The helper binary-searches a surrogate-safe prefix whose UTF-16 length stays within the limit.
There was a problem hiding this comment.
Pull request overview
Fixes Telegram outbound caption truncation to respect Telegram’s 1024 UTF-16 code unit limit (rather than Python codepoints), preventing Bad Request: message caption is too long failures when captions contain astral-plane characters (e.g., emoji).
Changes:
- Wire
gateway.platforms.base._prefix_within_utf16_limit(..., 1024)into all Telegram outbound caption/alt_text truncation sites that previously used[:1024]. - Add regression tests ensuring a 700-emoji caption (1400 UTF-16 units) is truncated to
<= 1024UTF-16 units forsend_documentandsend_video.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
gateway/platforms/telegram.py |
Replaces naive caption/codepoint slicing with UTF-16-aware prefix truncation across all outbound caption sites. |
tests/gateway/test_telegram_documents.py |
Adds regression coverage validating UTF-16-based truncation behavior for document/video captions containing emoji. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| assert result.success is True | ||
| connected_adapter._bot.send_document.assert_called_once() | ||
| sent_caption = connected_adapter._bot.send_document.call_args[1]["caption"] | ||
| # Codepoint slice would leave 1024 codepoints == 2048 UTF-16 units. |
alt-glitch
added
type/bug
The inline comment claimed a codepoint slice would leave '1024 codepoints == 2048 UTF-16 units', but the test input is only 700 emoji. A naive caption[:1024] slice leaves all 700 codepoints intact (700 < 1024) = 1400 UTF-16 units, not 1024/2048. Align the comment with the docstring and the actual 700-emoji case so the regression is clear to future readers. Comment-only; behavior unchanged.
|
@copilot Fixed in f3bb337. You were right — the input is 700 emoji, so a naive |
|
Draining our contribution queue to keep it lean and reviewable \u2014 closing this as stale (no reviewer traction in 2+ weeks). If the underlying issue is still live on current main, we'll re-file a fresh, focused fix. |
What does this PR do?
Telegram measures caption length in UTF-16 code units (1024 cap) — the same way it measures the 4096-unit message-text cap. Characters outside the Basic Multilingual Plane (emoji, CJK Extension B, musical symbols, …) encode as surrogate pairs and consume two UTF-16 code units each, while a Python
strslice counts codepoints.The nine outbound caption sites in
gateway/platforms/telegram.pytruncated with a naivecaption[:1024](and onealt_text[:1024]) codepoint slice. A caption of ≤1024 codepoints containing astral-plane characters therefore passes the slice unchanged yet exceeds 1024 UTF-16 units, and Telegram rejects the entire send withBad Request: message caption is too long.Repro: a caption of
'😀' * 700is 700 codepoints but 1400 UTF-16 units — the codepoint slice leaves it untouched, the send fails.The repo already solved this exact discrepancy for message text via
utf16_len()and_prefix_within_utf16_limit()ingateway/platforms/base.py(the latter binary-searches a surrogate-safe prefix). But_prefix_within_utf16_limithad zero call sites — it was written and never wired in. This PR wires the existing helper into the nine caption sites, preserving the existingif caption else Noneguards.Related Issue
Type of Change
Changes Made
gateway/platforms/telegram.py: import_prefix_within_utf16_limitfromgateway.platforms.base(alongside the already-importedutf16_len), and replacecaption[:1024]/alt_text[:1024]at the nine outbound caption sites (send_voice,send_audio,send_multiple_imagesalt_text,send_image_file,send_document,send_video, thesend_image/ animation paths) with_prefix_within_utf16_limit(..., 1024).tests/gateway/test_telegram_documents.py: regression tests forsend_documentandsend_videoasserting a 700-emoji caption (1400 UTF-16 units) is truncated to ≤1024 UTF-16 units and remains a valid surrogate-safe prefix.How to Test
'😀' * 700(1400 UTF-16 units).Bad Request: message caption is too long. After: the caption is truncated to ≤1024 UTF-16 units and the send succeeds.pytest tests/gateway/test_telegram_documents.py -q— all pass; the two new tests fail before the prod hunk (caption stays at 1400 units) and pass after.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/A