fix(line): inbound media caching + outbound media serving (allowed-roots copy, 24h token TTL) - #44019
Open
benagentai93-dot wants to merge 4 commits into
Open
Conversation
LINE inbound media was funneled through cache_image_from_bytes for all
message types. Its image magic-byte validation rejected mp4/m4a payloads
("Refusing to cache non-image data as .mp4"), so videos, voice clips,
and file attachments were silently dropped — the agent only saw a bare
"[video]" placeholder with no local path to analyze.
Route downloads through cache_media_bytes (same as Telegram), which
classifies by kind, keeps the HTML-error-page guard for images, and
returns an agent-visible cache path. Video and file messages now surface
the cached path in the transcript note so the agent can pass it to
video_analyze; file messages honor the webhook fileName.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
alt-glitch
added
type/bug
Collaborator
… to 24h Two outbound-media failures surfaced in production: 1. _handle_media's allowed-roots guard (tmp dirs + HERMES_HOME) 403'd files the agent legitimately sent from elsewhere (e.g. an Obsidian vault on Google Drive). The video message was created, but LINE's fetch of originalContentUrl failed, so the player showed an unplayable 0:00 bubble. Senders now route through _ensure_servable(), which copies out-of-root files into HERMES_HOME/cache/line_media/ (temp-tracked, unlinked when the token expires) so the guard holds without breaking delivery. 2. LINE clients fetch the video URL when the user taps play — often well after send. The 30-minute media token TTL meant delayed playback hit 410 gone. TTL is now 24h. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
benagentai93-dot
changed the title
… per agent init The notice is stashed in _compression_warning at agent init and replayed on the first turn. Gateways rebuild the agent on restarts, session rotation, and provider fallback, so the "one-time" banner re-fired on every rebuild and spammed chat platforms with the same text. Persist a marker file (HERMES_HOME/cache/codex_gpt55_autoraise_notified) when the notice is printed (CLI) or stashed for replay (gateway), and skip both paths once it exists. The autoraise behavior itself is unchanged — only the notification is deduplicated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The button timer lived only in the adapter's _keep_typing override, but the gateway pipeline drives send_typing directly and never awaits _keep_typing — so the slow-LLM postback button never fired. Turns slower than the ~50s reply-token TTL always fell back to metered Push, and were lost entirely once the monthly Push quota was exhausted. Arm the timer in _handle_message_event against each fresh reply token: fast turns consume the token first and the timer no-ops; slow turns convert the token into a 'Get answer' Template Buttons bubble before it expires, and the eventual response routes into the postback cache for free reply-token delivery. Timers are tracked per chat (re-armed by newer messages, cancelled on disconnect). _keep_typing now delegates to the same arming helper for callers that do await it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 3, 2026
teknium1
reviewed
Jul 14, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the LINE media investigation. The inbound cache-routing defect is real on current main: plugins/platforms/line/adapter.py:956-961 sends all media through _download_media(), and _download_media() still calls cache_image_from_bytes() for every kind at :1055-1070.
Problems
- The new webhook-level timer arm at
plugins/platforms/line/adapter.py:966bypasses the existingtyping_indicatorgate. Main starts_keep_typing()only when that setting is enabled (gateway/platforms/base.py:4857-4869), and the PR's_keep_typing()also re-arms atadapter.py:1287. - The completion callback at
adapter.py:1267-1269can remove a newer replacement task after cancelling an older one. shutil.copyfile()atadapter.py:1334runs synchronously from async send paths; LINE permits 200 MB audio/video (adapter.py:1476).agent/agent_init.pyis unrelated to the stated LINE fix and current main already has a different, state-aware Codex notice marker (agent/agent_init.py:128-175).
Suggested changes
- Salvage the focused
cache_media_bytesinbound path and tests. - Drop or correctly gate the webhook timer arm; make task cleanup identity-safe.
- Move file copying off the event loop and split the unrelated agent-init change.
Automated hermes-sweeper review.
| # no-ops; otherwise the token becomes a "Get answer" button | ||
| # before it expires, keeping slow turns deliverable without | ||
| # spending metered Push quota. | ||
| self._arm_slow_response_button(chat_id) |
Contributor
There was a problem hiding this comment.
BasePlatformAdapter._process_message_background() only starts _keep_typing() when typing_indicator is enabled (gateway/platforms/base.py:4857-4869), and the existing LINE override already arms the timer from that loop. Arming here bypasses that setting and then gets armed a second time by _keep_typing; please remove this hook or preserve the same gate and avoid duplicate arming.
| task = asyncio.create_task(self._fire_postback(chat_id)) | ||
| self._slow_button_tasks[chat_id] = task | ||
| task.add_done_callback( | ||
| lambda t, c=chat_id: self._slow_button_tasks.pop(c, None) |
Contributor
There was a problem hiding this comment.
A cancelled prior task can run this callback after its replacement has been stored and remove the replacement entry. Only pop when self._slow_button_tasks.get(c) is t; otherwise later re-arms and disconnect cannot reliably find the live task.
| cache_dir = Path.home() / ".hermes" / "cache" / "line_media" | ||
| cache_dir.mkdir(parents=True, exist_ok=True) | ||
| copy_path = cache_dir / f"{uuid.uuid4().hex[:12]}_{resolved.name}" | ||
| shutil.copyfile(resolved, copy_path) |
Contributor
There was a problem hiding this comment.
_ensure_servable() is called from async media send methods, and LINE permits inputs up to 200 MB. Move this synchronous copy off the event loop (for example via asyncio.to_thread) so a large outbound file does not stall gateway processing.
teknium1
added
sweeper:risk-message-delivery
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The LINE adapter funnels all inbound media (image/audio/video/file) through
cache_image_from_bytes, whose image magic-byte validation rejects anything that isn't PNG/JPEG/GIF/BMP/WEBP:Videos (e.g. iPhone uploads, served by LINE as mp4), voice clips (m4a), and file attachments are all silently dropped — the agent only sees a bare
[video]/[audio]/[file]placeholder with no local path, so tools likevideo_analyzeand the STT pipeline never get the media.Fix
Route inbound downloads through
cache_media_bytes(the same helper the Telegram adapter uses), which classifies by kind, keeps the HTML-error-page guard for images, and returns an agent-visible cache path:.mp4with MIMEvideo/mp4; the transcript text becomes theCachedMedia.context_note()([video 'line_video.mp4' saved at: …]) so the agent learns the local path — the gateway has no enrichment pipeline for video, unlike images/voice.m4a, reaching the VOICE/STT pathfileNameand routes to the document pipelineTests
7 new regression tests in
tests/gateway/test_line_plugin.py(TestInboundMedia), including one that reproduces the exact production mp4 bytes (ftypmp42) and a fetch-failure fallback case. All 83 LINE plugin tests pass.🤖 Generated with Claude Code