Skip to content

fix(feishu): pass quoted message media to Hermes when replying - #37262

Open
wengxiaoxiong wants to merge 2 commits into
NousResearch:mainfrom
wengxiaoxiong:fix/feishu-reply-media-passthrough
Open

fix(feishu): pass quoted message media to Hermes when replying#37262
wengxiaoxiong wants to merge 2 commits into
NousResearch:mainfrom
wengxiaoxiong:fix/feishu-reply-media-passthrough

Conversation

@wengxiaoxiong

@wengxiaoxiong wengxiaoxiong commented Jun 2, 2026

Copy link
Copy Markdown

Problem

When a user replies to an image or file message in Feishu and @mentions Hermes, the quoted message's media was silently dropped — Hermes only saw the reply text, never the original image or file.

Root Cause

_fetch_message_text() only extracted text from the quoted (parent) message and ignored any media attachments.

Fix

  • Added _fetch_parent_message_with_media() which fetches the quoted message and also downloads its images/files via the existing _download_feishu_message_resources() helper, returning (text, media_urls, media_types).
  • Replaced _fetch_message_text() call in _process_inbound_message() with the new method.
  • Merged parent media_urls/media_types into the current message's media lists so they are forwarded to the agent.
  • If the current message is plain text but the parent has media, upgrade inbound_type to the parent's media type (falls back to PHOTO on unknown types).

How to Test

  1. In Feishu, send an image to a chat where Hermes is present.
  2. Reply to that image message and @mention Hermes with a question (e.g. "what's in this image?").
  3. Hermes should now receive and describe the image. Previously it would respond as if no image was present.

Testing

  • ✅ Verified manually in a live Feishu workspace — replying to an image message and @mentioning Hermes now correctly surfaces the image to the agent.
  • ✅ 6 new unit tests added in tests/gateway/test_feishu.py (TestFeishuFetchParentMessageWithMedia): text message, image message, cache hit, API failure, no-client guard, and end-to-end inbound merge.
  • ✅ All 6 tests pass locally.

Platform

Tested on Linux. This change is Feishu-platform-specific (no file I/O, no process management, no cross-platform concerns).

When a user replies to an image/file message and @mentions Hermes,
the quoted message's media was silently dropped — Hermes only saw
the reply text, never the original image or file.

Root cause: _fetch_message_text() only extracted text from the
quoted message, ignoring any media attachments.

Fix:
- Add _fetch_parent_message_with_media() which fetches the quoted
  message and also downloads its images/files via the existing
  _download_feishu_message_resources() helper, returning
  (text, media_urls, media_types).
- Call _fetch_parent_message_with_media() instead of
  _fetch_message_text() in the inbound message handler.
- Merge parent media_urls/media_types into the current message's
  media lists so they are forwarded to the agent.
- If the current message is plain text but the parent has media,
  upgrade inbound_type to the parent's media type (e.g. PHOTO)
  so downstream handlers treat it as a media message.
Also fix a bug where MessageType enum was called with an unsupported
'default' keyword argument — use try/except ValueError instead.

Tests cover:
- text message returns text + empty media lists
- image message returns text + populated media lists
- cache hit skips API call and returns empty media
- API failure returns (None, [], [])
- no client returns (None, [], [])
- _process_inbound_message merges parent media into the event
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists platform/feishu Feishu / Lark adapter comp/gateway Gateway runner, session dispatch, delivery labels Jun 2, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing a real Feishu reply-context gap. Current main still fetches only parent text at plugins/platforms/feishu/adapter.py:3259, while the existing resource helper at :3771-3800 can hydrate the parent attachments.

Problems

  • gateway/platforms/feishu.py:4018-4020 returns empty media on a text-cache hit. After one reply to a quoted image/file, every subsequent reply to that same parent loses the attachment again; tests/gateway/test_feishu.py:5007-5020 currently expects that loss.
  • gateway/platforms/feishu.py:3082-3084 passes MIME strings such as image/jpeg to MessageType; these are not enum values, so all parent media falls back to PHOTO. On current main, PHOTO enters Feishu media batching (plugins/platforms/feishu/adapter.py:3307-3324), changing the semantics of a text reply and misclassifying audio/document/video parents.
  • The implementation path was relocated to plugins/platforms/feishu/adapter.py by 560010547.

Suggested changes

  • Port this to the bundled Feishu plugin, preserve the triggering reply as MessageType.TEXT, and merge parent media only.
  • Store media with the cached parent context (or bypass the text-only cache), then cover repeated replies and image/audio/video/document parents.

Automated hermes-sweeper review.

# If the current message is plain text but the quoted message has media, upgrade type
if inbound_type == MessageType.TEXT and parent_media_types:
try:
inbound_type = MessageType(parent_media_types[0])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parent_media_types[0] is a MIME value (for example image/jpeg), not a MessageType value (photo, audio, etc.), so this always falls into the PHOTO fallback. Please keep the triggering reply as TEXT and let the existing per-attachment MIME routing handle the merged parent media.

if not self._client or not message_id:
return None, [], []
# Text-only result may already be cached; re-use it (media already empty for pure-text).
if message_id in self._message_text_cache:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cache hit drops media for every later reply to the same quoted image/file. Cache the complete parent context including media paths and MIME types, or bypass the text-only cache for this lookup; add a repeated-reply regression test.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/feishu Feishu / Lark adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants