Skip to content

fix(whatsapp): resolve original media for quoted-media replies - #77660

Open
ishangodawatta wants to merge 2 commits into
NousResearch:mainfrom
ishangodawatta:ishangodawatta/whatsapp-quoted-media-context
Open

fix(whatsapp): resolve original media for quoted-media replies#77660
ishangodawatta wants to merge 2 commits into
NousResearch:mainfrom
ishangodawatta:ishangodawatta/whatsapp-quoted-media-context

Conversation

@ishangodawatta

@ishangodawatta ishangodawatta commented Aug 3, 2026

Copy link
Copy Markdown

Summary

Baileys' contextInfo.quotedMessage only ever carries a thumbnail-sized stub for media, or nothing at all for an uncaptioned attachment — never a way to fetch the original file. When a user replies to an earlier photo/video/document/voice note that had no caption (e.g. "did you save this?" quoting an uncaptioned wedding invite image someone else shared), the agent saw no text and no media reference at all: it looked like the message never had an attachment, and the agent would say "I don't see any image."

This reimplements #52875 (credit: dhruvkej9) against current main. That PR predates 11627fdcb ("native Baileys polls, clarify-as-poll, locations, and rich inbound metadata"), which restructured this exact code path into bridge_helpers.js and moved several call sites — the original diff no longer applies cleanly and would leave gaps against the current shape.

Approach, matching the original's idea:

  • createQuotedMediaCache (bridge_helpers.js) — a bounded in-memory cache (keyed by chatId:messageId) of each inbound message's already-downloaded media and text, populated as extractBridgeEvent processes every message.
  • When a later message quotes one of these, extractBridgeEvent resolves quotedMediaUrls/quotedMediaType from the cache, and falls back to a human-readable quotedText ("sent an image", etc.) when the quote had no caption to extract from.
  • The adapter (plugins/platforms/whatsapp/adapter.py) folds resolved quoted media into the event's own media_urls/media_types — reusing the existing vision/audio pipeline and the existing _is_allowed_bridge_path path-safety check, rather than adding a parallel reply-media code path or new MessageEvent fields.

Follow-up fix in this PR: the pre-existing empty-message guard in bridge.js (if (!event.body && !event.hasMedia)) ran before this resolution was consulted, so a bare quote-reply with no text or media of its own — a caption-less quote of an uncaptioned image — was still dropped as "empty" even after extractBridgeEvent had resolved its quotedMediaUrls. The guard now also checks quotedMediaUrls.length so that case reaches the queue.

Test plan

  • scripts/run_tests.sh tests/gateway/test_whatsapp_formatting.py — 10/12 passed. The 3 TestBridgeEventMetadata cases covering this change (quoted-reply metadata preserved, uncaptioned-quote resolves to cached media, quoted-media path-safety guard) all pass. The 2 failures (TestSendChunking) are pre-existing and unrelated — No module named 'aiohttp' in this venv, confirmed present via git stash before this change too.
  • node --test scripts/whatsapp-bridge/bridge.native.test.mjs — 19/19 passed, including a new case asserting a bare quote-reply (empty own body, no own media) still carries the resolved quotedMediaUrls, proving the guard fix is exercised. Existing suite unaffected.
  • node --check scripts/whatsapp-bridge/bridge.js — parses OK.

@ishangodawatta

ishangodawatta commented Aug 6, 2026

Copy link
Copy Markdown
Author

@teknium1 could you review this when you get a chance? A review request also unblocks the fork CI runs. Thanks.


Update: pushed 00dbd6a07 addressing @spfcraze's bot-triage finding above — the empty-message guard in bridge.js only checked event.body/event.hasMedia, so a bare quote-reply to cached media (no text or media of its own) was still dropped even after extractBridgeEvent resolved quotedMediaUrls for it. The guard now also checks quotedMediaUrls.length. Added a regression test in bridge.native.test.mjs for this case; PR description's test plan updated.

@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins platform/whatsapp WhatsApp Business adapter P3 Low — cosmetic, nice to have sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Aug 6, 2026
@spfcraze

spfcraze commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Summary:
A quote-reply with no text of its own is dropped as 'empty' by the guard at bridge.js:753 in this PR's head — whose condition checks only event.body and event.hasMedia — even though extractBridgeEvent has just resolved quoted media for it.

Problems:

  • extractBridgeEvent resolves quotedMediaUrls/quotedText from the new cache (bridge_helpers.js:367-386 in this PR's head), but the pre-existing guard if (!event.body && !event.hasMedia) at bridge.js:753 runs before messageQueue.push and reads only the reply's own body and media flags.
  • For a reply whose own text is empty (a bare quote of an uncaptioned image), body is '' and hasMedia is false, so the event is dropped as 'empty' at bridge.js:753-760 — the failure the PR summary describes ("no text and no media reference at all") still applies to that shape.

Solution:
Extend the emptiness test to the resolved quoted media — if (!event.body && !event.hasMedia && !quotedMediaUrls.length) continue; — so the guard passes a reply whose only content is the quoted media.


Checked against d763a89 — the PR head when this was written — and 0957277, main at the same moment.

@ishangodawatta
ishangodawatta force-pushed the ishangodawatta/whatsapp-quoted-media-context branch from 00dbd6a to 4494d97 Compare August 17, 2026 09:09
Baileys' contextInfo.quotedMessage only ever carries a thumbnail-sized
stub for media, or nothing at all for an uncaptioned attachment — never
a way to fetch the original file. When a user replies to an earlier
photo/video/document/voice note with no caption on it (e.g. "did you
save this?" quoting an uncaptioned wedding invite image), the agent
saw no text and no media reference at all: it looked like the message
never had an attachment.

Add createQuotedMediaCache, a bounded in-memory cache (keyed by
chatId:messageId) of each inbound message's already-downloaded media
and text, populated as extractBridgeEvent processes every message.
When a later message quotes one of these, extractBridgeEvent resolves
quotedMediaUrls/quotedMediaType from the cache and falls back to a
human-readable quotedText ("sent an image", etc.) when the quote had
no caption to extract. The adapter folds resolved quoted media into
the event's own media_urls/media_types — reusing the existing
vision/audio pipeline and the existing _is_allowed_bridge_path path
validation — rather than adding a parallel reply-media code path.

Reimplements the same feature as NousResearch#52875 (credit: dhruvkej9) against
current main, whose 11627fd refactor (native polls, locations, rich
inbound metadata) moved this code into bridge_helpers.js and made that
PR's diff no longer apply cleanly.
The empty-message guard only checked the reply's own body/hasMedia,
so a caption-less quote of a cached image (no text, no media of its
own) was dropped even though extractBridgeEvent had already resolved
quotedMediaUrls for it.
@ishangodawatta
ishangodawatta force-pushed the ishangodawatta/whatsapp-quoted-media-context branch from 4494d97 to 2fb0956 Compare August 25, 2026 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have platform/whatsapp WhatsApp Business adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants