fix(whatsapp): resolve original media for quoted-media replies - #77660
fix(whatsapp): resolve original media for quoted-media replies#77660ishangodawatta wants to merge 2 commits into
Conversation
|
@teknium1 could you review this when you get a chance? A review request also unblocks the fork CI runs. Thanks. Update: pushed 00dbd6a07 addressing @spfcraze's bot-triage finding above — the empty-message guard in |
|
This was generated by AI during triage. Summary: Problems:
Solution: Checked against |
00dbd6a to
4494d97
Compare
Baileys' contextInfo.quotedMessage only ever carries a thumbnail-sized
stub for media, or nothing at all for an uncaptioned attachment — never
a way to fetch the original file. When a user replies to an earlier
photo/video/document/voice note with no caption on it (e.g. "did you
save this?" quoting an uncaptioned wedding invite image), the agent
saw no text and no media reference at all: it looked like the message
never had an attachment.
Add createQuotedMediaCache, a bounded in-memory cache (keyed by
chatId:messageId) of each inbound message's already-downloaded media
and text, populated as extractBridgeEvent processes every message.
When a later message quotes one of these, extractBridgeEvent resolves
quotedMediaUrls/quotedMediaType from the cache and falls back to a
human-readable quotedText ("sent an image", etc.) when the quote had
no caption to extract. The adapter folds resolved quoted media into
the event's own media_urls/media_types — reusing the existing
vision/audio pipeline and the existing _is_allowed_bridge_path path
validation — rather than adding a parallel reply-media code path.
Reimplements the same feature as NousResearch#52875 (credit: dhruvkej9) against
current main, whose 11627fd refactor (native polls, locations, rich
inbound metadata) moved this code into bridge_helpers.js and made that
PR's diff no longer apply cleanly.
The empty-message guard only checked the reply's own body/hasMedia, so a caption-less quote of a cached image (no text, no media of its own) was dropped even though extractBridgeEvent had already resolved quotedMediaUrls for it.
4494d97 to
2fb0956
Compare
Summary
Baileys'
contextInfo.quotedMessageonly ever carries a thumbnail-sized stub for media, or nothing at all for an uncaptioned attachment — never a way to fetch the original file. When a user replies to an earlier photo/video/document/voice note that had no caption (e.g. "did you save this?" quoting an uncaptioned wedding invite image someone else shared), the agent saw no text and no media reference at all: it looked like the message never had an attachment, and the agent would say "I don't see any image."This reimplements #52875 (credit: dhruvkej9) against current
main. That PR predates11627fdcb("native Baileys polls, clarify-as-poll, locations, and rich inbound metadata"), which restructured this exact code path intobridge_helpers.jsand moved several call sites — the original diff no longer applies cleanly and would leave gaps against the current shape.Approach, matching the original's idea:
createQuotedMediaCache(bridge_helpers.js) — a bounded in-memory cache (keyed bychatId:messageId) of each inbound message's already-downloaded media and text, populated asextractBridgeEventprocesses every message.extractBridgeEventresolvesquotedMediaUrls/quotedMediaTypefrom the cache, and falls back to a human-readablequotedText("sent an image", etc.) when the quote had no caption to extract from.plugins/platforms/whatsapp/adapter.py) folds resolved quoted media into the event's ownmedia_urls/media_types— reusing the existing vision/audio pipeline and the existing_is_allowed_bridge_pathpath-safety check, rather than adding a parallel reply-media code path or newMessageEventfields.Follow-up fix in this PR: the pre-existing empty-message guard in
bridge.js(if (!event.body && !event.hasMedia)) ran before this resolution was consulted, so a bare quote-reply with no text or media of its own — a caption-less quote of an uncaptioned image — was still dropped as "empty" even afterextractBridgeEventhad resolved itsquotedMediaUrls. The guard now also checksquotedMediaUrls.lengthso that case reaches the queue.Test plan
scripts/run_tests.sh tests/gateway/test_whatsapp_formatting.py— 10/12 passed. The 3TestBridgeEventMetadatacases covering this change (quoted-reply metadata preserved, uncaptioned-quote resolves to cached media, quoted-media path-safety guard) all pass. The 2 failures (TestSendChunking) are pre-existing and unrelated —No module named 'aiohttp'in this venv, confirmed present viagit stashbefore this change too.node --test scripts/whatsapp-bridge/bridge.native.test.mjs— 19/19 passed, including a new case asserting a bare quote-reply (empty own body, no own media) still carries the resolvedquotedMediaUrls, proving the guard fix is exercised. Existing suite unaffected.node --check scripts/whatsapp-bridge/bridge.js— parses OK.