Skip to content

fix(bluebubbles): dedup inbound webhook events by message GUID - #30996

Closed
briandevans wants to merge 2 commits into
NousResearch:mainfrom
briandevans:fix/bluebubbles-inbound-dedup-30708
Closed

fix(bluebubbles): dedup inbound webhook events by message GUID#30996
briandevans wants to merge 2 commits into
NousResearch:mainfrom
briandevans:fix/bluebubbles-inbound-dedup-30708

Conversation

@briandevans

Copy link
Copy Markdown
Contributor

What does this PR do?

BlueBubbles fires both new-message and updated-message for the same iMessage (the second on delivered/read/edit echoes). The adapter currently has no inbound dedup — unlike slack / dingtalk / wecom / weixin / mattermost / feishu, all of which use the shared MessageDeduplicator helper. As a result the same inbound message is processed twice, and because the two payloads carry the chat reference differently (new-message includes chatGuid, updated-message typically omits it and falls back to chatIdentifier), the gateway derives two distinct session keys (any;-;<addr> vs bare <addr>) and spins up two parallel sessions for one chat — producing interleaved/duplicate replies and, e.g., two "Session reset" confirmations for a single /new.

Fix: wire in the shared MessageDeduplicator at the top of _handle_webhook, keyed by the message GUID resolved with the same priority already used for MessageEvent.message_id (guidmessageGuidid). The dedup runs before session-key derivation, so both the double-processing and the dual-session symptoms disappear without changing the event subscription — updated-message still flows through for the edits/retractions work tracked in #8513.

This is intentionally scoped to the exact-GUID echo case from #30708. The complementary text-then-attachment two-event coalescing described in #30989 is orthogonal (different GUIDs, requires a debounce-and-merge step modeled on Telegram's media-group handling) and is intentionally left out of this PR.

Related Issue

Fixes #30708

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • gateway/platforms/bluebubbles.py — import MessageDeduplicator alongside strip_markdown, instantiate self._dedup = MessageDeduplicator() in __init__, and call self._dedup.is_duplicate(msg_guid) near the top of _handle_webhook (after the is_from_me guard, before any chat-key derivation). The GUID is resolved with the same priority list already used for MessageEvent.message_id.
  • tests/gateway/test_bluebubbles.py — new TestBlueBubblesInboundDedup class:
    • test_same_guid_new_then_updated_message_dedups — drives the exact new-messageupdated-message sequence from the issue and asserts handle_message is called once, not twice. Pre-fix this test fails with two delivered events whose source.chat_id is any;-;+15551234567 and +15551234567 respectively (the two parallel sessions the issue describes).
    • test_different_guids_both_delivered — negative case proving distinct GUIDs still both reach handle_message.

How to Test

  1. uv run --with pytest --with pytest-xdist --with pytest-asyncio --with pytest-timeout --with aiohttp python3 -m pytest tests/gateway/test_bluebubbles.py -v
  2. All 51 tests pass (49 pre-existing + 2 new).
  3. Regression check: temporarily revert the production change and rerun the two new tests — test_same_guid_new_then_updated_message_dedups fails with assert 2 == 1 and a diff showing the two distinct source.chat_id values from the issue body.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(bluebubbles): …)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix (no unrelated commits)
  • I've run focused tests for the touched code and all pass
  • I've added tests for my changes
  • I've tested on my platform: macOS 15.x

Documentation & Housekeeping

  • I've updated relevant documentation — N/A (internal adapter wiring; no user-facing surface or config key changes)
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact — N/A (server-side adapter; runs wherever the gateway runs)
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Salvage-with-widening

Audited siblings: slack.py, dingtalk.py, wecom.py, weixin.py, mattermost.py, and feishu.py already use MessageDeduplicator (the helper docstring lists exactly this set as the adapters it consolidated). bluebubbles.py was the only inbound-message adapter missing the wiring. No widening needed.

Copilot AI review requested due to automatic review settings May 23, 2026 15:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds inbound message GUID deduplication to the BlueBubbles webhook handler and introduces regression tests to ensure new-message + updated-message events for the same iMessage do not create duplicate deliveries/sessions.

Changes:

  • Add MessageDeduplicator usage in BlueBubblesAdapter._handle_webhook keyed by message GUID.
  • Add regression tests covering same-GUID dedup and different-GUID delivery behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
tests/gateway/test_bluebubbles.py Adds async regression tests validating same-GUID webhook events are deduped while distinct GUIDs are delivered.
gateway/platforms/bluebubbles.py Deduplicates inbound webhook processing by message GUID prior to session-key derivation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread gateway/platforms/bluebubbles.py Outdated
Comment on lines +132 to +137
# Suppress duplicate inbound webhooks for the same message GUID.
# BlueBubbles fires both `new-message` and `updated-message` on
# delivered/read/edit echoes for the same message; without dedup the
# second event normalizes to a different chat key (chatIdentifier vs
# chatGuid) and spins up a parallel session for the same chat.
self._dedup = MessageDeduplicator()
Comment thread tests/gateway/test_bluebubbles.py Outdated
Comment on lines +747 to +750
r1 = await adapter._handle_webhook(_FakeRequest(new_msg, "secret"))
await asyncio.sleep(0)
r2 = await adapter._handle_webhook(_FakeRequest(upd_msg, "secret"))
await asyncio.sleep(0)
Comment thread tests/gateway/test_bluebubbles.py Outdated
Comment on lines +713 to +720
class _FakeRequest:
def __init__(self, payload, password):
self._payload = payload
self.query = {"password": password}
self.headers = {}

async def read(self):
return json.dumps(self._payload).encode("utf-8")
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery labels May 23, 2026
@chrisberthe

Copy link
Copy Markdown

Came to the repo to fix this same issue. Thanks @briandevans

@briandevans
briandevans force-pushed the fix/bluebubbles-inbound-dedup-30708 branch from 5ba088d to 4121a31 Compare May 24, 2026 19:13
@briandevans

Copy link
Copy Markdown
Contributor Author

@copilot All three findings addressed in c166153e6:

  • MessageDeduplicator bounds: now passing max_size=2000, ttl_seconds=300 explicitly so the cache cap is visible at the call site (the helper's defaults are bounded but were not obvious from the BlueBubbles code).
  • asyncio.sleep(0) flakiness: replaced with an asyncio.Event set inside the patched handle_message, awaited via asyncio.wait_for(..., timeout=2.0). Removes the slow-CI race where the background task scheduled by _handle_webhook had not yet run when assertions executed.
  • _FakeRequest duplication: hoisted to module-level _FakeWebhookRequest; both tests now share the same helper.

The slice-3 CI failure (tests/acp/test_server.py::TestSlashCommands::test_model_switch_uses_requested_provider 'custom' == 'anthropic') is a pre-existing baseline flake on clean main, not touched by this PR.

BlueBubbles fires both `new-message` and `updated-message` for the same
iMessage (the second on delivered/read/edit echoes). The adapter
currently has no inbound dedup — unlike slack/dingtalk/wecom/weixin/
mattermost/feishu — so both events are processed and, because the two
payloads carry the chat reference differently (`new-message` includes
`chatGuid`, `updated-message` typically omits it and falls back to
`chatIdentifier`), the gateway derives two distinct session keys
(`any;-;<addr>` vs bare `<addr>`) and spins up two parallel sessions
for one chat.

Wire in the shared `MessageDeduplicator` helper at the top of
`_handle_webhook`, keyed by the message GUID resolved with the same
priority already used for `MessageEvent.message_id`
(`guid`/`messageGuid`/`id`). This drops the duplicate before
session-key derivation, so both the double-processing and the
dual-session symptoms disappear, without changing the event
subscription (`updated-message` still flows through for the
edits/retractions tracked in NousResearch#8513).

Regression coverage in `tests/gateway/test_bluebubbles.py`:
- Same GUID via `new-message` then `updated-message` → one delivery.
- Two distinct GUIDs → both delivered (no false positives).

Refs: NousResearch#30708
Address Copilot review on NousResearch#30996:

- Pass `max_size=2000, ttl_seconds=300` to `MessageDeduplicator()`
  explicitly so the long-running cache cap is visible at the call site
  instead of relying on the helper's defaults.
- Replace `await asyncio.sleep(0)` synchronization in the dedup tests
  with an `asyncio.Event` set inside the patched `handle_message`, awaited
  via `asyncio.wait_for(..., timeout=2.0)`. Eliminates the slow-CI race
  where the background task scheduled by `_handle_webhook` had not yet
  run when the assertions executed.
- Hoist the inline `_FakeRequest` helper to module-level
  `_FakeWebhookRequest` so both tests share one definition.
@tmchow

tmchow commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

I opened #38379 as a narrower replacement/complement after testing this approach against the BlueBubbles group/DM alias ordering case.

This PR is directionally right for exact GUID replay dedup, but it uses first-seen semantics before chat routing. That means it fixes “two replies” but can still keep the wrong route if BlueBubbles delivers the sparse updated-message first and the richer new-message second:

  1. updated-message has the same message GUID but only chatIdentifier / sender-like routing data
  2. new-message later carries the real group identity under data.chats[0]["[auth-key]"]
  3. first-seen GUID dedup can preserve the DM-like event and drop the later group event

#38379 instead ignores updated-message echoes before they touch the dedup cache, then lets the later new-message resolve the full group route, including nested [auth-key]. It also keeps bounded GUID dedup for true duplicate new-message replays.

So: #30996 removes duplicate replies, but #38379 closes the routing-correctness hole too.

@briandevans

Copy link
Copy Markdown
Contributor Author

Closing to focus the queue on security/file-safety work where civilian merges are landing. Happy to reopen if maintainers want this picked up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BlueBubbles adapter lacks inbound dedup → duplicate processing + two parallel sessions per message

5 participants