Skip to content

BlueBubbles: native iMessage voice memo support + inbound webhook dedup - #11654

Closed
cypres0099 wants to merge 3 commits into
NousResearch:mainfrom
cypres0099:ben/bluebubbles-voice-memo-support
Closed

BlueBubbles: native iMessage voice memo support + inbound webhook dedup#11654
cypres0099 wants to merge 3 commits into
NousResearch:mainfrom
cypres0099:ben/bluebubbles-voice-memo-support

Conversation

@cypres0099

Copy link
Copy Markdown
Contributor

Summary

Three related fixes to the BlueBubbles adapter that together make iMessage voice memos work end-to-end and eliminate duplicate inbound turn processing. Each is a standalone commit so they can be cherry-picked or split during review.

The three changes

1. Route voice memos through the `private-api` send path

iMessage only renders an attachment as a voice bubble (waveform + play button) when the underlying IMCore send carries the voice-memo marker. The AppleScript send path in BlueBubbles-server doesn't set this marker -- only the Private API send path does. Setting `method="private-api"` in `_send_attachment` when `is_audio_message=True` fixes this.

Without this change, audio attachments always arrive as generic file attachments even with a correct codec and the `isAudioMessage` flag. See the BlueBubbles adapter source (`packages/server/src/server/api/apple/actions.ts`): the AppleScript `sendMessage` path never forwards `isAudioMessage` into the AppleScript itself.

2. Transcode audio to Opus-in-CAF before voice send

iMessage's voice-memo renderer specifically requires Opus codec inside a CAF container -- MP3, AAC, and PCM-in-CAF all fall back to generic file rendering. BlueBubbles-server's own `convertMp3ToCaf` helper currently produces PCM-in-CAF (tracked in BlueBubblesApp/bluebubbles-server#793), and callers shipping any other audio format (AAC, WAV, Opus-in-OGG) never trigger the converter at all.

This pre-converts anything non-.caf to Opus-in-CAF via `afconvert`, the native macOS tool. Ships on every macOS version BlueBubbles-server runs on, so no new dependency. On conversion failure, the original path is sent unchanged -- matches today's fallback behavior.

Mirrors the Opus-in-OGG transcode `tools/tts_tool.py` already does for Telegram voice messages, just targeting CAF instead of OGG.

3. Dedup inbound webhook events by message guid

BlueBubbles fires more than one webhook per iMessage in several conditions:

  • A `new-message` event followed ~1s later by an `updated-message` event for the same guid (delivery receipt / edit / reaction notification).
  • Group-chat fanout: when the local user participates in both a DM and a group, a single inbound can trigger webhooks carrying different chat GUIDs (e.g. `any;-;+phone` and `+phone`) but the same message guid.
  • Reconnect / retry paths where BlueBubbles redelivers recent events.

Without dedup, each duplicate webhook produced an independent agent turn -- most visibly: a voice memo landing in the DM and a separate text reply landing in the group chat, both triggered by one user send.

This adds a small in-memory LRU (guid -> monotonic timestamp, 60s TTL) that short-circuits the handler if the guid has already been processed. At-least-once delivery is the norm for HTTP webhooks; idempotent handling on the receiver is the right place to solve this rather than trying to suppress duplicates upstream (which would also lose legitimate edit/receipt signal on future `updated-message` handling).

Testing

Verified end-to-end on Apple Silicon macOS 26.3, Hermes v0.10.0 native install, BlueBubbles-server v1.9.9 with Private API helper connected:

  1. Send TTS-generated audio via adapter's `send_voice` → iMessage renders as a waveform voice bubble with play button on the receiving device.
  2. Send user message from phone → one inbound turn, one agent response, even though BlueBubbles fires two webhooks (`new-message` + `updated-message`) ~1s apart. Debug log shows `dropping duplicate webhook for guid ...`.
  3. Non-audio attachments (images, videos, documents) unchanged -- still use server-default send method.

Prerequisites for end users

Voice memos require the BlueBubbles Private API Helper to be connected (`helper_connected: true` in `/api/v1/server/info`). On Apple Silicon macOS 14+, this means:

  1. Boot into 1 True Recovery (hold power button, Options → Continue while holding Left Shift)
  2. Startup Security Utility → Reduced Security
  3. Recovery Terminal: `csrutil enable --without debug`
  4. In normal macOS: `sudo defaults write /Library/Preferences/com.apple.security.libraryvalidation.plist DisableLibraryValidation -bool true`
  5. Quit and reopen BlueBubbles; it will inject the helper dylib into Messages.app

Users without Private API see the same fallback behavior as today (voice send lands as a file attachment), so there's no regression for the non-Private-API path.

Related

iMessage only renders an attachment as a voice bubble (waveform + play
button) when the underlying send includes the voice-memo marker on the
IMCore attachment, and the BlueBubbles AppleScript send path does not
set that marker -- only the Private API send path does. Without
method=private-api, an audio attachment with isAudioMessage=true still
arrives as a generic file attachment even when the bytes are valid
Opus-in-CAF.

This sets method="private-api" whenever is_audio_message is True so
the adapter targets the send path that can actually produce a voice
bubble. Non-audio attachments continue to use the server default.

Note: this requires the BlueBubbles Private API Helper to be
connected on the server (helper_connected: true in /api/v1/server/info).
Users without it will see the attachment fall back to a file -- same
behavior as before this change -- and should follow the BlueBubbles
Private API install docs to enable it.
iMessage's voice-memo renderer is strict about the attachment format:
it specifically requires Opus audio inside a CAF container. Any other
codec (MP3, AAC, PCM -- including PCM wrapped inside CAF) arrives as
a generic file attachment even when the voice-memo flag is set
correctly on the send.

BlueBubbles-server has its own MP3 -> CAF converter that fires when
an .mp3 is sent with isAudioMessage=true, but it currently produces
16-bit PCM in CAF (see BlueBubblesApp/bluebubbles-server#793), which
hits the same iMessage fallback. So even with a patched upstream BB,
callers that ship any other audio format (AAC, WAV, Opus-in-OGG)
still end up with file attachments.

This pre-converts anything non-.caf to Opus-in-CAF via afconvert, the
native macOS tool. afconvert ships on every macOS version the
BlueBubbles server runs on, so there is no new dependency. If
afconvert is missing or the conversion fails, the original path is
sent unchanged -- matches today's behavior, no regression.

This mirrors the Opus-in-OGG transcode already done for Telegram
voice messages in tools/tts_tool.py, just targeting the CAF container
instead of OGG.
BlueBubbles fires more than one webhook per iMessage in several
conditions:

1. A new-message event followed ~1s later by an updated-message event
   for the same guid (delivery receipt / edit / reaction notification).
2. Group-chat fanout: when the local user is a participant in both a
   DM and a group chat, a single inbound message can trigger webhooks
   carrying different chat GUIDs (e.g. "any;-;+phone" and "+phone")
   but the same message guid.
3. Reconnect / retry paths where BlueBubbles redelivers recent events.

Without dedup, each of these duplicate webhooks was being processed by
the gateway as an independent inbound turn, producing multiple agent
responses for a single user message (most visibly: a voice memo
landing in the DM *and* a separate text reply landing in the group
chat, both triggered by one user send).

This adds a small in-memory LRU (guid -> monotonic timestamp, 60s TTL)
that tracks recently-seen iMessage guids and short-circuits the
handler if the same guid has already been processed. At-least-once
delivery is the norm for HTTP webhooks; idempotent handling on the
receiver side is the right place to solve this rather than trying to
suppress the duplicate upstream (which would also lose legitimate
edit/receipt signal on updated-message events).

60s is generous; iMessage guid collisions are impossible in that
window. Eviction is opportunistic on each inbound to keep the dict
from growing unbounded.
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery tool/tts Text-to-speech and transcription labels Apr 24, 2026
@mwhuss

mwhuss commented Apr 28, 2026

Copy link
Copy Markdown

Would love to get the de-dup fix here.

@cypres0099

Copy link
Copy Markdown
Contributor Author

Closing this older bundled branch in favor of the dedup-only replacement in #47262. That keeps the maintainer-requested BlueBubbles webhook de-dup fix isolated from the unrelated voice-memo send/transcode work.

@cypres0099 cypres0099 closed this Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants