fix(bluebubbles): prevent duplicate inbound turns on v0.19.0

[Oceanswave]

Problem

BlueBubbles can deliver one incoming iMessage as both new-message and updated-message, sometimes with different chat-ID shapes. Hermes could therefore start two agent turns and send duplicate replies.

The prior review-driven branch reduction fixed that inbound problem, but it also removed the existing BlueBubbles reply experience while leaving its configuration in place. Reactions stopped, the configured typing cadence was not used, paragraph replies were split into several iMessage bubbles, and YAML behavior values could be overwritten by defaults.

Related: #30708, #34372, #34378.

What changed

Provider-realistic inbound delivery

• Treat new-message and the legacy message alias as user-message events.
• Acknowledge updated-message events without starting an agent turn.
• Deduplicate stable message GUIDs with the shared bounded TTL deduplicator.
• Reserve GUIDs before attachment I/O and coalesce concurrent deliveries.
• Retry each attachment download inside BlueBubbles' single webhook request.
• Preserve captions and successfully downloaded sibling attachments after retry exhaustion or an unexpected per-attachment metadata error.
• Hand off an explicit unavailable-attachment placeholder when every attachment fails.
• Release failed or cancelled handoff claims so a local retry is not suppressed.
• Fail the adapter connection when webhook lookup, listener startup, or registration fails. Clean up the listener/client on bind failure or cancellation so the gateway can retry instead of reporting a connected-but-dead inbound path.
• Keep same-text messages with distinct GUIDs as separate user turns.

Reply experience

• Add one immediate native like reaction when processing starts. No text acknowledgement is sent. Reaction aliases are normalized to BlueBubbles' canonical set, invalid values fall back to like, and unsuccessful response envelopes are not reported as success.
• Keep BasePlatformAdapter as the only typing scheduler. BlueBubbles supplies its configured refresh interval and one-shot start/stop API calls.
• Enable reactions, typing indicators, and read receipts by default. Each setting remains configurable.
• Keep a normal response in one iMessage bubble by default. Paragraph splitting is an explicit opt-in. Over-limit messages are still split safely.
• Deliver all chunks when the first message creates a new chat. If the new chat cannot be resolved for the remaining chunks, return a failure instead of reporting truncated delivery as successful.
• Parse string boolean values correctly, including "false".
• Preserve BlueBubbles behavior values from config.yaml; only explicit environment values override supported environment-backed settings.
• Isolate BlueBubbles tests from ambient machine configuration.

Reviewer guidance preserved

• The shared base lifecycle is the sole typing scheduler. There is no adapter-local refresh task.
• Attachment dedupe is retry-safe.
• Attachment recovery does not depend on provider webhook redelivery.
• Same-URL webhook registrations are compared against the exact new-message event set. Stale registrations are migrated with shielded cancellation and rollback/reconciliation after ambiguous failures.
• BlueBubbles' fixed-URL POST is idempotent, so POST plus readback cannot prove which concurrent process created an exact row. The exact registration is therefore durable across reconnects and is never deleted on disconnect/cancellation. Fresh and migrated concurrent-winner regressions verify this boundary.

Validation on current main

• Current final BlueBubbles suite: 61 passed.
• Current adjacent config, platform-base, typing, deduplicator, registry, and send-message validation: 210 passed, 1 skipped.
• Earlier full gateway sweep on the same base: 4,475 passed and 30 skipped. The remaining seven failures were unrelated current-upstream optional-plugin or environment failures in Discord, WeCom, URL safety, and session-home isolation. The one BlueBubbles environment-isolation failure found by that sweep was fixed and rerun successfully.
• Ruff: passed.
• Python compilation: passed.
• Windows portability scan: passed.
• git diff --check and conflict-marker scan: passed.

[tonydwb]

Code Review Summary

Verdict: Approved

Clean, well-scoped fix/feature with comprehensive tests. No issues found.

• Logic is correct and focused
• Tests cover the new behavior
• No security concerns
• Good error handling

---

Reviewed by Hermes Agent

[Oceanswave]

Maintainer note: I updated the PR description to match the final behavior after the follow-up commits. The final version uses an immediate BlueBubbles tapback/reaction when processing starts and intentionally does not send a delayed One sec progress bubble. I also tightened the duplicate-content fallback to a short window so immediate BlueBubbles DM webhook variants are still deduped without suppressing legitimate repeated messages later, and made the BlueBubbles tests isolate ambient BLUEBUBBLES_* environment variables.

[Oceanswave]

Follow-up after review: I found and fixed two edge-case regressions in the rebased PR. Distinct rapid attachment-only messages now include attachment identifiers in the short-lived content-dedupe key, so photo bursts are preserved while duplicate webhook variants still collapse. Existing/legacy inbound type: message webhook payloads are also accepted for compatibility, and retraction updates are acknowledged without routing deleted content as a new agent prompt. I also removed new behavior env fallbacks for the reply-UX options so those remain config.yaml-driven.

[Oceanswave]

Follow-up from searching related BlueBubbles issues/PRs: I found several directly related bugs that were not fully represented in the PR body, so I expanded the PR and added code coverage for the ones that fit this consolidation.

New required fixes included here:

• Preserve explicit 127.0.0.1 in the BlueBubbles webhook URL so macOS/Node does not resolve localhost to IPv6 ::1 while Hermes is listening on IPv4 (BlueBubbles _webhook_url normalizes 127.0.0.1 → localhost, breaking IPv4 webhook delivery (inbound attachments dropped) #45308 / [Bug]: BlueBubbles webhook fails on macOS due to IPv6 localhost resolution #8512).
• Treat data.chats[0]["[auth-key]"] as stable group identity and group evidence when top-level BlueBubbles chat fields are missing (BlueBubbles group webhooks can lose chat identity when only chats[0]["[auth-key]"] is populated #38339).
• Remove participant-address fallback from outbound chat GUID resolution so a DM target cannot be guessed as a group chat just because that contact is a participant ([Bug]: BlueBubbles outbound chat resolution can misroute DM replies to group chats. Privacy leak! #24157).

I also updated the PR body to distinguish direct closes from partial/related follow-ups: #30989, #31534, #8513 are partially addressed; #46624, #33489, and #23409 remain intentionally out of scope.

Validation after the new changes:

ruff check: All checks passed
pytest tests/gateway/test_bluebubbles.py: 82 passed
pytest BlueBubbles + config regression sweep: 200 passed

[thesammyagent]

I pulled this PR locally, merged current origin/main, and resolved the remaining conflict in gateway/config.py.

Resolution summary:

• kept the PR's config-preserving behavior for BlueBubbles extra values from config.yaml
• kept explicit env vars as overrides when provided
• kept default webhook host on 127.0.0.1
• used env_int("BLUEBUBBLES_WEBHOOK_PORT", 8645) for the env override path instead of a raw int(...)

Verification on the resolved branch:

uv run --with pytest --with pytest-asyncio --with aiohttp --with httpx pytest tests/gateway/test_bluebubbles.py -q -o 'addopts='
82 passed in 3.78s

uv run --with ruff ruff check gateway/config.py gateway/platforms/bluebubbles.py tests/gateway/test_bluebubbles.py
All checks passed!

uv run python -m py_compile gateway/config.py gateway/platforms/bluebubbles.py
# passed

I tried to push directly to the PR head branch, but GitHub denied write access to Oceanswave/hermes-agent for this account (403). I pushed the resolved merge commit here instead:

https://github.com/thesammyagent/hermes-agent/tree/fix/bluebubbles-consolidated-hardening-resolved

Commit: 9b0cf1cfdb39c6a6b88f5b5a556308690b045272

You should be able to cherry-pick/merge that commit or use the branch as a conflict-resolution reference.

[Oceanswave]

Status bump (July 10): verified the PR head still defaults typing_indicators to true and includes a regression test for that default. Current upstream main still lacks the config-driven typing_indicators gate/default and the broader consolidated hardening, so this is not implemented upstream. I also corrected the stale PR-body wording that still said typing defaulted off and linked the existing phantom-typing report (#31534) with the important distinction: enabled-by-default is the intended UX, while the BlueBubbles/IMCore stop race remains a separate lifecycle bug.

[Oceanswave]

Rebased onto current upstream main (base b8880f124) and resolved the BlueBubbles listener/config/routing test conflicts while preserving upstream’s webhook body cap, IPv4 loopback behavior, channel overrides, safe env parsing, and strict DM GUID resolution. Validation: 84 passed in tests/gateway/test_bluebubbles.py; Python compile and git diff --check also pass. GitHub now reports the PR as MERGEABLE (the remaining BLOCKED state is policy/review, not a merge conflict).

[Oceanswave]

Senior review follow-up: fixed three additional correctness/documentation smells after reviewing the rebased integration end-to-end. (1) GUID-less inbound payloads no longer get a 60-second text key that could swallow two legitimate repeated replies; they use only the short content-dedupe window. (2) Existing BlueBubbles webhook registrations are now reused only when their event set exactly matches config; stale/duplicate registrations are replaced so webhook_events changes actually take effect. (3) The user guide no longer contradicts code—the documented typing default is now on with an explicit opt-out. Added regression tests. Fresh validation: Ruff check passed, Python compile passed, git diff --check passed, and the BlueBubbles + adjacent config sweep is 263 passed. PR remains MERGEABLE; BLOCKED is review policy only.

[Oceanswave]

Second senior-review pass (independent reviewer findings incorporated): rapid same-text messages with distinct GUIDs/same payload shape are now both dispatched; content fallback only collapses cross-shape BlueBubbles webhook variants and no longer extends a sliding suppression window. send_read_receipts: "false" now parses correctly. Unexpected listener startup failures close the HTTP client and leave the adapter disconnected, while owned-listener webhook registration failures now fail/clean up rather than reporting a false healthy connection. Fresh expanded validation: 264 passed; Ruff, Python compile, and diff checks pass. Head: 958657c01, MERGEABLE.

[Oceanswave]

Typing persistence follow-up pushed in d007f4a54. BlueBubbles now refreshes the typing event every 4 seconds while a turn is active, deduplicates refresh tasks per chat, cancels them before stop-typing, and cleans them up on disconnect. Added configurable typing_refresh_interval, docs, and regression coverage. Focused BlueBubbles/config validation: 265 passed; Ruff, py_compile, and diff checks passed.

[Oceanswave]

Applied the latest BlueBubbles review feedback and rebased onto current main.

Changes in 501ee5756:

• moved duplicate suppression ahead of attachment downloads, so replayed webhooks no longer repeat download/cache side effects;
• preserved listener address families in webhook URLs (0.0.0.0 → 127.0.0.1, :: → [::1], and bracketed explicit IPv6 literals);
• added regression coverage proving failed stale-webhook cleanup aborts before replacement POST (the implementation already returned False on failed DELETE);
• added exact-replay and cross-shape attachment download-count assertions.

Fresh validation after the rebase:

91 passed — focused BlueBubbles suite
269 passed — BlueBubbles + adjacent config sweep
ruff check — passed
py_compile — passed
git diff --check / conflict-marker scan — passed

The PR body now reflects the current behavior and validation counts.

[Oceanswave]

Thanks @thesammyagent — your conflict-resolution notes and branch were useful, especially the gateway/config.py guidance around preserving YAML values, explicit env override authority, the 127.0.0.1 default, and safe webhook-port parsing.

I have since rebased #45317 onto current main and carried those behaviors forward while preserving newer upstream config/listener logic. The current head also includes the subsequent review fixes for pre-download deduplication, IPv6-safe webhook URLs, stale-registration cleanup, lifecycle failure handling, and typing refresh.

Current verification:

91 passed — focused BlueBubbles suite
269 passed — BlueBubbles + adjacent config sweep
ruff check / py_compile / git diff --check — passed

GitHub now reports the PR as MERGEABLE; the remaining BLOCKED state is review/workflow policy rather than a conflict. Thanks again for taking the time to reproduce and validate the resolution.

[teknium1]

Thanks for consolidating the BlueBubbles hardening and for preserving the existing strict chat-GUID behavior.

Problems

• gateway/platforms/bluebubbles.py:1287-1298 commits dedupe state before attachment retrieval. An attachment-only webhook whose download fails reaches the missing-fields path at line 1377, but a retry is then dropped for the dedupe TTL. Please commit the key only after the event is viable, or remove it on that failure path.
• The new adapter-local typing task at gateway/platforms/bluebubbles.py:1008-1012 overlaps the shared typing loop. gateway/platforms/base.py:4851-4869 starts _keep_typing() for each inbound event, and that loop invokes send_typing() every two seconds (gateway/platforms/base.py:3837-3859), so typing_refresh_interval cannot control the normal inbound cadence.

Suggested changes

• Add a retry regression for a transient attachment-download failure.
• Make the base lifecycle the sole typing scheduler, with a platform-specific interval if needed, and test through the background processing path.

Automated hermes-sweeper review.

[Oceanswave]

Rebased onto current main and completed a fresh architecture/reliability pass.

Key follow-up fixes:

• attachment dedupe reservations are released on transient download failure and the webhook returns retryable 503, including captioned media, so retries cannot be silently suppressed;
• the shared base adapter is now the sole typing scheduler; BlueBubbles only supplies its heartbeat interval and typing_indicators: false disables the shared lifecycle correctly;
• unexpected aiohttp listener startup failures clean up the runner before propagating;
• added boundary regressions for retry, captioned attachments, duplicate replay, typing disablement, YAML config authority, and listener cleanup.

Validation: 216 focused/adjacent gateway tests passed; Ruff, py_compile, and git diff --check passed.

[Oceanswave]

Resubmitted on current main after a senior correctness/design review. The issue still reproduces on v0.19.0: one iMessage can arrive as both new-message and updated-message, producing two agent turns. I replaced the previous broad hardening bundle with one focused commit covering event filtering, stable-GUID/in-flight deduplication, safe webhook registration migration, retry-safe attachment/handoff behavior, and aiohttp boundary regressions.

[thesammyagent]

Requesting changes because the attachment retry contract does not match BlueBubbles’ actual behavior. BlueBubbles performs one fire-and-forget Axios POST and only logs non-2xx failures; it does not retry the webhook. Returning 503 whenever any attachment download fails therefore permanently drops captioned messages and successful siblings in multi-attachment messages that current main still delivers. The new test passes only by manually invoking _handle_webhook() again, which production does not do.

Please use bounded internal attachment retries, then preserve valid caption text and successfully downloaded attachments after exhaustion rather than relying on redelivery. Please also make registration migration cancellation-safe/ownership-aware and add provider-realistic single-delivery coverage. The GUID/in-flight dedup itself looks sound.

[Oceanswave]

Addressed in 3c9805a77 after verifying BlueBubbles’ actual server implementation.

Attachment delivery now matches the provider contract:

• each attachment is retried internally with a bounded three-attempt schedule during the single webhook request;
• exhausted failures no longer return 503 in hopes of redelivery;
• valid caption text and successfully downloaded sibling attachments are preserved;
• an all-failed attachment-only webhook is still handed off as (attachment unavailable) so the user turn is not silently lost;
• provider-realistic tests invoke _handle_webhook() once and cover retry recovery, caption preservation, partial multi-attachment success, and total exhaustion.

Registration migration is now cancellation-safe and ownership-aware:

• the migration accounts for BlueBubbles addWebhook() being idempotent by URL and not updating events;
• stale delete → replacement create runs as a shielded unit;
• true replacement failure restores the prior event set only after GET confirms the canonical URL is empty;
• a replacement that committed before a response timeout is reconciled as success;
• an unexpected same-URL owner is not deleted or claimed;
• idempotent POST response IDs are not treated as proof of ownership;
• partial stale-cleanup failures restore the prior registration if the URL became empty;
• connection cancellation cleans local resources and removes only independently proven owned IDs.

Fresh validation on current main:

79 passed — tests/gateway/test_bluebubbles.py
303 passed — BlueBubbles + adjacent config + shared deduplicator suites
ruff check . — passed
py_compile — passed
git diff --check — passed

[Oceanswave]

Rebased onto current main and narrowed the branch in d9322959f to the intended single-delivery inbound fix. The unrelated webhook registration ownership/migration rewrite and associated docs/tests were removed. Retained behavior covers GUID dedupe, updated-message suppression, bounded attachment retries, recoverable sibling/caption preservation, attachment-unavailable fallback, and failed/cancelled handoff release. All 26 BlueBubbles adapter tests pass; Ruff, py_compile, Windows footgun scan, ancestry, and diff checks pass.