Fix Photon iMessage voice replies and threading

[keithdperez]

What changed

• Normalize outbound Photon voice media to AAC/M4A with the documented audio/mp4 shape.
• Route CAF audio through native audio senders so generated iMessage voice notes reach Photon send_voice instead of send_document.
• Pass reply anchors through the Photon adapter and sidecar for native threaded replies.
• Add regression coverage for voice normalization, reply metadata, and CAF routing.

Why

Hermes TTS normally emits MP3, while one iMessage fallback emitted CAF. CAF was omitted from the shared native-audio extension set, so the normal gateway path treated it as a document and bypassed Photon voice normalization. Manual MP3-to-CAF conversion was also unreliable.

Impact

Photon iMessage voice replies are normalized to an iMessage-compatible M4A voice payload, including when the source is MP3 or CAF. Replies can retain their native message anchor.

Verification

• pytest -q tests/plugins/platforms/photon/test_outbound_media.py
• pytest -q tests/gateway/test_platform_base.py -k ShouldSendMediaAsAudio
• Node syntax check for the Photon sidecar
• Live CAF-origin Photon voice send accepted with a Spectrum message ID

[teknium1]

Thanks for tracing the Photon media path; current main does drop reply_to in plugins/platforms/photon/adapter.py:1083-1143, and .caf is absent from gateway/platforms/base.py:30.

Problems

• The usual gateway auto-TTS path calls play_tts() without reply_to at gateway/platforms/base.py:5054-5059; normal response media similarly calls send_voice() without it at gateway/platforms/base.py:5159-5164 and gateway/run.py:13516-13521. The new adapter plumbing therefore does not thread those common voice/media replies.
• plugins/platforms/photon/adapter.py:1083-1111 deterministically writes <stem>-imessage.m4a using ffmpeg -y. It can overwrite an existing sibling file, then delete it on a failed conversion.
• The attachment is threaded at plugins/platforms/photon/sidecar/index.mjs:771, but its caption remains a flat space.send() at line 777.

Suggested changes

• Propagate the event reply anchor through the BasePlatformAdapter and streamed-media dispatch paths, with end-to-end dispatch tests.
• Use a unique temporary/cache conversion output and clean up only the file created for this send.
• Send captions via sendWithOptionalReply and cover the caption-plus-reply case.

Automated hermes-sweeper review.

[teknium1]

-y overwrites an existing <stem>-imessage.m4a sibling. If conversion then fails, lines 1103-1111 unlink that path, so sending voice.mp3 can replace and delete a pre-existing voice-imessage.m4a. Please use a unique temporary/cache output and clean up only the file created by this invocation.

[teknium1]

This threads the attachment, but the caption immediately below is still sent with bare space.send() at line 777. For any attachment with a caption and replyToId, the caption escapes the native reply. Route it through sendWithOptionalReply as well and add coverage.

[redeye1011]

One deployment gap in the current voice-normalization path: _prepare_imessage_voice() falls back to sending the original MP3/CAF whenever shutil.which("ffmpeg") is false or conversion fails. A clean Hermes/Photon install does not guarantee a system ffmpeg, so this fallback can recreate the original 0:00/unplayable iMessage voice bubble instead of fulfilling the PR's normalization contract.

We reproduced this with normal Hermes TTS MP3 output. A robust fix was to ship a deterministic ffmpeg binary with the Photon sidecar (ffmpeg-static as a real package.json + lockfile dependency), convert to AAC/M4A before voice(), force the upload basename to end in .m4a, and fail the voice send rather than silently send incompatible source bytes. The smallest regression check is: no system ffmpeg + MP3 input still produces an audio/mp4 .m4a payload (or an explicit send failure), never a source-format fallback.

Also worth preserving in the final implementation: Spectrum keeps the supplied content name, so M4A bytes with an .mp3 upload name can still render as a zero-second/unplayable bubble even when MIME is corrected.

[redeye1011]

I opened #65836 as a focused companion for the remaining final-hop gap: it installs Spectrum’s optional static-ffmpeg peer, aligns transcoded upload filenames to .m4a, and ensures the Photon sidecar actually ships in wheel/sdist artifacts. It does not touch #65322’s CAF routing or threading work. If both proceed, it may be worth checking whether adapter-level _prepare_imessage_voice still needs to pre-transcode every voice attachment.

[andrexibiza]

Verification (Vox Lockin lane 08 — platform adapter voice class): verified against origin/main @ 70db671.

• Issue Photon iMessage voice notes bypass STT pipeline and interrupt active tasks #66325 (Photon voice notes bypass STT) is already CLOSED (COMPLETED) on GitHub. On main, inbound Photon voice is promoted to MessageType.VOICE with cached local media + audio/* MIME (plugins/platforms/photon/adapter.py::_normalize_binary_payload), so gateway/run.py's _event_media_is_stt_input routes it into the central STT pipeline exactly like every other adapter — including CAF attachments (.caf/audio/x-caf → VOICE).
• This PR is a draft and its head does not merge cleanly with main: git merge-tree reports 9 conflict regions (base.py + adapter.py + sidecar/index.mjs), all touched by later main work (reaction-ack policy refactor, secret-scope credential reads, message-editing opt-out).
• Scope note: the PR's content (media-directive leak prevention, CAF→native voice senders, outbound threading) is the outbound/send half of the Photon voice class; the inbound STT half this lane owns is already complete on main (tests/plugins/platforms/photon/test_inbound.py::test_caf_attachment_named_promoted_to_voice passes — 7 passed).

Recommendation: rebase the draft on main if the outbound half is still desired; no inbound-STT work remains for it to carry. No duplicate PR will be opened.