fix: encode Matrix auto-TTS voice as Ogg Opus

[ciabata-git]

Summary

• transcode Matrix and Telegram auto-TTS voice replies to Ogg/Opus before sending native voice messages
• add Matrix native voice metadata (duration + waveform) for Element-compatible playback
• cover the Matrix/Telegram auto voice paths and Matrix MSC3245 metadata with targeted tests

Testing

• PYTHONPATH="/tmp/hermes-agent-matrix-voice-pydeps${PYTHONPATH:+:$PYTHONPATH}" python -m pytest tests/gateway/test_voice_command.py::TestSendVoiceReply tests/gateway/test_matrix_voice.py::TestMatrixSendVoiceMSC3245 -q -o 'addopts=' → 8 passed in 1.34s
• python -m py_compile gateway/run.py plugins/platforms/matrix/adapter.py tests/gateway/test_voice_command.py tests/gateway/test_matrix_voice.py → passed
• uv run --active --no-sync --with ruff==0.15.10 ruff check gateway/run.py plugins/platforms/matrix/adapter.py tests/gateway/test_voice_command.py tests/gateway/test_matrix_voice.py → All checks passed!
• git diff --check origin/main...HEAD -- gateway/run.py plugins/platforms/matrix/adapter.py tests/gateway/test_voice_command.py tests/gateway/test_matrix_voice.py → passed

Notes

• This fixes a Matrix/Element behavior where MP3 bytes marked as a native Matrix voice event can render as a voice bubble but fail playback.

[alt-glitch]

This was generated by AI during triage.

Competing with #14900 for the same Matrix voice-bubble bug (#14841). This PR transcodes the generated MP3 to Ogg/Opus after generation (keeping the .mp3 path, which it notes is safer for Edge TTS) and additionally adds native Matrix MSC3245 voice metadata (ffprobe duration + MSC1767 waveform) for Element-compatible playback. #14900 takes the narrower one-line approach of extending the want_opus predicate to include matrix, without the post-generation transcode or native metadata. Flagging the cluster so a maintainer can pick the canonical fix.

[teknium1]

Thanks for addressing the Matrix native-voice playback path; the current main implementation still requests an MP3 for Matrix in gateway/run.py:13153-13159 and marks it as MSC3245 voice in plugins/platforms/matrix/adapter.py:1964-1981.

Problems

• The new gateway conversion does not cover direct text_to_speech delivery. tools/tts_tool.py:2202 still enables Opus only for Telegram, while gateway media delivery routes recognized Matrix audio to send_voice (gateway/run.py:13293-13301). Matrix can therefore still receive a native voice event containing non-Opus audio outside _send_voice_reply.
• _matrix_voice_metadata_for_file() runs ffprobe and ffmpeg synchronously, but the PR invokes it from async _send_local_file. Its subprocess timeouts can block the Matrix adapter event loop during every voice upload. The gateway conversion already uses asyncio.to_thread; metadata extraction should do likewise.

Suggested changes

• Centralize Matrix voice conversion at the Matrix send_voice boundary, or update the tool-level Matrix Opus routing as well.
• Offload metadata probing and add coverage for the non-blocking path and direct Matrix TTS routing.

Automated hermes-sweeper review.

[teknium1]

This covers only _send_voice_reply. Direct text_to_speech media on Matrix still follows tools/tts_tool.py's Telegram-only want_opus predicate and is routed to MatrixAdapter.send_voice; please enforce the Matrix codec at the adapter boundary or extend the tool-level routing too.

[teknium1]

_matrix_voice_metadata_for_file() may synchronously run ffprobe for 10s and ffmpeg for 15s. Calling it in this async method blocks the Matrix event loop; await it through asyncio.to_thread(...) instead.

[malaiwah]

Downstream confirmation + a concrete assist. We hit #14841 with a self-hosted OpenAI-compatible TTS (Moshi/Kyutai) and ran this PR's approach (Opus container + info.duration + org.matrix.msc1767.audio duration/waveform) in production — voice bubbles render and play correctly in Element X iOS, verified on-device. Field-by-field diff against an Element-recorded voice note is in #14841: the metadata this PR adds is exactly what Element needs; container-only fixes leave a dead Play button with no length in our testing.

To help this land, I've opened #68063, which stacks on this branch unchanged (your commit stays yours) and resolves the two blockers from the automated review:

1. Direct text_to_speech coverage — extends want_opus to Matrix in tools/tts_tool.py (native-Opus providers emit Ogg directly; MP3-native providers go through the existing tool-level _convert_to_opus, which writes a distinct .ogg path — no Edge in/out collision), plus enforcement at the MatrixAdapter.send_voice boundary: any non-Ogg audio reaching it is transcoded to Ogg/Opus best-effort (original sent unchanged if ffmpeg is missing).
2. Non-blocking metadata — _matrix_voice_metadata_for_file() is awaited via asyncio.to_thread(...) from _send_local_file; the boundary transcode runs off-loop the same way. (Your existing test still passes unmodified — the patch target resolves at call time.)

Tests: the affected suites pass (13 passed — your 8 plus new cases for Matrix tool routing, boundary transcode, and skip-when-already-Ogg); ruff clean.

Whichever way the maintainers prefer — merge this PR then #68063 as a follow-up, or take #68063 wholesale (it contains this branch) — happy to rebase either way.

[malaiwah]

One small observation while validating this downstream — not a blocker, just so it's a conscious trade-off on the record.

The mp3-first + post-convert design (rightly recommended in review for Edge safety) has a minor side effect on the auto-reply path for native-Opus providers (openai / elevenlabs / mistral / gemini): previously a Telegram auto-reply requested .ogg and got one-step native Opus (response_format=opus, no ffmpeg); now those providers emit MP3 which is then re-encoded to Opus — a lossy double-encode plus an ffmpeg dependency on a path that didn't previously need it.

In practice it's modest for 64k voice audio, and the Edge-safety rationale is sound, so this seems like a reasonable trade to land as-is. Flagging it mainly in case anyone later wonders why auto-reply audio differs slightly from direct text_to_speech output (which still gets one-step native Opus via want_opus). A possible future refinement — out of scope here — would be for _send_voice_reply to request .ogg when the configured provider is known to support native Opus, falling back to mp3-first otherwise.

[teknium1]

The root cause here (gateway auto-TTS choosing MP3 vs Ogg/Opus via the cleared HERMES_SESSION_PLATFORM contextvar, so opus platforms got audio attachments instead of native voice bubbles) was fixed class-wide in #73508: platform-awareness now comes from the caller via build_auto_tts_output_path(platform) keyed off OPUS_VOICE_PLATFORMS (based on @giladbau's #62040), with the central container repair guaranteeing real Ogg/Opus bytes.

(Landed via #73508, merge f440a44753.) Closing.