fix: send voice replies as Opus/OGG on Matrix

[Kailigithub]

Summary

Matrix requires Opus codec in an OGG container for proper voice bubbles per MSC3245. When sent as MP3, Matrix treats voice replies as generic file attachments — on mobile clients they appear as broken or unplayable.

Two code paths were affected:

1. tools/tts_tool.py — model-invoked text_to_speech tool

The want_opus predicate at line 960 only matched "telegram", so when the model calls the TTS tool directly on a Matrix session the output was always MP3 regardless of provider capabilities.

Fix: Extended the check to platform in ("telegram", "matrix").

2. gateway/run.py — auto voice-reply in _send_voice_reply()

The output path was hardcoded to .mp3 regardless of the target platform. For providers that support native Opus output (OpenAI, ElevenLabs, Mistral, Gemini), passing .ogg as the extension allows the tool to request response_format=opus directly — no ffmpeg conversion needed.

Fix: Platform-aware extension selection (.ogg for Telegram/Matrix, .mp3 otherwise).

Testing

• py_compile passes for both modified files
• All 7 tests in tests/gateway/test_matrix_voice.py pass
• All TTS-related tests pass (43 passed, 3 pre-existing failures unrelated to this change)
• Broader gateway + tools test suite: 312 passed (3 pre-existing failures)

Closes #14841

[e-shizz]

Hey! I was affected by this exact same issue on my Matrix instance. I can confirm that these two patches (tts_tool.py + gateway/run.py) fix Matrix voice delivery — voice messages now arrive as proper OGG Opus voice bubbles via MSC3245.

The patch has been running stable for me. Anything blocking this from getting merged? Happy to help test or rebase if needed. ★

[teknium1]

Thanks for isolating both affected paths. The premise remains valid on current main: Matrix still follows the MP3 auto-reply branch in gateway/run.py:13035, and Matrix is absent from want_opus in tools/tts_tool.py:2202.

Problems

• gateway/run.py:6308 calls .lower() on event.source.platform, but SessionSource.platform is a Platform enum (gateway/session.py:75), so this path will raise before TTS generation.
• The .ogg generation path at gateway/run.py:6311 is unsafe for Edge TTS. Edge writes its MP3 stream to the supplied path (tools/tts_tool.py:961-963), while _convert_to_opus() derives its output as the same .ogg path (tools/tts_tool.py:912-916).
• No Matrix regression test accompanies the change; existing format coverage is Telegram/Slack-only in tests/gateway/test_auto_voice_reply_format.py:17-72.

Suggested changes

• Compare against Platform.TELEGRAM / Platform.MATRIX directly.
• Generate to a distinct compatible path, then post-convert to a distinct Ogg/Opus file when required; preserve already-native Opus output.
• Add Matrix coverage for both auto voice replies and model-invoked TTS. The linked #54488 documents and tests this post-generation conversion direction.

Automated hermes-sweeper review.

[teknium1]

event.source.platform is a Platform enum (gateway/session.py:75), not a string, so .lower() raises here. Compare directly against {Platform.TELEGRAM, Platform.MATRIX} (or normalize through .value) before choosing the extension.

[teknium1]

Generating Edge TTS directly to .ogg does not make it Opus: Edge writes MP3 bytes to the requested path, and _convert_to_opus() derives its output by replacing the suffix with .ogg, producing the same input/output path here. Generate to a distinct path and post-convert to a separate OGG file.

[malaiwah]

Downstream confirmation the direction is right (Telegram→Matrix for want_opus / audio_ext). Two quick notes on the bot's blockers, both easy:

• tools/tts_tool.py's platform local is already get_session_env(...).lower() (a str), so platform in ("telegram", "matrix") needs no .lower() on a Platform enum. In gateway/run.py compare the enum directly: event.source.platform in (Platform.TELEGRAM, Platform.MATRIX).
• The Edge-TTS .ogg in/out collision only bites providers that emit MP3-then-convert; native-Opus providers (OpenAI-compatible / ElevenLabs) write .ogg directly.

That said, for reconcilers: #54488 fixes the same bug more completely — an Edge-safe post-generation transcode (rather than this one-line want_opus extension) plus the MSC3245/MSC1767 duration+waveform metadata that Element needs, plus tests — and the automated review already cites it as the documented/tested direction. Correcting my earlier note: no separate metadata follow-up is needed; #54488 has it. Endorsing that as the one to land.

[Kailigithub]

Closing as superseded — the same fix landed upstream via the salvage track in commit b2a8a45591. No further action needed on this branch. 🤖

[Kailigithub]

Closing per supersede comment above. The fix landed via salvage track.