Skip to content

fix(gateway): use OGG/Opus for Matrix voice replies - #58716

Closed
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:liuhao/cron-bugfix-58715-matrix-tts-opus
Closed

fix(gateway): use OGG/Opus for Matrix voice replies#58716
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:liuhao/cron-bugfix-58715-matrix-tts-opus

Conversation

@liuhao1024

Copy link
Copy Markdown
Contributor

What does this PR do?

Matrix MSC3245 voice messages require Opus audio format, but both the _send_voice_reply path in gateway/run.py and the want_opus check in tools/tts_tool.py only triggered Opus conversion for Telegram. Matrix voice replies were sent as MP3/WAV, which Element X silently refuses to play in the voice bubble.

Related Issue

Fixes #58715

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • gateway/run.py: Add Platform.MATRIX to the _voice_platforms tuple so _send_voice_reply generates .ogg files for Matrix sessions
  • tools/tts_tool.py: Add "matrix" to the want_opus condition so the TTS tool triggers ffmpeg Opus conversion for Matrix platform sessions
  • tests/tools/test_tts_opus_routing.py: Add regression test verifying Edge TTS converts to Opus when HERMES_SESSION_PLATFORM=matrix

How to Test

  1. Configure a Matrix gateway with any TTS provider (e.g., Edge TTS or piper)
  2. In a room, send /voice tts to enable voice replies
  3. Send a text message and wait for the voice reply
  4. The voice bubble should now play correctly in Element X iOS — previously it showed the bubble with correct duration but playback did nothing
  5. Run pytest tests/tools/test_tts_opus_routing.py -v — all 3 tests should pass, including the new test_edge_matrix_converts_to_opus_voice

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/tools/test_tts_opus_routing.py -v and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 15 (Apple Silicon)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Matrix MSC3245 voice messages require Opus audio format, but both the
_send_voice_reply path in gateway/run.py and the want_opus check in
tools/tts_tool.py only applied Opus conversion for Telegram.

- gateway/run.py: add Platform.MATRIX to the voice_platforms tuple so
  _send_voice_reply generates .ogg files for Matrix
- tools/tts_tool.py: add "matrix" to the want_opus condition so the
  TTS tool triggers ffmpeg Opus conversion for Matrix sessions
- tests/tools/test_tts_opus_routing.py: add regression test verifying
  Edge TTS converts to Opus when HERMES_SESSION_PLATFORM=matrix

Fixes NousResearch#58715
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery platform/matrix Matrix adapter (E2EE) sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jul 5, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #14900, an earlier open PR fixing the same bug (#14841) via the same mechanism: adding matrix to the .ogg extension logic in gateway/run.py and to the want_opus check in tools/tts_tool.py. #14900 is the canonical fix. Also related: #54488 (a competing approach that transcodes MP3->Ogg/Opus after generation and adds native MSC3245 metadata). A maintainer should pick between the predicate-extension approach (#14900/this) and the transcode approach (#54488).

@liuhao1024

Copy link
Copy Markdown
Contributor Author

Closing as duplicate of #14900 (Kailigithub) — same Matrix OGG/Opus voice reply fix in gateway/run.py + tools/tts_tool.py, created 2+ months earlier.

This PR's regression test (test_edge_matrix_converts_to_opus_voice) may be worth porting to #14900 if the maintainer requests it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists platform/matrix Matrix adapter (E2EE) sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Matrix TTS voice replies sent as non-Opus audio — Element X voice bubble renders but won't play

2 participants