Skip to content

fix(tts): improve Opus encoding quality for Telegram voice messages - #18861

Closed
shellybotmoyer wants to merge 1 commit into
NousResearch:mainfrom
shellybotmoyer:fix/tts-opus-encoding-voice-quality
Closed

fix(tts): improve Opus encoding quality for Telegram voice messages#18861
shellybotmoyer wants to merge 1 commit into
NousResearch:mainfrom
shellybotmoyer:fix/tts-opus-encoding-voice-quality

Conversation

@shellybotmoyer

Copy link
Copy Markdown
Contributor

Summary

Fixes #18818 — suboptimal Opus encoding for Gemini/Edge TTS causing quality loss in Telegram voice messages.

Problem

Both _convert_to_opus() and _generate_gemini_tts() use ffmpeg with suboptimal Opus encoding settings:

ffmpeg -i input -acodec libopus -ac 1 -b:a 64k -vbr off output.ogg -y

Issues:

  • -vbr off forces Constant Bitrate (CBR) — wastes bits on silence, starves complex segments
  • Missing -application voip — Opus encoder defaults to audio mode, not optimized for speech
  • Missing -compression_level 10 — lower compression quality

This causes:

  1. Noticeably lower speech quality compared to properly-encoded Opus
  2. Inconsistent Telegram voice bubble rendering (some files appear as regular audio instead of voice messages)

Fix

Changed ffmpeg encoding parameters at both call sites:

Parameter Before After Why
-b:a 64k 48k 48k VBR is standard for VoIP; VBR uses bits more efficiently
-vbr off (CBR) on (VBR) VBR allocates bits where needed for speech
-application (missing) voip Optimizes Opus for voice intelligibility
-compression_level (missing) 10 Maximum encoding quality

Testing

  • ffmpeg -application voip is a standard libopus option (value 2048) that enables CELT+SILK hybrid mode optimized for speech
  • 48k VBR is the standard bitrate for high-quality VoIP Opus
  • No functional changes to the conversion pipeline — only encoding parameters changed

Affected Code

  • tools/tts_tool.py:712_convert_to_opus() (used by Edge TTS, xAI TTS, and other providers)
  • tools/tts_tool.py:1201_generate_gemini_tts() inline conversion

Switch from CBR 64k to VBR 48k with -application voip and -compression_level 10

The previous settings (CBR 64k, no voip optimization) produced lower quality Opus audio

that failed to trigger Telegram voice bubble rendering consistently.

Fixes NousResearch#18818
@alt-glitch alt-glitch added type/perf Performance improvement or optimization P2 Medium — degraded but workaround exists tool/tts Text-to-speech and transcription platform/telegram Telegram bot adapter labels May 2, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for addressing both Telegram Opus conversion paths. The current-main premise is confirmed: tools/tts_tool.py:916 and tools/tts_tool.py:1795 still use -vbr off.

Problems

  • The change has no regression coverage for the resulting ffmpeg argv. tests/tools/test_tts_opus_routing.py:47-70 mocks _convert_to_opus rather than its subprocess call, and tests/tools/test_tts_gemini.py currently has no ffmpeg/.ogg assertion.
  • The same Telegram encoder settings remain duplicated in the two paths (tools/tts_tool.py:915-916 and tools/tts_tool.py:1792-1797). The linked fix(tts): optimize Opus output for Telegram voice #30066 discussion identifies a shared command builder plus tests as the more thorough form of this fix.

Suggested changes

  • Add mocked-subprocess tests for both paths, asserting -vbr on, -application voip, and -compression_level 10.
  • Centralize the shared Telegram Opus argv while retaining each caller’s existing subprocess options.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
teknium1 pushed a commit that referenced this pull request Jul 28, 2026
Salvaged from PR #18861 (@shellybotmoyer). Switch the central Opus
transcoders from CBR 64k (-vbr off) to VBR 48k with -application voip and
-compression_level 10. The old CBR settings produced lower-quality speech
audio and occasionally failed to trigger Telegram's native voice-bubble
rendering. Applied to _ffmpeg_transcode_to_opus (the single transcoder
behind _convert_to_opus and _repair_ogg_container), the Gemini WAV→OGG
path, and the Matrix adapter boundary transcoder added in this branch.

Fixes #18818
teknium1 pushed a commit that referenced this pull request Jul 28, 2026
Salvaged from PR #18861 (@shellybotmoyer). Switch the central Opus
transcoders from CBR 64k (-vbr off) to VBR 48k with -application voip and
-compression_level 10. The old CBR settings produced lower-quality speech
audio and occasionally failed to trigger Telegram's native voice-bubble
rendering. Applied to _ffmpeg_transcode_to_opus (the single transcoder
behind _convert_to_opus and _repair_ogg_container), the Gemini WAV→OGG
path, and the Matrix adapter boundary transcoder added in this branch.

Fixes #18818
@teknium1

Copy link
Copy Markdown
Contributor

Merged into main via consolidated salvage PR #73508 (merge f440a44753). Your Opus VBR/voip encode settings were cherry-picked with your authorship and applied to the current central transcoders.

Your contribution is credited to you in git history. Thank you! Closing this PR as merged-via-salvage.

@teknium1 teknium1 closed this Jul 29, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Salvaged from PR NousResearch#18861 (@shellybotmoyer). Switch the central Opus
transcoders from CBR 64k (-vbr off) to VBR 48k with -application voip and
-compression_level 10. The old CBR settings produced lower-quality speech
audio and occasionally failed to trigger Telegram's native voice-bubble
rendering. Applied to _ffmpeg_transcode_to_opus (the single transcoder
behind _convert_to_opus and _repair_ogg_container), the Gemini WAV→OGG
path, and the Matrix adapter boundary transcoder added in this branch.

Fixes NousResearch#18818
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TTS: Suboptimal Opus encoding for Gemini/Edge TTS causes quality loss in Telegram voice messages

3 participants