fix(tts): transcode OpenAI-compatible TTS to OGG/Opus when backend lacks opus - #56873
fix(tts): transcode OpenAI-compatible TTS to OGG/Opus when backend lacks opus#56873Bartok9 wants to merge 1 commit into
Conversation
27a8e54 to
403bd07
Compare
|
Thanks for the focused compatibility fix. The premise remains present on current main: Problems
Suggested changes
This is an automated hermes-sweeper review. |
…cks opus Closes NousResearch#54589 _generate_openai_tts() derives response_format from the output extension, so any .ogg target requests opus. OpenAI-*compatible* backends that only encode mp3/flac/wav/pcm (e.g. self-hosted Speaches/Kokoro) reject that and no voice bubble is delivered. Honor an optional tts.openai.response_format. When set to a non-opus format for an .ogg target, synthesize in that format and transcode to OGG/Opus locally via ffmpeg (mirroring the Edge provider's _convert_to_opus path). Default behavior is unchanged. Rebased onto the refactored _generate_openai_tts (3-tuple _resolve_openai_audio_client_config) and documented the option in website/docs/user-guide/features/tts.md per review.
403bd07 to
5fd0dc2
Compare
|
Thanks @teknium1 — addressed both the CI failure and the docs gap:
Default behavior unchanged: |
|
Resolved at the class level by PR #73072. Rather than a per-provider transcode, |
Summary
_generate_openai_tts()hardcodedresponse_format="opus"for any.oggtarget, breaking OpenAI-compatible backends that can't encode opus.tts.openai.response_format; when set to a non-opus format for an.oggtarget, synthesize in that format and transcode to OGG/Opus locally via ffmpeg.Motivation
Closes #54589.
Self-hosted OpenAI-compatible TTS backends (e.g. Speaches/Kokoro) only support
mp3/flac/wav/pcm. Because Telegram/Matrix voice bubbles use an.oggoutput path,_generate_openai_tts()sentresponse_format="opus"straight to the endpoint, which rejected it — so no voice bubble was delivered. This mirrors the Edge provider, which already synthesizes MP3 and transcodes to OGG/Opus via_convert_to_opus().Fix
tts.openai.response_formatis set to a non-opus format and the target is.ogg: synthesize to a temp file in that format, transcode to.oggwith_convert_to_opus(), and clean up the intermediate file. If ffmpeg transcode fails, raise a clear error..oggtargets still requestopusdirectly (real OpenAI API path).Verification
python3 -m pytest tests/tools/test_tts_openai_opus_transcode.py tests/tools/test_tts_opus_routing.py— 4 passedtest_ogg_target_defaults_to_opusproves the default path is untouched (still requests opus, no transcode).test_ogg_target_transcodes_when_backend_lacks_opusproves the new path asks the backend for mp3, transcodes to.ogg, and removes the temp mp3.Did NOT change
want_opusrouting intext_to_speech_tool()— this fix is scoped to the synthesis format negotiation inside_generate_openai_tts().