Skip to content

fix(tts): synthesize mp3 then transcode to opus for .ogg targets #55278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alaamohanad169-ship-it wants to merge 1 commit into from
Closed

fix(tts): synthesize mp3 then transcode to opus for .ogg targets #55278

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions tools/tts_tool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1100,7 +1100,7 @@ def _generate_openai_tts(
speed_default = tts_config.get("speed", 1.0) if isinstance(tts_config, dict) else 1.0
speed = float(oai_config.get("speed", speed_default))

# The managed OpenAI audio gateway only proxies MANAGED_OPENAI_TTS_MODELS.
# The managed OpenAI audio gateway only proxies MANAGED_OPENAI_TTS_MODELS.
# A model set for direct OpenAI (e.g. "tts-1-hd") 400s there with
# "Unsupported managed OpenAI speech model", so coerce it — unless the user
# redirected base_url to their own endpoint, in which case respect it.
Expand All @@ -1118,7 +1118,13 @@ def _generate_openai_tts(
)
model = DEFAULT_OPENAI_MODEL

response_format = _tts_response_format_from_path(output_path)
# Determine response format from extension.
# Always request mp3 from the API — then transcode to OGG/Opus locally
# for .ogg targets. This avoids breaking non-opus-compatible backends
# (e.g. Speaches/Kokoro) that reject response_format="opus".
wants_opus = output_path.endswith(".ogg")
response_format = "mp3"

@teknium1 teknium1 Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes direct OpenAI too: current documentation states OpenAI produces native Opus (website/docs/user-guide/features/tts.md:172). Please preserve an Opus-first path and use MP3 plus local conversion only after an endpoint rejects Opus; otherwise ffmpeg becomes a new dependency for a route that currently needs none.

synth_path = (output_path[:-4] + ".mp3") if wants_opus else output_path

OpenAIClient = _import_openai_client()
client = OpenAIClient(api_key=api_key, base_url=base_url)
Expand All @@ -1134,8 +1140,19 @@ def _generate_openai_tts(
create_kwargs["speed"] = max(0.25, min(4.0, speed))
response = client.audio.speech.create(**create_kwargs)

response.stream_to_file(output_path)
return output_path
response.stream_to_file(synth_path)
if wants_opus:
converted = _convert_to_opus(synth_path)
# Clean up the intermediate mp3 if transcoding succeeded
if converted and converted != synth_path:
try:
os.remove(synth_path)
except OSError:
pass
return converted
# Transcoding failed — return the mp3 as best-effort

@teknium1 teknium1 Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This return path is currently discarded by the caller: text_to_speech_tool() invokes _generate_openai_tts(...) at tools/tts_tool.py:2295 without assigning its result, then validates the original .ogg path. On conversion failure the promised MP3 fallback will still report no output. Propagate the returned path into the dispatcher state.

return synth_path
return synth_path
finally:
close = getattr(client, "close", None)
if callable(close):
Expand Down
Loading