Skip to content

fix(tts): transcode xAI ogg requests to Opus - #57222

Closed
ishaanlabs-gg wants to merge 1 commit into
NousResearch:mainfrom
ishaanlabs-gg:codex/xai-tts-ogg-opus
Closed

fix(tts): transcode xAI ogg requests to Opus#57222
ishaanlabs-gg wants to merge 1 commit into
NousResearch:mainfrom
ishaanlabs-gg:codex/xai-tts-ogg-opus

Conversation

@ishaanlabs-gg

Copy link
Copy Markdown

Fixes #57213.

Summary

  • synthesize xAI TTS .ogg requests to a temporary MP3 because xAI does not provide native Opus/OGG output
  • transcode that temporary MP3 to the requested .ogg target so Telegram auto voice replies receive real Opus audio
  • cover the provider path with a regression test that verifies .ogg requests do not keep MP3 bytes under the .ogg name

Tests

  • RED: scripts/run_tests.sh tests/tools/test_tts_xai_speech_tags.py -q failed with out.read_bytes() == b"mp3" before the fix
  • scripts/run_tests.sh tests/tools/test_tts_xai_speech_tags.py -q
  • scripts/run_tests.sh tests/tools/test_tts_xai_speech_tags.py tests/tools/test_tts_opus_routing.py tests/gateway/test_auto_voice_reply_format.py -q
  • .venv/bin/python -m py_compile tools/tts_tool.py tests/tools/test_tts_xai_speech_tags.py
  • git diff --check

@alt-glitch alt-glitch added type/bug Something isn't working comp/tools Tool registry, model_tools, toolsets tool/tts Text-to-speech and transcription provider/xai xAI (Grok) P2 Medium — degraded but workaround exists labels Jul 2, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: implementing fix for #57213 (xAI TTS writes MP3 bytes into .ogg voice files). Part of the broader .ogg/Opus TTS cluster — sibling of #57190 (Edge branch, #57048) and the OpenAI-compatible fixes #54597 (earliest canonical transcode)/#56873. Same symptom class, different provider branch in tools/tts_tool.py. Not a duplicate.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the xAI provider path. The reported behavior is still present on current main: Telegram auto voice replies request .ogg in gateway/run.py:13300-13311, while xAI writes MP3 response bytes to the requested output path in tools/tts_tool.py:1370-1413.

Problems

  • tools/tts_tool.py:1288 raises when _convert_to_opus() returns no path, before the temporary MP3 cleanup at :1292-1295. _convert_to_opus() returns None without ffmpeg on current main (tools/tts_tool.py:917-918), leaving the UUID-named MP3 behind. Gateway finalization only removes the requested/returned paths (gateway/run.py:13357-13362).

Suggested changes

  • Move intermediate-file cleanup into a finally block and add a regression test for the failed-conversion path.

Automated hermes-sweeper review.

Comment thread tools/tts_tool.py

if needs_opus_transcode:
opus_path = _convert_to_opus(api_output_path)
if not opus_path:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When _convert_to_opus() returns None (including when ffmpeg is unavailable), this raises before the cleanup below, leaving the UUID-named temporary MP3 behind. Please clean up api_output_path in a finally block and add a failure-path regression test.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Resolved at the class level by PR #73072. Rather than a per-provider transcode, text_to_speech_tool now sniffs the audio magic bytes once after every synthesis and repairs any MP3/WAV-bytes-in-.ogg centrally (in-place ffmpeg transcode to real Ogg/Opus, honest-extension rename fallback when ffmpeg is missing) — covering this provider and all others, including command providers and plugins. Your diagnosis of the container mismatch was correct and helped shape the central fix — thanks for the contribution.

@teknium1 teknium1 closed this Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists provider/xai xAI (Grok) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

3 participants