Skip to content

fix(tts): transcode xAI TTS output to Opus when .ogg requested (#57213) - #57518

Closed
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:fix/xai-tts-ogg-opus
Closed

fix(tts): transcode xAI TTS output to Opus when .ogg requested (#57213)#57518
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:fix/xai-tts-ogg-opus

Conversation

@AlexFucuson9

Copy link
Copy Markdown
Contributor

Problem

xAI /v1/tts API only supports mp3/wav codecs. When caller requests .ogg output (Telegram auto voice replies), MP3 bytes silently written into .ogg file -> broken voice bubbles.

This is xAI instance of known bug class (#57048, #54589). Existing fixes (#55278, #57069, #56873) target other providers but none covers xAI.

Root Cause

tools/tts_tool.py::_generate_xai_tts:
codec = wav if .wav else mp3 -- no .ogg handling. xAI API has no native Opus codec.

Fix

When output_path ends with .ogg:

  1. Write API response to temp .mp3 file
  2. Transcode to Opus/OGG via existing _convert_to_opus (ffmpeg)
  3. Clean up temp file, return .ogg path
  4. Fall back to mp3 if ffmpeg unavailable

Same pattern as dispatcher-level opus conversion for other providers.

Closes #57213

…esearch#57213)

xAI's /v1/tts API only supports mp3 and wav codecs. When the caller
requests an .ogg output path (e.g. Telegram auto voice replies via
_send_voice_reply), MP3 bytes were silently written into the
.ogg-named file, producing broken/unplayable voice bubbles.

Fix: detect .ogg output paths in _generate_xai_tts, write the API
response to a temp .mp3 file, then transcode to Opus/OGG using the
existing _convert_to_opus helper (ffmpeg). Falls back to the mp3
file if ffmpeg is unavailable so the caller still gets audio.

This follows the same pattern already used by the dispatcher-level
opus conversion for other providers (edge, openai, etc.).

Closes NousResearch#57213
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/tools Tool registry, model_tools, toolsets tool/tts Text-to-speech and transcription provider/xai xAI (Grok) duplicate This issue or pull request already exists labels Jul 3, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.
Duplicate of #57222 — same fix at the same site: both edit tools/tts_tool.py::_generate_xai_tts to synthesize to a temp MP3 then transcode to Opus/OGG via _convert_to_opus for .ogg targets. #57222 (2026-07-02T16:29) is the earliest open xAI-specific fix; #57250 was already deduped into it. Consolidating on #57222.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting a real xAI/Telegram incompatibility: current main still writes xAI's MP3 response into a requested .ogg path at tools/tts_tool.py:1370-1413, while Telegram auto replies request .ogg at gateway/run.py:13300-13310.

Problems

  • tools/tts_tool.py:1300 returns the OGG beside the temporary MP3. _convert_to_opus() derives that name from its input (tools/tts_tool.py:920), so it is not the requested output path.
  • The xAI dispatch at tools/tts_tool.py:2443-2445 ignores that return value, then validates the original .ogg path at tools/tts_tool.py:2528-2533. The proposed successful-conversion path therefore reports no output.
  • This PR changes only tools/tts_tool.py; please add a regression test for this .ogg path.

Suggested changes

  • Move/replace the converted OGG onto output_path before returning, or propagate the actual returned path through the dispatcher and correctly handle the MP3 fallback.
  • Add a test that verifies the requested .ogg path is the converted output, including ffmpeg-unavailable behavior.

This is an automated hermes-sweeper review.

Comment thread tools/tts_tool.py
os.unlink(_mp3_tmp)
except OSError:
pass
return opus_path

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_convert_to_opus() derives an OGG beside this temporary MP3, but text_to_speech_tool() ignores the xAI generator's return value (tools/tts_tool.py:2443-2445) and still validates the original requested .ogg path. Move the converted file to output_path before returning, or propagate the returned path through the dispatcher.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Resolved at the class level by PR #73072. Rather than a per-provider transcode, text_to_speech_tool now sniffs the audio magic bytes once after every synthesis and repairs any MP3/WAV-bytes-in-.ogg centrally (in-place ffmpeg transcode to real Ogg/Opus, honest-extension rename fallback when ffmpeg is missing) — covering this provider and all others, including command providers and plugins. Your diagnosis of the container mismatch was correct and helped shape the central fix — thanks for the contribution.

@teknium1 teknium1 closed this Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/xai xAI (Grok) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

3 participants