fix(tools): transcode to OGG/Opus for OpenAI-compatible TTS backends without opus (#54589)

[PRATHAMESH75]

What does this PR do?

Fixes a bug where _generate_openai_tts() in tools/tts_tool.py hardcoded response_format="opus" for any .ogg target (Telegram voice bubbles). Real OpenAI encodes opus natively, but many OpenAI-compatible backends — e.g. a self-hosted Speaches/Kokoro endpoint — only support mp3/flac/wav/pcm and reject the opus request, so the audio.speech.create() call errors and no voice bubble is delivered.

The fix makes the OpenAI path try native opus first, then fall back to synthesizing mp3 and transcoding to OGG/Opus locally via the existing _convert_to_opus() helper — mirroring how the Edge provider already handles this. I chose try-opus-first over an unconditional mp3+transcode specifically to avoid regressing real-OpenAI users, who would otherwise gain an ffmpeg dependency and a lossy mp3→opus re-encode on every voice reply. If ffmpeg is unavailable, the mp3 is kept so the caller still has playable audio (delivered as a document instead of a voice bubble) rather than failing with "no output".

This is the inverse of #14841 (which assumes an opus-capable backend); both share this function. It is complementary to and does not overlap #54488, which transcodes at the Matrix/gateway layer (gateway/run.py) rather than in the TTS synthesis tool.

Related Issue

Fixes #54589

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• tools/tts_tool.py — _generate_openai_tts(): try response_format="opus" for .ogg targets, and on failure re-synthesize as mp3 and transcode to OGG/Opus via _convert_to_opus(); fall back to the mp3 when ffmpeg is absent. Non-.ogg targets are unchanged (direct mp3).
• tools/tts_tool.py — text_to_speech_tool(): capture the path returned by _generate_openai_tts() (consistent with the command/plugin providers) so the fallback path is honored downstream.
• tests/tools/test_tts_openai_opus.py — new tests covering native opus (no transcode), opus-rejection → mp3+transcode, no-ffmpeg → keep mp3, and plain .mp3 targets.

How to Test

1. scripts/run_tests.sh tests/tools/test_tts_openai_opus.py tests/tools/test_tts_speed.py tests/tools/test_tts_max_text_length.py tests/tools/test_managed_media_gateways.py
   → Summary: 4 files, 62 tests passed, 0 failed.
2. Manual repro: point tts.openai.base_url at a Speaches/Kokoro endpoint (no opus encoder) and trigger a Telegram voice reply. Before: backend rejects opus and no bubble is sent. After: Hermes synthesizes mp3 and transcodes to OGG/Opus locally, delivering a playable voice bubble.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS 15 (Darwin 25.5.0)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Backend error before the fix:

Unsupported response_format "opus". Supported formats: mp3, flac, wav, pcm.

[alt-glitch]

This was generated by AI during triage.

Fix PR for #54589. Related (same _generate_openai_tts, distinct bug): #37337 (honor wav/flac output). Complementary, different layer: #54488 (transcodes at the Matrix/gateway layer rather than in the TTS synthesis tool). Not a duplicate of either.

[PRATHAMESH75]

Heads-up on CI: the only failing check is Build&Test Docker image / build-arm64. That job writes a registry build cache to ghcr.io/nousresearch/hermes-agent:buildcache-arm64, which fails on all fork PRs because GitHub issues fork PRs a read-only GITHUB_TOKEN (denied: installation not allowed to Write organization package) regardless of the workflow's packages: write block. It's unrelated to this change and resolves once the code is on main.

The required aggregate check (All required checks pass) is green and the PR is MERGEABLE. The earlier red on Python tests / Run tests slice 6/8 was a flake in test_session_create_no_race_keeps_worker_alive (a TUI-gateway thread-timing test, untouched by this PR — passes locally); it went green on re-run.

[teknium1]

Thanks for the targeted fallback and regression coverage. The premise is verified on current main: tools/tts_tool.py:1050-1071 still sends response_format="opus" whenever a target ends in .ogg, despite _convert_to_opus() already existing at tools/tts_tool.py:899.

Problems

• The new test stubs _resolve_openai_audio_client_config() with two values at tests/tools/test_tts_openai_opus.py:33, but current main destructures three values at tools/tts_tool.py:1027; the resolver contract changed in b53ba0e188.
• except Exception at proposed tools/tts_tool.py:1062 retries as MP3 after any create or file-stream failure. Limit fallback to a confirmed unsupported-opus response, and propagate unrelated authentication, transport, or write errors.

Suggested changes

• Salvage against the current managed-gateway flow and update mocks to return (api_key, base_url, False).
• Add a regression test that an unrelated error does not issue an MP3 retry.

Automated hermes-sweeper review.

[teknium1]

Current main's resolver now returns (api_key, base_url, is_managed) and _generate_openai_tts destructures all three values (tools/tts_tool.py:1027). Update this and the other new mocks to return a three-tuple so the salvaged tests exercise the current contract.

[teknium1]

This catches failures from both the API request and stream_to_file, then issues a second synthesis request even for auth, transport, or local-write errors. Restrict the fallback to a confirmed unsupported-opus format response and propagate unrelated exceptions.

[teknium1]

Resolved at the class level by PR #73072. Rather than a per-provider transcode, text_to_speech_tool now sniffs the audio magic bytes once after every synthesis and repairs any MP3/WAV-bytes-in-.ogg centrally (in-place ffmpeg transcode to real Ogg/Opus, honest-extension rename fallback when ffmpeg is missing) — covering this provider and all others, including command providers and plugins. Your diagnosis of the container mismatch was correct and helped shape the central fix — thanks for the contribution.