Skip to content

fix(tts): never truncate spoken replies — split long text and scale playback waits - #78234

Open
andrexibiza wants to merge 3 commits into
NousResearch:mainfrom
andrexibiza:vox/10-tts-streaming
Open

andrexibiza wants to merge 3 commits into
NousResearch:mainfrom
andrexibiza:vox/10-tts-streaming

Conversation

@andrexibiza

@andrexibiza andrexibiza commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Related #17973 #53587 #53589 #75201

What changed and why

The streaming-TTS truncation class had four silent user-facing failure modes on main:

  1. 4000-char pre-cap in speak_text and cli._voice_speak_response — every voice-mode/TUI reply was sliced to 4000 characters before synthesis, so long answers were cut off mid-sentence (#53587).
  2. Provider-level truncation in text_to_speech_tool — over-cap text was truncated to the per-provider request limit (OpenAI 4096, xAI 15000, MiniMax 10000, …) with only a log line, silently dropping the tail of user-facing speech (#17973).
  3. Streaming per-sentence truncation — a single sentence over the provider cap was sliced (cleaned[:stream_max_len]) in stream_tts_to_speaker, losing the end of run-on replies.
  4. Flat 300s ffplay waitplay_audio_file killed ffplay after 300 seconds, cutting long TTS files mid-playback even when synthesis succeeded.

The fix

  • No arbitrary truncation: speak_text and _voice_speak_response now pass the full prepared text; length enforcement is deferred to the provider layer, which now splits instead of truncating.
  • Split long speech: text_to_speech_tool splits over-cap text into provider-safe chunks (sentence-boundary aware; run-on words hard-sliced so no content is ever dropped) and concatenates the per-chunk audio with ffmpeg into the single requested output file. Without ffmpeg it degrades to the first chunk with a loud warning (never worse than the old behavior).
  • Streaming path: an oversized sentence is split into cap-sized parts, each prefetched in order, so run-on replies are spoken in full.
  • Waits scaled to content: play_audio_file probes the file duration (ffprobe) and waits duration + 30s slack instead of a flat 300s cap.
  • websockets v15: the xAI streaming TTS path passes additional_headers (the extra_headers kwarg was removed in websockets 14/15 — this path was crashing on every streaming xAI synthesis).

How to test

# Targeted suites (from repo root with the project venv)
python -m pytest tests/tools/test_tts_streaming.py \
  tests/tools/test_tts_streaming_e2e.py \
  tests/gateway/test_streaming_tts_consumer.py \
  tests/gateway/test_streaming_tts_gateway_regression.py \
  tests/tools/test_tts_max_text_length.py -q --no-header -p no:cacheprovider

# New regression coverage (all pass):
#  - test_split_text_for_tts_*            : splitting never drops content
#  - test_text_to_speech_tool_splits_long_text_and_concatenates : 5000 chars → [4096, 904], temp cleanup
#  - test_speak_text_no_longer_precaps_at_4000 : full 5000-char reply reaches the provider layer
#  - test_hybrid_oversized_sentence_split_not_truncated : 200-char sentence with 40-char cap → 5 parts, joined == original
#  - test_ffplay_wait_scales_to_file_duration : wait == duration + 30s slack
#  - test_xai_websocket_uses_additional_headers : websockets v15 kwarg

# Manual smoke (optional, with a TTS provider configured):
#   speak_text("…a 5000+ character reply…") in the TUI voice loop should now speak to the end.

Expected: all of the above pass except the known pre-existing timing-flaky test_hybrid_prefetch_fires_http_immediately (fails on pristine origin/main too — it's an environment-timing test, not related to this change; verified against a clean checkout).

Platforms tested

  • Windows 10 (git-bash), CPython 3.11 — full targeted suites green.
  • ffmpeg/ffprobe present (Windows path); the no-ffmpeg degrade path is covered by the fallback branch and unit-tested via mocked concat.

Why this matters to users

Voice mode and TUI TTS used to silently stop reading long replies at 4000 characters, drop the tail of anything over the provider's request cap, cut playback at 300 seconds, and crash xAI streaming entirely under websockets v15. After this change, whatever the model says is spoken in full — split into provider-safe chunks and reassembled — and playback waits as long as the audio actually is.

Credits

  • Splitting + chunk concatenation approach salvaged from #17973, authored by @TKCen.
  • 4000-char cap removal + ffprobe-scaled ffplay wait salvaged from #53589, authored by @Nomadcxx.
  • websockets v15 additional_headers fix salvaged from #75201, authored by @pluton74mac.

Fixes #53587
Fixes #17973

Part of #40010
Part of #78207
Part of #79890

…ayback waits

The streaming-TTS truncation class had four silent user-facing failure
modes on main:

1. speak_text and cli._voice_speak_response pre-capped every reply at
   4000 characters, so long answers were cut off mid-sentence (NousResearch#53587).
2. text_to_speech_tool truncated over-cap text to the provider limit
   (OpenAI 4096, xAI 15k, ...) with only a log line, dropping the tail
   of user-facing speech (NousResearch#17973).
3. The streaming speaker truncated a single over-cap sentence instead
   of splitting it, losing the end of run-on replies.
4. play_audio_file killed ffplay after a flat 300s, cutting long TTS
   files mid-playback.

This change eliminates the class:

- speak_text and _voice_speak_response now pass the full prepared text;
  length enforcement is deferred to the provider layer, which splits.
- text_to_speech_tool splits over-cap text into provider-safe chunks
  (sentence-boundary aware, hard-slicing run-on words) and concatenates
  the per-chunk audio with ffmpeg into the single requested output file
  (salvaged from NousResearch#17973, authored by @TKCen). No content is dropped;
  without ffmpeg it degrades to the first chunk with a warning.
- stream_tts_to_speaker splits an oversized sentence into cap-sized
  parts and prefetches each, so run-on replies are spoken in full.
- play_audio_file probes the file duration (ffprobe) and waits
  duration + 30s slack instead of a flat 300s (salvaged from NousResearch#53589,
  authored by @Nomadcxx).
- XAI streaming TTS now passes additional_headers for websockets v15
  (extra_headers was removed upstream; salvaged from NousResearch#75201, authored
  by @pluton74mac).

Regression tests cover splitting (no content loss, run-on words,
concat + cleanup, single-chunk short path), speak_text no longer
pre-capping at 4000, streaming oversized-sentence splitting, the
duration-scaled ffplay wait, and the websockets v15 header kwarg.

Fixes NousResearch#53587
Fixes NousResearch#17973

Signed-off-by: andrexibiza <84248988+andrexibiza@users.noreply.github.com>
The NousResearch#53587 duration probe added to voice_mode playback spawns via
subprocess.run, whose internal Popen collides with tests that mock
subprocess.Popen: the env-scrub test captured the probe's env-less call
as the player call (env=None -> assertion fail), and the macOS policy
test's fake proc lacks stdout/stderr, raising AttributeError that the
outer handler misread as a failed player.

- probe catches Exception broadly: unknown duration -> fallback wait;
  it can never raise or abort a successful playback
- env-scrub test identifies the system-player call by its flags instead
  of assuming a single Popen call

Fixes the two CI failures on this PR (slices 2/8 + 7/8). Pre-existing
Windows-local failures (PulseSocket/WSL, hybrid-prefetch timing) verified
identical on pristine main via stash-test.

Signed-off-by: andrexibiza <84248988+andrexibiza@users.noreply.github.com>
Five test_voice_mode failures + one tts_streaming failure, all Windows-
local, all in the platform-mocking/timer-resolution class documented in
the repo's own test conventions:

- TestPulseSocketReachable: AF_UNIX does not exist on Windows Python ->
  skipif(win32); the feature (PulseAudio socket reachability) is POSIX-only.
- test_wsl_without_pulse_blocks_voice: on Windows, powershell.exe exists,
  so _wsl_powershell_tts_available() is True and the code (correctly)
  downgrades to a notice. The test simulates WSL without that fallback, so
  pin _wsl_powershell_tts_available to False regardless of host platform.
- TestWSL2PowerShellFallback (both): the PowerShell fallback is gated on
  platform.system() == "Linux" (WSL IS Linux), but the tests only mocked
  _is_wsl2_env. On a Windows runner the gate never opens. Patch
  platform.system -> "Linux" (+ pin _import_audio ImportError in the
  temp-filename test so a host with sounddevice installed cannot take the
  playback path early) — the repo's documented sys.platform/platform.system
  patch-together pattern.
- test_hybrid_prefetch_fires_http_immediately: time.monotonic() has
  ~15.6ms tick resolution on Windows; two prefetch-adjacent starts land on
  the same tick and break the strict-ordering assert. Record with
  time.perf_counter() (QPC, high-res) — ordering intent unchanged.

Result: tests/tools/test_voice_mode.py + test_tts_streaming.py fully green
on Windows (111 passed, 2 skipped). Full campaign voice surface: 300
passed, 2 skipped; the one remaining local failure
(test_stderr_progress_extends_beyond_timeout) is fixed by NousResearch#78221 pending
main-merge.

Signed-off-by: andrexibiza <84248988+andrexibiza@users.noreply.github.com>
@andrexibiza

Copy link
Copy Markdown
Contributor Author

CI failures (slices 2/8 + 7/8) — root-caused and fixed (8e86264ea6)

Both failures came from the #53587 duration probe interacting with tests that mock subprocess.Popen:

  • test_play_audio_file_scrubbed_envsubprocess.run (the probe) internally calls the mocked Popen, so the probe's env-less call overwrote the test's captured player env (env=None → assertion fail). Fix: the test now identifies the system-player call by its flags (-nodisp/-autoexit) instead of assuming a single Popen call.
  • test_play_audio_file_skips_sounddevice_on_macos — the fake proc lacks stdout/stderr, so the probe raised AttributeError, which the outer handler misread as "player failed" and aborted a successful playback. Fix: the probe catches Exception broadly — best-effort by contract, it can never raise or affect playback.

Windows-local failure class — closed (277693ee42)

The 6 tests that fail on a Windows host (deterministic, verified identical on pristine main) are all in the repo's documented platform-mock/timer class:

Test Root cause Fix
TestPulseSocketReachable ×2 socket.AF_UNIX doesn't exist on Windows Python skipif(win32) — feature is POSIX-only
test_wsl_without_pulse_blocks_voice Windows has real powershell.exe, so the PowerShell TTS fallback is "available" and the code correctly downgrades to a notice pin _wsl_powershell_tts_available → False (the scenario under test)
TestWSL2PowerShellFallback ×2 fallback gated on platform.system() == "Linux" (WSL IS Linux) but tests only mocked _is_wsl2_env patch platform.system → "Linux" (repo's documented sys.platform/platform.system pattern)
test_hybrid_prefetch_fires_http_immediately time.monotonic() ~15.6ms tick on Windows; prefetch-adjacent starts land on the same tick record with time.perf_counter() (QPC, high-res)

Verification

CI is re-running on the new head (277693ee42).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

voice-mode TTS truncates long replies: 4000-char pre-cap in speak_text and 300s ffplay wait

2 participants