fix(tts): per-sentence prefetch pipeline with PortAudio resilience

[beardedeagle]

Problem

Three issues in stream_tts_to_speaker:

1. Inter-sentence gaps (3-5s) — each sentence triggered its own API call, but the HTTP request only fired when the playback worker reached the generator, not when the sentence was enqueued.

2. Scattered audio fragments — OpenAI streaming PCM yields chunks on arbitrary byte boundaries not aligned to int16 (2 bytes). numpy.frombuffer raised on odd-length chunks and silently dropped them.

3. Process hang on PortAudio errors — transient PaErrorCode -9986 errors on macOS device state changes killed the playback thread and hung the pipeline join. The 30s worker join timeout was also too short.

Solution

Per-sentence prefetch pipeline — each sentence gets its own streamer.stream() call immediately upon completion. A background thread fires the HTTP request right away, buffering PCM chunks into a per-segment queue. A single playback worker drains these in FIFO order — sentence N+1's HTTP fires while sentence N is still playing.

PCM chunk alignment — carry leftover bytes into the next chunk. Use dtype='<i2' (explicit little-endian) per the StreamingTTSProvider contract.

PortAudio write error resilience — wrap output_stream.write() in try/except. On error: reinitialize the stream (up to 3 attempts), re-attempt the failed write, fall back to temp-file playback if reinit is exhausted.

Worker join timeout — 30s to 300s (a 1000-char response is ~70s of audio).

Regression tests

• test_streamer_path_handles_misaligned_pcm_chunks
• test_streamer_path_survives_portaudio_write_error
• test_streamer_reinit_after_portaudio_error_plays_remaining_sentences
• test_streamer_tempfile_fallback_after_reinit_exhausted
• test_hybrid_subsequent_sentences_prefetched_individually
• test_hybrid_playback_serialized_no_overlap
• test_hybrid_prefetch_fires_http_immediately
• test_display_callback_not_called_when_streaming_enabled

[Copilot]

Pull request overview

Improves the low-latency streaming TTS path in tools/tts_tool.py by introducing a per-sentence prefetch pipeline and adding resilience around PortAudio write failures, with accompanying regression tests for the streaming dispatch behavior.

Changes:

• Add per-sentence prefetching so each sentence’s streamer.stream() request fires immediately and is buffered for a single FIFO playback worker.
• Add PortAudio write error handling with stream reinit attempts and fallback to temp-file playback when reinit is exhausted.
• Expand tests/tools/test_tts_streaming.py with new regression tests for misaligned PCM chunks, PortAudio failures, and per-sentence prefetch behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
tools/tts_tool.py	Implements per-sentence streaming prefetch + PortAudio error recovery in stream_tts_to_speaker.
tests/tools/test_tts_streaming.py	Adds regression tests covering streaming dispatch, misaligned PCM chunks, and PortAudio error scenarios.

Comments suppressed due to low confidence (2)

tools/tts_tool.py:3023

• The no-audio-device temp-file path also passes raw streaming PCM chunks straight into the WAV writer. If the provider yields odd-length chunks (common for HTTP chunking), the WAV output can contain partial int16 samples or drop bytes. Apply the same int16 alignment here before writing/playing the tempfile.

                    # Materialize the prefetched chunks for the temp-file path.
                    _chunks = []
                    while True:
                        chunk = chunk_queue.get()
                        if chunk is None:
                            break
                        _chunks.append(chunk)
                    _play_via_tempfile(
                        iter(_chunks), stop_event, streamer.sample_rate
                    )

tests/tools/test_tts_streaming.py:156

• These new OpenAIStreamer tests assume managed-gateway availability via tools.tts_tool._has_openai_audio_backend(), but OpenAIStreamer.available() currently only checks OPENAI_API_KEY (tools/tts_streaming.py:204-206) and never consults tts_tool. As written, this test will fail (get_env_value is patched to None, so available() returns False). Either implement the managed-gateway availability path in OpenAIStreamer (and include that file change in this PR) or update/remove this test.

def _drain_queue(sentences):
    q = queue.Queue()
    for s in sentences:
        q.put(s)
    q.put(None)
    return q


def _sd_mock():
    sd = MagicMock()
    out = MagicMock()
    sd.OutputStream.return_value = out
    return sd, out


def test_streamer_path_writes_pcm_to_output(monkeypatch):
    from tools import tts_tool

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[teknium1]

Thanks for tackling the real sequential-streaming and PCM-alignment issues: current main still blocks on a sentence's streamer.stream() iterator at tools/tts_tool.py:3445-3468, and directly converts each chunk at tools/tts_tool.py:3463.

Problems

• Cancellation can strand a prefetch thread. In PR head tools/tts_tool.py:2880, _consume_to_queue() unconditionally blocks on chunk_queue.put(None), but the worker skips a queued segment when stop_event is set (tools/tts_tool.py:2937-2938). If the bounded queue is full, nobody drains it and _prefetch_sem.release() is unreachable.
• The branch predates main's deliberate macOS output policy. a1bc12f191 added the Darwin guard now at tools/tts_tool.py:3393-3399; the PR baseline opens OutputStream unconditionally. A salvage must retain that tempfile/afplay route and the current output-activity lifecycle (tools/tts_tool.py:3449-3465).

Suggested changes

• Make cancellation drain/unblock every segment queue and add a saturated-queue cancellation regression.
• Reapply the prefetch design onto current main while retaining the newer macOS and tempfile-cleanup behavior.

This is an automated hermes-sweeper review.

[teknium1]

Cancellation can deadlock this producer: _playback_worker skips a queued segment when stop_event is set, so after the bounded queue fills there is no consumer for this unconditional sentinel put. The thread then never reaches _prefetch_sem.release(). Drain cancelled queues or make completion signaling cancellation-aware, and add a saturated-queue stop regression.

[kshitijk4poor]

Merged via #76623. Your commits cherry-picked with authorship preserved — the prefetch pipeline architecture is unchanged, with fixes applied on top for regressions found during review (mark_audio_output_active, tmp.close, exception-path deadlock, PCM leftover bug) and code dedup (_create_output_stream, _align_int16_chunks).

Thanks for the contribution!