Skip to content

fix(tts): prevent stalled streaming prefetch from wedging voice barge-in - #76655

Open
Drexuxux wants to merge 1 commit into
NousResearch:mainfrom
Drexuxux:codex/fix-tts-prefetch-barge-in-deadlock
Open

Drexuxux wants to merge 1 commit into
NousResearch:mainfrom
Drexuxux:codex/fix-tts-prefetch-barge-in-deadlock

Conversation

@Drexuxux

@Drexuxux Drexuxux commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

The per-sentence streaming TTS prefetch pipeline from #71084, salvaged onto current main by #76623, caps in-flight requests with a semaphore and plays each bounded chunk queue in order. If three provider iterators stop yielding, the fourth sentence blocks forever in Semaphore.acquire(). Barge-in only sets stop_event, so it cannot release that wait; the playback worker can also remain blocked in chunk_queue.get(), and cleanup waits up to 300 seconds while the old pipeline keeps the audio device.

This makes every prefetch-capacity and playback-queue wait interruption-aware. Normal completion still drains and joins all prefetches. On barge-in, Hermes abandons stalled upstream iterators, closes playback promptly, sets the done event, and allows the next voice turn to start. Full bounded queues are drained just enough to let producer threads terminate instead of leaking.

Related Issue

No issue filed. Follow-up to #71084 / #76623; open and closed PRs/issues were searched for this behavior.

Fixes #

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Security fix
  • Documentation update
  • Tests (adding or improving test coverage)
  • Refactor (no behavior change)
  • New skill (bundled or hub)

Changes Made

  • tools/tts_tool.py: make prefetch semaphore, chunk-queue writes, playback reads, and interrupted cleanup responsive to stop_event.
  • tests/tools/test_tts_streaming.py: reproduce three stalled prefetches plus the blocked fourth sentence and prove barge-in completes promptly.

How to Test

  1. Start four streaming TTS sentences with the first three provider iterators stalled.
  2. Set the real pipeline stop_event while sentence four is waiting for prefetch capacity.
  3. On current main, tts_done_event remains unset; on this branch it is set within one second and the consumer exits.

Regression proof:

main:   FAILED test_hybrid_interrupt_unblocks_stalled_prefetch_pipeline
        AssertionError: TTS interruption remained blocked on stalled prefetch requests
branch: 1 passed in 0.37s

Focused validation:

123 passed, 1 deselected in 5.63s
ruff: passed
compileall: passed
Windows footgun scan: passed
git diff --check: passed

The deselected test is an existing Windows-only clock-resolution assertion in test_hybrid_prefetch_fires_http_immediately; the focused test and all other voice/TTS tests pass.

Checklist

Code

  • I've read the Contributing Guide
  • My commit message follows Conventional Commits
  • I searched open and closed PRs/issues for duplicates
  • My PR contains only changes related to this fix
  • I've run pytest tests/ -q and all tests pass - focused voice/TTS suites were run instead
  • I've added tests for my changes

Documentation & Housekeeping

  • Documentation update - N/A; no user-facing contract changed
  • cli-config.yaml.example update - N/A; no config keys changed
  • CONTRIBUTING.md / AGENTS.md update - N/A
  • Cross-platform impact considered; synchronization uses stdlib queues/events/semaphores
  • Tool descriptions/schemas update - N/A

Screenshots / Logs

See the main-vs-branch regression proof and validation output above.

@alt-glitch alt-glitch added type/bug Something isn't working tool/tts Text-to-speech and transcription P2 Medium — degraded but workaround exists labels Aug 2, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused barge-in fix. The premise is present on current main: tools/tts_tool.py:3611 performs an unbounded prefetch-semaphore acquire, while the playback worker blocks on chunk_queue.get() (tools/tts_tool.py:3532) and cleanup waits for it before signaling completion (tools/tts_tool.py:3763-3775). The stop-aware polling in this PR addresses those waits without adding a new configuration or tool surface.

Problems

  • tests/tools/test_tts_streaming.py:649 only proves three producer generators started. It does not prove the consumer reached the fourth blocked semaphore acquire before stop.set(), so the regression can pass without exercising the reported failure.

Suggested changes

  • Add a deterministic fourth-acquire synchronization point before setting the stop event, then retain the prompt done assertion.

Automated hermes-sweeper review.

daemon=True,
)
consumer.start()
assert three_prefetches_started.wait(timeout=2.0)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This event proves only that three prefetch generators started. The consumer can be descheduled before it reaches the fourth _prefetch_sem.acquire(), allowing stop.set() to make the test pass without exercising the blocked-capacity path. Please synchronize on entry to the fourth acquire (or an equivalent observable) before interrupting.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform area/streaming Streaming responses: gateway delivery, provider wire labels Aug 2, 2026
@andrexibiza

Copy link
Copy Markdown
Contributor

Adversarial verification — Vox Lockin lane 01 (barge-in echo class)

Verified head 199b408f33 against origin/main @ 70db671fac. Covers the stalled-prefetch wedge; mergeable (CI clean: Python tests 8/8 slices, lints, footguns, attribution all SUCCESS); composes with #75792 and #64086 — no file overlap.

Live probe

  • tests/tools/test_tts_streaming.py::test_hybrid_interrupt_unblocks_stalled_prefetch_pipeline: passes on this head (new regression test — a provider whose prefetch stalls indefinitely must not block stop_event; done fires within 1s after interrupt).
  • The regression is real: the test does not exist on origin/main (0 collected there), and on main the blocking chunk_queue.put(timeout=30.0) / _prefetch_sem.acquire() would hold the producer until the 30s put-timeout per stalled sentence — barge-in stop_event was ignored during prefetch.
  • test_hybrid_prefetch_fires_http_immediately fails here AND identically on origin/main — pre-existing Windows-env baseline (HTTP mock timing), not caused by this PR.

Composition with the echo guard (#75792)

#75792 touches cli.py + tools/voice_mode.py; this PR touches tools/tts_tool.py + tests/tools/test_tts_streaming.py — zero overlap. Together they close the CLI barge-in class: playback-phase echo transcripts dropped (#75780) and interruption no longer wedged behind a stalled prefetch (#76655's issue). Both are conflict-free against current main.

Note (out of scope, cross-referenced)

The _next_chunk_or_stop/queue.Full drain loop keeps barge-in responsive; the same surface in tui_gateway/server.py (lane 02/07 territory) still emits playback-phase transcripts unconditionally — flagged on #75792.

No changes needed. Good fix — the test asserts the actual unblock behavior, not a mocked seam.

@andrexibiza

Copy link
Copy Markdown
Contributor

Verification comment (Vox Lockin lane 10 — adversarial check)

Verified this PR against current origin/main (70db671). Verdict: mergeable, CI green, covers the stalled-prefetch wedge — no duplicate opened.

Premise confirmed on main: tools/tts_tool.py stream_tts_to_speaker still had the wedging pattern this PR removes:

  • chunk_queue.put(chunk, timeout=30.0) — a full bounded queue could block the prefetch producer for 30s with no stop-event escape
  • blocking chunk_queue.get() in the playback worker — barge-in stop_event couldn't break the wait
  • unconditional t.join(timeout=10.0) on prefetch threads at teardown

Checked against this PR's head 199b408f33:

  • _consume_to_queue now polls stop_event while retrying put (0.1s) ✓
  • sentinel None insert made barge-in-safe (drains on full queue when stopped) ✓
  • _next_chunk_or_stop / prefetch-semaphore acquire poll stop_event ✓
  • teardown skips join when stop is set (Python can't cancel a thread blocked in network I/O) ✓
  • regression test test_hybrid_interrupt_unblocks_stalled_prefetch_pipeline

CI: all required checks pass (run 30737994064), mergeable_state CLEAN.

Relation to #40010 (Stop TTS on PTT): this is the piece that makes PTT-stop actually responsive during a stalled upstream — the TUI already wires _voice_tts_stop.set() + stop_playback() on PTT (cli.py full-duplex listener). With this merged, #40010's wedge is closed.

Composition note (lane 10): orthogonal to my truncation work (#78234, touches the same file but different hunks — sentence splitting vs. queue protocol). No conflicts expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants