feat(tts): streaming playback for edge and openai providers

[a1398394385]

Summary

Adds streaming playback for two providers that previously took the slow
file-then-play path in CLI voice mode, and generalizes the streaming
dispatch surface so future built-in TTS providers plug in via one
elif branch instead of touching cli.py or stream_tts_to_speaker.

What this PR actually delivers per provider

Provider	Streaming nature	User-visible win
Edge (free, no key)	True streaming — edge_tts.Communicate + miniaudio on-demand decode	First-audio latency: full-reply-wait → ~first sentence
OpenAI (paid)	HTTP-body chunked only — server generates full clip before first byte	Skips temp-file round-trip; no latency win
ElevenLabs	Unchanged	Hard-coded branch removed; now goes through generic dispatch

Why the dispatch surface matters

Before this PR, cli.py only activated streaming for ElevenLabs and
stream_tts_to_speaker was hard-coded around the ElevenLabs SDK. The
three built-in providers (edge/openai/elevenlabs) now route
through a single generic dispatch surface (_iter_pcm_chunks), so a
future built-in plugs in via one elif branch instead of touching
cli.py or stream_tts_to_speaker.

Plugin providers (registered via the TTSProvider ABC) are not
dispatched to stream() by this PR — _dispatch_to_plugin_provider
still routes to synthesize() only. Wiring it to stream() is a
follow-up that needs its own format-negotiation contract (sample rate
/ channel / format advertisement).

Cross-references

• Partial progress on Plugin TTS providers define stream(), but current TTS dispatch and CLI never consume it generically #47896 — plugin TTS providers define
  stream() but CLI never consumes it generically. This PR delivers
  the built-in half: generic dispatch path for edge/openai/elevenlabs,
  CLI no longer hard-coded, refactored stream_tts_to_speaker. The
  plugin dispatcher still routes to synthesize() only — wiring it to
  stream() is a follow-up.
• Partial progress on Feature: TTS Speed Control — Wire native rate params for Edge/OpenAI, add ffmpeg atempo fallback for NeuTTS/ElevenLabs #6926 — native speed params now wired on
  the edge and openai streaming paths (Tier 1 of that issue).
• Supersedes [Bug]: Signal replies are duplicated when gateway streaming is enabled #4647's symptom for the CLI case — the
  suppress_token_streaming_for_voice helper eliminates the duplicate
  ⚕ Hermes response box that voice streaming produced when both
  display_callback (sentence-level) and stream_delta_callback
  (token-level) ran simultaneously.

What changed

• agent/tts_provider.py — minor docstring tweak on
  TTSProvider.stream() ("the dispatcher falls back" → "the caller
  falls back") to reflect that the fallback decision lives in the
  caller, not the registry.
• tools/tts_tool.py
  • _stream_edge(text, *, voice, speed, tts_config) — sync bridge
    over edge_tts.Communicate().stream(); yields raw MP3 chunks.
  • _stream_openai(text, *, voice, model, speed, format, tts_config)
    — yields bytes from
    client.audio.speech.create(...).iter_bytes(chunk_size=4096);
    supports format="pcm" for direct sounddevice write. Note:
    the OpenAI endpoint is HTTP-body chunked only — see "What this PR
    actually delivers" above.
  • _decode_edge_mp3_to_pcm(mp3_iter) —
    miniaudio.StreamableSource wrapping a chunked MP3 iterator so
    PCM is decoded on demand (no full-file buffering).
  • _iter_pcm_chunks(text, provider, tts_config) — single dispatch
    by provider; all branches return Iterator[bytes] yielding the
    same PCM shape (24 kHz int16 mono LE).
  • _probe_streaming_deps(provider) — startup dep + cred check so
    the user gets a clean warning before the first sentence, not a
    per-call exception inside the speak loop.
  • stream_tts_to_speaker(..., provider="elevenlabs") — gains a
    provider arg and routes through _iter_pcm_chunks instead of
    building an ElevenLabs client inline.
  • suppress_token_streaming_for_voice(agent, use_streaming_tts) —
    clears agent.stream_delta_callback while voice streaming is
    rendering sentence-by-sentence, so the user doesn't see two
    ⚕ Hermes boxes with the same content (same root cause as
    [Bug]: Signal replies are duplicated when gateway streaming is enabled #4647's double delivery class).
• cli.py — _voice_tts activation no longer hard-codes ElevenLabs;
  reads tts.<provider>.streaming config and calls
  _probe_streaming_deps(provider). Restores
  stream_delta_callback in a finally block.
• hermes_cli/config.py — adds streaming: bool to tts.edge,
  tts.elevenlabs, tts.openai sub-dicts. Defaults:
  elevenlabs=true, edge=false, openai=false (edge needs
  miniaudio; opt-in keeps pip install minimal). Schema version
  31 → 32.
• pyproject.toml — declares miniaudio>=1.71,<2 in the voice
  extra. Lazy-imported at call time so non-streaming installs don't
  pay the import cost.

New tests

• tests/hermes_cli/test_tts_streaming_config.py
• tests/tools/test_stream_tts_to_speaker.py — multi-provider
  dispatch; pytest.importorskip("numpy") at module top so the file
  is skipped on lean CI.
• tests/tools/test_tts_streaming.py
• tests/tools/test_tts_streaming_playback.py
• tests/tools/test_voice_cli_integration.py::TestStreamingTTSActivation
  rewritten to cover all three providers instead of only ElevenLabs.

Out of scope / follow-ups

• xAI streaming playback. The tts.xai.* config knobs from
  upstream commit e00b96540 work for the file path; this PR does
  not wire _iter_pcm_chunks(xai=) because the current xAI REST
  endpoint advertises a streaming URL but is non-chunked per docs.
  Tracked separately.
• Tier 3 of Feature: TTS Speed Control — Wire native rate params for Edge/OpenAI, add ffmpeg atempo fallback for NeuTTS/ElevenLabs #6926 (per-stream PCM playback speed via resample or
  samplerate change) is a follow-up — this PR only sends native
  provider speed params on the streaming path.
• Format metadata on the playback path — Plugin TTS providers define stream(), but current TTS dispatch and CLI never consume it generically #47896 mentions that
  providers may yield raw PCM with varying sample rate / channel
  counts. The 24 kHz int16 mono LE contract is documented in
  _iter_pcm_chunks docstring but not yet passed as explicit format
  metadata. PR keeps the existing implicit contract to avoid
  breaking callers; a follow-up can lift it to a StreamFormat
  dataclass when the second non-PCM-shape provider lands.
• MiMo streaming ([Feature]: Native MiMo TTS streaming support (Xiaomi MiMo-V2.5-TTS) #43700). The author of [Feature]: Native MiMo TTS streaming support (Xiaomi MiMo-V2.5-TTS) #43700 wanted MiMo as a
  fourth provider in _iter_pcm_chunks. The dispatch table here is
  one elif branch away from supporting that — MiMo's
  OpenAI-compatible request shape means the _stream_openai branch
  is the reference implementation.

Verification

scripts/run_tests.sh \
  tests/tools/test_tts_streaming.py \
  tests/tools/test_stream_tts_to_speaker.py \
  tests/tools/test_tts_streaming_playback.py \
  tests/hermes_cli/test_tts_streaming_config.py \
  tests/tools/test_voice_cli_integration.py

Result on this branch: 112 tests passed, 0 failed in ~1.7s.

Manual smoke (CLI, Edge TTS, free, no key):

# config.yaml
tts:
  provider: edge
  edge:
    streaming: true

hermes → /voice on → expect first-audio latency < 1.5 s on a
short reply; no second ⚕ Hermes box.

Manual smoke (CLI, OpenAI TTS, paid):

# config.yaml
tts:
  provider: openai
  openai:
    streaming: true
    model: gpt-4o-mini-tts
    voice: alloy

Expect no temp-file write/read on the playback path; first-audio
latency unchanged from non-streaming because the OpenAI endpoint
synthesizes the full clip server-side (see "What this PR actually
delivers" above).

Risks

• miniaudio becomes a runtime dep when tts.edge.streaming=true.
  Documented in the config default (streaming: false for edge).
• _config_version bump 31 → 32 is additive — streaming keys
  merge in via the existing deep-merge path for users on 31 with no
  schema migration required.

Commits

feat(tts): streaming playback for edge and openai providers
build(pyproject): add miniaudio to voice extra for edge streaming decode

[teknium1]

Thanks for generalizing the CLI streaming path. The premise remains valid on current main: cli.py:12248 only activates it for ElevenLabs.

Problems

• tools/tts_tool.py:1157 unpacks two values from _resolve_openai_audio_client_config(), but current main defines that helper as a three-tuple at tools/tts_tool.py:2525-2553; the existing synchronous caller correctly unpacks all three at tools/tts_tool.py:1027. OpenAI streaming will raise before playback. The added test masks this by mocking a two-tuple at tests/tools/test_tts_streaming_playback.py:33-34.
• tests/hermes_cli/test_tts_streaming_config.py:27 asserts version 32, while this PR sets the default to 34 in hermes_cli/config.py:3309. This is both self-failing and a config-version snapshot rather than a behavioral test.

Suggested changes

• Preserve the three-value OpenAI resolver contract, including managed-gateway model handling, and test direct plus managed resolution.
• Replace the fixed-version assertion with a migration/deep-merge invariant, and document the new opt-in Edge/OpenAI flags in website/docs/user-guide/features/tts.md:43-58.

Automated hermes-sweeper review.

[teknium1]

_resolve_openai_audio_client_config() returns (api_key, base_url, is_managed) on current main (tools/tts_tool.py:2525-2553), so this two-value unpack raises ValueError for every OpenAI streaming call. Preserve the third value and the managed-gateway handling used by _generate_openai_tts.

[teknium1]

This assertion cannot pass with this PR: hermes_cli/config.py:3309 sets the default schema version to 34. Please remove the literal version snapshot and assert migration behavior against DEFAULT_CONFIG instead.

[a1398394385]

Sweeper feedback addressed in a4c646a0d:

• OpenAI streaming crash — _stream_openai now unpacks the 3-tuple from _resolve_openai_audio_client_config() (api_key, base_url, is_managed) and mirrors the sync _generate_openai_tts path's managed-gateway model coercion — when is_managed and no custom base_url, an unsupported model (e.g. tts-1-hd) is coerced to a MANAGED_OPENAI_TTS_MODELS entry so the gateway doesn't 400. The original test masked the bug by mocking a 2-tuple; both test mocks corrected to 3-tuples, and a new test_iter_pcm_chunks_openai_managed_coerces_unsupported_model covers the managed path. Direct + managed resolution are now both exercised.
• Brittle version snapshot — dropped test_config_version_bumped_to_32. The two migration tests (test_v31_config_migrates_streaming_fields, test_v31_config_preserves_user_set_streaming) already cover the deep-merge invariant that matters.
• Docs — added tts.<provider>.streaming (with the edge/miniaudio opt-in note) to website/docs/user-guide/features/tts.md.

Also rebased onto latest main; 112 PR-scoped tests pass.

[a1398394385]

Closing — superseded by #69511 (merged), which shipped a more complete provider-agnostic streaming core (StreamingTTSProvider ABC + registry), barge-in, and CLI/TUI/Desktop coverage.

Most of what this PR delivered overlaps with #69511:

• ElevenLabs + OpenAI streaming PCM paths
• stream_tts_to_speaker refactored into a generic dispatcher
• CLI streaming gate broadened beyond ElevenLabs-only

The one piece #69511 deliberately left out is Edge TTS streaming (Edge only outputs MP3, so true streaming needs miniaudio decode, and per-sentence sync fallback is what their dispatch does today). I will open a smaller follow-up that registers Edge as a StreamingTTSProvider instead of re-implementing the whole dispatch surface.

Thanks for the hermes-sweeper review — the _resolve_openai_audio_client_config 3-tuple catch was a real bug.