feat(plugins): pluggable TTS backends + Volcengine (Doubao) reference provider - #17589
Hypnus-Yuan wants to merge 11 commits into
Conversation
Pure refactor in preparation for pluggable TTS backends. Moves the post-synthesis finalization block (file existence check, Opus conversion, MEDIA tag construction, response JSON) into a dedicated helper. No behavior change — all existing TTS tests pass unchanged.
Mirrors agent/image_gen_provider.py in shape. Concrete backends subclass TtsProvider, implement name + synthesize(text, output_path, config), and optionally override is_available / max_text_length / list_voices / default_voice / get_setup_schema. synthesize() returns a result dict documenting file_path, format, native_opus, and voice_compatible so the text_to_speech tool can route Telegram voice-bubble delivery without hardcoded provider lists.
Mirrors agent/image_gen_registry.py with one deliberate divergence: get_active_provider() returns None when tts.provider is unset, rather than auto-selecting a legacy-default or sole-registered backend. This preserves the existing Edge TTS default for users who never configured tts.provider — the text_to_speech dispatcher treats None as 'fall through to the hardcoded branch'. Thread-safe under a module-level Lock. _reset_for_tests() provided for pytest isolation.
Mirrors register_image_gen_provider at line ~423. Validates the provider is a TtsProvider subclass; delegates to agent.tts_registry. Plugins call ctx.register_tts_provider(MyTtsProvider()) from their register(ctx) function to attach a new TTS backend without touching core tool code.
…atcher Add LEGACY_TTS_PROVIDERS frozenset and _dispatch_to_plugin_tts_provider function that fires before the legacy if-elif chain in text_to_speech_tool. The dispatcher returns None for empty or legacy provider names so existing users (tts.provider: edge/elevenlabs/openai/etc.) see zero change. Non-legacy names consult agent.tts_registry; unregistered names surface a helpful 'run hermes plugins list' error. _finalize_tts_response_from_plugin consumes the result dict returned by TtsProvider.synthesize() — file_path, native_opus, voice_compatible — so plugin backends can produce any format without hardcoded gateway voice-bubble drift. check_tts_requirements consults the plugin registry after all hardcoded checks, so users with only a plugin backend configured still see TTS as 'available'. _resolve_max_text_length falls through to the plugin registry when provider is not in the hardcoded PROVIDER_MAX_TEXT_LENGTH table.
stream_tts_to_speaker must not consult the plugin registry or the prepend dispatcher. Two source-level assertions lock this in: no tts_registry reference, no _dispatch_to_plugin_tts_provider reference.
Add _plugin_tts_providers() row builder mirroring _plugin_image_gen_providers. Plugin providers with names in LEGACY_TTS_PROVIDERS are skipped to avoid duplicate picker entries. _visible_providers injects TTS plugin rows when cat is 'Text-to-Speech'. _is_provider_active recognizes 'tts_plugin_name' marker. _configure_provider writes tts.provider when tts_plugin_name is present.
When tts.provider names a non-legacy backend, the setup wizard's tool-status section now consults the TTS plugin registry to determine availability. Shows display_name and 'missing env vars' when is_available() returns False, falls through to Edge TTS default when the named provider is not registered.
…backend Implements the first plugin-registered TTS provider using the new TtsProvider ABC + tts_registry infrastructure. - plugins/voice/volcengine/client.py: 868-line WebSocket bidirectional streaming client (seed-tts-2.0 protocol), production-tested - plugins/voice/volcengine/tts.py: VolcengineTtsProvider implementing the ABC contract; returns result dict; lazy-imports .client inside synthesize() so discovery doesn't require websockets - plugins/voice/volcengine/__init__.py: register(ctx) hook - plugins/voice/volcengine/plugin.yaml: manifest Design notes: - is_available() checks only VOLCENGINE_APP_ID + VOLCENGINE_ACCESS_TOKEN via get_env_value (profile-aware), never imports .client/websockets - All os.getenv calls replaced with get_env_value; _reload_dotenv removed - audio_format=ogg_opus → native_opus=True, voice_compatible=True - Config > env > default resolution via _env_or helper
- pyproject.toml: add voice-volcengine = ["websockets>=12.0,<15"] optional dependency and include it in the [all] extra - scripts/release.py: add hypnus.yuan@gmail.com → Hypnus-Yuan to AUTHOR_MAP
- hermes_cli/tools_config.py: add tts_plugin_name handling to _reconfigure_provider so reconfigure flow correctly sets tts.provider for plugin backends - hermes_cli/setup.py: distinguish unregistered non-legacy plugin names from the Edge TTS default fallback in status probe; show 'plugin not registered' with helpful guidance - plugins/voice/volcengine/tts.py: derive correct output path based on audio_format (ogg_opus → .ogg, wav → .wav) to avoid writing Ogg/Opus bytes into a .mp3 file - pyproject.toml: tighten websockets version to >=14.0,<16 (additional_headers API requires websockets 14+)
|
Closing this PR — saw that #17843 (command-type provider registry) shipped Migration path for the Volcengine work: I've reshaped the 868-line Volcengine WebSocket-streaming client into a tts:
provider: volcengine
providers:
volcengine:
type: command
command: 'python ~/.hermes/scripts/volcengine_tts.py
--text-file {input_path} --out {output_path}
--voice {voice} --speed {speed} --format {format}'
output_format: mp3
max_text_length: 1024
timeout: 30This works for non-streaming use cases (cold-start ~1-2s per call to open Branch stays on my fork at |
Summary
Mirrors the
image_genplugin refactor for TTS:TtsProviderABC +tts_registryinagent/PluginContext.register_tts_provider()hooktext_to_speechtool (mirror ofimage_generation_tool._dispatch_to_plugin_provider)hermes_cli/tools_config.py), setup-wizard status probe (hermes_cli/setup.py)plugins/voice/volcengine/Why
Adding a new TTS backend today requires editing
tools/tts_tool.py(1500+ LOC). This PR makes that a plugin drop, matching howimage_genalready works. The dispatcher is prepend-style so all existing users see zero change — onlytts.provider: <non-legacy-name>routes through the plugin path.What's NOT in this PR
tests/tools/test_tts_streaming_untouched.pylocks this in)Behavior preservation
tts.provider: edge/elevenlabs/openai/minimax/mistral/gemini/xai/kittentts/neutts→ zero changetts.provideris set to a non-legacy nameHow to test locally
Platforms tested
tests.ymlrun)Design notes
TtsProvider.synthesize()returns aDict[str, Any](mirror ofImageGenProvider.generate()) rather than writing and raising. This lets providers carrynative_opus/voice_compatiblemetadata forward without a hardcoded name list in_finalize_tts_response.tts_registry.get_active_provider()returnsNonewhentts.provideris unset — deliberate divergence fromimage_gen_registry(which auto-selects FAL). The divergence preserves the existing unset-config → Edge-default flow; documented in the registry module docstring.Volcengine's
client.pyWebSocket streaming code is lazily imported insidesynthesize()so machines without the optionalvoice-volcengineextra installed can still discover the plugin without ImportError.Output path derivation in Volcengine provider: when
audio_format=ogg_opus, the provider writes to a.oggpath (not the caller-suggested.mp3) and returns the actual path infile_path.Closes
(no open issue — happy to split / revert / rework if you prefer a different architecture)