feat(tts): add text preprocessing for natural speech output

[kas-cor]

Problem

When TTS reads text aloud, it pronounces markdown syntax, emoji characters, and pseudo-graphics literally — making voice messages sound robotic and unnatural across all platforms (Telegram, Discord, WhatsApp, CLI).

Solution

Add _preprocess_tts_text() — a multilingual text normalization layer that runs before any TTS provider receives the text.

Features

1. Strips markdown: code blocks, bold/italic, inline code, links, headings, horizontal rules
2. Localized emoji speech: 28 emojis with spoken equivalents in 13 languages (en, ru, de, es, fr, pt, ja, zh, ko, it, pl, uk, tr)
3. Smart language detection via _get_tts_language():
   • tts.language config (explicit override)
   • stt.local.language config (reuse STT language — same user intent)
   • TTS voice name extraction (e.g. ru-RU-SvetlanaNeural → ru)
   • 'en' as fallback
4. Strips remaining Unicode emojis (10 major ranges covered)
5. Converts box-drawing chars and table pipes to pauses/spaces
6. Removes MEDIA: tags, collapses whitespace

Config options

tts:
  preprocess: true    # enable/disable text preprocessing (default: true)
  language: ru        # explicit override for TTS preprocessing language

If neither tts.language nor stt.local.language is set, language is auto-detected from the configured TTS voice name.

Supported languages

Code	Language	✅	⬆️	⚠️
en	English	done	update	warning
ru	Russian	готово	обновление	внимание
de	German	fertig	Update	Warnung
es	Spanish	hecho	actualización	atención
fr	French	fait	mise à jour	attention
pt	Portuguese	feito	atualização	atenção
ja	Japanese	完了	更新	警告
zh	Chinese	完成	更新	注意
ko	Korean	완료	업데이트	주의
it	Italian	fatto	aggiornamento	attenzione
pl	Polish	gotowe	aktualizacja	uwaga
uk	Ukrainian	готово	оновлення	увага
tr	Turkish	tamam	güncelleme	dikkat

Testing

from tools.tts_tool import _preprocess_tts_text, _get_tts_language, _load_tts_config

# Auto-detect language from config (voice = ru-RU-SvetlanaNeural)
config = _load_tts_config()
lang = _get_tts_language(config)  # → 'ru'

# Russian: emoji speech in Russian
_preprocess_tts_text('✅ **Обновлено** до v0.8 ⬆️', lang='ru')
# → 'готово Обновлено до v0.8 обновление'

# English: emoji speech in English
_preprocess_tts_text('✅ **Updated** to v0.8 ⬆️', lang='en')
# → 'done Updated to v0.8 update'

# Table pipes → spaces
_preprocess_tts_text('| Name | Value |', lang='en')
# → 'Name Value'

Impact

• Zero config changes needed — enabled by default, works out of the box
• All 6 TTS providers supported (Edge, ElevenLabs, OpenAI, MiniMax, Mistral, NeuTTS)
• No breaking changes — config load moved slightly earlier, no functional impact
• 28 emojis with localized speech in 13 languages, ~30 emojis silently removed
• Config toggle: tts.preprocess: false to disable

[alt-glitch]

Related to #13311 (open) and #9005 (closed) — all three add TTS text preprocessing. This PR is the most comprehensive (13-language emoji localization, language detection). Consider consolidating with #13311.

[teknium1]

Thanks for the comprehensive normalization proposal. The underlying cleanup gap is still present on current main, but the salvage needs to cover a live sibling path and the current configuration surface.

Problems

• The new invocation is only in text_to_speech_tool(), while ElevenLabs streaming calls _strip_markdown_for_tts() directly at tools/tts_tool.py:2674; that path would still send emoji and pseudo-graphics to its provider.
• The PR documents tts.preprocess and tts.language, but current defaults live in hermes_cli/config.py:2066-2123 and the diff does not add either key there.
• No tests are added. Existing coverage in tests/tools/test_voice_cli_integration.py:48-123 covers basic Markdown stripping only.

Suggested changes

• Centralize config-aware normalization and apply it to both synchronous and streaming provider submission paths.
• Add defaults plus focused behavior tests, including preprocess: false, language resolution, and streaming TTS.

Automated hermes-sweeper review.

[kas-cor]

Fixes applied — addressing sweeper review

Thanks @teknium1 for the review! Pushed a fix (c111062) addressing all three points:

1. Centralize config-aware normalization for both paths

The streaming path (_speak_sentence in stream_tts_to_speaker()) was calling _strip_markdown_for_tts() directly, bypassing emoji speech substitution and pseudo-graphics stripping. Now both paths use _preprocess_tts_text():

# In _speak_sentence (streaming path):
if tts_config.get("preprocess", True):
    lang = _get_tts_language(tts_config)
    cleaned = _preprocess_tts_text(sentence, lang=lang).strip()
else:
    cleaned = _strip_markdown_for_tts(sentence).strip()

• text_to_speech_tool() — already had preprocessing (unchanged)
• stream_tts_to_speaker() → _speak_sentence() — now uses _preprocess_tts_text() when preprocess: true (default)

2. Add config defaults

Added preprocess and language to the TTS defaults in hermes_cli/config.py:

"tts": {
    "provider": "edge",
    "preprocess": True,   # set false to pass raw text to TTS
    "language": "",        # ISO 639-1 code; "" = auto-detect
    ...
}

3. Tests added (40 tests, all passing)

New file: tests/tools/test_tts_preprocess.py

Suite	Tests	Coverage
TestPreprocessBasic	9	markdown, code blocks, links, headers, MEDIA: tags, whitespace
TestEmojiSpeech	10	localized emoji speech (en/ru/de/ja), multiple emojis, silent removal
TestPseudoGraphics	2	table pipes, box-drawing chars
TestGetTtsLanguage	7	explicit override, STT fallback, voice extraction, empty string, fallback en
TestDetectLangFromVoice	6	Edge/OpenAI voice names, short names, empty/None
TestLookupEmojiSpeech	5	exact match, en fallback, silent, unknown, lang prefix
TestPreprocessDisabled	1	preprocess toggle sanity

============================== 40 passed in 1.49s ==============================

[teknium1]

Superseded by the broader normalizer from @AlexxRussell's #46497, merged in #73513 as the shared prepare_spoken_text cleaner. Thanks for the early work on this!

(Landed via #73513, merge 4aac89b429.) Closing.