feat: strip emoji and markdown formatting from TTS output

[srithanupriyadharsan]

Summary

Text-to-speech produces garbled output when the text contains emoji and markdown formatting. This commit strips them before synthesis.

Changes

• Remove emoji characters (emoticons, symbols, flags, dingbats, transport)
• Strip markdown: **bold**, *italic*, `code`, ~~strikethrough~~
• Clean up zero-width joiners and variation selectors

Why

When Hermes speaks responses that contain emoji (e.g., "Here's the update! 🔥") or markdown (e.g., "the main issue"), the TTS engine reads out the raw unicode/code. This produces noise like "fire emoji" or "asterisk asterisk main asterisk asterisk".

Before/After

• Before: "Here asterisk asterisk is asterisk asterisk the update fire emoji"
• After: "Here is the update"

Test Plan

• Send a message that triggers TTS with emoji
• Verify clean speech output
• Test with markdown-heavy responses

[alt-glitch]

Related to #8205 and #9005 (prior TTS text preprocessing PRs). Consider consolidating these TTS cleanup efforts.

[alt-glitch]

Related to #8205 and #9005.

[teknium1]

Thanks for targeting a real speech-cleanup gap. The current implementation still sends raw text to Edge TTS, but this patch needs restructuring before it can provide the stated behavior.

Problems

• The added logic is only in _generate_edge_tts; the generic dispatcher forwards raw text to command, plugin, and other built-in providers at tools/tts_tool.py:2174-2378. The ElevenLabs streaming path separately calls _strip_markdown_for_tts() at tools/tts_tool.py:2674, so it would not receive the emoji cleanup.
• The proposed range \U00010000-\U0010ffff in the PR diff (tools/tts_tool.py:220, right side) matches all non-BMP Unicode, not just emoji, and would remove valid non-emoji text.
• No tests are included. Existing Markdown-only helper coverage is in tests/tools/test_voice_cli_integration.py:48-123.

Suggested changes

• Centralize Unicode-safe normalization before provider dispatch and reuse it for streaming TTS.
• Preserve non-emoji astral Unicode, replace removed symbols with spacing, and add direct-dispatch plus streaming regression tests.

Automated hermes-sweeper review.

[teknium1]

This range matches every code point above U+FFFF, not only emoji. It would strip valid non-emoji astral Unicode; use a narrow emoji matcher instead.

[teknium1]

This runs only in the Edge provider. text_to_speech_tool() also dispatches to command, plugin, and other built-in providers, while streaming ElevenLabs has its own preprocessing path. Please centralize normalization before dispatch and reuse it for streaming.

[GottZ]

This was generated by AI during triage.

Summary

Two PRs address the TTS preprocessing gap. #13311 adds Edge-only emoji and Markdown stripping but misses other providers and over-removes all non-BMP Unicode, while #25956 describes broader provider-facing cleanup but has no implementation in the available diff.

Related pull requests

• #13311 related — (+25/-0) — keep open for rework, not merge-ready: the diff preprocesses only _generate_edge_tts, so command, plugin, streaming, and other built-in provider paths remain unaffected; its \U00010000-\U0010ffff range also deletes valid non-emoji astral Unicode. This follows the keep_open review on #13311: normalization should be centralized, Unicode-safe, shared with streaming TTS, and covered by dispatch and streaming regression tests.
• #25956 [closed] related — (+0/-0) — retain closed as a duplicate reference only: its description targets broader emoji, diagram, code, and table cleanup, but the available diff contains no changes that can be evaluated or merged. It remains relevant because the contributor discussion identifies it as overlapping #13311 and as a duplicate of the more comprehensive #8205 effort.

Duplicates

#13311 and #25956 substantially overlap on stripping emoji and formatting before TTS; contributor discussion additionally identifies both as overlapping the broader #8205 preprocessing work.

Suggested consolidation

Do not merge either current diff. Keep #13311 as the active consolidation target only if it is reworked to centralize Unicode-safe preprocessing across every provider and streaming path with regression tests; keep #25956 closed as a duplicate/reference, and reconcile the revised scope with #8205 before merge to avoid duplicating the repository's more comprehensive preprocessing effort.

Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 2 kB of PR diffs, 2 kB of issue/PR text, 2 kB of discussion (4 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[teknium1]

Emoji stripping is now covered on every TTS path by the shared prepare_spoken_text cleaner merged in #73513 (base normalizer from @AlexxRussell's #46497). Thanks for the contribution!

(Landed via #73513, merge 4aac89b429.) Closing.