Skip to content

feat(tts): normalize spoken text (units, symbols, markdown) before synthesis - #46497

Closed
AlexxRussell wants to merge 2 commits into
NousResearch:mainfrom
AlexxRussell:tts-text-normalize
Closed

feat(tts): normalize spoken text (units, symbols, markdown) before synthesis#46497
AlexxRussell wants to merge 2 commits into
NousResearch:mainfrom
AlexxRussell:tts-text-normalize

Conversation

@AlexxRussell

@AlexxRussell AlexxRussell commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

What

Routes text destined for TTS through a new normalizer, tools/tts_text_normalize.prepare_spoken_text, used by both text_to_speech_tool and the auto-TTS path in the base adapter. It expands units and symbols into spoken words and flattens Markdown into a transcript-like script.

Why

Auto-TTS sent raw chat Markdown and compact symbols straight to the speech provider. Units such as a temperature written with the degree symbol were read as stray letters, and headings or bullet lists ran together without pauses. A dedicated normalizer produces a script that sounds like speech rather than a rendered Markdown message.

Behavior

  • Best-effort: if the normalizer ever raises, the code falls back to the previous markdown-strip behavior, so auto-TTS never breaks.
  • The normalized script is used for synthesis only. Telegram caption eligibility and payload stay on the original reply text, so a long reply is still delivered in full as its own message even when its spoken script would fit the 1024 character caption limit (review follow-up).

Testing

Includes tests/tools/test_tts_text_normalize.py and a long-original/short-normalized regression case in the existing Telegram auto-TTS caption tests (tests/gateway/test_base_topic_sessions.py). Verified locally: those two suites, 8 passed. Also running in production: spoken output reads units and symbols correctly and pauses between sections.

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have tool/tts Text-to-speech and transcription comp/gateway Gateway runner, session dispatch, delivery labels Jun 15, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Feature normalizes spoken text (units, symbols, markdown) before TTS synthesis. 319/6 — moderate scope, clean implementation. No security concerns. The text normalization logic looks correct.


Reviewed by Hermes Agent

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused TTS normalization work. The premise is valid: current main still uses only a minimal Markdown-character regex in gateway/platforms/base.py:3381-3386 before calling TTS at gateway/platforms/base.py:4988-4993.

Problems

  • gateway/platforms/base.py:4317 uses the lossy spoken caption_text for Telegram caption eligibility and payload. The new normalizer removes URL content (tools/tts_text_normalize.py:70), so an original reply over the 1024-character caption limit can shrink below it, be sent only as a normalized caption, and suppress the original visible reply. Current behavior intentionally sends the original reply separately when it exceeds the caption limit (tests/gateway/test_base_topic_sessions.py:299-326).

Suggested changes

  • Keep normalization on the synthesis path, but retain text_content for the Telegram caption and its 1024-character check.
  • Add the long-original/short-normalized regression case to the existing Telegram auto-TTS caption tests.

Automated hermes-sweeper review.

Comment thread gateway/platforms/base.py
):
telegram_tts_caption = text_content
telegram_tts_caption = caption_text
tts_result = await self.play_tts(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes the 1024-character caption gate measure lossy TTS text rather than the visible reply. prepare_spoken_text removes link destinations and URLs, so an original reply over 1024 characters can shrink below the limit, be captioned here, and suppress delivery of the original text. Keep text_content for the user-visible caption/gate; use normalized text only for synthesis.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
…nthesis

Auto-TTS previously fed raw chat Markdown and compact symbols straight to the
speech provider, so units were read as stray letters and headings or bullets ran
together. This routes spoken text through a new normalizer,
tools/tts_text_normalize.prepare_spoken_text, that expands units (for example a
temperature written with the degree symbol becomes "degrees Celsius") and
flattens Markdown into a transcript-like script with sentence pauses.

The normalizer is best-effort: if it ever fails the code falls back to the
previous markdown-strip behavior, so auto-TTS keeps working. The Telegram voice
caption uses the same normalized text. Includes a unit test.
Review follow-up: the spoken script is for synthesis only. Caption
eligibility and payload stay on the original reply, so a long reply
whose normalized script fits the 1024 char limit is still delivered
in full as its own message. Adds the long-original/short-normalized
regression case to the auto-TTS caption tests.
@AlexxRussell

Copy link
Copy Markdown
Contributor Author

Thanks for the review. The caption concern was a real bug and the update fixes it:

  • Caption eligibility and payload now stay on the original reply text (text_content). The normalized script is used only for synthesis, so a long reply whose spoken form shrinks below the 1024 character limit is still delivered in full as its own message, matching current main behavior.
  • Rebased onto current main.
  • Added the requested long-original/short-normalized regression case to the existing Telegram auto-TTS caption tests: test_long_original_with_short_spoken_script_still_sends_full_reply asserts the caption is skipped and the full formatted reply is sent separately.

Test run: tests/tools/test_tts_text_normalize.py plus tests/gateway/test_base_topic_sessions.py::TestTelegramAutoTtsCaptionDelivery, 8 passed.

@AlexxRussell

Copy link
Copy Markdown
Contributor Author

Closing this out: the work landed on main as a9a9005 (feat(tts): normalize spoken text) and ef274a4 (fix(tts): keep Telegram caption on the original reply text), so there is nothing left for this PR to merge.

Thanks for picking it up, and for 4aac89b building on it. Routing the CLI, voice mode and web dashboard through the same prepare_spoken_text cleaner is a better home for it than the two call sites I had originally patched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants