Skip to content

fix(tools): gate OGG conversion on platform and strip markdown from media paths - #11457

Open
nsyring wants to merge 1 commit into
NousResearch:mainfrom
nsyring:fix/tts-voice-delivery
Open

fix(tools): gate OGG conversion on platform and strip markdown from media paths#11457
nsyring wants to merge 1 commit into
NousResearch:mainfrom
nsyring:fix/tts-voice-delivery

Conversation

@nsyring

@nsyring nsyring commented Apr 17, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes two TTS delivery issues:

  1. OGG/Opus conversion was applied to all platforms — only Telegram requires Opus for voice bubbles. Other platforms (Discord, Nextcloud Talk, etc.) work better with the original MP3/WAV. The fix adds a want_opus flag derived from the platform.

  2. Markdown artifacts in MEDIA paths — some models (e.g., Mistral) wrap MEDIA tags in bold/italic markdown (**MEDIA:/tmp/file.mp3**). The fix strips * and _ from media path extraction.

Related Issue

No existing issue.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

Changes Made

  • tools/tts_tool.py: Add want_opus flag gated on platform == "telegram", applied at all three OGG conversion decision points
  • gateway/platforms/base.py: Add *_ to lstrip/rstrip in extract_media path parsing
  • tests/gateway/test_send_image_file.py: 4 tests for markdown artifact stripping (bold, italic, underscore, mixed)

How to Test

  1. Generate TTS on a non-Telegram platform (e.g., Nextcloud Talk)
  2. Audio should be delivered as MP3/WAV, not converted to OGG
  3. Send a message with **MEDIA:/tmp/test.mp3** — path should extract cleanly
  4. Run pytest tests/gateway/test_send_image_file.py -v — all tests pass

Checklist

  • I have read the Contributing Guide
  • My commits follow the Conventional Commits format
  • I have searched for existing PRs to avoid duplicates
  • This PR contains only related changes
  • pytest tests/ -q passes
  • I have added tests for my changes
  • I have tested on: Debian 13 (LXC, amd64)
  • N/A — no new docs needed
  • N/A — no config key changes
  • N/A — no architecture changes
  • Cross-platform: this IS a cross-platform fix
  • N/A — no tool schema changes

@nsyring
nsyring force-pushed the fix/tts-voice-delivery branch 4 times, most recently from f91c10a to 0e4f75a Compare April 21, 2026 09:29
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists tool/tts Text-to-speech and transcription comp/gateway Gateway runner, session dispatch, delivery labels Apr 24, 2026
@nsyring
nsyring force-pushed the fix/tts-voice-delivery branch 7 times, most recently from 492ef8c to 774a574 Compare May 1, 2026 06:18
@nsyring
nsyring force-pushed the fix/tts-voice-delivery branch 9 times, most recently from 5139e1d to 4d6fad3 Compare May 9, 2026 05:47
@nsyring
nsyring force-pushed the fix/tts-voice-delivery branch 6 times, most recently from a504be9 to d9d9cfb Compare May 14, 2026 06:20
@nsyring
nsyring force-pushed the fix/tts-voice-delivery branch 12 times, most recently from 4f4541e to 14ff52b Compare June 1, 2026 06:21
@nsyring
nsyring force-pushed the fix/tts-voice-delivery branch 8 times, most recently from a842a2c to 9aa8bd4 Compare June 6, 2026 21:20
@nsyring
nsyring force-pushed the fix/tts-voice-delivery branch 6 times, most recently from a279e45 to 537eff0 Compare June 13, 2026 06:20
teknium1
teknium1 reviewed Jul 12, 2026
View reviewed changes

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for covering the direct tool call path. Current main already has the Telegram-only Opus routing from ed9087fce (tools/tts_tool.py:2202, :2421-2430), while text_to_speech_tool still sends raw Markdown after its truncation logic (tools/tts_tool.py:2177-2194); the proposed normalization addresses that remaining path.

Problems

  • skip_markdown_strip is a new user-facing tts.providers.<name> setting, but the command-provider optional-key table omits it (website/docs/user-guide/features/tts.md:309-317). Users need its default and SSML/raw-markup purpose documented.

Suggested changes

  • Add skip_markdown_strip to the command-provider documentation, stating that it defaults to false and preserves raw text for providers that require markup.

Automated hermes-sweeper review.

Comment thread tools/tts_tool.py Outdated
# the Auto-TTS path in gateway/platforms/base.py) already strip markdown;
# this brings the agent-callable tool path in line with them.
# Command-providers can opt out via tts.providers.<name>.skip_markdown_strip
# for SSML-aware CLIs that want raw markup passed through.

@teknium1 teknium1 Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skip_markdown_strip is a public command-provider config option, but the documented optional-key table at website/docs/user-guide/features/tts.md:309-317 does not list it. Please document its default and SSML/raw-markup use case.

@nsyring

nsyring commented Jul 12, 2026

Copy link
Copy Markdown
Author

Review feedback addressed in d0c0d7abc:

  • skip_markdown_strip is now documented in the command-provider optional-keys table (website/docs/user-guide/features/tts.md), including its false default and the raw-markup/SSML use case.
  • Rebased onto current main; the Telegram-only Opus routing that ed9087fce landed is no longer part of this diff — what remains is the text_to_speech_tool markdown normalization plus the opt-out.

@nsyring

nsyring commented Aug 10, 2026

Copy link
Copy Markdown
Author

Rebased onto current main. Upstream has since restructured the TTS path — text normalization (prepare_spoken_text) now runs in the text_to_speech_tool wrapper before chunk splitting, and the inner _text_to_speech_single no longer normalizes. Moved the skip_markdown_strip opt-out accordingly: the command-provider config is now resolved in the wrapper before normalization, so an SSML-aware CLI still receives raw markup untouched (both the normalizer and the markdown stripper are gated by the flag). Tests green (7/7 in test_tts_markdown_strip.py).

hermes-lxc
fix(tts): strip markdown in text_to_speech_tool entry point
846d444 
The agent-callable text_to_speech_tool() did not strip markdown before
provider dispatch. Result: Edge TTS (and other providers) verbalized raw
markdown artifacts like *bold* ("asterisk asterisk Bold asterisk asterisk"),

Two other TTS call sites already strip markdown:
- gateway/run.py:_send_voice_reply via _strip_markdown_for_tts()
- gateway/platforms/base.py Auto-TTS via re.sub regex

This brings the third call site (the model-callable tool) in line with
them. Strip happens after empty-check and before max_len truncation, so
the per-provider character budget applies to spoken length, not raw
markdown length.

Command-providers can opt out via tts.providers.<name>.skip_markdown_strip
for SSML-aware CLIs that need raw markup passed through.

Tests: tests/tools/test_tts_markdown_strip.py — 7 cases covering bold,
headers, inline code, list markers, truncation interaction, and the
skip-opt-out flag for command providers.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@teknium1 teknium1 teknium1 left review comments

Assignees

No one assigned

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nsyring @teknium1 @alt-glitch