Skip to content

fix(telegram): preserve replied voice-note semantics for STT - #95740

Open
djagya wants to merge 1 commit into
NousResearch:mainfrom
djagya:fix/telegram-replied-voice-stt
Open

djagya wants to merge 1 commit into
NousResearch:mainfrom
djagya:fix/telegram-replied-voice-stt

Conversation

@djagya

@djagya djagya commented Aug 26, 2026

Copy link
Copy Markdown

What does this PR do?

When a Telegram text message replies to a voice note, _cache_replied_media() stores the media through _attach_cached. Cached audio is mapped via _CACHED_KIND_TO_MESSAGE_TYPE, which always yields MessageType.AUDIO.

The gateway runs automatic STT for VOICE and skips AUDIO. A replied-to native voice note therefore arrives as an audio file instead of entering STT.

This change remaps a cached audio kind back to MessageType.VOICE when the replied Telegram message is a native voice note. Generic replied audio files stay MessageType.AUDIO.

Related Issue

No tracking issue.

Related prior work: #18887 closed after replied-to media caching landed; the remaining gap is this classification. This PR does not overlap #86040 (reply anchoring after transcript echo) or #94007 (keeping an audio reference after successful STT). It acts earlier, when replied Telegram media is classified for the gateway.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • plugins/platforms/telegram/adapter.py — after _attach_cached, if cached.kind == "audio" and reply_msg.voice is set, set MessageType.VOICE.
  • tests/gateway/test_telegram_group_gating.py — replied voice notes enter gateway STT; replied audio files do not.

How to Test

  1. Focused regressions (this rebase, macOS 26 / Python 3.11 via uv):

    uv run --extra dev --extra messaging pytest -q \
      tests/gateway/test_telegram_group_gating.py::test_replied_voice_keeps_voice_semantics_for_gateway_stt \
      tests/gateway/test_telegram_group_gating.py::test_replied_audio_file_remains_non_stt_audio

    Result: 2 passed.

  2. Lint the changed files:

    uv run --extra dev ruff check \
      plugins/platforms/telegram/adapter.py \
      tests/gateway/test_telegram_group_gating.py

    Result: All checks passed!

  3. Manual: reply to a Telegram voice note with text; the gateway should treat the attachment as STT input. Reply to a regular audio file; it should stay a non-STT audio attachment.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass — focused affected suite only; full suite not run
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 26 (arm64), Python 3.11

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — Telegram classification only
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

N/A. Classification is pinned by the two regressions above.

@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins platform/telegram Telegram bot adapter tool/tts Text-to-speech and transcription P3 Low — cosmetic, nice to have labels Aug 26, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference; please use your judgment.

PR #95740 — fix(telegram): preserve replied voice-note semantics for STT

Diff evidence (plugins/platforms/telegram/adapter.py:8-19, tests/gateway/test_telegram_group_gating.py:30-142):

What it does: In adapter._cache_replied_media, replied media with cached.kind=="audio" previously collapsed to MessageType.AUDIO unconditionally. Now distinguishes Telegram voice notes: VOICE if getattr(reply_msg, "voice", None) is not None else AUDIO. The gateway transcribes VOICE via configured Hermes STT but not generic AUDIO, so the old collapse pushed transcription onto the agent instead of the gateway provider.

Reviewed:

  • Correctness: Check reply_msg.voice is not None matches Telegram Bot API semantics where voice notes populate voice field, not audio. getattr(..., None) safe if attribute missing. cached.kind already normalized to "audio" by cache_media_bytes, so voice-note files (often audio/ogg) arrive as kind="audio" — the runtime voice field is the only disambiguator, correctly used here.
  • Boundary: No change to photo/video branches; document-wrapped audio stays AUDIO path (Telegram sends audio files as audio, not voice). reply_msg existence assumed — _cache_replied_media is only called when reply_to_message exists and media cached, so safe.
  • Test: test_replied_voice_keeps_voice_semantics_for_gateway_stt asserts VOICE + media_urls/media_types + _event_media_is_stt_input(event,0) is True; companion test_replied_audio_file_remains_non_stt_audio asserts AUDIO + is False for audio field — directly covers the STT routing regression.

Non-blocking nits:

  • display_name in the appended note still uses cached.display_name ("voice.ogg" vs "recording.mp3") — fine for debugging, not user-visible typing.
  • If a voice note arrives as document (rare client quirk), it would still be AUDIO — acceptable edge; Telegram API normalizes voice notes to voice.

No security impact. Minimal, targeted fix.

Verdict: Approve — LGTM.

@djagya
djagya force-pushed the fix/telegram-replied-voice-stt branch from 34ff934 to 79f61d0 Compare September 8, 2026 22:25
@djagya

djagya commented Sep 8, 2026

Copy link
Copy Markdown
Author

Rebased onto current main (b1f003e1).

_cache_replied_media now uses _attach_cached + _CACHED_KIND_TO_MESSAGE_TYPE. After attach, a cached audio kind that came from reply_msg.voice is remapped to MessageType.VOICE so gateway STT still treats replied voice notes as speech.

Tests: test_replied_voice_keeps_voice_semantics_for_gateway_stt, test_replied_audio_file_remains_non_stt_audio.

@kvnloo

kvnloo commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Exact-head KEEP on #95740 head 79f61d0.

KEEP remapping a cached replied Telegram voice note from MessageType.AUDIO back to VOICE when the replied message was a native voice note, so automatic STT runs (gateway STTs VOICE, skips AUDIO). KEEP adapter change in plugins/platforms/telegram/adapter.py + tests/gateway/test_telegram_group_gating.py.

CHECK: plain audio-file replies stay AUDIO (no forced STT). CHECK: group gating tests still pass. Author djagya not kvnloo. No competing PR.

djagya added a commit to djagya/hermes-agent that referenced this pull request Sep 15, 2026
Keep native replied Telegram voice notes as VOICE so the gateway STT
pipeline sees them; generic replied audio files stay AUDIO.

Rebuilt for PR NousResearch#95740 on 5910de2: the upstream KEEP head (79f61d0)
sits 2522 commits behind current main, so these are independently
justified bytes at the drifted seam, not a cherry-pick.

Co-authored-by: Cursor <cursoragent@cursor.com>
@djagya
djagya force-pushed the fix/telegram-replied-voice-stt branch from 79f61d0 to e3d34a2 Compare September 15, 2026 22:47
@djagya djagya closed this Sep 19, 2026
@djagya
djagya deleted the fix/telegram-replied-voice-stt branch September 19, 2026 02:12
@djagya
djagya restored the fix/telegram-replied-voice-stt branch September 19, 2026 14:22
@djagya djagya reopened this Sep 19, 2026
djagya added a commit to djagya/hermes-agent that referenced this pull request Sep 19, 2026
Keep native replied Telegram voice notes as VOICE so the gateway STT
pipeline sees them; generic replied audio files stay AUDIO.

Rebuilt for PR NousResearch#95740 on 5910de2: the upstream KEEP head (79f61d0)
sits 2522 commits behind current main, so these are independently
justified bytes at the drifted seam, not a cherry-pick.

Co-authored-by: Cursor <cursoragent@cursor.com>
@djagya
djagya force-pushed the fix/telegram-replied-voice-stt branch from e3d34a2 to 04a7a01 Compare September 19, 2026 14:22
Keep native replied Telegram voice notes as VOICE so the gateway STT
pipeline sees them; generic replied audio files stay AUDIO.

Rebuilt for PR NousResearch#95740 on 5910de2: the upstream KEEP head (79f61d0)
sits 2522 commits behind current main, so these are independently
justified bytes at the drifted seam, not a cherry-pick.

Co-authored-by: Cursor <cursoragent@cursor.com>
@djagya
djagya force-pushed the fix/telegram-replied-voice-stt branch from 04a7a01 to dee21b8 Compare September 22, 2026 17:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have platform/telegram Telegram bot adapter tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants