Skip to content

fix(cli): only mark speech-to-text messages as voice input - #65925

Closed
brian717 wants to merge 1 commit into
NousResearch:mainfrom
brian717:fix/voice-typed-input-provenance-65827
Closed

fix(cli): only mark speech-to-text messages as voice input#65925
brian717 wants to merge 1 commit into
NousResearch:mainfrom
brian717:fix/voice-typed-input-provenance-65827

Conversation

@brian717

Copy link
Copy Markdown
Contributor

What does this PR do?

Voice mode stays active while you type, but the [Voice input ...] prefix in chat() was selected from self._voice_mode alone. Every message sent while voice mode was on therefore reached the model labelled as a speech-to-text transcript, so the model had no way to tell a real transcript from typed text. In the reported case it took a typed troubleshooting message as evidence that transcription had started working, when it hadn't.

A transcript is only produced in one place (_voice_stop_and_transcribe), so that's where I tag it. VoiceTranscript is a str subclass, which is what keeps the change small: it rides the existing _pending_input queue and satisfies every isinstance(..., str) check unchanged, so none of the other producers or consumers need to learn a new payload shape. process_loop reads the tag off the raw queue item and passes it to chat(voice_input=...). That read has to happen where it does — the sanitizers immediately after it (paste-wrapper stripping, file-drop rewrites) return plain str and the subclass doesn't survive them.

One thing worth a second opinion: that prefix was doing two jobs at once, provenance and conciseness. The reply is still spoken aloud for as long as voice mode is on, so I kept the brevity instruction for typed messages and only changed the label to Typed input (not transcribed). The issue suggested either leaving typed messages unmarked or giving each message explicit provenance; I went with explicit provenance, since dropping the prefix entirely would make spoken replies to typed messages long again. The prefix for a real transcript is byte-for-byte what it was before, and chat() defaults to voice_input=False so no caller can claim voice input by accident.

Related Issue

Fixes #65827

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • cli.py: add a VoiceTranscript(str) marker for messages that came from speech-to-text.
  • cli.py (_voice_stop_and_transcribe): tag the transcript when queueing it.
  • cli.py (process_loop): read the tag off the queue item before the sanitizers flatten it, and pass it through to chat().
  • cli.py (chat): new voice_input parameter, defaulting to False.
  • cli.py (_build_voice_prefix): pulled the prefix construction out of chat() into its own method so it can be tested directly, and picked the provenance label from voice_input rather than _voice_mode.
  • tests/tools/test_voice_cli_integration.py: new TestVoiceInputProvenance covering both paths.

How to Test

No microphone needed — the STT call is mocked, so the reproduction is deterministic:

pytest tests/tools/test_voice_cli_integration.py::TestVoiceInputProvenance -q

To watch it catch the original bug, change the provenance line in _build_voice_prefix back to the old behaviour:

provenance = "Voice input"

test_typed_message_in_voice_mode_is_not_marked_voice_input and test_typed_and_voice_prefixes_are_distinguishable then fail with the reported symptom, a typed message arriving as [Voice input — respond concisely ...].

Manually, against a real session: start hermes, turn on voice mode, record a message, then type a follow-up without leaving voice mode. The recorded one reaches the agent as [Voice input — ...], the typed one as [Typed input (not transcribed) — ...]. Session history is unaffected either way; the prefix stays API-call-local as before.

The existing TestVoiceMessagePrefix tests re-implement the prefix logic inline rather than calling into cli.py, so they pass either way — the new tests exercise the real methods.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Windows 11
  • I've run pytest tests/ -q and all tests pass — ran the affected suites rather than the whole tree: tests/cli/, tests/tools/test_voice_cli_integration.py, tests/tools/test_clipboard.py and the voice/prefix/persist tests in tests/run_agent/. tests/cli/ has 13 failures on Windows (symlink privileges, and source-reading tests that hit cp1252), but the set is identical with and without this change, and they reproduce on a clean main.

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — docstrings only; no user-facing docs describe this prefix
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A, no config keys
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — pure Python, no platform-specific paths; reported on macOS, fixed and tested on Windows
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Voice mode stays active while the user types, but the "[Voice input ...]"
prefix was selected from `self._voice_mode` alone. Every message sent while
voice mode was on therefore reached the model labelled as a transcript, so
the model could not tell speech-to-text output from typed text -- and read a
typed troubleshooting message as evidence that transcription had started
working when it had not.

Tag the transcript at the one place it is produced (`_voice_stop_and_transcribe`)
with a `VoiceTranscript` str subclass, so it rides the existing `_pending_input`
queue and every `isinstance(..., str)` check unchanged. `process_loop` reads the
tag off the raw queue item -- before the input sanitizers rebuild it as a plain
str -- and passes it to `chat(voice_input=...)`.

Provenance and conciseness are now decided separately: the reply is spoken
aloud for as long as voice mode is on, so typed messages keep the brevity
guidance but are labelled "Typed input (not transcribed)" instead of claiming
to be voice. The prefix for a real transcript is unchanged, and `chat` defaults
to `voice_input=False` so no caller can claim voice input by accident.

Fixes NousResearch#65827
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard tool/tts Text-to-speech and transcription labels Jul 16, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to #11744 and #65827: this implements the same typed-versus-STT provenance goal with a lightweight str-subclass marker, while #11744 uses structured queue payloads and also adds reply-mode policy.

@brian717
brian717 marked this pull request as ready for review July 16, 2026 20:55
@brian717

Copy link
Copy Markdown
Contributor Author

Looks like a dupe of #11744. I'll close this one out.

@brian717 brian717 closed this Jul 16, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Comment

Looks Good

  • Adds VoiceTranscript(str) marker class to distinguish speech-to-text messages from typed input
  • _build_voice_prefix only applies to actual STT transcripts (not typed text in voice mode), fixing the STT failure masking bug (#65827)
  • Clean design: subclasses str so it passes isinstance(..., str) checks unchanged
  • No security or performance concerns

Reviewed by Hermes Agent

@brian717
brian717 deleted the fix/voice-typed-input-provenance-65827 branch July 17, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CLI voice mode marks typed messages as voice input

3 participants