Skip to content

fix(cli): route voice reply behavior by input origin - #11744

Closed
KeroZelvin wants to merge 1 commit into
NousResearch:mainfrom
KeroZelvin:fix/voice-reply-mode-routing-v2
Closed

fix(cli): route voice reply behavior by input origin#11744
KeroZelvin wants to merge 1 commit into
NousResearch:mainfrom
KeroZelvin:fix/voice-reply-mode-routing-v2

Conversation

@KeroZelvin

@KeroZelvin KeroZelvin commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

Bug Description

The classic CLI voice path did not distinguish typed turns from STT-originated turns while voice mode was enabled. Typed prompts could therefore inherit the concise voice-only prompt and spoken-output behavior, and the old routing attempt lacked a complete input-origin path.

Related Issue

Fixes #65827

Current-Main Validation

The bug remains present on current upstream main (7b5ba2054721dde998ed47fd4a0f031955278e99):

  • _voice_stop_and_transcribe() queues STT output as an unmarked string.
  • The CLI process loop understands only strings and legacy (text, images) tuples.
  • chat() gates its voice prefix and both streaming/batch TTS only on global voice/TTS state.

Newer gateway message-type routing and provider-level streaming TTS do not carry origin metadata into the classic CLI, so they do not supersede this fix.

Fix

  • Add a typed _CLIQueuedInput payload and _CLIInputOrigin enum.
  • Normalize structured turns and legacy string/(text, images) queue payloads at one process-loop boundary.
  • Mark successful STT transcripts as voice-originated.
  • Thread the normalized origin into chat().
  • Centralize spoken-output policy so both streaming and batch TTS obey voice.message_reply_mode.
  • Apply the concise voice prompt only to actual voice-origin turns while preserving the clean transcript through persist_user_message.
  • Preserve current-main _voice_speak_response_async() completion signaling, beep configuration, transcription-failure recording retention, terminal-input recovery, and legacy image queue handling.

Teknium1 Feedback Addressed

  • Updated tests/tools/test_clipboard.py for the structured STT queue payload.
  • Added production-path assertions proving voice_only microphone turns are spoken and typed turns remain unprefixed, do not initialize streaming TTS, and do not invoke batch TTS.
  • Added queue-normalization and reply-policy coverage.
  • Documented voice.message_reply_mode, accepted values (all, voice_only), default behavior, CLI scope, and TTS-disabled behavior in both CLI configuration and voice-mode documentation.
  • Removed the unrelated dead _clear_current_input helper from the old patch.

Test Plan

  • Current-main negative control confirms the new routing tests fail before the patch.
  • 250 passed across tests/cli/test_cli_init.py, tests/tools/test_voice_cli_integration.py, and tests/tools/test_clipboard.py.
  • 117 passed across tests/tools/test_voice_mode.py and tests/hermes_cli/test_voice_wrapper.py.
  • Ruff passed on all changed Python files.
  • Python compile checks passed.
  • git diff --check passed.
  • Rebased onto current upstream main; conflicts resolved semantically.

A repository-wide test run was also attempted. The full suite has unrelated baseline/environment failures and did not complete within ten minutes; no failure implicated these changed paths.

Risk Assessment

Low-to-moderate and localized to classic CLI queue normalization and voice-output policy. Legacy string and (text, images) payloads remain supported, the default message_reply_mode: all preserves existing spoken-output behavior, and newer async TTS completion handling remains intact.

@KeroZelvin
KeroZelvin marked this pull request as ready for review April 17, 2026 19:13
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard tool/tts Text-to-speech and transcription labels Apr 24, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused voice-routing fix. The current remote main still queues STT as an unmarked string (cli.py:11293) and applies the voice prefix/TTS based solely on mode state (cli.py:12291, cli.py:12715), so the core premise is valid.

Problems

  • The changed enqueue payload will break the unmodified clipboard regression: tests/tools/test_clipboard.py:1082 still expects "hello", while this PR changes the payload to (transcript, [], "voice").
  • voice.message_reply_mode is user-facing config, but the CLI voice config reference at website/docs/user-guide/configuration.md:1574-1581 is not updated.

Suggested changes

  • Update the clipboard test and add an end-to-end typed-turn assertion for voice_only routing.
  • Document message_reply_mode and its accepted values.

This is an automated hermes-sweeper review.

Comment thread cli.py Outdated
if hasattr(self, '_app') and self._app:
self._app.invalidate()
self._pending_input.put(transcript)
self._pending_input.put((transcript, [], "voice"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also update tests/tools/test_clipboard.py:1082: it exercises this same successful-transcription path and still asserts the old raw-string queue payload, so it will fail after this three-tuple change.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 12, 2026
@KeroZelvin
KeroZelvin force-pushed the fix/voice-reply-mode-routing-v2 branch from b9005f8 to 0694614 Compare July 12, 2026 18:36
@KeroZelvin

Copy link
Copy Markdown
Contributor Author

Rebased and refactored onto current main (7b5ba2054721dde998ed47fd4a0f031955278e99), with the PR head now at 0694614ab73b69dbc0c1dac77e658c9fd305cbee.

Teknium1 feedback is addressed:

  • updated the clipboard/STT regression for the structured queue payload;
  • added production-path typed and voice turn assertions for voice_only, including streaming-TTS setup and batch-TTS suppression;
  • documented voice.message_reply_mode and its accepted/default behavior in both CLI configuration and the voice-mode guide.

The old tuple extension was refactored into a typed queued-turn object plus one normalization boundary. The conflict resolution preserves current-main _voice_speak_response_async() completion signaling, terminal-input recovery, beep configuration, transcription-failure recording retention, and legacy (text, images) queue behavior.

Verification:

  • focused CLI/voice/clipboard tests: 250 passed;
  • supporting voice-mode/wrapper tests: 117 passed;
  • Ruff, compile checks, and git diff --check: passed;
  • current-main negative control: fails before the patch because input origin is still absent.

The full repository suite was attempted but has unrelated baseline/environment failures and timed out; no failure implicated this PR's changed paths.

@brian717

Copy link
Copy Markdown
Contributor

@KeroZelvin this fixes #65827. When you get a chance can you link that issue to avoid others picking it up and attempting the same fix?

@KeroZelvin

Copy link
Copy Markdown
Contributor Author

Thanks for catching that — I’ve linked #65827 in the PR body with Fixes #65827, so GitHub now tracks this PR as the closing fix.

@teknium1 teknium1 closed this in 0062107 Jul 28, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
… instruction (NousResearch#65827)

Typed messages sent while voice mode was active were also getting the
'[Voice input — respond concisely...]' API-local prefix, because the gate
checked only self._voice_mode. Route STT transcripts through a
_VoiceInputMessage sentinel in _pending_input (both the PTT/continuous
transcription path and the barge-in utterance path), unwrap it in
process_loop, and thread voice_input= through chat() so the prefix applies
only to genuinely voice-transcribed messages.

Re-cut of PR NousResearch#65961 (@webtecnica) — the original diff had the sentinel
class embedded inside __init__'s docstring. Credit also to the earliest
route-by-origin attempt in PR NousResearch#11744 (@KeroZelvin).

Fixes NousResearch#65827
Closes NousResearch#65961
Closes NousResearch#11744
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CLI voice mode marks typed messages as voice input

4 participants