Skip to content

feat(voice): wake word enters continuous hands-free conversation - #79696

Open
ruhipiano wants to merge 2 commits into
NousResearch:mainfrom
ruhipiano:feat/wake-continuous-conversation
Open

feat(voice): wake word enters continuous hands-free conversation#79696
ruhipiano wants to merge 2 commits into
NousResearch:mainfrom
ruhipiano:feat/wake-continuous-conversation

Conversation

@ruhipiano

Copy link
Copy Markdown

Summary

When the wake word is detected, the CLI now enters continuous hands-free conversation instead of single-utterance capture.

Before

Wake word → one utterance → reply → listener re-arms immediately. The user had to say the wake word again for every exchange.

After

Wake word → multi-turn conversation:

  • _voice_continuous = True is set on wake, so process_loop keeps auto-restarting recording after each reply
  • A voice stop phrase ("stop" or configured voice.stop_phrases) or 3×silence ends the session
  • The wake-word watchdog (_start_wake_watchdog) keeps the detector paused while continuous mode holds the microphone, and only re-arms once the conversation actually ends

Why the watchdog change matters

Without it, the detector could fire mid-conversation (hearing its own TTS output) and corrupt the session — a busy-check that included _voice_continuous prevents that.

Tests

  • tests/tools/test_wake_word.py: 27 passed (22 existing + 5 new)
    • test_wake_enters_continuous_voice — wake sets _voice_continuous = True
    • test_wake_watchdog_holds_listener_during_continuous — detector stays paused while continuous
    • test_wake_watchdog_resumes_listener_when_idle — re-arms after conversation ends
  • Related voice suites: test_voice_stop_phrase.py 20 passed, test_voice_cli_integration.py 28 passed
  • ruff check cli.py tests/tools/test_wake_word.py — clean

Notes

  • Config already supports voice.stop_phrases; behavior is now consistent between push-to-talk continuous mode and wake-word continuous mode.

When the wake word is detected, enter continuous voice mode instead of single-utterance capture. The user can keep talking after each reply (process_loop auto-restarts recording); a voice stop phrase ("stop" or config voice.stop_phrases) ends the session. The wake-word watchdog keeps the detector paused while continuous mode holds the microphone, and only re-arms once the conversation ends.
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard tool/tts Text-to-speech and transcription labels Aug 5, 2026
@spfcraze

spfcraze commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Summary:
The wake failure path leaves _voice_continuous True, and the busy-check added here reads that as busy, so the wake watchdog holds the listener paused after a failed capture start.

Problems:

  • _on_wake_word sets _voice_continuous = True (cli.py:12974) before the try that calls _voice_start_recording(); the except handler (cli.py:12978-12979) prints "Wake capture failed" and returns, and the flag is still True when the watchdog next polls.
  • _voice_start_recording() can raise: recorder creation fails on no input device or PortAudio init error (cli.py:12193-12197), and it resets _voice_recording to False before re-raising. _on_wake_word's own guard (cli.py:12927) already excluded _agent_running / _voice_recording / _voice_processing at entry, so on this path the added _voice_continuous term is what keeps busy True.
  • The watchdog busy-check now includes or self._voice_continuous (cli.py:12999); while busy it resets idle_polls to 0, and resume_listening runs only after three consecutive non-busy polls (cli.py:13006-13011). With the flag stuck True, the handler's own comment "Leave _wake_suspended set; the watchdog resumes once idle" (cli.py:12979) cannot hold — the listener stays paused until the user toggles wake off/on.

Solution:
Reset _voice_continuous (and _voice_mode) in the except branch of _on_wake_word, mirroring how _voice_start_recording resets _voice_recording on failure, or set _voice_continuous = True only after _voice_start_recording() succeeds.


Checked against 07b3c11 — the tip of feat/wake-continuous-conversation when this was written — and 52a5fc0, main at the same moment.

@ruhipiano

Copy link
Copy Markdown
Author

感谢 triage 的详细分析,问题确认属实并已修复 ✅

问题复现确认_on_wake_word_voice_start_recording() 之前就设置了 _voice_continuous = True(cli.py:12975),如果录音启动失败(无输入设备 / PortAudio init 错误),except 分支直接返回,flag 保持 True → watchdog busy-check(or self._voice_continuous)永远 busy → 监听器卡死直到用户手动 toggle。

修复(commit eeebcb983):在 except 分支回滚 _voice_continuous_voice_mode,镜像 _voice_start_recording 失败时对 _voice_recording 的重置:

except Exception as e:
    _cprint(f"{_DIM}Wake capture failed: {e}{_RST}")
    # Recording never started — roll back the continuous-voice flags so
    # the watchdog doesn't read busy forever...
    with self._voice_lock:
        self._voice_mode = False
    self._voice_continuous = False
    # Leave _wake_suspended set; the watchdog resumes once idle.

测试:新增 test_wake_capture_failure_rolls_back_continuous_flags(验证录音失败后 _voice_mode/_voice_continuous 都回滚为 False),28/28 通过,ruff 干净。

@ruhipiano

Copy link
Copy Markdown
Author

Hi maintainers — quick ping on this one. The AI triage review above confirmed the reported wake-failure bug is real, and I've since fixed it in eeebcb983 (rolls back _voice_continuous / _voice_mode in the except path so the watchdog doesn't stay busy after a failed capture start) with a regression test (test_wake_capture_failure_rolls_back_continuous_flags, 28/28 passing, ruff clean). Could someone take a look / approve CI when you get a chance? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have tool/tts Text-to-speech and transcription type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants