Skip to content

fix(desktop): never replay hydrated session history in auto-speak; lock mute/re-arm semantics - #78194

Open
andrexibiza wants to merge 1 commit into
NousResearch:mainfrom
andrexibiza:vox/09-desktop-voice
Open

andrexibiza wants to merge 1 commit into
NousResearch:mainfrom
andrexibiza:vox/09-desktop-voice

Conversation

@andrexibiza

@andrexibiza andrexibiza commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Related #64390 #73649 #73691 #73880 #74337 #78098 #78118

What changed and why

Desktop voice-conversation reliability residuals (Lane 09 of the Vox Lockin campaign).

1. Auto-speak replays hydrated session history after a session switch (fixes #64390)

With Read Responses Aloud enabled, switching sessions could speak the newly opened session's last reply from history — an old reply the user already heard — because:

  • the sessionId re-arm in useAutoSpeakReplies consumes whatever reply sits in the store at switch time, but
  • session history hydrates asynchronously after the switch, so a completed reply arriving a beat later still looks "unspoken" to the dedupe and gets read aloud.

Fix: gate auto-speak on the session having started a fresh turn. While the gate is armed (after a session switch / toggle-on / mount), completed replies are consumed silently; only a pending assistant reply or a new turn (busy) opens the gate. Ported from #64390's approach (authored by @kannishk) — the open PR conflicts with current main, so the change is rebased onto main's composer-scope wiring (useComposerScope().$messages, ownsAmbientCue) and the repo's eslint conventions.

2. Mute semantics locked by regression tests (#74337 class)

The "can remain muted after reply playback" report shares its root with #73649 (fixed by merged #73880 — re-arm after playback is now covered by the existing use-voice-conversation-rearm.test.tsx). This PR adds the complementary mute-contract regressions to use-voice-conversation.test.tsx:

  • muting while listening cancels the recorder and the mic stays off until unmute;
  • a reply completing while muted is never spoken, and unmute re-arms into the next turn (held reply plays, then the mic re-arms);
  • muting during reply playback never wedges the loop — unmute returns it to listening.

No production code change was needed for the mute path: on current main muted only ever flips via user toggleMute (never spontaneously), and the re-arm class is covered by #73880. The tests lock that contract so the class can't silently regress.

How to test

cd apps/desktop
npx vitest run src/app/chat/composer/hooks/use-auto-speak-replies.test.tsx \
  src/app/chat/composer/hooks/use-voice-conversation.test.tsx \
  src/app/chat/composer/hooks/use-voice-conversation-rearm.test.tsx \
  src/lib/voice-stop-word.test.ts src/lib/speech-text.test.ts \
  src/store/voice-prefs.test.ts src/app/settings/voice-field-visible.test.ts \
  src/app/settings/voice-provider-fields.test.ts

Expected: 8 files, 60 tests, all passing (was 55; +2 auto-speak session-switch regressions, +3 mute-semantics regressions). npx tsc -p tsconfig.json --noEmit, npx eslint on the touched files, and git diff --check are all clean.

Platforms tested

  • Windows 10 (git-bash), vitest UI project, Node 24 — full targeted suite green.
  • No Python/gateway files touched; desktop hooks only.

Why this matters to users

  • Opening a session with history no longer triggers an unexpected spoken replay of an old reply.
  • Voice conversation mute stays predictable: mute is respected and reversible, and the loop returns to listening after playback — no manual stop/start or mute toggling needed.

Fixes #64390

Related: #74337 (class regression-locked; re-arm root fixed by #73880), #73649 (verified fixed on main), #73691 (verified covered — desktop auto-speak is client-side, closed NOT_PLANNED), #78098 (verified fixed by #78118).

Credit: session-switch gating approach salvaged from #64390, authored by @kannishk.

Part of #74337

Part of #78207
Part of #79890

…ck mute/re-arm semantics

Switching sessions with Read Responses Aloud enabled could speak the newly
opened session's last reply from HISTORY: the sessionId re-arm consumed
whatever the store held at switch time, but history hydrates asynchronously,
so a completed reply arriving afterwards looked unspoken and got read aloud.

Gate auto-speak on the session having started a fresh turn: completed replies
that hydrate before any new turn (or pending reply) are consumed silently.
Approach ported from NousResearch#64390 (authored by @kannishk), rebased onto current main
with the composer-scope wiring and repo lint conventions.

Also adds mute-semantics regressions for the NousResearch#74337 class: muting mid-turn
cancels the recorder and holds the reply, unmute returns the loop to
listening after settle, and muting during playback never wedges the loop.

Tests: 60 vitest UI tests pass (8 files) including new
use-auto-speak-replies.test.tsx (2) and mute semantics (3); tsc + eslint
clean; git diff --check clean.

Signed-off-by: andrexibiza <84248988+andrexibiza@users.noreply.github.com>
@andrexibiza

Copy link
Copy Markdown
Contributor Author

The current head f518538b2efee84c4eec55d93cdbcb1104f4ffda closes the auto-speak and mute-semantics update. useAutoSpeakReplies now keeps hydrated history silent after a session switch, mount, or voice toggle until a pending reply or newly started turn opens the gate; this prevents an old completed reply from being treated as new audio. The companion regressions exercise muting while listening, a reply completing during mute, unmute re-arming, and muting during playback, so recorder, held-reply, and playback transitions remain explicit. The covered implementation commit is f518538b2efee84c4eec55d93cdbcb1104f4ffda. Exact-head check rollup: SUCCESS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/desktop Electron desktop app (apps/desktop/*) P3 Low — cosmetic, nice to have tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants