Skip to content

fix(desktop): stop replaying a turn when the spoken anchor is rewritten - #110326

Open
Jabar42 wants to merge 1 commit into
NousResearch:mainfrom
Jabar42:fix/desktop-voice-spoken-anchor
Open

Jabar42 wants to merge 1 commit into
NousResearch:mainfrom
Jabar42:fix/desktop-voice-spoken-anchor

Conversation

@Jabar42

@Jabar42 Jabar42 commented Sep 13, 2026

Copy link
Copy Markdown

What happens

With the voice conversation loop on, one turn can be read aloud twice: the final reply first, and then — exactly when that audio drains — the whole turn again (narration interstitials and the reply).

Captured at synthesis time on a real session (one line per sentence handed to TTS):

when sentences what was spoken
15:56:37 → 15:56:51 7 final reply only
15:57:25 → 15:57:39 9 narration + final reply — the same turn, again

The second pass starts the moment the first one ends, which is what makes it audible as a repeat rather than as a duplicate.

Root cause

spoken-reply.ts stores the spoken anchor as {id, ordinal} because the row id is not stable: the live-tail row is rewritten to a durable id on hydration, and assistantReplyOrdinal() exists to survive that rewrite.

collectUnspokenTurnSpeech() still resolved the boundary by id alone:

let spokenIndex = lastSpokenId ? messages.findLastIndex(m => m.id === lastSpokenId) : -1
if (spokenIndex < 0) { /* fallback: everything after the last user message */ }

A reassigned id makes that lookup miss, spokenIndex stays -1, and the fallback re-collects the turn that was just read. use-composer-voice.ts made it easier to hit by passing anchor?.id ?? null — dropping the ordinal the anchor type exists to carry.

Earlier fixes closed neighbouring variants of this class (a2e0ca7fc9 "don't replay earlier turns in voice conversation", 63565fa26b "keep auto-speak silent across the stream-id rewrite"). This is the remaining current turn variant.

Fix

Resolve the anchor by id, then by its ordinal (the durable slot); if neither lands on a visible assistant bubble, there is nothing to say. Replaying a turn is the failure a listener notices; staying silent is not, because the text is already on screen.

Tests

In apps/desktop/src/lib/chat-messages.test.ts:

  • the reassigned-id case from production (narration row rewritten to a durable id → only what follows it is spoken);
  • a dead anchor whose ordinal no longer matches → null instead of the whole turn;
  • the healthy path, unchanged.

Proven red on the base: with the old code the new cases fail with expected 'assistant-stream-n1' to be 'durable-answer-9' — the slice started at the unread narration, i.e. the turn replayed. Green after the fix: 89 tests across chat-messages + spoken-reply, no errors.

Note on authorship

The diff was produced by Hermes Agent (an AI coding agent) at the direction of the reporter, who reviewed the diagnosis, the failing case and the numbers above. Submitted from the account that hit the bug in daily use — happy to adjust style or scope, or to hand it to a maintainer.

Voice playback could read the same turn twice: the final reply first, and then
the whole turn (narration + reply) once the audio drained.

`spoken-reply.ts` stores the anchor as `{id, ordinal}` because the id is not
stable: the live-tail row is rewritten to a durable id on hydration, which is
why `assistantReplyOrdinal` exists. `collectUnspokenTurnSpeech` still resolved
the boundary by id alone, so a reassigned id made the lookup miss and the
"clamp to the current turn" fallback treated an already-read turn as unread.

Resolve by id, then by the anchor's ordinal (the durable slot), and treat an
anchor that cannot be located at all as nothing to say — never as "everything
since the last user message". Replaying a turn is the failure a listener
hears; staying silent is not.
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/desktop Electron desktop app (apps/desktop/*) tool/tts Text-to-speech and transcription labels Sep 13, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Related: #109077 (open) targets the same re-speak-after-id-rewrite class from spoken-reply.ts (absorbSpokenReplyRewrite); this PR fixes the boundary lookup in collectUnspokenTurnSpeech() instead. Worth reviewing together to avoid overlapping fixes.

@kvnloo

kvnloo commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Comment assumes head 96cb7e0b718efae26b9532ad32e566d3adc0c6b2; void if moved.

KEEP — Desktop spoken-anchor collection boundary (voice-loop replay)

  • Resolving collectUnspokenTurnSpeech() by spoken id, then by ordinal slot, and staying silent when neither lands — instead of falling back to “everything after the last user message” — is the right fix for the current-turn double-read (final reply, then narration+reply when audio drains) after hydrate rewrites the live-tail id.
  • Passing the ordinal through use-composer-voice (not anchor?.id ?? null) matches why spoken-reply stores {id, ordinal} in the first place.
  • Prefer this as the collection-boundary spoken-anchor vehicle; peer complementary fix is fix(desktop): don't re-speak replies after hydrate rewrites durable ids #109077 (absorbSpokenReplyRewrite migrate) — neither replaces the other (zero shared files). Earlier neighbours a2e0ca7fc9 / 63565fa26b closed adjacent variants; this is the remaining current-turn slot.

CHECK

One-click close / ordering

Landing-evidence

  • When merged, please reply with the merge commit SHA on main (not “completed”).

@kvnloo

kvnloo commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Comment assumes head 96cb7e0b718efae26b9532ad32e566d3adc0c6b2; void if moved.

KEEP — Desktop spoken-anchor collection boundary (voice-loop replay)

  • Resolving collectUnspokenTurnSpeech() by spoken id, then by ordinal slot, and staying silent when neither lands — instead of falling back to “everything after the last user message” — is the right fix for the current-turn double-read (final reply, then narration+reply when audio drains) after hydrate rewrites the live-tail id.
  • Passing the ordinal through use-composer-voice (not anchor?.id ?? null) matches why spoken-reply stores {id, ordinal} in the first place.
  • Prefer this as the collection-boundary spoken-anchor vehicle; peer complementary fix is fix(desktop): don't re-speak replies after hydrate rewrites durable ids #109077 (absorbSpokenReplyRewrite migrate) — neither replaces the other (zero shared files). Earlier neighbours a2e0ca7fc9 / 63565fa26b closed adjacent variants; this is the remaining current-turn slot.

CHECK

One-click close / ordering

Landing-evidence

  • When merged, please reply with the merge commit SHA on main (not “completed”).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/desktop Electron desktop app (apps/desktop/*) P3 Low — cosmetic, nice to have tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants