fix(desktop): dedupe auto-spoken replies by content fingerprint - #75649
fix(desktop): dedupe auto-spoken replies by content fingerprint#75649lilShawtty-byte wants to merge 1 commit into
Conversation
Spoken-reply dedupe tracked the last spoken message by its id, but message ids are not stable across the post-turn hydrate: during streaming the bubble carries a runtime id (assistant-stream-<ts>-<seq>), and hydrateFromStoredSession rebuilds it with a persisted id (<timestamp>-<index>-<role>). The id-based check then misses the same reply and reads it aloud a second time. Add speechFingerprint()/latestSpokenReply() to lib/chat-messages.ts and switch the auto-speak path (pendingResponse/consumePendingResponse in use-composer-voice.ts) to fingerprint-based dedupe. The voice-conversation path keeps id semantics: its live speech session binds to a message id for the turn and depends on it staying stable while streaming. Adds unit tests covering the streaming→hydrated id rewrite case.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing the runtime-to-hydrated id transition; current main does have the id-only selector at apps/desktop/src/app/chat/composer/hooks/use-composer-voice.ts:75-109, and hydration replaces the message list at apps/desktop/src/app/contrib/wiring.tsx:321-327.
Problems
apps/desktop/src/lib/chat-messages.ts:211makes trimmed text the entire persistent spoken identity. A later distinct reply with the same content as the preceding reply is therefore returned asnulland never read aloud.useAutoSpeakRepliesis intended to read each completed assistant turn (apps/desktop/src/app/chat/composer/hooks/use-auto-speak-replies.ts:29-34).
Suggested changes
- Rework the hydration dedupe to retain or reconcile a per-turn identity rather than using content alone, and add a regression case for two distinct same-text replies. The current new-reply test uses different text, so it does not exercise this collision.
Automated hermes-sweeper review.
|
|
||
| const fingerprint = text | ||
|
|
||
| if (fingerprint === lastSpokenFingerprint) { |
There was a problem hiding this comment.
This permanently treats equal text as the same reply. A later, distinct turn returning the same text (for example, two separate Done. replies) will be suppressed rather than spoken. Please retain or reconcile a per-turn identity across hydration and add a same-text, distinct-turn regression case.
|
Superseded by #89815. That salvage keeps the stream-id rewrite fix, rebased onto current main, and keys spoken state on the assistant-turn ordinal instead of reply text so two You're credited via Co-authored-by. Thanks for tracing the live-to-durable id swap. |
Supersedes NousResearch#75649, NousResearch#86637, NousResearch#87672, NousResearch#88642. Fixes NousResearch#86601 Fixes NousResearch#87652 Fixes NousResearch#87823 Co-authored-by: Charmmy <lilShawtty@qq.com> Co-authored-by: chelsealong <chelsealong@126.com> Co-authored-by: Olympusbuildz <Olympus.roots@outlook.com> Co-authored-by: Ricardo Mendes <ricardo.mendes@maiolabs.ai>
Supersedes NousResearch#75649, NousResearch#86637, NousResearch#87672, NousResearch#88642. Fixes NousResearch#86601 Fixes NousResearch#87652 Fixes NousResearch#87823 Co-authored-by: Charmmy <lilShawtty@qq.com> Co-authored-by: chelsealong <chelsealong@126.com> Co-authored-by: Olympusbuildz <Olympus.roots@outlook.com> Co-authored-by: Ricardo Mendes <ricardo.mendes@maiolabs.ai>
Problem
Auto-speak (voice.auto_tts) sometimes reads the same reply aloud twice.
Root cause: spoken-reply dedupe tracked the last spoken message by its id, but message ids are NOT stable across the post-turn hydrate:
assistant-stream-<ts>-<seq>)message.complete,completeAssistantMessagesettles that bubble and auto-speak reads it (first playback)hydrateFromStoredSession→toChatMessagesrebuilds the list with persisted ids (<timestamp>-<index>-<role>)pendingReplysees a 'new' id ≠ `lastSpokenIdRef` → plays the same text a second timeBoth existing dedupe layers fail:
lastSpokenIdRef(id rewritten by hydrate) andownsAmbientCue(1s window, the two triggers are a network round-trip apart).Fix
lib/chat-messages.ts: addspeechFingerprint()(trimmed text) andlatestSpokenReply()— an auto-speak selector that dedupes on content, which survives the hydrate id rewrite, and returns the fingerprint as the ambient-cue key.use-composer-voice.ts: auto-speak path (pendingResponse/consumePendingResponse) uses the fingerprint ref. The voice-conversation path keeps id semantics — its live speech session binds to a message id for the turn and depends on it staying stable while streaming.Verification
vitest run src/lib/chat-messages.test.ts→ 57/57 (5 new tests incl. the streaming→hydrated id-rewrite case)vitest run src/app/chat/composer/hooks/use-voice-conversation*.test.tsx→ 11/11 (voice-conversation path unaffected)tsc --noEmit: no new errors (11 pre-existing errors in user-edit-composer.tsx, unrelated @assistant-ui version drift)