fix(desktop): durably retain streamed assistant responses

[JoaoMarcos44]

Summary

Fixes #95514

An empty terminal completion must not erase assistant text that was already streamed to the Desktop, and a streamed answer must remain durable when the turn settles after a tool result.

This PR closes both sides of that failure class without adding a second journal, changing prompt/tool caching, or copying the implementation from the related open PRs.

Confirmed root cause

The issue report is a real data-loss report, not only a rendering glitch: it records an empty assistant row in state.db after a tool result. The exact remote Desktop build and logs were not supplied, so the original Windows/secondbrain run cannot be reproduced against its authenticated environment.

The current code nevertheless contains a deterministic reproduction of the same loss:

1. message.delta adds text to the Desktop's live assistant parts.
2. message.complete can carry empty terminal text.
3. mergeFinalAssistantText(parts, '') treated that empty value as an authoritative replacement and removed the streamed text.
4. completeAssistantMessage then allowed session hydration, so a stale/empty persisted row could overwrite the only visible copy.
5. On the agent side, a finalizer can receive an empty final_response while the stream buffer contains text. If the tail was already flushed incrementally, the old fill path had no reliable live _row_id and could append a duplicate while leaving the empty row in SQLite.

The invariant fixed here is:

Once assistant text has been delivered, an empty terminal completion is non-destructive, and finalization leaves exactly one canonical assistant response in the durable transcript.

Shift+Tab is a timing trigger rather than a direct Desktop transcript command: plain Shift+Tab is not bound as a Hermes action; Ctrl+Shift+Tab is the session-cycle shortcut. Switching sessions can expose the empty-completion/hydration race, but it must not be able to remove durable assistant content.

Fix

• Make mergeFinalAssistantText a no-op for empty or whitespace-only final text, preserving streamed text, reasoning, and tool timeline parts.
• Add the shared hasVisibleAssistantText predicate and scope the hydration guard to the current streamId (or the current interim boundary), so a previous turn's text cannot suppress legitimate hydration for a reasoning-only turn.
• Recover non-empty _current_streamed_assistant_text in finalize_turn when the terminal response is empty, before completion/result calculation and persistence.
• Generalize finalizer tail repair to empty assistant tails, not only assistant rows with tool_calls.
• Add SessionDB.ensure_assistant_message_content, a parameterized BEGIN IMMEDIATE transaction that fills only an active blank assistant row and returns the durable canonical content. A concurrent winner is adopted into the live message and returned result instead of creating another row.
• Propagate _row_id from the temporary batch row back to the live message dict so finalization can update an incrementally persisted row in place.

No new state file, model tool, environment variable, toolset, or cache-invalidating behavior is introduced.

Causal flow

flowchart LR
  D[message.delta\nstreamed text] --> L[Desktop live parts]
  E[message.complete\nempty text] --> M[non-destructive merge]
  L --> M
  M --> H{local current text?}
  H -- yes --> N[skip destructive hydrate]
  H -- no --> R[normal hydrate]
  S[finalizer\nempty response] --> B[stream buffer recovery]
  B --> Q{persisted empty row id?}
  Q -- yes --> U[conditional in-place DB update]
  Q -- no --> A[normal append]
  U --> V[one canonical assistant row]
  A --> V

Loading

Non-duplicate comparison

The following related work was inspected before implementation:

• Fix desktop assistant text disappearing after empty completion event #62504 and fix(desktop): preserve streamed text when provider sends empty completion #41922: Desktop-only empty-completion preservation. They patch the hook locally and cannot repair an already-empty backend row or prevent append-only finalizer duplication.
• fix(desktop): preserve multi-round streamed text in message.complete #56104: multi-round streamed-text UI preservation; it does not own the durable empty-row boundary.
• fix(desktop): treat interrupted completions as metadata #63292: interrupted-completion metadata/sequence handling; it does not establish the invariant that delivered stream text becomes the canonical persisted assistant content.
• fix(desktop): preserve session history after interrupted API calls #78058: stale-history protection after interrupted API calls; it addresses snapshot desynchronization, not empty final-row materialization.
• fix(desktop): prevent cache eviction of optimistic user message during session switch #92730: optimistic user-message cache eviction during session switching; it concerns the user prompt, not the assistant final row.
• Recover undelivered tool-call content #53544 and fix(cli): re-emit assistant text after tool output prevents scroll-away #68576: undelivered tool-call content and CLI re-emission; different delivery paths.

This PR owns the missing cross-layer enforcement point: stream buffer → finalizer → existing SQLite row identity, plus the shared Desktop merge contract. It does not copy those implementations.

Test plan

RED evidence

The new regressions were run before their corresponding production changes and failed for the intended reasons:

• Empty Desktop completion removed the previously rendered text.
• Empty completion still invoked session hydration over the local response.
• Finalizer recovery returned an empty response when the stream buffer was non-empty.
• The real incremental-flush reproduction left three assistant rows (including the empty row) instead of one final response.
• The scoped hydration regression initially failed when a previous turn's text was mistakenly considered current.
• The concurrency regression initially exposed a mismatch between the live response and a concurrent durable winner.

Focused GREEN results

• scripts/run_tests.sh — 9 affected/adjacent Python files, 59 tests passed, 0 failed.
• npm exec --workspace apps/desktop -- vitest run --project ui src/lib/chat-messages.test.ts src/app/session/hooks/use-message-stream/interim-sealing.test.tsx src/app/session/hooks/use-message-stream/session-info-side-effects.test.tsx src/app/session/hooks/use-message-stream/delta-flush.test.tsx — 113 tests passed, 0 failed.
• Real SessionDB reopen test verifies the persisted final content and absence of a duplicate empty assistant tail.
• uvx ruff==0.15.10 check on changed Python files — passed; an existing invalid # noqa warning at run_agent.py:108 remains.
• Prettier on changed Desktop files — passed.
• ESLint on changed Desktop files — 0 errors; the full project lint also reported only existing warnings.

Known validation limitations

• The full Desktop UI run completed with 5,981/5,999 tests passing and 18 unrelated failures/timeouts in billing, provider settings, messaging, locale formatting, canvas support, and session-unread infrastructure. The affected stream/message files passed their focused suite.
• The final rebased npm run --workspace apps/desktop typecheck is blocked by an existing origin/main error at apps/desktop/src/app/contrib/hooks/use-background-sync.test.ts:660 (updateSessionState is missing from that test fixture); that file is not changed here. Typecheck passed before the rebase when the branch was based on 9aa7530....
• The full npm run check exceeded the local 420-second execution ceiling and has no final result; it is not reported as green.
• No authenticated reproduction against the reporter's secondbrain profile or production state.db was performed. The issue did not provide a Desktop build, logs, or a safe test account, so production data was neither accessed nor mocked.

Hardening record

GitHub labels: P1, type/bug, comp/desktop, platform/windows, area/sessions, sweeper:risk-session-state, sweeper:risk-platform-windows.

Three required P1 hardening passes were completed:

1. Premise/root-cause pass: traced Desktop events, finalizer persistence, SQLite append behavior, and searched open/merged related PRs.
2. Failure-mode pass: added real SQLite reopen coverage, tested empty/whitespace/failure/previous-turn/tool-tail cases, and verified row counts/counter-preserving updates.
3. Concurrency/simplification pass: scoped the UI guard to the current turn, added canonical-winner coverage, removed the redundant pure-tool helper, and reran focused tests, typecheck/lint/format checks, and independent review.

Risk: Medium. The change is limited to empty-completion reconciliation and final assistant-row settlement. It uses the existing persistence lock/transaction machinery, preserves tool-call rows and session counters, and falls back to the existing append path when no row identity is available.

[andrexibiza]

Topology reconciliation for #95514: this carrier is superseded by #95886 rather than a second landing owner.

I re-read both live objects again. #95822 still carries the older split persistence shape (ensure_assistant_message_content before the ordinary append path). #95886 is now at exact head 970cd83146677b7c0d3b2a2798e7650537b2de9c, where the previously open 2K/ownership blocker is also closed: agent/transcript_repair.py owns the repair policy while SessionDB.append_messages_batch() remains the single guarded transaction owner. The rollback, active watermark-clone resolution, concurrent canonical-winner adoption, and post-commit live-state synchronization regressions remain in that object.

Exact-head hosted receipts for #95886 are still green: CI 33085107817, Docker 33085106690, and Nix 33085106669 all succeeded.

The landing edge has moved since that review, so I am not transferring those green receipts to current main. Live main@35328345d5e3b5badc47271bdb8828e1fd2d25f4 is 192 commits beyond #95886's recorded base, and that interval now overlaps two of #95886's submitted production paths: hermes_state.py and run_agent.py. #95886 therefore remains the canonical implementation owner for #95514, but it needs a semantic rebase/reconciliation of those two paths and fresh exact-head execution before landing.

Disposition: #95822 remains superseded and should stay out of the landing order; preserve its unique tests/history as provenance. #95886 owns delivery, subject only to current-main reconciliation plus fresh all-green exact-object proof.