fix(desktop): persist @image: refs instead of vision-enrichment text so attachments survive session switch and restart - #70720
Conversation
|
Note: |
|
Added unit tests covering the attached-image persist fix (#70720):
92 TS tests + 5 Python tests pass. |
57ad2f7 to
2c11e6c
Compare
Root cause: server.py persisted the vision-enriched (model-only) message text containing 'image_url:<path>' instead of the original '@image:<path>' directive the desktop client parses. Adds _build_persist_message_with_image_refs and wires persist_user_message through run_kwargs so the clean, UI-recognizable text is what actually lands in session history. This surfaced two dormant frontend bugs once @image: refs finally reached real history: a duplicated user bubble on session-switch reconciliation (fixed via textWithoutImageRefs normalization, class of NousResearch#67603), and images rendering clamped/undersized inline in text instead of as a proper attachment row (fixed by extracting @image: refs into attachmentRefs in chat-messages.ts and rendering them block-level at full size in directive-text.tsx).
…ge segment filter type
…rfectionist/sort-imports)
Adds unit tests for the @image directive handling that fixes attached-image loss/reorder on session switch and app restart.
2c11e6c to
94379fa
Compare
|
Update on both points raised above: Re: scoping — I removed Re: CI — Fixed. There were two independent causes:
All tests pass locally now ( |
Restores display_kind on the in-memory model-switch entry, display_kind/display_metadata forwarding in _history_to_messages, and the async-delegation metadata + stamping path in _notification_poller_loop / _run_prompt_submit. Keeps the diff scoped to the attached-image persistence fix.
|
Pushed a follow-up commit restoring the remaining timeline wiring that was unintentionally dropped:
Verified with |
|
Superseded by #71121, which builds directly on your commit — thanks for the clean diagnosis, Two things I found while reviewing that the follow-up covers:
Your authorship is preserved in the history of #71121. |
What does this PR do?
Fixes attached images being lost or misordered in the desktop chat after switching sessions or restarting the app.
Root cause:
tui_gateway/server.pypersisted the vision-enriched, model-only message text (containingimage_url:<path>) as the user's turn in session history, instead of the original@image:<path>directive the desktop client actually parses._enrich_with_attached_imagesbuilds that enriched text for the LLM, but it was also being passed straight intoagent.run_conversationas the persisted turn — even thoughrun_conversation/build_turn_contextalready support a separatepersist_user_messageparameter for exactly this "what the model sees" vs "what gets stored" split (previously used for #48677-class issues), it was simply never wired up for the image-attachment path.This PR adds
_build_persist_message_with_image_refs, which builds a clean,@image:<path>-formatted version of the message, and passes it aspersist_user_messageinrun_kwargs. That one change is the actual fix for both original symptoms:Fixing this exposed two dormant frontend bugs that had never been exercised before, because
@image:refs had never actually reached persisted history until now:textWithoutImageRefsnormalization before comparison inpreserveLocalPendingTurnMessages/appendLiveSessionProjection.toChatMessagesnever populatedattachmentRefsfor server-loaded messages, so@image:stayed inline in the clamped text body. Fixed by extracting@image:refs intoattachmentRefs(mirroring how the local composer already represents attachments) inchat-messages.ts, and rendering them as a proper block-level row at full size indirective-text.tsxinstead of a small inline chip.Related Issue
Fixes #70772
Type of Change
Changes Made
tui_gateway/server.py— added_build_persist_message_with_image_refs; wiredpersist_user_messageintorun_kwargsso@image:<path>(not the vision-enrichedimage_url:text) is what gets persisted to session historyapps/desktop/src/lib/embedded-images.ts— addedtextWithoutImageRefsandextractImageRefshelpers for stripping/extracting@image:directive linesapps/desktop/src/app/session/hooks/use-session-actions/utils.ts—preserveLocalPendingTurnMessagesandappendLiveSessionProjectionnow compare text viatextWithoutImageRefsinstead of raw text, preventing duplicated user bubbles once@image:refs are presentapps/desktop/src/lib/chat-messages.ts—toChatMessagesnow extracts@image:refs from persisted user text intoattachmentRefs(with guards so an image-only message, with no caption, is no longer dropped)apps/desktop/src/components/assistant-ui/directive-text.tsx—@image:directives render as a block-level thumbnail row below the text instead of inline in the flow, atmax-h-48 max-w-full(matching embedded images) instead of the oldmax-h-32 max-w-48chip sizeHow to Test
Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — N/A, no config keys changedCONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/A, no architecture/workflow changeScreenshots / Logs
Before

After
