Skip to content

fix(web): render assistant images inline in chat - #9126

Merged
t3dotgg merged 3 commits into
mainfrom
fix/assistant-inline-images
Sep 3, 2026
Merged

fix(web): render assistant images inline in chat#9126
t3dotgg merged 3 commits into
mainfrom
fix/assistant-inline-images

Conversation

@t3dotgg

@t3dotgg t3dotgg commented Sep 1, 2026

Copy link
Copy Markdown
Member

Problem

When an agent viewed or produced an image, the chat timeline hid it twice. The tool row folded under the "Worked for ..." summary once the turn settled, and even unfolded, the image only appeared after the user clicked the row open. The agent thought it had shown a screenshot; the user saw nothing.

What changed

Rows that preview an image now render on their own. They stay out of tool groups, stay out of the settled-turn fold, and show the image without an expand click. This reuses the existing signed asset route, so no image bytes travel over the websocket.

The change is in apps/web, which desktop wraps. Mobile has a parallel work-log row and still gates its image behind expand; that is a separate fix.

Markdown images with local paths

Absolute paths and file:// URIs already resolve through the signed asset route after #9023, which introduced the media-file resource. The reporter tested a build from before that merge. I added a POSIX regression test for ![x](/tmp/embed-test/2.png) and ![x](file:///tmp/embed-test/5.png) because the existing coverage only exercised Windows path forms.

Still unsupported: data URIs

data: images in Markdown and in HTML <img> stay unsupported on purpose. Large base64 in a message is a performance problem twice over: it inflates every websocket frame that carries the message, and it inflates the DOM node that renders it. Agents should write the file and reference its path instead.

Provider coverage

Claude and the ACP providers (Cursor, Grok) are the only ones that put real image data on the wire. T3 Code does not carry those bytes through. It carries the file path and re-reads the file through the signed asset route, which is why this fix is a rendering change and not a contract change. Codex view_image and OpenCode only emit a label, no pixels, so path-based rendering is all they could ever support.

Tests

  • apps/web/src/components/chat/MessagesTimeline.logic.test.ts covers the row placement: the command row folds, the image row stays visible and does not collapse into a tool group.
  • apps/web/src/components/ChatMarkdown.workspace-images.test.tsx covers POSIX absolute and file:// sources resolving to a signed media-file asset.

vp test run on the touched files plus the neighboring work-log and markdown-image suites: 147 passed. Web typecheck and lint clean.

No before/after images. This was not run in a browser.

Refs #9094

Built by Claude Opus 5 in the Claude Code harness.


Note

Low Risk
UI-only timeline and markdown rendering changes with broad test coverage; no auth, API, or data contract changes.

Overview
Image-preview work rows (agent viewed/produced screenshots) now stay visible in the chat timeline instead of folding into “Worked for …” or hiding behind tool groups and expand clicks.

Timeline derivation adds workEntryRendersImagePreview (via workEntryViewedImagePath) and threads it through group visibility, settled-turn folds, live activity grouping, and standalone row emission so image rows behave like first-class answers. MessagesTimeline.tsx renders ChatMarkdownAssetImage below the row header without requiring expand; tool output text remains behind expand.

Adds regression tests for POSIX/file:// markdown images resolving to signed media-file assets, plus timeline tests that image rows survive folding and in-progress turns.

Reviewed by Cursor Bugbot for commit 54b2d49. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Render assistant images inline in chat timeline work rows

  • Adds workEntryRendersImagePreview predicate in MessagesTimeline.logic.ts to classify work entries with a viewed image path as image-preview rows.
  • Updates workEntryIsVisibleInGroup, deriveTurnFolds, and deriveMessagesTimelineRows so image-preview entries pass the group-visibility filter, are excluded from settled-turn folds, and are emitted as standalone rows rather than absorbed into active or adjacent tool groups.
  • Moves the viewed-image rendering in MessagesTimeline.tsx out of the expanded-detail container so it displays without expanding the row; expanding still controls the separate tool body.
  • Adds tests covering POSIX/file-URI asset loading in ChatMarkdown.workspace-images.test.tsx and image-preview visibility in MessagesTimeline.logic.test.ts.
  • Behavioral Change: work entries previously folded or grouped when neutral will now appear as visible image rows when they have a viewed image path.

Macroscope summarized 54b2d49.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Sep 1, 2026
Comment thread apps/web/src/components/chat/MessagesTimeline.logic.ts
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.6 KiB 13.7 KiB +134 B (+1.0%) 15.1 KiB
Codex Thread snapshot wire 7.0 KiB 7.0 KiB +6 B (+0.1%) 7.3 KiB
Codex Live turn WebSocket wire 6.6 KiB 6.7 KiB +128 B (+1.9%) 7.8 KiB
Codex Live turn WebSocket decoded 57.1 KiB 58.5 KiB +1.4 KiB (+2.5%) 66.4 KiB
Codex Live turn messages 10 10 0 (0.0%) 21
Claude Total thread wire 13.5 KiB 13.6 KiB +55 B (+0.4%) 15.1 KiB
Claude Thread snapshot wire 7.0 KiB 7.0 KiB −6 B (−0.1%) 7.3 KiB
Claude Live turn WebSocket wire 6.5 KiB 6.6 KiB +61 B (+0.9%) 7.8 KiB
Claude Live turn WebSocket decoded 57.8 KiB 57.9 KiB +88 B (+0.1%) 66.4 KiB
Claude Live turn messages 8 10 +2 (+25.0%) 21

Baseline: 4b26132 · PR result: 54b2d49 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.8 KiB
  • Claude decoded thread snapshot: 114.5 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@macroscopeapp

macroscopeapp Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 1a1d3b1

Macroscope's review found this PR approvable — This is a contained web chat rendering fix that keeps existing image-preview entries visible and renders them through the existing signed asset loader. Production changes are limited to timeline grouping and row presentation, with regression tests added and no schema, security, billing, default, or deployment changes.

No code changes detected at 54b2d49. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

@t3dotgg
t3dotgg force-pushed the fix/assistant-inline-images branch from bcfe6ea to 2ef0c74 Compare September 2, 2026 00:04

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 2ef0c74. Configure here.

Comment thread apps/web/src/components/chat/MessagesTimeline.logic.ts
@t3dotgg
t3dotgg force-pushed the fix/assistant-inline-images branch from 1a1d3b1 to 0cdfff6 Compare September 3, 2026 09:25
t3dotgg and others added 3 commits September 3, 2026 02:26
An image the agent viewed or produced was hidden twice in the timeline.
The row folded under "Worked for ..." when the turn settled, and the image
itself only rendered after the user expanded the row.

Image preview rows now render on their own, outside tool groups and outside
the settled-turn fold, and the image shows without an expand click.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
While the turn was still working, a trailing image view was collected
into the live tool row, which renders only a label. The image appeared
only after the turn settled. Image preview rows now stop the active
collection the same way agent spawns and errors do.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Standalone image rows sit outside any expanded group, so the neutral
filter hid them while the tool was still in progress and left an empty
gap. Image preview rows now bypass that filter.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@t3dotgg
t3dotgg force-pushed the fix/assistant-inline-images branch from 0cdfff6 to 54b2d49 Compare September 3, 2026 09:27
@t3dotgg
t3dotgg merged commit 652515a into main Sep 3, 2026
27 checks passed
@t3dotgg
t3dotgg deleted the fix/assistant-inline-images branch September 3, 2026 09:33
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Sep 3, 2026
## What's Changed
* fix(web): keep trailing tool groups out of "Worked for" accordion by @Yash-Singh1 in pingdotgg/t3code#9384
* feat(marketing): put named-developer quotes on the landing page by @t3dotgg in pingdotgg/t3code#9385
* fix(claude): expand slash commands when a message has attachments by @t3dotgg in pingdotgg/t3code#9122
* fix(web): stop the resting composer layout loop by @t3dotgg in pingdotgg/t3code#9393
* fix(web): render assistant images inline in chat by @t3dotgg in pingdotgg/t3code#9126
* feat(providers): add Google Antigravity via the official ACP agent by @t3dotgg in pingdotgg/t3code#9348


**Full Changelog**: pingdotgg/t3code@v0.0.39-nightly.20260903.1267...v0.0.39-nightly.20260903.1268

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.39-nightly.20260903.1268
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant