Render images in agent threads by rtfeldman · Pull Request #46167 · zed-industries/zed

rtfeldman · 2026-01-06T16:13:53Z

Now when the agent reads images (which the tool now explicitly advertises to agents that it is capable of; previously the tool said it could only read text even though it can actually read images), we see them in the thread, and also they are auto-expanded by default so you can see them when scrolling through:

This also adds a visual regression test that verifies images render correctly in the agent thread view.

Unlike our previous visual tests, this one only renders the agent panel, not the entire Zed window.

The "screenshot" it generates (rendered to a Metal texture) is from completely mocked/simulated data structures, and looks like this:

Changes

New visual test: agent_thread_with_image renders an AcpThreadView containing a tool call with image content (the Zed app icon)
Test infrastructure: Added StubAgentServer helper and required feature flags for visual testing
Test-support API: Added expand_tool_call() method to AcpThreadView to allow expanding tool calls for visual testing
Baseline screenshot: Included baseline image showing the Zed logo rendered in a tool call output

How to run

# Run the visual tests
cargo run -p zed --bin visual_test_runner --features visual-tests

# Update baselines if UI intentionally changed  
UPDATE_BASELINE=1 cargo run -p zed --bin visual_test_runner --features visual-tests

Release Notes:

N/A

zed-industries-bot · 2026-01-06T16:14:21Z

	Warnings
⚠️	This PR is missing release notes. Please add a "Release Notes" section that describes the change: `Release Notes: - Added/Fixed/Improved ...` If your change is not user-facing, you can use "N/A" for the entry: `Release Notes: - N/A`

Generated by 🚫 dangerJS against 8902bc4

- Use AgentPanel instead of bare AcpThreadView for full UI chrome - Add test-only methods to AgentPanel for injecting stub server - Add StubModelSelector to show model dropdown in tests - Load assets so icons render properly - Dismiss onboarding banner before screenshots - Update docs to clarify baselines are gitignored

- Use include_bytes! for test image instead of reading from disk at runtime - Extract create_test_png_base64 to acp_thread's test_support module for reuse - Make StubModelSelector private (only exposed via Rc<dyn AgentModelSelector>) - Add doc comments to AgentPanel test helpers explaining they're test-only - Add explicit error if tool_call_id is None instead of silently skipping - Add module-level docs for test_support explaining its purpose and gating Amp-Thread-ID: https://ampcode.com/threads/T-019b945e-1dd3-734d-97ea-a28c49446512 Co-authored-by: Amp <amp@ampcode.com>

The visual test already covers this functionality with better end-to-end coverage.

- Add db/test-support to visual-tests feature so KEY_VALUE_STORE uses an in-memory database instead of the real user database - Remove the workaround that was writing to a non-existent key

The visual test works because it uses stubbed data, but in production: 1. The read_file tool description doesn't mention image support 2. Image output is shown as JSON instead of rendered because the tool only sends acp content for text results, not images

- Add agent/test-support, language_model/test-support, fs/test-support to visual-tests feature - Rewrite run_agent_thread_view_test() to actually run the real ReadFileTool - Capture tool events via ToolCallEventStream::test() and use real output - Test now fails if tool code is broken (verified by temporarily commenting out event_stream.update_fields) - Remove unused base64 dependency from visual-tests feature

…l tests - Images in tool calls are no longer auto-expanded (collapsed by default like other tool calls) - Remove collapsed_image_tool_calls field since it's no longer needed - Add two visual tests: agent_thread_with_image_collapsed and agent_thread_with_image_expanded - Both tests verify the real ReadFileTool output is used

Replace multiple if-let statements with one big match on (self, block) tuple. Also make block_string_contents a static method since it doesn't use self.

Turns out we were creating an ImageDecoder on every frame (added in zed-industries#46167) when a tool returned an image as output, because we were trying to get its dimensions. That is now cached on `ContentBlock::Image`. Release Notes: - N/A

Now when the agent reads images (which the tool now explicitly advertises to agents that it is capable of; previously the tool said it could only read text even though it can actually read images), we see them in the thread, and also they are auto-expanded by default so you can see them when scrolling through: <img width="725" height="1019" alt="Screenshot 2026-01-06 at 2 57 11 PM" src="https://github.com/user-attachments/assets/5c908bad-48f2-46c2-afaa-7f189a178e05" /> This also adds a visual regression test that verifies images render correctly in the agent thread view. Unlike our previous visual tests, this one only renders the agent panel, not the entire Zed window. The "screenshot" it generates (rendered to a Metal texture) is from completely mocked/simulated data structures, and looks like this: <img width="546" height="984" alt="Screenshot 2026-01-06 at 2 54 41 PM" src="https://github.com/user-attachments/assets/89a0921f-59e9-4dfe-94b2-4c3b625a851b" /> ## Changes - **New visual test**: `agent_thread_with_image` renders an `AcpThreadView` containing a tool call with image content (the Zed app icon) - **Test infrastructure**: Added `StubAgentServer` helper and required feature flags for visual testing - **Test-support API**: Added `expand_tool_call()` method to `AcpThreadView` to allow expanding tool calls for visual testing - **Baseline screenshot**: Included baseline image showing the Zed logo rendered in a tool call output ## How to run ```bash # Run the visual tests cargo run -p zed --bin visual_test_runner --features visual-tests # Update baselines if UI intentionally changed UPDATE_BASELINE=1 cargo run -p zed --bin visual_test_runner --features visual-tests ``` Release Notes: - N/A --------- Co-authored-by: Zed Zippy <234243425+zed-zippy[bot]@users.noreply.github.com> Co-authored-by: Amp <amp@ampcode.com>

Turns out we were creating an ImageDecoder on every frame (added in zed-industries#46167) when a tool returned an image as output, because we were trying to get its dimensions. That is now cached on `ContentBlock::Image`. Release Notes: - N/A

Now when the agent reads images (which the tool now explicitly advertises to agents that it is capable of; previously the tool said it could only read text even though it can actually read images), we see them in the thread, and also they are auto-expanded by default so you can see them when scrolling through: <img width="725" height="1019" alt="Screenshot 2026-01-06 at 2 57 11 PM" src="https://github.com/user-attachments/assets/5c908bad-48f2-46c2-afaa-7f189a178e05" /> This also adds a visual regression test that verifies images render correctly in the agent thread view. Unlike our previous visual tests, this one only renders the agent panel, not the entire Zed window. The "screenshot" it generates (rendered to a Metal texture) is from completely mocked/simulated data structures, and looks like this: <img width="546" height="984" alt="Screenshot 2026-01-06 at 2 54 41 PM" src="https://github.com/user-attachments/assets/89a0921f-59e9-4dfe-94b2-4c3b625a851b" /> ## Changes - **New visual test**: `agent_thread_with_image` renders an `AcpThreadView` containing a tool call with image content (the Zed app icon) - **Test infrastructure**: Added `StubAgentServer` helper and required feature flags for visual testing - **Test-support API**: Added `expand_tool_call()` method to `AcpThreadView` to allow expanding tool calls for visual testing - **Baseline screenshot**: Included baseline image showing the Zed logo rendered in a tool call output ## How to run ```bash # Run the visual tests cargo run -p zed --bin visual_test_runner --features visual-tests # Update baselines if UI intentionally changed UPDATE_BASELINE=1 cargo run -p zed --bin visual_test_runner --features visual-tests ``` Release Notes: - N/A --------- Co-authored-by: Zed Zippy <234243425+zed-zippy[bot]@users.noreply.github.com> Co-authored-by: Amp <amp@ampcode.com>

Turns out we were creating an ImageDecoder on every frame (added in zed-industries#46167) when a tool returned an image as output, because we were trying to get its dimensions. That is now cached on `ContentBlock::Image`. Release Notes: - N/A

cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Jan 6, 2026

rtfeldman force-pushed the agent-panel-images branch from cd7f0e8 to 02056a5 Compare January 6, 2026 16:16

rtfeldman added 2 commits January 6, 2026 11:38

Add inline image rendering in thread view

7364ea9

Add planning doc for visual tests

b95401c

rtfeldman force-pushed the agent-panel-images branch from 670a561 to b95401c Compare January 6, 2026 16:38

zed-zippy Bot and others added 2 commits January 6, 2026 16:54

Autofix

49f54b6

rtfeldman changed the title ~~Add visual test for image rendering in agent thread view~~ Render images in agent threads Jan 6, 2026

rtfeldman and others added 16 commits January 6, 2026 12:30

Remove unused assistant_text_thread dependency from visual-tests

5609577

Autofix

08980c6

Remove redundant unit test for image in tool call output

92f7ad2

The visual test already covers this functionality with better end-to-end coverage.

Fix outdated comment for embedded test image

9d02eee

Use in-memory test DB for visual tests

ee0506a

- Add db/test-support to visual-tests feature so KEY_VALUE_STORE uses an in-memory database instead of the real user database - Remove the workaround that was writing to a non-existent key

Fix formatting

461845b

Merge origin/main into agent-panel-images

30760eb

Fix formatting

0d0053f

Don't show Raw Input/Output labels for image tool calls

df7ca41

Refactor ContentBlock::append to use single match expression

f5d80bb

Replace multiple if-let statements with one big match on (self, block) tuple. Also make block_string_contents a static method since it doesn't use self.

Merge origin/main and resolve conflicts

f06ff86

Fix dependency issues from merge

8902bc4

rtfeldman marked this pull request as ready for review January 7, 2026 00:48

rtfeldman merged commit b5ac0f9 into main Jan 7, 2026
36 of 38 checks passed

rtfeldman deleted the agent-panel-images branch January 7, 2026 00:48

rtfeldman added the area:ai Related to Agent Panel, Edit Prediction, Copilot, or other AI features label Jan 9, 2026

bennetbo mentioned this pull request May 15, 2026

agent: Do not decode images during render #56866

Merged

Navigation Menu

Uh oh!

Uh oh!

Render images in agent threads - #46167

Render images in agent threads#46167
rtfeldman merged 20 commits into
mainfrom
agent-panel-images

rtfeldman commented Jan 6, 2026 •

edited

Loading

Uh oh!

zed-industries-bot commented Jan 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

rtfeldman commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

How to run

Uh oh!

zed-industries-bot commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rtfeldman commented Jan 6, 2026 •

edited

Loading

zed-industries-bot commented Jan 6, 2026 •

edited

Loading