Skip to content

Paste images into the composer - #38

Closed
guilimasp wants to merge 1 commit into
milind-soni:mainfrom
guilimasp:feat/paste-images
Closed

Paste images into the composer#38
guilimasp wants to merge 1 commit into
milind-soni:mainfrom
guilimasp:feat/paste-images

Conversation

@guilimasp

Copy link
Copy Markdown
Contributor

The composer only takes text, so anything visual — a screenshot of a failing build, a chart, a photo of a whiteboard — has to be described in words, or saved to a file and referenced by path, which only works when the bot happens to have filesystem tools.

Pasting an image now stages it as a thumbnail above the input (hover for an ✕ to drop it), sends with or without accompanying text, and renders in the user's own bubble. The Claude driver turns the attachments into Messages-API image blocks on the stream-json prompt; a turn with no images sends the same plain string it does today, and a driver that cannot carry images ignores them and sends the text alone.

Sizing happens before anything leaves the browser: 1568px on the long edge (past that the providers downscale anyway), re-encoded as PNG, or JPEG when the PNG would be large — GIFs pass through untouched so animation survives. server/attachments.ts re-validates independently of the client (known image types, at most 6 per message, 5MB of base64 each) and drops what fails rather than throwing away the user's message.

Composer, before and after a paste:

composer before

composer after

The turn itself — pasted chart, and the bot reading it:

conversation

New behavior is covered by server/attachments.test.ts (wire validation and the block shape) and server/drivers/claude-images.test.ts (the driver contract against the scripted fake CLI, skipIf(win32) like its neighbours). pnpm typecheck && pnpm test pass, and there are no new dependencies.

Two things deliberately left out, happy to add either if you'd rather they ship together: the + button is still inert — this is only about paste — and only the Claude driver forwards images today.

The composer only ever accepted text, so anything visual — a screenshot of
a failing build, a chart, a photo of a whiteboard — had to be described in
words or saved to disk and referenced by path, which only works when the
bot happens to have filesystem tools.

Pasting an image now stages it as a thumbnail above the input (with an ✕ to
drop it), sends with or without accompanying text, and renders in the user's
own bubble. The Claude driver turns the attachments into Messages-API image
blocks on the stream-json prompt; a turn with no images keeps sending a
plain string, and drivers that cannot carry images ignore them and send the
text alone.

Images are downscaled client-side to 1568px on the long edge (the point past
which the providers resize anyway) and re-encoded as PNG, or JPEG when the
PNG would be large — GIFs pass through untouched so animation survives. The
server re-validates independently of the client: known image types only, at
most 6 per message, 5MB of base64 each, anything else dropped rather than
throwing away the user's message.

No new dependencies.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkHmFg3BcN3RfPoSjTc3W3
@guilimasp

Copy link
Copy Markdown
Contributor Author

Closing this one — it went conflicting when the chat overhaul landed, and rebasing it means rewriting the image wiring across 12 files against code that has moved a long way. Not worth carrying it red in your queue while it waits.

For files that live on disk, #69 covers the same ground from a different angle: dropping a file on the window attaches it by path, which every driver can open without an attachment protocol. What that does not cover is an image straight from the clipboard, which is what this PR was for.

Happy to redo the clipboard-image path against current main if you want it — say the word and I'll open a fresh one.

@guilimasp guilimasp closed this Aug 13, 2026
guilimasp added a commit to guilimasp/OpenMausBot that referenced this pull request Aug 13, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 15, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
guilimasp added a commit to guilimasp/OpenMausBot that referenced this pull request Aug 18, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 19, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 20, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 21, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 22, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 23, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 24, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 25, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 26, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 27, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 28, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
github-actions Bot pushed a commit to guilimasp/OpenMausBot that referenced this pull request Aug 29, 2026
050 stopped applying when the chat overhaul moved the fold, so every turn
has been settling an idle desktop into the transcript again. Refreshed
from the rewritten upstream PR (milind-soni#37), which also clears the mark on every
turn.completed instead of only when a frame was captured.

Only 060-paste-images stays dead now; its PR (milind-soni#38) is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197wYuWWpF21iBeNgHZ8n3X
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant