Skip to content

feat(web): paste images directly into dashboard Chat - #57912

Closed
bird wants to merge 1 commit into
NousResearch:mainfrom
bird:feat/dashboard-chat-image-paste
Closed

bird wants to merge 1 commit into
NousResearch:mainfrom
bird:feat/dashboard-chat-image-paste

Conversation

@bird

@bird bird commented Jul 3, 2026

Copy link
Copy Markdown

What does this PR do?

Makes image paste work in the dashboard Chat tab.

The Chat tab is an xterm mirror of a TUI running inside the gateway (often a
remote Docker container). Ctrl+V / /paste route to the server-side
clipboard.paste RPC, which shells out to xclip/wl-paste inside the
container. That container has no X11/Wayland display and no access to the
browser's clipboard, so pasting a screenshot always fails with "No image found
in clipboard".

This handles it fully client-side, reusing endpoints that already ship — no
new RPC, no server changes, no new deps
:

  1. Intercept the DOM paste event on the terminal host element.
  2. If the clipboard holds an image, preventDefault() and upload the bytes the
    browser already has via the existing POST /api/files/upload-stream
    (api.uploadFile). It lands at uploads/paste/paste_<timestamp>.<ext> under
    the gateway's HERMES_HOME.
  3. Drive the TUI's existing /image <path> slash command over the PTY
    WebSocket, using the same burst-then-Return timing as handleCopyLast.

Text paste is untouched — the handler only fires when an image is present and
otherwise lets the event fall through to the existing text path. This works
identically in loopback and gated/OAuth-cookie auth modes because
api.uploadFile → fetchJSON already attaches the session token / sends
credentials: 'include'.

Related Issue

Fixes #24860 (the image-paste half; text paste is tracked separately in #29975).

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • web/src/lib/chatImagePaste.ts (new) — firstImageFromClipboard()
    (DataTransfer parsing, incl. Safari/Firefox files[] fallback) and
    uploadPastedImage() (size cap, ext sniff, upload → absolute container path).
    Kept out of the component so the clipboard logic is unit-testable.
  • web/src/lib/chatImagePaste.test.ts (new) — 5 vitest cases for the
    clipboard-extraction logic.
  • web/src/pages/ChatPage.tsx — register/unregister the host paste listener
    inside the terminal effect; surface upload/connection errors inline in the
    terminal.

How to Test

  1. Start the dashboard: hermes dashboard --host 0.0.0.0 --tui --no-open
  2. Open it in a browser and go to the Chat tab (click into the terminal).
  3. Copy an image (screenshot) to your clipboard and press Ctrl+V.
  4. The image uploads and the TUI attaches it via /image — send your message
    and the agent receives the image. Pasting plain text still behaves as before.

Automated checks (run in web/):

  • npm run typecheck → passes (exit 0)
  • npx vitest run src/lib/chatImagePaste.test.ts → 5/5 pass
  • npx eslint on the changed files → 0 errors

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass — N/A (frontend-only change; ran web typecheck + vitest instead, see above)
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform:

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A (no config keys)
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A (browser-side; no OS-specific paths)
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Scope note for reviewers: the automated checks above cover typecheck, the
pure clipboard-parsing unit tests, and lint. I did not run a live
end-to-end paste against a running dashboard + gateway, so the runtime wiring
(DOM paste → upload → /image over the PTY) is validated by construction
rather than an interactive session. The upload endpoint, api.uploadFile, and
the /image TUI command are all pre-existing and exercised elsewhere; this PR
only composes them. A quick manual paste smoke-test before merge is worth doing.

The dashboard Chat tab is an xterm mirror of a TUI running inside the
gateway (often a remote Docker container). Ctrl+V / `/paste` route to the
server-side clipboard, which has no access to the browser's clipboard, so
pasting a screenshot always fails with "No image found in clipboard"
(issue NousResearch#24860).

Handle it fully client-side: intercept the DOM `paste` event on the
terminal host, and when the clipboard holds an image, upload the bytes the
browser already has to the gateway's managed-files root via the existing
/api/files/upload-stream endpoint, then drive the TUI's existing
`/image <path>` command over the PTY. No new RPC, no server changes.

Text paste is untouched — the handler only fires when an image is present
and otherwise lets the event through.

Adds web/src/lib/chatImagePaste.ts (clipboard parsing + upload helper) with
unit tests for the clipboard-extraction logic.
@alt-glitch alt-glitch added type/bug Something isn't working comp/dashboard Web dashboard / control panel UI (dashboard/, landing) comp/tui Terminal UI (ui-tui/ + tui_gateway/) tool/vision Vision analysis and image generation P2 Medium — degraded but workaround exists labels Jul 3, 2026
@bird

bird commented Jul 3, 2026

Copy link
Copy Markdown
Author

✅ Validated live (end-to-end)

Tested against a running dashboard + gateway (Hermes v0.18.0, Unraid/Docker, loopback mode) by building this branch's web bundle, pointing HERMES_WEB_DIST at it, and dispatching a real paste ClipboardEvent carrying a PNG at the terminal host element.

Confirmed the full chain fires:

  1. firstImageFromClipboard matched the image; the handler called preventDefault() (defaultPrevented === true).
  2. Bytes uploaded via POST /api/files/upload-stream — file physically written to disk:
    /opt/data/uploads/paste/paste_20260703_155042_740.png (matches the paste_<timestamp>.png scheme).
  3. /image <path> reached the PTY and the TUI attached it — terminal rendered:
    · /image /opt/data/uploads/paste/paste_20260703_155042_740.png
    · 📎 Attached image: paste_20260703_155042_740.png · 1x1 · ~85 tok
    

Text paste is unaffected (handler no-ops when the clipboard holds no image).

Note: exercised via a synthetic ClipboardEvent rather than a physical Ctrl+V keystroke, but the browser populates clipboardData identically for both, and the DOM paste listener is the same entry point. The earlier "not validated" scope note in the description is now superseded by this run.

@OutThisLife

Copy link
Copy Markdown

Superseded by #61929 — thanks @bird!

Kept your client composition (chatImagePaste helper + /image over the PTY) and unit tests. The relative uploads/paste/… managed-files path 400s on local dashboards without a locked root (the Discord :9119 case), so the salvage writes to HERMES_HOME/images/ via a dedicated upload endpoint (from #48563), adds capture-phase paste + drop, and makes Ctrl/Cmd+Shift+V try clipboard.read() for images before readText() (that chord's preventDefault otherwise suppresses the DOM paste event).

Closing in favor of #61929.

@OutThisLife

Copy link
Copy Markdown

Superseded by #61929.

Bartok9 pushed a commit to Bartok9/hermes-agent that referenced this pull request Jul 10, 2026
Dashboard Chat is an xterm mirror of a TUI inside the gateway, so
server-side clipboard.paste never sees the browser clipboard. Upload
pasted/dropped images to the profile's images/ dir (same place
clipboard.paste / image.attach use), then drive /image over the PTY.

Uses a dedicated /api/chat/image-upload endpoint (magic-byte check,
25MB cap, profile scope) instead of relative managed-files uploads that
400 on local dashboards without a locked root. Ctrl/Cmd+Shift+V also
tries clipboard.read() for images before falling back to text, since
preventDefault on that chord suppresses the DOM paste event.

Salvages NousResearch#57912 (client composition + /image PTY drive) and folds in
NousResearch#48563's upload endpoint + drop path.

Co-authored-by: bird <6666242+bird@users.noreply.github.com>
Co-authored-by: tt-a1i <53142663+tt-a1i@users.noreply.github.com>
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
Dashboard Chat is an xterm mirror of a TUI inside the gateway, so
server-side clipboard.paste never sees the browser clipboard. Upload
pasted/dropped images to the profile's images/ dir (same place
clipboard.paste / image.attach use), then drive /image over the PTY.

Uses a dedicated /api/chat/image-upload endpoint (magic-byte check,
25MB cap, profile scope) instead of relative managed-files uploads that
400 on local dashboards without a locked root. Ctrl/Cmd+Shift+V also
tries clipboard.read() for images before falling back to text, since
preventDefault on that chord suppresses the DOM paste event.

Salvages NousResearch#57912 (client composition + /image PTY drive) and folds in
NousResearch#48563's upload endpoint + drop path.

Co-authored-by: bird <6666242+bird@users.noreply.github.com>
Co-authored-by: tt-a1i <53142663+tt-a1i@users.noreply.github.com>
justemu pushed a commit to justemu/hermes-agent that referenced this pull request Jul 18, 2026
Dashboard Chat is an xterm mirror of a TUI inside the gateway, so
server-side clipboard.paste never sees the browser clipboard. Upload
pasted/dropped images to the profile's images/ dir (same place
clipboard.paste / image.attach use), then drive /image over the PTY.

Uses a dedicated /api/chat/image-upload endpoint (magic-byte check,
25MB cap, profile scope) instead of relative managed-files uploads that
400 on local dashboards without a locked root. Ctrl/Cmd+Shift+V also
tries clipboard.read() for images before falling back to text, since
preventDefault on that chord suppresses the DOM paste event.

Salvages NousResearch#57912 (client composition + /image PTY drive) and folds in
NousResearch#48563's upload endpoint + drop path.

Co-authored-by: bird <6666242+bird@users.noreply.github.com>
Co-authored-by: tt-a1i <53142663+tt-a1i@users.noreply.github.com>
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
Dashboard Chat is an xterm mirror of a TUI inside the gateway, so
server-side clipboard.paste never sees the browser clipboard. Upload
pasted/dropped images to the profile's images/ dir (same place
clipboard.paste / image.attach use), then drive /image over the PTY.

Uses a dedicated /api/chat/image-upload endpoint (magic-byte check,
25MB cap, profile scope) instead of relative managed-files uploads that
400 on local dashboards without a locked root. Ctrl/Cmd+Shift+V also
tries clipboard.read() for images before falling back to text, since
preventDefault on that chord suppresses the DOM paste event.

Salvages NousResearch#57912 (client composition + /image PTY drive) and folds in
NousResearch#48563's upload endpoint + drop path.

Co-authored-by: bird <6666242+bird@users.noreply.github.com>
Co-authored-by: tt-a1i <53142663+tt-a1i@users.noreply.github.com>
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
Dashboard Chat is an xterm mirror of a TUI inside the gateway, so
server-side clipboard.paste never sees the browser clipboard. Upload
pasted/dropped images to the profile's images/ dir (same place
clipboard.paste / image.attach use), then drive /image over the PTY.

Uses a dedicated /api/chat/image-upload endpoint (magic-byte check,
25MB cap, profile scope) instead of relative managed-files uploads that
400 on local dashboards without a locked root. Ctrl/Cmd+Shift+V also
tries clipboard.read() for images before falling back to text, since
preventDefault on that chord suppresses the DOM paste event.

Salvages NousResearch#57912 (client composition + /image PTY drive) and folds in
NousResearch#48563's upload endpoint + drop path.

Co-authored-by: bird <6666242+bird@users.noreply.github.com>
Co-authored-by: tt-a1i <53142663+tt-a1i@users.noreply.github.com>
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Dashboard Chat is an xterm mirror of a TUI inside the gateway, so
server-side clipboard.paste never sees the browser clipboard. Upload
pasted/dropped images to the profile's images/ dir (same place
clipboard.paste / image.attach use), then drive /image over the PTY.

Uses a dedicated /api/chat/image-upload endpoint (magic-byte check,
25MB cap, profile scope) instead of relative managed-files uploads that
400 on local dashboards without a locked root. Ctrl/Cmd+Shift+V also
tries clipboard.read() for images before falling back to text, since
preventDefault on that chord suppresses the DOM paste event.

Salvages NousResearch#57912 (client composition + /image PTY drive) and folds in
NousResearch#48563's upload endpoint + drop path.

Co-authored-by: bird <6666242+bird@users.noreply.github.com>
Co-authored-by: tt-a1i <53142663+tt-a1i@users.noreply.github.com>
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
Dashboard Chat is an xterm mirror of a TUI inside the gateway, so
server-side clipboard.paste never sees the browser clipboard. Upload
pasted/dropped images to the profile's images/ dir (same place
clipboard.paste / image.attach use), then drive /image over the PTY.

Uses a dedicated /api/chat/image-upload endpoint (magic-byte check,
25MB cap, profile scope) instead of relative managed-files uploads that
400 on local dashboards without a locked root. Ctrl/Cmd+Shift+V also
tries clipboard.read() for images before falling back to text, since
preventDefault on that chord suppresses the DOM paste event.

Salvages NousResearch#57912 (client composition + /image PTY drive) and folds in
NousResearch#48563's upload endpoint + drop path.

Co-authored-by: bird <6666242+bird@users.noreply.github.com>
Co-authored-by: tt-a1i <53142663+tt-a1i@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/dashboard Web dashboard / control panel UI (dashboard/, landing) comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dashboard Chat: Ctrl+V paste broken, image paste not supported

3 participants