Skip to content

cmux-next Home: drag pictures into the Chief chat from any app - #22965

Merged
lawrencecchen merged 3 commits into
feat-cmux-nextfrom
feat-cmux-next-chief-image-drop
Oct 21, 2026
Merged

lawrencecchen merged 3 commits into
feat-cmux-nextfrom
feat-cmux-next-chief-image-drop

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Lawrence (2026-10-20): "I want people to drag images into Chief properly."

What changes

A drag of pictures or files into Home (the Chief chat) attaches them like a paste, anywhere on the pane:

  • File promises (the screenshot thumbnail, Photos, a browser's or Preview's picture): the drop asks the source for its file at once; the file arrives on a background queue (HomePromisedFiles, 30 s bound by a DemandTimer on an injected clock). Meanwhile the notice above the field says "Adding the dropped files…"; then the chip appears in its place in the intake order. The drag's own picture bytes (read at the drop) stand in when no file comes. The main thread never waits for a source.
  • A drag offers copy for a promise of a file (not only folders). A promise that does not read during the drag (Chrome's picture) does not hide the drag's own picture bytes.
  • Picture bytes: GIF, WebP and PDF next to PNG, JPEG, HEIC and TIFF (Preview's selection, a browser).
  • The part limit is checked in the intake order, so a paste after a pending drop counts its files.
  • One drop target for the whole pane: MessagesLab's pane and transcript register no drag types, so every drag outside the field reaches HomeNativeTranscriptView; the field reports its own drags (Compose patch: acceptsDropPasteboard, onDropTargetChange); Home's conversation list forwards drags to the shown conversation. No plain string or bare link is registered, so text drags keep the field's handling and links keep the panes' drop zones.
  • Drop highlight: the theme's focus ring as a rounded outline over a faint tint of it, over the pane, while a drag the composer takes is over it (Motion.home fades; none with motion off).
  • Paste never asks for promises (only a drop can). debug.home.drive {action: draft} reports the draft, pending drops, the notice and the highlight.
  • scripts/cmux-next/home-image-drop-proof.py: real CGEvent drags from Finder, the screenshot thumbnail, Chrome, Safari, Preview and a cmux browser tab into the DEBUG Home fixture (nx-remote --needs input).

Changelog

  • Added: Drag pictures into the Chief chat from the screenshot thumbnail, Photos, Preview and web pages, anywhere on the Home pane, with a drop highlight.

Tests

HomeImageDropTests (CmuxNextAppTests): real Home views (dock, Home tab, conversation tab) get drags as AppKit delivers them (the deepest registered view, entered, updated when implemented, drop, end). The drag contents are a value read through HomeNativeTranscriptView.dropContents / HomeFilePromise (CI pasteboard servers answer inconsistently). Cases: Finder files (1 and 5) on the transcript rows, header, field and list; a file promise; TIFF, PDF and GIF bytes with the picture's link; a browser picture whose promise does not read; link drags stay out. DragWhileStreamingTests follows the new routing.

Observed / Not run

  • Red: package-test-lane.sh suite Packages/macOS/CmuxNext HomeImageDropTests at the red commit (test and seam only): Finder, promise and picture-bytes cases fail (88+12+9 issues at 60c7e92 / earlier red heads), the link case passes.
  • Green: the same command at head 42b1f75 passed (with HomeTypeToFocusTests). Gate at the rebased head: see the landing comment.
  • HomeTypeToFocusTests crashes silently on the current tip 7b41b2a too (base red, not this change).
  • Real pointer drags on cmux-lawrence-2 (--needs input), table in the landing comment. The real-drag harness is not deterministic (the same Finder drag attached in one run and not in the next, at the same points), so the table counts only drags that attached.

@lawrencecchen lawrencecchen added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 21, 2026
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@teamleaderleo

Copy link
Copy Markdown
Collaborator

Bead cx-8xpz1 created for this PR by the cmux coordinator.

@lawrencecchen
lawrencecchen force-pushed the feat-cmux-next-chief-image-drop branch from 3d3e2a4 to 47a942c Compare October 21, 2026 05:19
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Correctness review (subagent, read-only, at 3d3e2a4)

No blockers. Findings and how they are resolved at 47a942c:

  1. should-fix, fixed: the part-limit room for a sync paste was counted at call time, before a pending promise drop's files landed (draft could exceed 16 parts). attach now checks inside the intake chain after earlier intakes (checked(_:clearing:)).
  2. should-fix, fixed: the stand-in picture bytes were read from the drag pasteboard after the promise wait (a later drag could replace them). They are read at the drop now.
  3. should-fix, fixed: any file promise showed copy and the highlight. A drag now offers copy only for a promise of a file (not only folders; the data side takes any file), cached per drag sequence by the callers.
  4. should-fix, fixed (test): the availability observation could re-hide the transcript in the test. The harness shows the conversation again before each drag (showConversation). The drag emulation also keeps the entered answer for a destination with no draggingUpdated: of its own (as AppKit does); the first red run showed that a direct draggingUpdated call on such a view returns none.
  5. evidence: the red commit stands in for the highlight and pending-notice checks (they do not exist on the base); recorded in the PR body with the red and green runs.
  6. nit, fixed: the field now reports onDropTargetChange on draggingUpdated too (the host ignores a repeat).
  7. nit, fixed: HomePromisedFiles keeps its receivers and queue until the files came.
  8. nit, partly fixed: a drop where only some promised files came shows "The file couldn't be attached." Not done: no sweep of cmux-home-drops/* folders left by a crash (they are in the temporary folder).
  9. nit, not changed: on the New Message page the conversation list still registers the picture types, so a browser image drag over the list there is refused instead of reaching a pane link zone. Rare; left as is.

Checked correct by the reviewer: concurrency of the promise reader (@Sendable, nonisolated Arrival), drag routing after PaneHost stopped registering (no other Home view registers types; text and link drags are not intercepted), highlight cleared on exit/end/perform on all three paths with no NSView super calls, promise flow inside performDragOperation, folder cleanup only after the blob-cache copy, paste never using promises, vendored patch consistent (sync-messageslab.sh --check green).

@lawrencecchen
lawrencecchen force-pushed the feat-cmux-next-chief-image-drop branch 6 times, most recently from 583ea7a to 52f7b09 Compare October 21, 2026 09:48
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@lawrencecchen
lawrencecchen force-pushed the feat-cmux-next-chief-image-drop branch from 42b1f75 to 19939bd Compare October 21, 2026 11:27
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 405ee64: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr22965 -f ref=feat-cmux-next-chief-image-drop -f platform=macos -f backend_mode=local

Not verified: macOS jobs did not run: compile and app tests were skipped on 405ee649ab; ci-status is green without them.

CI passes on 405ee649ab (run 39631068390 attempt 1).
macOS jobs did not run: compile and app tests were skipped: Fast static checks skipped (the macos job needs both to succeed). Re-run CI once they pass, before merging.

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way, seen on other PRs that it failed on another pull request's run lately.

web and size checks: green

Tour of 405ee649ab (run): no media. The tour job failed before the tour ran (build or setup); the run log has the output.

@blacksmith-sh

blacksmith-sh Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Found 6 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmpapxgo5em/summary-fail.json (live):/summary-fail.json (live): View Logs
/tmp/tmpy32h4crs/summary-missing.json (live):/summary-missing.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@lawrencecchen
lawrencecchen force-pushed the feat-cmux-next-chief-image-drop branch from d6962be to beb3dab Compare October 21, 2026 14:38
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Second correctness review (subagent, read-only, at cca6db8) and resolution

  1. blocker, fixed: HomeDropState.install took its inset frame from zero bounds (a null rect that autoresizing never recovers), so the highlight never drew though isShowing was true. The highlight now takes the pane's inset frame each time it shows. Confirmed on a real drag: the outline draws around the pane during a Chrome drag at d6962be (cmux-lawrence-2, --needs input).
  2. should-fix, fixed: the test now checks the highlight's frame (finite, inside the pane when the pane has room), not only isShowing.
  3. nit, fixed: the theme re-applies to the highlight with the rest of the view.
  4. nit, fixed: a drop asks only promises of files (promisesAFile), so a folder-only promise next to picture bytes attaches the bytes.
  5. nit, not changed: the conversation list recomputes its drop target per call; a conversation switch in the middle of a drag could leave the old view's highlight on until its next drag. Rare.
  6. nit, fixed: every changed Compose.swift line is marked cmux.

Rebase onto f3fbb9c (#22698, attachments checks off the main actor): the promise flow now uses that change's admit and off-main HomeComposerCheck.problem stat; the part limit is counted in intake order; a clean batch keeps "Adding the dropped files…" while a drop's files still come and keeps a notice with an action (the mic's).

… into the message box

Drags with real CGEvents from Finder (png, jpg, heic, gif, pdf, several at once), the screenshot
thumbnail, Chrome, Safari, Preview and a cmux browser tab into the DEBUG Home fixture, and records
what the draft holds and what a send attaches.
…into the message box

Real Home views (dock, Home tab, conversation tab) get a drag as AppKit delivers it: the deepest
view under the pointer registered for one of the drag's types, entered, moves (the entered answer
stands for a view with no draggingUpdated of its own), the drop and the end. What the drag carries
is a value the intake reads through a new seam (HomeNativeTranscriptView.dropContents, and
HomeFilePromise for NSFilePromiseReceiver): the CI Macs' pasteboard server answers reads
inconsistently. No behavior change.

Cases: Finder files (1 and 5) on the transcript, header, field and conversation list; a file
promise (the screenshot thumbnail, Photos); TIFF, PDF and GIF bytes with the picture's link (Preview,
a browser); link drags that must stay out. The drop highlight and the pending notice are checked by
the fix commit (they do not exist here).
@lawrencecchen
lawrencecchen force-pushed the feat-cmux-next-chief-image-drop branch from beb3dab to efbd7ab Compare October 21, 2026 15:51
… bytes, the whole pane, a drop highlight)

Lawrence 2026-10-20: "I want people to drag images into Chief properly."

- File promises (the screenshot thumbnail, Photos, a browser's or Preview's picture): the drop asks
  the source for its file at once and the file arrives on a background queue (HomePromisedFiles,
  30 s bound by a DemandTimer on an injected clock); the notice says "Adding the dropped files…"
  meanwhile, and the file takes its place in the intake order. The drag's own picture bytes (read
  at the drop) stand in when no file comes. A drag offers copy only for a promise of a file, not
  of a folder; a promise that does not read during the drag (Chrome's picture) does not hide the
  drag's own picture bytes.
- Picture bytes: GIF, WebP and PDF next to PNG, JPEG, HEIC and TIFF (Preview's selection, a browser).
- The part limit is checked in the intake order, so a paste after a pending drop counts its files.
- One drop target for the whole pane: MessagesLab's pane and transcript register no drag types, so
  every drag outside the field reaches HomeNativeTranscriptView; the field reports its drags to it
  (Compose patch: acceptsDropPasteboard, onDropTargetChange); Home's conversation list forwards
  drags to the shown conversation. No plain string or bare link is registered: text drags keep the
  field's own handling and links keep the panes' drop zones. DragWhileStreamingTests follows: the
  transcript side of its sweep is the surrounding drop target's now.
- A drop highlight over the pane (the theme's focus ring as a rounded outline over a faint tint,
  Motion.home fades) while a drag the composer takes is over it.
- Paste never asks for promises (only a drop can); debug.home.drive draft reports the draft.
- Proof script: windows close through their own close buttons, front order before each drag.
@lawrencecchen
lawrencecchen force-pushed the feat-cmux-next-chief-image-drop branch from efbd7ab to 405ee64 Compare October 21, 2026 16:01
@lawrencecchen
lawrencecchen merged commit 860c5b4 into feat-cmux-next Oct 21, 2026
@lawrencecchen
lawrencecchen deleted the feat-cmux-next-chief-image-drop branch October 21, 2026 17:52
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Landed in feat-cmux-next

Landed through safe-push as 860c5b452fff (merge train 5d8e6dd6c5cc). It carries 59b94f7 (proof script), 63aed53 (failing drag tests and the test seam), and 405ee64 (the change).

Gate receipt at 405ee64 (gate-run.sh, all 5 steps green): CmuxMessagesLab suites (60), the CmuxNext suites gate-plan.py picked (including HomeImageDropTests, HomePasteAnywhereTests and HomeAttachmentPasteTests), Release compile, proof script py_compile, and check-no-godfiles.

Red/green: package-test-lane.sh suite Packages/macOS/CmuxNext HomeImageDropTests. On the red commit, the Finder, promise and picture-bytes cases fail (110 issues) and the link case passes. On the head, everything passes.

  • The red result for Finder drops over the transcript comes partly from the test seam. The base reads the real drag pasteboard, so it does not see the stand-in drag contents.
  • Real Finder drags already attached on the base (see the table below). On the base they did not attach over the conversation list.

Real pointer drags on cmux-lawrence-2 (nx-remote --needs input, CGEvents, DEBUG Home fixture). The table counts only drags that attached; a dash means no run attached anything for that source. The harness is not deterministic: the same Finder drag attached in one run and not in the next.

Source Base (6512ef1 / d2b35a2 + harness) After (19939bd / d6962be)
Finder png attached (1 of 4 runs) attached, highlight drawn (3 of 4 runs)
Finder jpg attached (1 of 4) attached, highlight drawn (1 of 3)
Finder heic — attached as image/heic, highlight drawn (3 of 4)
Finder gif — attached as image/gif (1 of 4); one run attached the wrong icon (harness)
Finder pdf — attached as application/pdf, highlight drawn (1 of 4)
Finder, 5 files at once — — (the harness Cmd+A selection did not take; covered by the behavior test only)
Finder png onto the message box — — (harness; covered by the behavior test only)
Screenshot thumbnail (file promise) — — (the drag did not start from the thumbnail position; covered by the behavior test only)
Chrome image attached as attachment.png (2 of 4) attached as hidp-web.png, the page's own file through its promise, highlight drawn (3 of 4)
Safari image — — (no drag reached Home)
Preview selection — — (no drag reached Home)
cmux browser tab — — (the tab opened; no drag reached Home)

The drop highlight on a real drag: a rounded outline in the theme's focus ring around the pane (window snapshots during the hover, at 19939bd and d6962be).

Base reds named by safe-push (not from this push): package conventions lint, main-actor work ratchet, CI target graph. HomeTypeToFocusTests failed in the suite batch on 804cef3 and passes on d5c7a22. The MessagesLab timing suites (HomeComposerTypingLatencyTests, HomeStreamingPerfTests) fail under host load on some minis, on the base and on the head. A run of the head alone on aws-m4pro-7 passed all three perf suites.

@github-actions

Copy link
Copy Markdown
Contributor

The feat-cmux-next push run https://github.com/manaflow-ai/cmux/actions/runs/39647862464 at 5818254 failed: cmux-next checks (god files, concurrency, crash safety, l10n).
Those jobs last passed at 8a06461. This pull request is one of 5 merged since: #22965, #23159, #23601, #22762, #22336.
If the failure is in your change, fix forward on feat-cmux-next. Pull requests run only the tiers their paths reach (docs/ci/cmux-next-tiers.md); label a batch PR full-ci to run them all before merging.

This branch had an error being deployed

1 failed deployment
ci-routing-agent — 405ee649 Deployed Oct 21, 2026 by lawrencecchen
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants