Skip to content

cmux mobile: newlines in the chat composers; tap any file to preview it - #24572

Merged
azooz2003-bit merged 9 commits into
feat-cmux-nextfrom
feat-cmux-next-ios-composer-files
Oct 23, 2026
Merged

azooz2003-bit merged 9 commits into
feat-cmux-nextfrom
feat-cmux-next-ios-composer-files

Conversation

@azooz2003-bit

@azooz2003-bit azooz2003-bit commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Two gaps in the cmux-next iPhone app, plus the shared preview they need.

1. Newlines in the composers

Composer On-screen Return Hardware Return Shift- or Option-Return Grows to
Agent chat (GUI) new line sends new line 8 lines, then scrolls
Chief (CmuxConversation) new line (unchanged) sends (new) new line (new) unchanged (header)
Terminal "Send to …" sends (unchanged, SPEC 10.2) sends new line (new) 6 lines, then scrolls
  • The agent chat field was a SwiftUI TextField, which decides Return by itself. It is now ChatComposerField, a UITextView over the shared ComposerReturnTextView and ComposerReturnPolicy in CmuxMobileDesign.
  • Hardware Return only does what the send button would: send, or queue/steer while the agent runs. It never sends an empty draft and never presses Stop.
  • While text is being composed (marked text, for example Japanese), Return belongs to the input method.
  • The terminal keeps Return = send, because a shell expects it. A multi-line draft still gets a new line: Shift-Return, Option-Return, or New Line in the field's edit menu. The draft goes to the program as one paste (ComposerSubmit, unchanged).

2. Preview any file everywhere

  • One preview: FilePreviewGallery, built from ArtifactPreview pages.
    • Images: pinch, double-tap, swipe down to close.
    • Video: plays.
    • PDF, syntax-colored code and text, rendered Markdown and HTML (with a Source toggle).
    • Audio, Office/iWork, RTF and anything else Quick Look renders: an embedded QLPreviewController (new .audio/.document kinds and ArtifactContent.quickLook).
    • Types nothing on the phone can show: a share sheet ("Share or Open In…").
    • Every page has Share or Save (Save to Files, Save Image, AirDrop) and a position line ("3 of 5"). Swipe between a message's attachments.
  • Wired into:
    • agent-chat composer attachments before sending;
    • sent agent-chat attachments (files were inert; images used an image-only viewer, now removed);
    • terminal composer attachments (UIKit presentation, like Files and Links);
    • Files and Artifacts / Files and Links;
    • diffs: Open File in the diff screen and the diff card menus;
    • tool calls: long press opens the files the call touched;
    • files agents mention: links and path:line inline code in answers open the preview, web links open as before;
    • Chief: file links (onOpenLink), queued composer attachments (Quick Look), and a message's documents (Quick Look, swipeable).
  • Feed: a link in an event's markdown no longer navigates the pooled web view (which went blank). Web links open in Safari; others open the event.
  • Trust: Mac files are read only through the link's file service (LinkFileArtifactSource), so the Mac's workspace-root and Chat Folders rules (cnx-ios: fix full-verify defects D1–D14 (Chief, teams, previews, Codex default, polish) #22354, D9) still decide. Chief resolves only absolute paths. Phone-held attachments use an attachment: reference that never goes to the Mac. Nothing widens access.

CmuxConversation changes land upstream, byte-identical, in #24573 (feat-imsg-parity-gui).

Changelog

  • Added: the iPhone agent chat composer takes several lines (Return starts a new line, the send button sends; on a hardware keyboard Return sends and Shift-Return starts a new line).
  • Added: tap any file in the iPhone app (attachments before and after sending, Files and Artifacts, diffs, files agents mention) to preview it, swipe between a message's files, and share or save it.

Tests

Failing tests first: c4c894d397dc. On mini-cmux11s with verify/termqa/cmux_mobile_kit-test-ios.sh:

  • Red on the tests-only commit: the CmuxMobileAgentChatTests build fails (ArtifactContent has no member quickLook, no ComposerReturnPolicy, and so on).
  • Green on head 32251815e7b3: CmuxMobileAgentChatTests 132/132, CmuxMobileLiveTests 262/262, CmuxMobileChiefTests 79/79. ComposerSubmitTests 5/5 and CmuxMobileShellTests passed on earlier heads.
  • Sent attachments: red on 3475760ebbbc (no recordedImage/recordedImageID), green on the fix.
  • New suites in ComposerFilesTests.swift:
    • Return policy, chat and terminal;
    • the text view's Return and New Line;
    • kinds for every common type;
    • local attachments decoded like Mac files (image, text, docx through Quick Look, video, binary, PDF);
    • missing bytes;
    • a Mac file through the fallback source;
    • local copies named and stable;
    • FileLink (web, app and anchor links refused; path:line, file://#L);
    • MarkdownFilePath (console.log is not a file).

Live check (real usage, real link, no shortcuts)

  • Setup:
    • Phone: headless iPhone 17 Pro simulator cmux-composer-files (iOS 26.5) on mini-cmux8s, Demo build of this branch, signed in with the dev credential.
    • Mac: the hidden rig Mac app on mini-cmux11s (fleet build cmux-mobile-e2e-e196c421, open -g -j -n), workspace live-files in a real project folder.
    • Link: WireGuard (path=Direct, then path=Turn).
    • Agent: a real Claude Code chat (Opus 5.5), started from the phone's New Agent Chat With….
    • Driving: axe HID input. Recording: simctl io recordVideo. Simulator.app was never opened, and nothing on the laptop took focus.
  • Newlines (agent chat): the on-screen Return typed "Reply with the line count of this prompt.\nline two\nline three". Claude answered "Your prompt has 3 lines of text" and described the attached photo.
  • Newlines (Chief): hardware Shift-Return and Option-Return, then Return, sent one three-line message to the Mac's Chief.
  • Composer attachments (photo, notes.txt, report.pdf, clip.mp4): each tap opened the full-screen preview (image, syntax-colored text, PDF, playing video), paging "1 of 5" to "5 of 5", with Share or Save. Claude received all of them; it ran a command on …/.cmux/attachments/phone-…/clip.mp4.
  • Sent message:
    • The live check found a bug: the sent bubble showed no attachments, because acpmux's user_message names images only. Fixed in 05f0ad0b0987/32251815e7b3 (tests first in 3475760ebbbc).
    • After the fix, the new turn's bubble shows the photo plus notes.txt, report.pdf and clip.mp4. Tapping each opens it, paging "1 of 4" to "4 of 4".
    • After an app relaunch, the earlier turn's photo, which now comes from the Mac's record through _acpmux/attachment, shows in the bubble and opens full screen.
  • Files agents mention: tapping report.pdf in Claude's answer opened the PDF read from the Mac over the link (Copy Path shown).
  • Mac files (terminal Files and Links):
    • At first the Mac refused with "isn't sharing any folders with this phone yet" (forbidden, "no workspace roots are shared"), which is the trust rule holding.
    • With the project folder set as a Chat Folder, these opened from the Mac: report.pdf, photo.png, page.html (rendered, with Source), notes.md (rendered), brief.docx (Quick Look) and main.swift (syntax colored).
  • Evidence: git-ignored, on the laptop under ~/cmux-mobile-builds/20261023-composer-files/evidence/ (videos 01–10, shots/).

Observed / Not run

  • Not observed: Chief's queued-attachment Quick Look (the agent-chat equivalent was observed); the Feed link change (no Feed event with a link on the rig); the diff "Open File" menu (unit-wired only).
  • Not finished: Claude's tool permission in the second turn asks for device authentication, which a simulator without a passcode can't answer, so that turn was declined.
  • No real iPhone (user direction: simulators only).

Review

A fresh correctness review subagent reviewed both rounds. Round 1 found no blockers and 6 should-fix items, fixed in 0909d699c39b. Round 2, on the sent-attachment fix, found no blockers and 2 should-fix items, fixed in 32251815e7b3. Both are in PR comments.

🤖 Generated with Claude Code

azooz2003-bit and others added 5 commits October 21, 2026 23:55
…y file kind

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Composer: the agent chat field is a growing UITextView (8 lines, then it
scrolls). The keyboard's Return starts a new line and the send button sends;
a hardware Return sends and Shift- or Option-Return starts a new line. Chief
(CmuxConversation) keeps Return as a new line and gains the same hardware
keys. The terminal composer keeps Return as send and gains Shift-Return and
New Line in its edit menu.

Previews: one FilePreviewGallery (ArtifactPreview pages) opens attachments
before and after sending (swipe between a message's files, Share or Save),
files agents name (links and code paths in answers, tool calls, diffs), and
Files and Artifacts. Audio, Office/iWork and other types open in Quick Look;
types nothing can show offer the share sheet. Mac files still go through the
link's file service and its workspace-root checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n actor (Swift 6 sending)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… once in its window

Chief: ConversationViewController.onOpenLink lets the host take a tapped
link; Chief opens file links in the file preview, read from Chief's Mac.
Feed: a link tapped in an event's markdown no longer navigates the pooled
web view (web links open in Safari, others open the event).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…f main, Quick Look of a message's documents)

- FileLink: only scheme-less links, file:// (with #L line) and name.ext:12 read as files;
  maps:, shortcuts: and other app links open with the system.
- Chat field: focus asked before the field has a window lands in didMoveToWindow; the
  scheduled change reads the binding again; a torn-down field drops its focus.
- ArtifactPreview writes its share copy off the main actor; each gallery page keeps its
  own Preview/Source choice; an empty attachment names itself, not its reference.
- CmuxConversation: a message's documents open with the tapped one first (abort if it
  fails), the rest fetched together; a queued attachment named .. stays in its folder.
- Chief ignores a file-link tap while something is presented. Feed: sms/facetime open
  with the system, in-page anchors stay put. Inline code like console.log is not a file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@azooz2003-bit azooz2003-bit added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 23, 2026
@cursor

cursor Bot commented Oct 23, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 3225181: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr24572 -f ref=feat-cmux-next-ios-composer-files -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: red check: Daemon capabilities export is fresh (+7 more) is not red on the base 36289e3; do not merge until green; see lane-rules

@azooz2003-bit

Copy link
Copy Markdown
Collaborator Author

Correctness review (fresh subagent), findings and resolution

This was a read-only review of origin/feat-cmux-next...HEAD at 2861a08391af. It found no blockers. It checked access: no Mac file can be read except through the link's file service. Chief's source has baseDirectory: nil, so only absolute paths reach the Mac, and attachment: references never do.

# Finding Resolution (0909d699c39b)
1 CmuxConversation edits were not on feat-imsg-parity-gui Upstream PR with the same files byte-identical (linked from the description)
2 FileLink treated app links (maps:, shortcuts:, x-apple-…:) as file paths Only scheme-less links, file://, name.ext:12 and Makefile:3 count as files; tests added
3 Composer focus: a stale captured value, a yield loop that could run out, no teardown The Task reads the binding again when it runs; focus asked before the field has a window lands in didMoveToWindow; dismantleUIView clears focus
4 Share copy written on the main actor for every page Written in a detached task
5 A message's documents waited for every download in sequence, and a failed tapped item opened another The tapped item loads first (opening stops if it fails); the others load concurrently
6 A Chief file-link tap was swallowed while something was presented Ignored while a controller is presented
nit console.log, res.json and similar read as files Ambiguous extensions need a /
nit file://…#L12 lost its line Line read from the fragment
nit A queued attachment named .. Sanitized to Attachment
nit Empty attachment showed its internal reference Shows the file name
nit One Preview/Source toggle shared across gallery pages Each page keeps its own
nit Feed: sms/facetime opened the event; #anchor links did too Both open with the system; anchors stay put
left VoiceOver label "Attachment" without the file name (CmuxConversation shelf) Not changed (the shelf item has no name)
left Diff Open File resolves a repo-relative path against the session folder Not changed; it matches Files and Artifacts
left Local copies keyed by reference only (two Macs, same path) Not changed (a temp cache; the next open rewrites it)

After the fixes, the iOS kit tests on mini-cmux11s passed: AgentChat 126/126, Live 247/247, Chief 74/74, ComposerSubmit 5/5.

@azooz2003-bit azooz2003-bit added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Oct 23, 2026
@blacksmith-sh

blacksmith-sh Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Found 10 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmpbh4ssck_/summary-fail.json (live):/summary-fail.json (live): View Logs
/tmp/tmpq3j14pzj/summary-missing.json (live):/summary-missing.json (live): View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
convention violations found (ERROR lines above)./
convention violations found (ERROR lines above).
View Logs
test_base_passes (main.NoticesPairTests.test_base_passes)/
test_base_passes (main.NoticesPairTests.test_base_passes)
View Logs
test_targets_outside_the_app_go_to_the_mac_canary (main.SwiftCanaryFleetCoverage.te
st_targets_outside_the_app_go_to_the_mac_canary)/
test_targets_outside_the_app_go_to_the_mac_canary (main.SwiftCanaryFleetCoverage.te
st_targets_outside_the_app_go_to_the_mac_canary)
View Logs
the committed export failed: check-daemon-capabilities-fresh: plans/cmux-next/
daemon-capabilities.json is stale:
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@azooz2003-bit azooz2003-bit added ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds and removed ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds labels Oct 23, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator

Bead cx-o2yrq created for this PR by the cmux coordinator.

@azooz2003-bit

Copy link
Copy Markdown
Collaborator Author

Non-green checks on head 3229117fb061, classified. This PR changes only cmux-next-ios/ and Packages/iOS/CmuxConversation/:

  • fast gates (prepush gate, Sidebar-Look): names the base-update merge commit 3229117fb06 with CmuxNextSidebar/Views files that come from the base. The checkout is shallow, so the gate reads the merge as adding the whole tree. That bug is fixed in fast gates: deepen a pull request's checkout so a head that brings the base in is not read as adding the whole tree #24539. A local prepush-gate.py run on the same head reports no new violations.
  • cmux-next checks: ThinScroller.swift:206 sleep, ThinScrollerFadeTests.swift:37 notification center, the perf-gate unlit test and stale daemon-capabilities.json are all base files (CmuxNextDesign, plans/), not in this diff.
  • nightly-guards / workflow-guard-tests: FileNotFoundError: xcodebuild on the runner, which is infrastructure.
    iOS kit tests on this head (mini-cmux11s): AgentChat 126/126, Live 261/261, Chief 79/79 (one MacHome flake on the first run passed on rerun; MacHome isn't touched here).

azooz2003-bit and others added 2 commits October 23, 2026 13:45
…k: the sent bubble had none)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d from the Mac

Live check over the real link (real Claude Code chat, 2026-10-23): a prompt sent with a photo
and three files reached the agent, but the sent bubble showed no attachments. acpmux's
user_message names a prompt's images only (no bytes, no files), and the reducer took that
record's empty list. The store now keeps what this phone sent, by prompt id; a prompt from
elsewhere (history, another device) shows the images its record names, read on demand with
_acpmux/attachment (only an image the session's own log names), in the bubble and the preview.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@azooz2003-bit azooz2003-bit removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Oct 23, 2026
…keep, two copies of one image)

- AcpmuxSessionSource caches recorded prompt images it read (newest 24), shares a read
  in flight, and never caches a failed read: scrolling a bubble back no longer downloads again.
- The store drops a prompt's kept attachments once its record arrives (the turn holds them),
  drops a returned draft's old id when it goes again under a new one, and keeps at most 8.
- Two copies of one image in a prompt keep two identities (#n in the id).
- A Mac-held image that can't be read says to connect to the Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@azooz2003-bit

Copy link
Copy Markdown
Collaborator Author

Review round 2 (fresh subagent): sent attachments, 3229117fb061..05f0ad0b0987

No blockers. Isolation is fine: the source is @mainactor, so the loader's weak capture is safe. Scope is fine: only acpmux-image: ids are fetched, and acpmux serves only images the session's own log names.

# Finding Resolution (32251815e7b3)
1 Thumbnails downloaded again on every cell rebuild AcpmuxSessionSource caches the 24 newest images it read, shares a read in flight, and never caches a failed read
2 sentAttachments was never pruned An entry is dropped when its record arrives or when a returned draft goes again under a new id; at most 8 are kept (test added)
nit The same image twice in one prompt shared one id The id carries #n, and recordedImageID strips it (test added)
nit A failed Mac fetch said "isn't on this iPhone anymore" It says "Connect to your Mac to see …" (test added)
left A peer Mac's relay refuses _acpmux/attachment Not changed: a peer-Mac session shows a placeholder for history images. A relay pass-through is a trust-policy change, so it is left to its owner

@azooz2003-bit azooz2003-bit added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Oct 23, 2026
@azooz2003-bit

Copy link
Copy Markdown
Collaborator Author

Merge note for head 32251815e7b3:

  • Reds that are not this PR's:
  • Still pending: CmuxMobileKit tests, queued about an hour for capacity. The same suites passed on this head on mini-cmux11s with an iOS simulator: AgentChat 132/132, Live 262/262, Chief 79/79.
  • Real-usage live check: passed (see the description). Both review rounds are resolved.

@azooz2003-bit
azooz2003-bit merged commit 3980404 into feat-cmux-next Oct 23, 2026
154 of 162 checks passed
@azooz2003-bit
azooz2003-bit deleted the feat-cmux-next-ios-composer-files branch October 23, 2026 21:21

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 32251815 Deployed Oct 23, 2026 by azooz2003-bit via namespace autofix #1982
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants