Repository navigation
cmux-next: Mac Screen tabs (see and control another Mac) and Share This Mac - #20426
Closed
azooz2003-bit wants to merge 60 commits into
Closed
azooz2003-bit wants to merge 60 commits into
azooz2003-bit wants to merge 60 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This comment has been minimized.
This comment has been minimized.
Contributor
|
Agents: |
Contributor
|
Probably yours: CI failed on
Matched log linesNot re-run automatically: Written by |
… (crash-class gate); workspace package groups
- Address bar: pairing links and cmux://mac-screen resolve to URLs, never a search (the room secret must not reach a search engine); pairing from the address bar needs a person. - Hidden tabs end their session (and pending reconnects) after the grace period on a DemandTimer; showing them reconnects. - Share This Mac: the menu bar indicator follows the viewer session, events arrive in order, a capture failure ends the viewer session, one permission prompt at a time, no blocking host lookup, an unreadable Keychain never resets the link. - The store reads and writes off the main thread; Mac tabs wait for it. - Other Mac Screen records navigate within the tab; a new-tab open by a person expires; weak session in observation; the card re-lays out. - Page service and debug method live in CmuxNextMacScreen (App size). - ScreenCaptureBackend: no display to capture fails instead of crashing. - Tests: Keychain failure keeps the link; pairing links never search.
- Plain keys (f) always reach the other Mac: only Command/Control chords can stay in cmux as its shortcuts. - Control-Option-Escape releases the keyboard on Mac Screen tabs. - The restored home page fills its tab (hosting view sizing off). - Quitting a sharing cmux ends the viewer session at once. - Renaming a Mac renames its open tabs; Forget asks first; New Mac Screen Tab always opens a new tab in the pane it ran on. - CmuxVNCWebRTC signaling uses an ephemeral session with no URL cache: the room secret header and TURN credentials never reach Cache.db.
This comment has been minimized.
This comment has been minimized.
Collaborator
Author
|
Correctness review (subagent, at 4cdaf56) and resolution The fixes are in 4408cc6 and 69c95b3, together with fixes from a live verification run on 4cdaf56.
Found during live verification and fixed: the letter f never reached the remote (plain keys could match cmux bindings; now only Command and Control chords stay local); Control-Option-Escape did not release the keyboard; a restored home page drew blank; a stale "connected" status after the host app quit; Rename did not update open tabs; Forget had no confirmation. Separately, signaling now uses an ephemeral URLSession with no URL cache, so the room secret header and TURN credentials are no longer written to |
- Control-Option-Escape releases the keyboard as a state: control stays off even when focus returns to the screen, until the next click on it. - A Mac Screen tab never hides its content view (a tab restored hidden collapsed the chrome and drew blank); Connect starts the stream even when no reveal reached the tab. - The failed card never shows the session's raw error (it can quote the signaling URL with the room id); debug state reports only 'failed'. - Share This Mac: Disconnect also turns sharing off (the viewer would reconnect with the link); one Stop Sharing action in the menu bar. - The keyboard hint shows only while the screen is live.
This comment has been minimized.
This comment has been minimized.
…loses the dropped stream Library bug fix (CmuxVNCWebRTC, CmuxScreenKit; both iOS demos and cmux-vnc-host build): ScreenStreamViewer.connect() left its room socket and peer link running after its open timeout, so when the host next joined the leftover viewer offered and opened a session nobody saw (a Share This Mac host reported a viewer that was not there). ScreenSession, giving up after its reconnect limit, also left the dropped stream's room socket reconnecting. Both now close.
A tab restored as its pane's selected tab is shown before the pane has its launch size; its browser chrome laid out at that size kept the stale frames (page area 0 high at the bottom, toolbar under it) until a tab switch. The pane now lays the content subtree out in its own layout pass whenever the host frame or the content frame changes. Seen with restored remote view and Mac Screen tabs.
…k handling - macOS ScreenVideoView enqueues decoded frames from libwebrtc's decode thread into an AVSampleBufferDisplayLayer (hardware frames are already CVPixelBuffers; I420 frames are copied to pooled NV12). RTCMTLNSVideoView drew on the main thread (texture uploads, drawable waits): a 1.97 s and a 323 ms main-thread hang under Animation Hitches. - ScreenCaptureBackend restarts capture when its stream stops with an error and re-reads the displays when they change (a display mode change froze the picture while the session stayed connected). - Copy Link marks the pasteboard item concealed (clipboard managers skip it). - Quitting the viewer ends its sessions, so the host stops showing a viewer.
- Library (additive, off by default): ScreenStreamViewer(pairing:relayOnly:)
and ScreenSession.relayOnly pass ICE transport policy 'relay' to the peer
link (the flag the throughput tests already used). Nothing changes for a
caller that does not set it; only the viewer side is restricted, so the
host keeps its normal candidates.
- DEBUG builds only (compiled out of Release and NIGHTLY): a 'Relay Only
(DEV)' toggle in a Mac Screen tab's quality menu (reconnects the session),
CMUX_NEXT_MAC_SCREEN_RELAY_ONLY=1 to start sessions relay-only, and
debug.mac_screen relay_only {enabled}. The toolbar's path reads Relay.
…elay Only - SettingsSection.development and a Development category (hammer) holding the Mac Screen group; the page export marks it dev_only and the host lists send dev_build (DevTools.isDebugBuild), so the React page shows the category, its search results and its rows only in DEV builds. The schema and the key stay the same in every build (the CLI and React embed one export); only DEV builds apply it. - macScreen.debug.relayOnly (bool, off; userOnly for agents): Mac Screen sessions use the TURN relay only. MacScreenPages follows the setting in DEBUG builds and reconnects running sessions when it changes. - The Relay Only toolbar toggle, the env var and the debug RPC switch from the previous commit are gone: Settings is the one place. - cmux app settings --section development; strings in 21 languages.
This comment has been minimized.
This comment has been minimized.
…ction schemas/settings/settings-schema.json and docs/mdm (exported on the fleet by scripts/measure/export-settings-artifacts.sh), webviews settings strings (gen-strings), the page parity test reads dev_only through devOnly. Settings page tests: 125 pass (bun, run on mini 2: Freestyle was unavailable).
azooz2003-bit
temporarily deployed
to
artifacts
October 14, 2026 18:13 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
azooz2003-bit
temporarily deployed
to
artifacts
October 14, 2026 18:38 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
… Screen fixes - Every route to the Development category (deep link, app settings --section development, settings.open, the relay key's focus link) opens the default page outside DEV builds: the page's shownSection and SettingsWindowService.route both check dev_build. The mock host reports a DEV build. Tests: categories (shownSection), a render test with dev_build false and true, and the Swift route test. - Reset Link re-masks the new link (it was shown with its QR code when Show was on before the reset). - Forgetting a Mac ends its open tabs' sessions; they show not paired. - A confirm or missing card never reports the keyboard as captured.
The callbacks run on the main thread; removing the callback and releasing the backend it held from another thread could free it while a callback was still running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
Independent verification: approved at 86fc8e4 (macscr26)A fresh reviewer, separate from the implementer and from the agent that captured the evidence, approved the 12 acceptance criteria against build Earlier rejections and their fixes. Each fix has a test that failed first.
Evidence from macscr26. Viewer on a cmux Mac mini; Swift and Rust reference hosts plus a cmux-next host on a second mini.
Not covered by runtime evidence
Failing checks. The base
The PR stays a draft until it's dogfooded and the trust decision on #13742 is made. 🤖 Generated with Claude Code |
Routes name only the record (home, mac:<id>), a restored Mac tab waits for Connect, a forgotten Mac's restored tab says it is not paired, the title follows renames, focus goes to the input surface, and visibility follows the view. Red until the page tab lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The home page, the toolbar and the cards resolve their colors in the theme scope they sit in and again when it changes (Settings > Appearance), not macOS's system styling. Red until the themed hosting lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mac Screen was a browser app page (cmux://mac-screen in a browser tab, under the address bar). It is now a native page tab (InternalPageID.macScreen) like Diff and Settings: its own title (Mac Screen, or the Mac's name, live on rename) and remote-display icon, no browser chrome or engine. - Routes are "home" or "mac:<saved Mac id>", never the pairing secret; a relaunched or reopened tab shows that record behind the Connect card. A restored tab of a forgotten Mac says it is not paired. - New Mac Screen Tab, the home page's Open in New Tab, debug.mac_screen open and pair, a cmux://mac-screen address and a pairing link a person types in an address bar all open the page tab (only a person's open connects at once). An old browser tab on a Mac Screen address is replaced by the page tab, waiting for Connect. - The tab view reports when it is shown (in a window and not hidden): hidden pauses (20 s grace), shown resumes. Its focus target is the screen's input surface. Closing the tab ends the session. - Every surface (home page, toolbar, cards, rename sheet) resolves Palette tokens in the tab's theme scope and again on every theme change, with Typography and Metrics, so Settings > Appearance shows at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page is clear over a window backdrop (Palette.paneFill), the rename sheet uses the popup surface, and the tab view draws no black of its own (the screen keeps its black letterbox). Imports the strip tab id alias. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RemoteSystemShortcuts holds the seam with today's behavior (the tap takes nothing, never installs); the tests say when it installs, which chords it takes (the system list, never the release chord or ordinary typing, the system list over cmux bindings) and what it sends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tuple arguments did not type-check; the chords are a small Sendable value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A setting, off by default (macScreen.systemShortcuts, Keyboard > Mac Screen, userOnly for agents), with a toggle in the toolbar's Send Keys menu and on the home page. While it is on, cmux has Accessibility, a connected Mac Screen tab has the keyboard, its window is key and cmux is active, one session keyboard event tap (RemoteSystemShortcutTap) takes the system chords before macOS acts on them and sends them through the tab's input surface: Cmd-Space, Cmd-Option-Space, Control-Space, Cmd-Tab, Cmd-`, Control-arrows, Cmd-Shift-3/4/5, Control-Cmd-Q, Cmd-Q/W/H/M. The tap is removed as soon as any condition stops (focus, the release chord, a tab hidden or closed, the session ending, a window or the app losing focus, the setting), and asks on every key too. Precedence: the system list goes to the other Mac even where it is a cmux binding (Cmd-W, Cmd-Q); other chords keep the capture view's rule, so cmux's other shortcuts stay here. The release chord is never taken. Modifiers and ordinary typing still go through the capture view, and a key the tap took never reaches it, so nothing is sent twice. Without Accessibility the home page and the menu offer Allow Accessibility (the system prompt, else System Settings in the background), rechecked when cmux becomes active. Secure Event Input hides keys from taps; then the chords act on this Mac as without the setting. debug.mac_screen state reports system_shortcuts, system_shortcuts_tap and accessibility. Strings in 21 languages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…shortcuts schemas/settings/settings-schema.json and docs/mdm exported on the fleet by scripts/measure/export-settings-artifacts.sh (step 5fb9dd64c63899dcc2d1b5bd at a82f69f); the settings page strings add the four schema keys of macScreen.systemShortcuts in gen-strings' order and format. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
azooz2003-bit
temporarily deployed
to
artifacts
October 15, 2026 05:45 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
azooz2003-bit
temporarily deployed
to
artifacts
October 15, 2026 05:48 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
…ome tab (red) A Mac opened in place from the home page kept the tab's route "home", so a relaunch showed the home page instead of the Mac. The home page now asks its host to open the Mac's own tab in place of the home tab. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Connect and Pair and Connect on the home page, and Home on a screen, used to swap what the tab shows in place, which kept the tab's store route: a home tab that showed a Mac came back as the home page after a relaunch. Now the host opens the record as a new Mac Screen tab right after it (selected, confirmed by the person: no Connect card) and closes the old tab once the new one exists (a store tab once the daemon made it), so the route names the Mac. The in-place swap stays only as the fallback when no pane lists the tab. debug.mac_screen state rows add route and store_tab (the store tab id, null for an app-only tab that does not restore). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The compiler could not type-check the mapped closure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Showing an ended tab with Relay Only on went to the failed card in half a second: the pane started a single attempt (session.connect) where a relay often needs a few tries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Showing an ended or failed tab, Connect on the card, and the toolbar's Connect and Try Again called session.connect(): one attempt, so a relay that needs a few tries showed the failed card at once. They now go through session.reconnect() (5 attempts with backoff), as the Relay Only switch already did. A tab the person disconnected stays down until Connect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every pane connect now retries, so the session's reconnecting flag is set on a first connect too and the card said Reconnecting. The seam keeps today's behavior; the test fails on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After Connect on the confirm card the tab held the keyboard, so the next keystrokes went to the remote Mac unasked. RemoteInputSurface gains waitForClick() (the release chord's state); the pane does not call it yet, so the test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…te Mac "to This Mac" in a session's menu read as the viewer; the item now matches the setting's title in every locale. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every newly made screen starts with the keyboard here until a click on the picture (only Connect on the card did that), and when a screen of a Mac starts connecting, the other connecting or connected screens of that Mac end, so a hidden tab in its grace no longer takes the host's one viewer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Every newly made screen starts with the keyboard here (released) until a click on the picture, whichever way it opened (home Connect, Pair and Connect, Open in New Tab, a pasted link, New Mac Screen Tab), not only Connect on the card. A reconnect in the same pane keeps the state. - Hosts take one viewer and the newest wins: when a screen starts connecting (showing the tab, Connect, Try Again, a person's new tab), the other connecting or connected screens of the same Mac end now, hidden (as if their grace ran out) or shown (an older split shows Disconnected rather than the two taking the host from each other). Showing an ended tab or pressing Reconnect connects it again and ends the others. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A click on a card's button over the picture sometimes reached the capture view under it, which forwarded it to the other Mac and took the keyboard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The input gate only checked that the screen was open, so the capture view stayed in control mode under every card (connecting, waiting, failed, Disconnected): a click on a card's button that reached it went to the other Mac and took the keyboard. Now control mode and the pointer follow "open and no card", synced whenever the card can change; under a card the capture view is not hit-tested (RemoteInputSurface.acceptsPointer), so a click there neither goes to the host nor takes the keyboard. A screen that had the keyboard before a drop keeps first responder (nothing is sent, keys stay in cmux) and uses it again once the picture is back; a released one still needs a click. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 16, 2026
Collaborator
Author
|
Superseded by #22788, which carries this Mac Screen work plus the device hub (no pairing links), merged with current feat-cmux-next and targeting feat-cmux-next directly. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. This adds Mac Screen tabs to cmux-next on macOS: see and control another Mac from a cmux tab. It also adds Share This Mac, so a Mac running cmux can be viewed by another. The screen code is CmuxScreenKit (
feat-cmux-screen-kit), which this PR is stacked on.What it adds
cmux://mac-screen(the home page):cmux-vnc://pairlink).cmux://mac-screen?mac=<id>(one Mac's screen):ScreenVideoView.RemoteInputSurface): physical keys, IME text, pointer, buttons and scroll. cmux's Command/Control shortcuts stay local; Control-Option-Escape releases the keyboard.ScreenStreamHostwith ScreenCaptureKit and CGEvent in-process.cmux-vnc-hoststill build):ScreenVideoView.ScreenStreamHost.onEventanddisconnectViewer().Cache.db.dev_onlycategory in the settings export,dev_buildfromDevTools.isDebugBuild). It holds Mac Screen > Relay Only (macScreen.debug.relayOnly, off): Mac Screen sessions use the TURN relay only (ICE policyrelay, viewer side), and connected tabs reconnect when it changes. Library: additiveScreenStreamViewer(pairing:relayOnly:)andScreenSession.relayOnly, off by default. Exports regenerated on the fleet (scripts/measure/export-settings-artifacts.sh); settings page tests 125/125.macScreen.open("New Mac Screen Tab"): in the palette, the File menu and the + menu. CLI:tab new-mac-screen.debug.mac_screenandscripts/cmux-next/mac-screen-live.pyfor live checks. Links move only through 0600 files.Verification
Known issues and decisions
RTC*Objective-C classes. Code binds within its own framework, so only lookups by class name are ambiguous, and the live runs work. The clean fix is a WebRTC build with a renamed prefix; that is a library dependency decision.feat-cmux-next.Changelog
🤖 Generated with Claude Code