Repository navigation
cmux NEXT: agents can screenshot windows, panes and tabs; browser pages come back masked - #23972
Conversation
…es come back masked Socket runs of window.screenshot, pane.screenshot and tab.screenshot are no longer person-only. Any run the control socket carries (whatever origin it claims) gets the window without child or floating windows, with every browser page area, agent chat and conversation filled with an opaque mask read from the view tree on every run loop turn around the capture. A window that keeps animating or resizing, or a mask that cannot be painted, refuses the capture. A browser tab no window shows stays refused to automation. Design mode and the raw page captures stay person-only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a16047-2854-7333-9aff-632422c609d7&host=cmux16
|
Owned by an agent session · session |
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
Not verified: macOS jobs did not run: compile and app tests were skipped on CI passes on Written by red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: green on the base 79cdae2; do not merge until green; see lane-rules |
|
Found 8 test failures on Blacksmith runners: Failures
|
|
|
…ug.toast runs Review findings on the masked socket capture: - App pages are masked unless listed in ScreenshotMask.shownAppPages (keys, changelog, icon picker, apps, diff, markdown, editor, files, debug settings), so Settings, Passwords, History, Home Channels, Agent Activity, the Chief pages and any new page are masked by default. - The Home list, dock, draft and transcript views are masked too (the list shows each conversation's latest message). - debug.toast's runs are socket runs: a screenshot it starts stays masked. - The handler's raw-or-masked choice is tested through ScreenshotHandlers.shown with real invocations, pane crop included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a16047-2854-7333-9aff-632422c609d7&host=cmux16
…fence withExtendedLifetime with an async body does not compile on CI's SDK. The header now names everything a socket capture masks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a16047-2854-7333-9aff-632422c609d7&host=cmux16
…l reader Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a16047-2854-7333-9aff-632422c609d7&host=cmux16
|
Bead |
|
Orphaned: no activity for 16 h, and no lane holds a lease on it. The orphan sweep could not merge |
…nshotPrivacyMask #23983 landed the same feature (agents screenshot the app, browser pages blacked out) five hours after this PR opened, so base now has a second, weaker mask. Resolved by keeping this PR's ScreenshotMask, which does everything ScreenshotPrivacyMask does and more, and deleting the duplicate: - the mask comes from the window's view tree, so a page a pane holds while its next tab paints, a sleeping page's picture and a page under no chrome view are covered; chats, Home views and every app page not in `shownAppPages` too; - it samples each run loop turn across the capture, retries an unsettled window and refuses rather than return pixels it could not mask; - `viaControlSocket` marks every socket run, so a caller claiming `origin: "user"` no longer gets the raw image (base's `applies(to:)` let it through). Kept from #23983, which this PR's `isPersonsRun` did not cover: a run that names a caller (`callerApp`, `callerAgentSession`, `callerTerminal`) is masked too, even carrying `user` from the person's gesture inside it. Takes back #23983's refusal of pane/tab screenshots over the socket when the pane or tab shows a browser: masking the page area whole keeps that guarantee without the refusal, which is this PR's point. `handlers.screenshot.browserNeedsREPL` went with it. Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a174ff-00c4-7cf2-b0d3-58b1149a6809&host=huge-blue
…pplies the view rules The app-size guardrail refuses new logic in CmuxNextApp, and refuses appending it to an existing App file too. ScreenshotMask could not move whole: it names BrowserChromeView, AgentPaneView, the Home views, PageWebView and PaneContentView, which the App owns until the Workbench rows of the refactor plan move. So the geometry and the timing (the regions, the settle wait, the attempts, the run-loop Sampler, whether a view's pixels are where its frame says) go to CmuxNextScreenshots beside WindowImageLayer, and the view types stay in the App as ScreenshotMask.Rules.app in AppWindowImage.swift: covers, area, isAppPage, coversContents. CmuxNextScreenshots gains one dependency, CmuxNextWakeups, for the DemandTimer the settle wait uses. Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a174ff-00c4-7cf2-b0d3-58b1149a6809&host=huge-blue
|
|
The fleet compile at 018aac1 found the two files the move left without it: ScreenshotHandlers' `catch let failure as ScreenshotMask.Failure` and ScreenshotStrings' `reason(_ failure: ScreenshotMask.Failure)`. Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a174ff-00c4-7cf2-b0d3-58b1149a6809&host=huge-blue
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
|
Review (Codex gpt-6-astra high fast, review-router): SHIP at 331c0be Independent final-head review: no blocking findings. Reviewed full diff and supporting code, including privacy masking, socket-origin handling, tests, and base-merge correctness. Static checks passed; macOS display tests were not run in this environment. |
|
Review (codex gpt-6-astra high, next-review): BLOCK at 331c0be
|
|
Review (Codex gpt-6-astra high fast, review-router): SHIP at 5aba021 Independent final-head verdict: the exact head masks |
…hose areas move An independent review at 7d7ec32 blocked on two fail-open paths: - a native page tab (InternalPageView without a PageWebView: Phone and Mac Screen frames, Passwords, Stashed, a new page) was not masked, against the rule that an app page is masked until someone checks it. It is now covered whole unless its page id is in shownNativePages. - the sampler kept every reading's areas but never noticed an area that moved between readings, whose pixels can sit between two frames. Such a move now makes the mask unsettled, so the capture retries or refuses. Agent-Session: https://cmux.com/deeplink/agent?id=aid-85ca59f74f02d81734c3 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review (Codex gpt-6-astra high fast, review-router): SHIP at 5d6d95a Round 1 at
Round 2 at Focused fleet run on a display worker with
|
The generated-files check found schemas/defaults/cmux-next.json stale at 5d6d95a. Only this PR's 24 keys (the screenshot mask, handlers and strings, DebugToast.run) are updated, from the shared build's regenerated file; keys that are equally stale on the base are left for the base's own fix. Agent-Session: https://cmux.com/deeplink/agent?id=aid-85ca59f74f02d81734c3 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review (Codex gpt-6-astra high fast, review-router): SHIP at e6b4f18
Red matching by test name at |
Only schemas/defaults/cmux-next.json conflicted: it takes the base's snapshot plus this PR's own 24 screenshot keys. Agent-Session: https://cmux.com/deeplink/agent?id=aid-85ca59f74f02d81734c3 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review (Codex gpt-6-astra high fast, review-router): SHIP at a752545
|
|
Shipped in:
|
|
Shipped in:
|
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
Summary
Agents and scripts can capture windows, panes and tabs with
cmux window|pane|tab screenshot. Automation receives an image with browser chrome, tab strips, chats, saved chat frames and unchecked app pages filled opaque gray. Raw page capture remains restricted to the browser host's guarded path.The screenshot uses the live view tree and samples mask regions across capture. It omits child/floating windows and refuses an image when masking fails or geometry does not settle. Every control-socket invocation is marked as automation, including one claiming
origin: user; only an unmarked in-app user invocation without caller metadata receives raw pixels.The existing screenshot mask from #23983 is consolidated into
ScreenshotMask. Geometry/timing live inCmuxNextScreenshots; app-specific view rules remain inAppWindowImage.swift.Review fix
Requested: “mask the strip, or mask by page-derived content instead of by one view” and “add a test where a tab title is masked in a window and a pane capture”.
TabStripViewis masked independently ofBrowserChromeView, covering the sibling strip in the default pane layout. The separate saved-launch-image rule remains reachable. Regression coverage verifies raw title/launch sentinels are visible, then masked in full-window and offset pane captures; a separate geometry test requires both mask regions. Crop assertions account for the pane's flipped coordinate system.Validation
7d7ec329, then5d6d95a5fixed two fail-open review findings: unchecked native page tabs are now masked whole (shownNativePages), and an area that moves between readings now refuses the capture.CMUX_TEST_REQUIRE_GUI=1) at5d6d95a5: MaskedScreenshotTests 12/12 and ActionOriginTests 11/11 pass. On base, ActionOriginTests 11/11 pass.5d6d95a5.Risk: high, because this changes automation screenshot privacy. Each capture owns a sampler stopped with
defer; settling uses a one-shot timer, and capture retries stop after three attempts. Frames are compared within one sampler only.Proof
No capture: this changes the returned automation image rather than app UI. The display regression captures a real window and checks raw versus masked pixels.
Changelog
Changed: agents and scripts can screenshot windows, panes and tabs with sensitive page and chat regions masked.
Agent session: agent session