cua-driver v0.1 — initial public release - #1359
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughIntroduces the Cua Driver—a macOS background computer-use driver that enables Claude to interact with native applications via accessibility APIs and input synthesis. Includes complete Swift package with CLI entry points, core driver library, MCP server implementation with ~25 tools, focus/window management, input handling, cursor overlay, recording/trajectory playback, and comprehensive skill documentation. Changes
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes The diff introduces a complete, production-grade macOS background automation driver spanning 70+ heterogeneous files with logic density in multiple complex domains: Accessibility framework integration with element indexing and caching, private SkyLight API event synthesis, AppKit focus control and window management, CoreGraphics/ScreenCaptureKit image capture, video encoding/decoding via AVAsset, Bezier animation and curve rendering, concurrent actor-based state management, MCP server protocol, CLI argument parsing, daemon IPC, and sophisticated recording/trajectory infrastructure. Each subsystem requires careful reasoning about macOS system behavior, concurrency safety, and error handling. While many tool implementations follow repetitive patterns, the underlying infrastructure (AX tree management, focus suppression, event synthesis, coordinate spaces) demands deep scrutiny due to intricate interactions between accessibility, window state, and input delivery semantics. Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests (beta)
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Note
Due to the large number of review comments, Critical severity comments were prioritized as inline comments.
🟡 Minor comments (29)
libs/cua-driver/Sources/CuaDriverCore/Permissions/Permissions.swift-37-53 (1)
37-53:⚠️ Potential issue | 🟡 MinorDocstring overstates what the return value represents.
AXIsProcessTrustedWithOptionsreturns the trust state at call time, not after the user interacts with the dialog — the prompt is async and Accessibility grants typically require a restart to take effect. Same forCGRequestScreenCaptureAccess. Right after calling these with a missing grant, the bool will almost always befalse, even when the user is about to (or just did) grant access.Consider rewording to "Returns the trust state at call time (not after user interaction)" so callers like
CheckPermissionsTool(which composes a summary immediately after) don't misinterpret the result.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Permissions/Permissions.swift` around lines 37 - 53, The docstrings for requestAccessibility() and requestScreenRecording() overstate the meaning of their return values; update the comments for these functions (requestAccessibility, which calls AXIsProcessTrustedWithOptions, and requestScreenRecording, which calls CGRequestScreenCaptureAccess) to state that the returned Bool is the trust/grant state at call time (the prompt is asynchronous and grants may require restart), e.g. reword to "Returns the trust state at call time (not after user interaction)"; ensure callers like CheckPermissionsTool are not misled by the comment.libs/cua-driver/Sources/CuaDriverCore/CuaDriverCore.swift-3-5 (1)
3-5:⚠️ Potential issue | 🟡 MinorVersion string disagrees with the "v0.1 initial release" label.
CuaDriverCore.version = "0.0.1"is what the MCP server will advertise to clients (per the summary, it's the defaultCuaDriverMCPServerversion). The PR title, branch, and release notes all say v0.1. Since this is the first public release and several downstream consumers (MCP clients, user-facing logs, recordings) are likely to key off this string, it's worth aligning before tagging.- public static let version = "0.0.1" + public static let version = "0.1.0"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/CuaDriverCore.swift` around lines 3 - 5, Update the exported version string on CuaDriverCore so it matches the release label: change CuaDriverCore.version from "0.0.1" to the intended release value (e.g., "0.1.0" or "0.1" depending on your versioning scheme) so the MCP server advertises the correct version to clients; ensure the literal assigned to the public static let version is updated in the CuaDriverCore enum.libs/cua-driver/Sources/CuaDriverCore/Input/SkyLightEventPost.swift-418-427 (1)
418-427:⚠️ Potential issue | 🟡 MinorAvoid force-cast on
kCGWindowNumbervalue.
info[kCGWindowNumber as String] as! Intwill trap if the CF value bridges to anything other thanInt(the underlying type isSInt32/UInt32, and historically it has bridged as differentNSNumberrepresentations across OS versions).CGWindowIDisUInt32, so going throughNSNumber.uint32Valueis both safer and more accurate. A crash here would take down the driver on an otherwise benign window enumeration.🛡️ Suggested fix
- return all.compactMap { info -> CGWindowID? in - guard (info[kCGWindowOwnerPID as String] as? Int32) == pid - else { return nil } - return CGWindowID(info[kCGWindowNumber as String] as! Int) - } + return all.compactMap { info -> CGWindowID? in + guard (info[kCGWindowOwnerPID as String] as? pid_t) == pid, + let num = info[kCGWindowNumber as String] as? NSNumber + else { return nil } + return CGWindowID(num.uint32Value) + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Input/SkyLightEventPost.swift` around lines 418 - 427, The force-cast in windowIDs(forPid:) is unsafe: replace the `info[kCGWindowNumber as String] as! Int` force-cast with a safe extraction that handles NSNumber/CFNumber and uses its uint32Value to create the CGWindowID; e.g., guard-let the value as NSNumber (or CFNumber bridged) and call uint32Value, then return CGWindowID(uint32Value) so the code won't crash if the underlying representation differs across OS versions.README.md-36-36 (1)
36-36:⚠️ Potential issue | 🟡 MinorBroken anchor link: Cua card points to a non-existent section.
The card links to
#cua---agentic-ui-automation--code-execution, but the corresponding section heading on line 68 is## Cua - Agent-Ready Sandboxes for Any OS, whose GitHub-generated anchor is#cua---agent-ready-sandboxes-for-any-os. Clicking the Cua card will not jump to its section.🔗 Proposed fix
- <a href="#cua---agentic-ui-automation--code-execution"> + <a href="#cua---agent-ready-sandboxes-for-any-os">🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@README.md` at line 36, The README has a broken anchor: the card href uses "#cua---agentic-ui-automation--code-execution" but the actual section heading is "## Cua - Agent-Ready Sandboxes for Any OS" (GitHub anchor "#cua---agent-ready-sandboxes-for-any-os"); update the href in the card (the link with anchor "#cua---agentic-ui-automation--code-execution") to the correct anchor "#cua---agent-ready-sandboxes-for-any-os" or alternatively rename the section heading to match the existing anchor so the two identifiers (the card href and the section heading) match.libs/cua-driver/Sources/CuaDriverCore/Input/CursorControl.swift-18-21 (1)
18-21:⚠️ Potential issue | 🟡 MinorCheck the
CGWarpMouseCursorPositionreturn value and document the Space/"no-foreground" contract.Two things worth tightening:
CGWarpMouseCursorPositionreturns aCGError— silently ignoring failures (e.g., when the daemon lacks the necessary permission, or on multi-display edge cases) meansmove_cursorreports success even when the warp was rejected. Propagating or at least logging the error would make tool failures actionable.- The PR summary emphasizes a "no-foreground contract: … the real cursor is not warped." This utility warps the real cursor. That is fine for an explicit
move_cursortool invoked by the user, but a brief doc comment clarifying thatCursorControlis intentionally the exception to the no-warp rule (and must not be used from the background-click path) would help future contributors avoid wiring it into places where it breaks the contract.♻️ Proposed fix
public static func move(to point: CGPoint) { - CGWarpMouseCursorPosition(point) - CGAssociateMouseAndMouseCursorPosition(1) + let err = CGWarpMouseCursorPosition(point) + if err != .success { + // Caller should surface this; warp can fail on permission or + // invalid coordinates and we don't want to claim success. + // Consider logging here once a logger is wired in. + } + CGAssociateMouseAndMouseCursorPosition(1) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Input/CursorControl.swift` around lines 18 - 21, Update the move(to point: CGPoint) implementation to check the CGWarpMouseCursorPosition return CGError and surface failures (either propagate the error from move(to:) or call the process logger/OSLog with the CGError) instead of ignoring it; keep the CGAssociateMouseAndMouseCursorPosition call but only after a successful warp. Also add a brief doc comment on the CursorControl/move(to:) API stating this intentionally warps the real cursor (exception to the "no-foreground" contract) and must NOT be used from background-click or other background paths.libs/cua-driver/Sources/CuaDriverServer/Tools/MoveCursorTool.swift-47-57 (1)
47-57:⚠️ Potential issue | 🟡 MinorDead code:
pointis constructed but never used.
let point = CursorPoint(x: x, y: y)on line 48 is unused — the success message interpolatesxandydirectly. Either drop the binding or use it to format the response (and consider including it as structured content for tool consumers, since the tool's value is returning the new position).♻️ Proposed fix
CursorControl.move(to: CGPoint(x: x, y: y)) - let point = CursorPoint(x: x, y: y) return CallTool.Result( content: [ .text( text: "✅ Moved cursor to (\(x), \(y)).", annotations: nil, _meta: nil ) ] )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/MoveCursorTool.swift` around lines 47 - 57, The code creates an unused CursorPoint instance (let point = CursorPoint(x: x, y: y)) after calling CursorControl.move(to:)—remove the dead binding or use it in the returned CallTool.Result; to fix, either delete the unused let point line or include the constructed CursorPoint in the result (e.g., embed it in the content or metadata of CallTool.Result) so the tool returns the new cursor position; update the MoveCursorTool/CursorControl.call implementation to reference CursorPoint when building the response instead of only interpolating x and y.libs/cua-driver/Sources/CuaDriverServer/Tools/SetAgentCursorEnabledTool.swift-51-65 (1)
51-65:⚠️ Potential issue | 🟡 MinorAvoid partial success when config persistence fails.
The live cursor state flips before persistence. If the config write fails, callers get an error even though the daemon state already changed and will diverge after restart.
Proposed ordering fix
- await MainActor.run { - AgentCursor.shared.setEnabled(enabled) - } - // Persist through to the on-disk config so the next daemon - // restart boots in the same enabled/disabled state. Live - // state already flipped above — this is purely for the - // durability promise. do { try await ConfigStore.shared.mutate { config in config.agentCursor.enabled = enabled } } catch { return errorResult( - "Agent cursor live state updated, but persisting to config failed: \(error.localizedDescription)" + "Persisting agent cursor setting failed: \(error.localizedDescription)" ) } + await MainActor.run { + AgentCursor.shared.setEnabled(enabled) + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/SetAgentCursorEnabledTool.swift` around lines 51 - 65, The code flips live state with AgentCursor.shared.setEnabled(enabled) before persisting via ConfigStore.shared.mutate, causing partial success if the mutate fails; change the ordering so you first persist the new enabled value by awaiting ConfigStore.shared.mutate { config in config.agentCursor.enabled = enabled } and only after that call await MainActor.run { AgentCursor.shared.setEnabled(enabled) }, or alternatively if you must set live state first then catch mutation errors and call AgentCursor.shared.setEnabled(!enabled) to roll back; update the errorResult usage (errorResult(...)) to reflect the chosen strategy so callers don't observe a successful live change with a failed durable write.libs/cua-driver/Sources/CuaDriverServer/Tools/SetRecordingTool.swift-80-88 (1)
80-88:⚠️ Potential issue | 🟡 MinorReject relative or whitespace-only recording directories.
The schema says
output_diris absolute or~-rooted, but the implementation accepts relative paths and resolves them against the daemon’s working directory.Suggested validation
- guard let rawDir = arguments?["output_dir"]?.stringValue, - !rawDir.isEmpty + guard let rawDir = arguments?["output_dir"]?.stringValue else { return errorResult( "`output_dir` is required when enabling recording.") } - let expanded = (rawDir as NSString).expandingTildeInPath + let trimmedDir = rawDir.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmedDir.isEmpty else { + return errorResult("`output_dir` must not be empty.") + } + let expanded = (trimmedDir as NSString).expandingTildeInPath + guard expanded.hasPrefix("/") else { + return errorResult("`output_dir` must be absolute or ~-rooted.") + } let url = URL(fileURLWithPath: expanded).standardizedFileURL🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/SetRecordingTool.swift` around lines 80 - 88, The code currently accepts relative or whitespace-only output_dir values; change the validation in SetRecordingTool (the guard around arguments?["output_dir"] -> rawDir) to first trim whitespace and reject empty/whitespace-only strings, then require that the raw (trimmed) value is either absolute (starts with "/") or begins with "~" before calling (rawDir as NSString).expandingTildeInPath and creating URL/standardizedFileURL; if it fails these checks return errorResult("`output_dir` must be absolute or start with '~' and cannot be relative or empty.").libs/cua-driver/Sources/CuaDriverServer/Tools/TypeTextTool.swift-81-89 (1)
81-89:⚠️ Potential issue | 🟡 Minor
window_idwithoutelement_indexis silently ignored.The current check only catches
element_indexwithoutwindow_id. If a caller passeswindow_idalone (reasonable mistake, the two are paired everywhere else in the surface), it's silently dropped and the write targets whatever happens to be focused. Consider rejecting the incomplete pair symmetrically so the mistake surfaces at call time rather than landing text in an unexpected element.🛠 Proposed fix
- if elementIndex != nil && rawWindowId == nil { - return errorResult( - "window_id is required when element_index is used — the " - + "element_index cache is scoped per (pid, window_id). Pass " - + "the same window_id you used in `get_window_state`.") - } + if elementIndex != nil && rawWindowId == nil { + return errorResult( + "window_id is required when element_index is used — the " + + "element_index cache is scoped per (pid, window_id). Pass " + + "the same window_id you used in `get_window_state`.") + } + if elementIndex == nil && rawWindowId != nil { + return errorResult( + "element_index is required when window_id is supplied; " + + "without element_index the write always targets the " + + "currently-focused element of the pid and window_id is ignored.") + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/TypeTextTool.swift` around lines 81 - 89, The code currently rejects element_index without window_id but silently ignores window_id without element_index; update the validation in TypeTextTool (the block reading arguments?["element_index"]?.intValue and arguments?["window_id"]?.intValue) to symmetrically reject a request that supplies window_id without element_index by returning an errorResult with a clear message (mirror the existing message referring to element_index cache scoping per (pid, window_id) and referencing get_window_state) so callers cannot accidentally pass only window_id and have input routed to the focused element.libs/cua-driver/Sources/CuaDriverServer/Tools/RightClickTool.swift-54-55 (1)
54-55:⚠️ Potential issue | 🟡 MinorDescription promises
modifierforces the CGEvent path — implementation silently drops modifiers in the element path.The tool description says
"modifierforces the CGEvent path (AX doesn't propagate modifier keys)"
butinvokenever inspectsmodifierswhen routing: ifelement_indexis provided, it unconditionally runsperformElementRightClickand the parsed modifiers are discarded without warning. Either drop that sentence from the description, explicitly reject the combination with an error, or fall through toperformPixelRightClickwhenmodifiers.isEmpty == false. Today users asking for, say,shift+right_clickon an element will see the modifier silently ignored.Proposed fix (prefer explicit error or fallthrough)
- if let index = elementIndex, let rawWindowId { + if let index = elementIndex, let rawWindowId, modifiers.isEmpty { return await performElementRightClick( pid: pid, windowId: UInt32(rawWindowId), index: index) } + // element_index + modifiers → fall through to the pixel path, + // which is the only route that propagates modifier keys. return await performPixelRightClick( pid: pid, windowId: rawWindowId.map { UInt32($0) }, x: x!, y: y!, modifiers: modifiers)Note the pixel fallthrough still requires
x/y; if those aren't provided withelement_index + modifier, you'll want to returnerrorResult(...)explaining the combination isn't supported rather than force-unwrap.Also applies to: 139-150
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/RightClickTool.swift` around lines 54 - 55, The implementation of RightClickTool.invoke claims "modifier forces the CGEvent path" but currently ignores parsed modifiers when element_index is present; update invoke to either (A) detect non-empty modifiers with an element_index and return an explicit error via errorResult explaining that element+modifier is not supported, or (B) when modifiers.isEmpty == false and element_index is present fall through to performPixelRightClick (requiring x/y) instead of calling performElementRightClick; ensure you reference and update the logic in RightClickTool.invoke, and handle/validate combinations of modifiers, element_index, and x/y so you never silently drop modifiers (use performElementRightClick, performPixelRightClick, and errorResult as appropriate).libs/cua-driver/Sources/CuaDriverCore/Windows/SpaceMigrator.swift-127-132 (1)
127-132:⚠️ Potential issue | 🟡 MinorEmpty
spacesarray is reported as.onAnotherSpacewith zero ids.If
spaceIDs(forWindowID:)returns a non-nil but empty array (SPI quirk, transient state during window creation, etc.),spaces.contains(active)is false and the function falls through to.onAnotherSpace(currentSpaceID: active, windowSpaceIDs: []), which downstream callers can't act on meaningfully. Treat empty-array the same as SPI-unresolved and return.unknown:Proposed fix
- guard let spaces = spaceIDs(forWindowID: primary.id) - else { return .unknown } + guard let spaces = spaceIDs(forWindowID: primary.id), + !spaces.isEmpty + else { return .unknown } if spaces.contains(active) { return .onCurrentSpace }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Windows/SpaceMigrator.swift` around lines 127 - 132, The function currently treats a non-nil but empty result from spaceIDs(forWindowID:) as .onAnotherSpace with an empty windowSpaceIDs array; change the logic in the method that queries spaceIDs(forWindowID: primary.id) so that after the guard unwrapping you also check if spaces.isEmpty and, if so, return .unknown (same behavior as the nil/SPI-unresolved case) instead of falling through to .onAnotherSpace(currentSpaceID: active, windowSpaceIDs: spaces); reference the call to spaceIDs(forWindowID:), the variables primary.id and active, and the enum cases .unknown and .onAnotherSpace(...) when making the change..github/workflows/cd-swift-cua-driver.yml-119-135 (1)
119-135:⚠️ Potential issue | 🟡 Minor
grep -c … || echo "0"can produce a multi-line result and break the numeric comparison.When
grep -cfinds zero matches it still prints0to stdout and exits with status 1, so the||branch fires and appends another0. The capturedCERT_COUNT/INSTALLER_COUNTthen becomes"0\n0", which causes[ "$CERT_COUNT" -eq 0 ]to error out withinteger expression expectedrather than succeed/fail cleanly. Usegrep -c ... || trueand suppress grep's own exit status, or skip the fallback entirely:Proposed fix
- CERT_COUNT=$(security find-identity -v -p codesigning build.keychain | grep -c "Developer ID Application" || echo "0") - INSTALLER_COUNT=$(security find-identity -v build.keychain | grep -c "Developer ID Installer" || echo "0") + CERT_COUNT=$(security find-identity -v -p codesigning build.keychain | grep -c "Developer ID Application" || true) + INSTALLER_COUNT=$(security find-identity -v build.keychain | grep -c "Developer ID Installer" || true)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/cd-swift-cua-driver.yml around lines 119 - 135, The CERT_COUNT and INSTALLER_COUNT assignments can produce duplicate lines because using "grep -c … || echo '0'" lets grep both output and fail, resulting in values like "0\n0" which break the numeric test; change those assignments so grep's non-zero exit doesn't append another 0 (for example, suppress grep's failure with "|| true" or remove the fallback echo and rely on grep -c alone), ensuring CERT_COUNT and INSTALLER_COUNT are single numeric strings before the numeric comparisons in the script..github/workflows/cd-swift-cua-driver.yml-283-283 (1)
283-283:⚠️ Potential issue | 🟡 MinorBump
softprops/action-gh-releaseto the latest version.
softprops/action-gh-release@v1is outdated and flagged by actionlint as running on a retired runner image. Upgrade to v2.6.2 (the latest stable version) to keep this step runnable on GitHub-hosted runners.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/cd-swift-cua-driver.yml at line 283, The workflow step currently uses softprops/action-gh-release@v1 which is outdated and flagged by actionlint; update the action reference to softprops/action-gh-release@v2.6.2 in the GitHub Actions workflow (replace the uses: softprops/action-gh-release@v1 entry) so the release step runs on supported GitHub-hosted runners and avoids the retired runner image warning.libs/cua-driver/Sources/CuaDriverCLI/ConfigCommand.swift-397-414 (1)
397-414:⚠️ Potential issue | 🟡 MinorDisabling updates leaves the running LaunchAgent loaded until logout.
run()removes the plist from~/Library/LaunchAgents, but an already-loaded LaunchAgent keeps running until the user logs out or explicitly unloads it. Consider callinglaunchctl bootout gui/$(id -u)/com.trycua.cua_driver_updater(or legacylaunchctl unload <plist>before removing) so "disable" is effective immediately. Also note the "will be removed" line prints even when the plist doesn't exist — small UX nit.Sketch
if fileManager.fileExists(atPath: plistPath) { + // Best-effort: unload before removing so the running agent stops now. + let unload = Process() + unload.executableURL = URL(fileURLWithPath: "/bin/launchctl") + unload.arguments = ["bootout", "gui/\(getuid())/com.trycua.cua_driver_updater"] + try? unload.run() + unload.waitUntilExit() do { try fileManager.removeItem(atPath: plistPath) print("Removed LaunchAgent.") } catch { print("Note: Failed to remove LaunchAgent (you can do this manually):") print(" rm \(plistPath)") } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCLI/ConfigCommand.swift` around lines 397 - 414, ConfigCommand.run currently disables auto-update and deletes the plist at plistPath but does not unload a loaded LaunchAgent or guard the "will be removed" message; update run() to (1) if the plist exists call launchctl to unload/bootout the agent for the current user (e.g., "launchctl bootout gui/$(id -u)/com.trycua.cua_driver_updater" or fallback to "launchctl unload <plist>") before removing it, handling errors and logging them via the same print/error path, and (2) only print "The LaunchAgent will be removed from your system." (or similar) when the plist actually exists (use fileManager.fileExists(atPath: plistPath) to decide); keep ConfigStore.setAutoUpdateEnabledSync(false) as-is and ensure plistPath and the agent label "com.trycua.cua_driver_updater" are the referenced symbols..github/workflows/cd-swift-cua-driver.yml-75-84 (1)
75-84:⚠️ Potential issue | 🟡 MinorUnreachable duplicate branch.
Both
elifblocks check the same${{ inputs.version }}expression, so the second branch is dead.workflow_dispatchandworkflow_callboth populateinputs.version, so a single branch suffices.Proposed fix
if [[ "$GITHUB_REF" == refs/tags/cua-driver-v* ]]; then VERSION="${GITHUB_REF#refs/tags/cua-driver-v}" echo "Using version from tag: $VERSION" elif [[ -n "${{ inputs.version }}" ]]; then VERSION="${{ inputs.version }}" echo "Using version from input: $VERSION" - elif [[ -n "${{ inputs.version }}" ]]; then - VERSION="${{ inputs.version }}" - echo "Using version from workflow_call input: $VERSION" else echo "Error: No version found in tag or input" exit 1 fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/cd-swift-cua-driver.yml around lines 75 - 84, The second elif checking the same `${{ inputs.version }}` is unreachable; remove the duplicate branch and consolidate into a single branch that sets VERSION from inputs and logs a clear message (e.g., "Using version from input: $VERSION") so both workflow_dispatch and workflow_call cases are handled by the same condition; update the surrounding conditional in the shell block to only test `${{ inputs.version }}` once and keep the final else to error out if neither tag nor input provided.libs/cua-driver/Sources/CuaDriverCore/Windows/SpaceMigrator.swift-28-54 (1)
28-54:⚠️ Potential issue | 🟡 MinorDeclare
CopySpacesForWindowsFnwithUnmanaged<CFArray>?and usetakeRetainedValue()to properly handle the +1 retained return from this Copy-named SPI.For manually declared
@convention(c)function pointers, Swift cannot infer Core Foundation ownership semantics from naming conventions. The SPISLSCopySpacesForWindowstransfers ownership (Copy → +1 retained), but declaring the return type asCFArray?treats it as Swift-managed with +0 ownership, causing one reference leak per call. Since this function is invoked per window on everylist_windowscall, the leak accumulates over time in long-lived processes.Proposed fix
- private typealias CopySpacesForWindowsFn = `@convention`(c) ( - Int32, Int32, CFArray - ) -> CFArray? + private typealias CopySpacesForWindowsFn = `@convention`(c) ( + Int32, Int32, CFArray + ) -> Unmanaged<CFArray>?At the call site (around line 90):
- let raw = r.copySpacesForWindows(cid, 7, widArray) as? [NSNumber] + guard let unmanaged = r.copySpacesForWindows(cid, 7, widArray) else { return nil } + let raw = unmanaged.takeRetainedValue() as? [NSNumber]Also applies to: 83-93
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Windows/SpaceMigrator.swift` around lines 28 - 54, The CopySpacesForWindowsFn currently returns CFArray? which causes a CF ownership leak for the Copy-named SPI; change its signature to return Unmanaged<CFArray>? (e.g., CopySpacesForWindowsFn = `@convention`(c) (Int32, Int32, CFArray) -> Unmanaged<CFArray>?) and at the call site where you invoke resolved.copySpacesForWindows(...) unwrap the optional Unmanaged and call takeRetainedValue() (safely using optional chaining or guard) to obtain the CFArray with correct +0 ownership before bridging to Swift types; update any handling of the return to account for the Unmanaged-to-CFArray conversion and nil case.libs/cua-driver/Sources/CuaDriverServer/Tools/ClickTool.swift-458-482 (1)
458-482:⚠️ Potential issue | 🟡 Minor
isWindowMinimized(pid:)is dead code and should be removed.The function defined at lines 461-482 is never called anywhere in the codebase. Based on references to
FocusGuardError.windowMinimizedand minimized-window pitfalls in documentation, it appears to be an incomplete advisory feature for pixel clicks (which silently no-op against minimized windows becauseSLEventPostToPiddoesn't reach a non-rendering tree). Either complete the wiring by warning when a pixel click targets a minimized window, or delete the unused helper.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/ClickTool.swift` around lines 458 - 482, The helper isWindowMinimized(pid:) is dead code (never referenced) — either wire it into the pixel-click flow or remove it; the simplest fix is to delete the unused function to avoid dead code. Remove the private static func isWindowMinimized(pid: Int32) implementation and any related unused imports or comments; if you choose to keep behavior instead, call isWindowMinimized(pid:) from the click path where SLEventPostToPid is used and raise or map to FocusGuardError.windowMinimized before attempting a pixel click so minimized windows are detected and handled.libs/cua-driver/Sources/CuaDriverCore/Focus/FocusGuard.swift-70-79 (1)
70-79:⚠️ Potential issue | 🟡 Minor
FocusGuardError.windowMinimizedis never thrown — the minimized branch silently continues.Lines 73–79 detect a minimized window and set
focusState = nil, then letbodyproceed. TheFocusGuardError.windowMinimizedcase at lines 126–131 — whose description provides user-facing remediation ("use type_text_chars/press_key for keyboard input … or unminimize first") — is never produced by this code. No downstream caller (TypeTextTool, SetValueTool, PressKeyTool, ScrollTool, ClickTool, RightClickTool) catches this error specifically, so users cannot access the documented guidance.Either throw the error when the window is minimized and let downstream tools surface the remediation advice, or delete the case to reflect current behavior.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Focus/FocusGuard.swift` around lines 70 - 79, The minimized-window branch in FocusGuard.swift currently sets focusState = nil and proceeds silently; instead, change that branch to throw FocusGuardError.windowMinimized so the user-facing remediation is surfaced. Specifically, inside the async block where you compute window and windowIsMinimized (the code that now does "if windowIsMinimized { focusState = nil } else { focusState = await enforcer.preventActivation(...)}"), replace the nil-assignment with throwing FocusGuardError.windowMinimized so the error propagates to callers (TypeTextTool, SetValueTool, PressKeyTool, ScrollTool, ClickTool, RightClickTool) and their user guidance is reachable.libs/cua-driver/Skills/cua-driver/SKILL.md-23-42 (1)
23-42:⚠️ Potential issue | 🟡 MinorDocument the
open -n -g CuaDriverbootstrap exception.The no-foreground section says every
openinvocation is forbidden, but the prerequisites and management commands later requireopen -n -g -a CuaDriver --args serve. Add an explicit carve-out for launching the driver daemon itself, otherwise the skill contradicts its own startup instructions.Also applies to: 219-224, 232-237
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Skills/cua-driver/SKILL.md` around lines 23 - 42, Add an explicit carve-out in the "no-foreground" section documenting the bootstrap exception for launching the driver daemon: state that while all forms of `open` are forbidden, invoking `open -n -g -a CuaDriver --args serve` (or equivalent `open -n -g CuaDriver`) is allowed for the initial bootstrap/daemon start because it launches the background driver without activating it; reference the `launch_app` guidance and explain that this exception is narrowly scoped to the driver bootstrap only (not general app launches) and must be used only when starting the CuaDriver daemon, keeping the rest of the `open` prohibition intact.libs/cua-driver/Sources/CuaDriverServer/Tools/DoubleClickTool.swift-74-78 (1)
74-78:⚠️ Potential issue | 🟡 MinorFix the
window_idschema description for pixel mode.Line 77 says
window_idis ignored in the pixel path, but lines 241-245 use it to anchor window-local pixel conversion. Update the description so callers know passingwindow_idis useful and recommended for pixel double-clicks.Also applies to: 241-249
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/DoubleClickTool.swift` around lines 74 - 78, The schema description for the "window_id" field in DoubleClickTool.swift incorrectly says it's ignored in the pixel path; instead update the description to state that "window_id" is used to anchor window-local pixel conversion and is recommended (and can be required) for pixel-based double-clicks so callers know to pass it when performing pixel actions; apply this same description update to the other occurrence of the "window_id" schema (the duplicate block that governs pixel conversion/anchoring and the get_window_state/element_index usage).libs/cua-driver/Skills/cua-driver/SKILL.md-244-251 (1)
244-251:⚠️ Potential issue | 🟡 MinorAdd languages to fenced code blocks.
These fences trigger MD040 and lose syntax highlighting. Mark the command examples as
bash.📝 Proposed markdown fix
-``` +```bash open -n -g -a CuaDriver --args serve cua-driver launch_app '{"bundle_id":"com.apple.calculator"}' # → {pid: 844, windows: [{window_id: 10725, ...}]} cua-driver get_window_state '{"pid":844,"window_id":10725}' cua-driver click '{"pid":844,"window_id":10725,"element_index":14}' cua-driver stop ``` @@ -``` +```bash launch_app(target) → pick window_id from the returned `windows` array @@ ``` @@ -``` +```bash # canonical, works in every capture mode — writes the image bytes @@ fi ``` @@ -``` +```bash osascript -e 'tell application "<App Name>" to activate' ```Also applies to: 348-355, 441-452, 715-717
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Skills/cua-driver/SKILL.md` around lines 244 - 251, Add explicit language annotations to the fenced code blocks that contain shell/CLI examples in SKILL.md: change the fences around the block beginning with "open -n -g -a CuaDriver --args serve" and the other command examples to use ```bash; likewise update the fenced blocks that start with "launch_app(target)", the canonical capture snippet (the block commented "# canonical, works in every capture mode — writes the image bytes"), and the osascript activation example to ```bash as well so they pass MD040 and get proper syntax highlighting.libs/cua-driver/Skills/cua-driver/SKILL.md-521-537 (1)
521-537:⚠️ Potential issue | 🟡 MinorReconcile the pixel-click guidance.
Lines 521-537 say screenshot pixels are the click coordinate system, and lines 586-587 say the pixel path animates the agent cursor. Lines 785-792 then say never translate screenshot pixels into clicks and that pixel clicks skip the overlay. Please qualify this section to mean “prefer
element_indexwhen AX is available; use screenshot pixels for pixel-only surfaces.”Also applies to: 586-587, 783-792
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Skills/cua-driver/SKILL.md` around lines 521 - 537, Update the SKILL.md wording to reconcile the pixel-click guidance: state clearly that click({pid, x, y}) (the "pixel path") uses window-local screenshot pixels from get_window_state and is intended only for surfaces that the AX tree cannot target (canvases, video players, WebGL, custom-drawn controls), whereas element_index (the AX path) should be preferred whenever accessibility nodes are available; explicitly note that pixel clicks drive the agent cursor animation and intentionally bypass the accessibility overlay/hit-testing, and recommend passing window_id to pin coordinate conversion to the correct screenshot. Also apply the same clarified language/qualification to the other mentions of the pixel path and overlay behavior (the sections that currently state the pixel path animates the agent cursor and that screenshot pixels should never be translated into clicks) so all references consistently say “prefer element_index when AX is available; use screenshot pixels only for pixel-only surfaces; pixel clicks animate the cursor and skip the overlay; pass window_id to anchor conversion.”libs/cua-driver/Sources/CuaDriverServer/Tools/LaunchAppTool.swift-88-92 (1)
88-92:⚠️ Potential issue | 🟡 MinorDon’t mark URL launches as idempotent.
Line 91 advertises this tool as idempotent, but
urlscan open documents, folders, or browser windows on repeated calls. That can mislead MCP clients into retrying or caching a mutating operation.- idempotentHint: true, // relaunching a running app is a no-op + idempotentHint: false, // url/document handoff can create side effects🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/LaunchAppTool.swift` around lines 88 - 92, The tool's annotations wrongly set idempotentHint: true in the annotations initializer (the LaunchAppTool tool definition), which can cause clients to retry or cache URL launches; change idempotentHint to false (or remove it) in the annotations .init call so URL launches are not advertised as idempotent, leaving readOnlyHint/destructiveHint/openWorldHint unchanged.libs/cua-driver/Sources/CuaDriverServer/Tools/GetWindowStateTool.swift-30-33 (1)
30-33:⚠️ Potential issue | 🟡 MinorAlign the tool description with off-Space behavior.
The description says off-Space windows return
isError: true, but the implementation intentionally accepts them and surfacesoff_space. Update the public tool text so agents do not avoid supported background-window flows.Also applies to: 122-125
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverServer/Tools/GetWindowStateTool.swift` around lines 30 - 33, Update the public description text in GetWindowStateTool to reflect actual behavior: instead of saying off‑Space windows return isError: true, document that the tool accepts window_id values that belong to the pid but are on a different Space and will surface an off_space status (not an error), and update both description blocks in this file (the short tool description and the longer usage text near the bottom) to mention off_space as the returned condition and that callers must handle background/off‑Space windows themselves.libs/cua-driver/Sources/CuaDriverCore/Config/ConfigStore.swift-177-206 (1)
177-206:⚠️ Potential issue | 🟡 MinorAvoid read-modify-write races in synchronous setters.
These setters bypass the actor and rewrite the full config file, so concurrent
config telemetry,config auto-update, or daemonset_configcalls can lose unrelated changes. Add an interprocess file lock or route the synchronous commands through a single serialized helper.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Config/ConfigStore.swift` around lines 177 - 206, The synchronous setters setTelemetryEnabledSync and setAutoUpdateEnabledSync do a read-modify-write without actor serialization, causing lost updates; wrap the whole loadSync -> modify -> encode -> write sequence in an interprocess exclusive file lock (or create/hold a lock file in the same directory) so concurrent CLI/daemon invocations serialize; specifically, before calling loadSync use a POSIX/flock-style exclusive lock on Self.fileURL (or a dedicated lock at Self.configDirectoryURL()), perform the modification and write to Self.fileURL atomically, then release the lock; ensure the lock is obtained and released even on error to avoid deadlocks.libs/cua-driver/Sources/CuaDriverCLI/CuaDriverCommand.swift-368-379 (1)
368-379:⚠️ Potential issue | 🟡 MinorDo not catch and replace the script’s
ExitCode.The
throw ExitCode(Int32(process.terminationStatus))inside thedoblock is immediately caught by the catch-all below, so all script failures becomeExitCode(1)and print a misleading “Error running update script”.Proposed fix
do { try process.run() - process.waitUntilExit() - - if process.terminationStatus != 0 { - print("Update check failed. See /tmp/cua_driver_updater.log for details.") - throw ExitCode(Int32(process.terminationStatus)) - } } catch { print("Error running update script: \(error)") throw ExitCode(1) } + process.waitUntilExit() + + if process.terminationStatus != 0 { + print("Update check failed. See /tmp/cua_driver_updater.log for details.") + throw ExitCode(Int32(process.terminationStatus)) + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCLI/CuaDriverCommand.swift` around lines 368 - 379, The catch-all is swallowing script ExitCode and replacing it with ExitCode(1); change the error handling so the original ExitCode from throw ExitCode(Int32(process.terminationStatus)) is not caught and replaced: either move the process terminationStatus check and throw outside the do-catch, or narrow the catch to only non-ExitCode errors and rethrow if the caught error is an ExitCode (inspect error as? ExitCode and throw it), referencing the existing throw ExitCode(Int32(process.terminationStatus)), the do { try process.run(); process.waitUntilExit() } block, and the current catch { print("Error running update script: \(error)"); throw ExitCode(1) } to implement the fix.libs/cua-driver/Sources/CuaDriverCLI/CuaDriverCommand.swift-45-56 (1)
45-56:⚠️ Potential issue | 🟡 MinorGenerate the MCP config as JSON instead of interpolating strings.
Executable paths can contain quotes or backslashes; direct interpolation can print invalid JSON. Build the object and serialize it.
Proposed fix
// itself was invoked via a `/usr/local/bin/` symlink. let binary = resolvedBinaryPath() - let snippet = """ - { - "mcpServers": { - "cua-driver": { - "command": "\(binary)", - "args": ["mcp"] - } - } - } - """ - print(snippet) + let payload: [String: Any] = [ + "mcpServers": [ + "cua-driver": [ + "command": binary, + "args": ["mcp"], + ] + ] + ] + let data = try JSONSerialization.data( + withJSONObject: payload, + options: [.prettyPrinted, .sortedKeys] + ) + print(String(decoding: data, as: UTF8.self))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCLI/CuaDriverCommand.swift` around lines 45 - 56, The code currently builds JSON by interpolating resolvedBinaryPath() into the multiline string (variable snippet) and printing it, which can produce invalid JSON when the path contains quotes or backslashes; instead construct a Swift Dictionary/struct representing the payload for "mcpServers" -> "cua-driver" with keys "command" and "args", then serialize it to JSON using JSONEncoder (or JSONSerialization.data(withJSONObject:options:)) and print the resulting UTF-8 string, handling/propagating any encoding errors; replace the snippet construction and print(snippet) with the serialization flow so resolvedBinaryPath() is inserted safely as the "command" value.libs/cua-driver/Sources/CuaDriverCore/Cursor/AgentCursor.swift-478-488 (1)
478-488:⚠️ Potential issue | 🟡 MinorAdd guards for non-finite and zero timing values before
UInt64conversion.The JSON schema declares bounds (
glide_duration_ms≥ 50,dwell_after_click_ms≥ 0,idle_hide_ms≥ 100), but the Swift invoke handler does not re-validate after extracting values. Non-finite values (NaN, infinity) are not explicitly prevented by the schema. Additionally, lines 480, 699-701 lack the defensive check present at line 670 (if dwellAfterClickSeconds > 0), creating inconsistency when converting toUInt64.The properties are public and mutable; direct assignment bypasses schema validation. A local defensive helper ensures safety:
Defensive helper
+ private func sleepSeconds(_ seconds: CFTimeInterval) async { + guard seconds.isFinite, seconds > 0 else { return } + try? await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000)) + } + @@ - try? await Task.sleep(nanoseconds: UInt64(duration * 1_000_000_000)) + await sleepSeconds(duration)Apply the same pattern to lines 672 and 701.
Also applies to: 647-647, 670-672, 699-701
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Cursor/AgentCursor.swift` around lines 478 - 488, Add defensive checks to ensure timing values are finite and positive (or non-negative where allowed) before converting to UInt64: validate glideDurationSeconds, dwellAfterClickSeconds, and idleHideSeconds with a helper like isFinitePositive(_:) or isFiniteNonNegative(_:) and return/skip the Task.sleep or UInt64 conversion when invalid (NaN, ±infinity, or zero where zero is disallowed). Apply this pattern around the animate(to:duration:options:) call and the Task.sleep conversion (the sleep at the end of the shown block), and the other places noted (the code paths using dwellAfterClickSeconds and idleHideSeconds) so conversions to UInt64 only occur after the values pass the finite/threshold checks.libs/cua-driver/Sources/CuaDriverCore/Config/CuaDriverConfig.swift-35-39 (1)
35-39:⚠️ Potential issue | 🟡 MinorDoc string contradicts itself.
"Anonymous telemetry opt-out. Default
true(opt-in)" mixes the two postures. WithtelemetryEnabled = trueby default, this is an opt-out posture (telemetry on; user must disable). The(opt-in)parenthetical is the opposite and will confuse readers auditing the privacy default.📝 Suggested wording
- /// Anonymous telemetry opt-out. Default `true` (opt-in) to match - /// lume's posture. Override at run time via + /// Anonymous telemetry, opt-out posture. Default `true` (enabled; + /// user must explicitly disable) to match lume's posture. Override at run time via /// `CUA_DRIVER_TELEMETRY_ENABLED={0|1}` or mutate persistently via /// `cua-driver config telemetry {enable|disable}`.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@libs/cua-driver/Sources/CuaDriverCore/Config/CuaDriverConfig.swift` around lines 35 - 39, The doc comment for the telemetryEnabled property is self-contradictory; update the comment on telemetryEnabled to clearly state the posture and default without mixing opt-in/opt-out terms (e.g., "Anonymous telemetry opt-out. Default true (telemetry enabled)." or invert the default if you meant opt-in), and remove the confusing "(opt-in)" parenthetical so the comment and the property's default value are consistent.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 5e831430-94d5-4302-b945-683b8608f410
⛔ Files ignored due to path filters (14)
img/card-cua-bench.pngis excluded by!**/*.pngimg/card-cua-driver.pngis excluded by!**/*.pngimg/card-cua-lume.pngis excluded by!**/*.pngimg/card-cua-sandbox.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_128x128.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_128x128@2x.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_16x16.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_16x16@2x.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_256x256.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_256x256@2x.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_32x32.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_32x32@2x.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_512x512.pngis excluded by!**/*.pnglibs/cua-driver/App/CuaDriver/AppIcon.iconset/icon_512x512@2x.pngis excluded by!**/*.png
📒 Files selected for processing (119)
.github/workflows/cd-swift-cua-driver.yml.github/workflows/ci-swift-cua-driver.ymlREADME.mdlibs/cua-driver/.gitignorelibs/cua-driver/App/CuaDriver/AppIcon.icnslibs/cua-driver/App/CuaDriver/Info.plistlibs/cua-driver/Package.resolvedlibs/cua-driver/Package.swiftlibs/cua-driver/README.mdlibs/cua-driver/Skills/cua-driver/README.mdlibs/cua-driver/Skills/cua-driver/RECORDING.mdlibs/cua-driver/Skills/cua-driver/SKILL.mdlibs/cua-driver/Skills/cua-driver/TESTS.mdlibs/cua-driver/Skills/cua-driver/WEB_APPS.mdlibs/cua-driver/Sources/CuaDriverCLI/CallCommand.swiftlibs/cua-driver/Sources/CuaDriverCLI/ConfigCommand.swiftlibs/cua-driver/Sources/CuaDriverCLI/CuaDriverCommand.swiftlibs/cua-driver/Sources/CuaDriverCLI/DiagnoseCommand.swiftlibs/cua-driver/Sources/CuaDriverCLI/RecordingCommand.swiftlibs/cua-driver/Sources/CuaDriverCLI/RecordingRenderCommand.swiftlibs/cua-driver/Sources/CuaDriverCLI/ServeCommand.swiftlibs/cua-driver/Sources/CuaDriverCore/AppState/AppState.swiftlibs/cua-driver/Sources/CuaDriverCore/Apps/AppEnumerator.swiftlibs/cua-driver/Sources/CuaDriverCore/Apps/AppInfo.swiftlibs/cua-driver/Sources/CuaDriverCore/Apps/AppLauncher.swiftlibs/cua-driver/Sources/CuaDriverCore/Capture/DebugCrosshair.swiftlibs/cua-driver/Sources/CuaDriverCore/Capture/ScreenInfo.swiftlibs/cua-driver/Sources/CuaDriverCore/Capture/WindowCapture.swiftlibs/cua-driver/Sources/CuaDriverCore/Config/ConfigStore.swiftlibs/cua-driver/Sources/CuaDriverCore/Config/CuaDriverConfig.swiftlibs/cua-driver/Sources/CuaDriverCore/CuaDriverCore.swiftlibs/cua-driver/Sources/CuaDriverCore/Cursor/AgentCursor.swiftlibs/cua-driver/Sources/CuaDriverCore/Cursor/AgentCursorOverlayWindow.swiftlibs/cua-driver/Sources/CuaDriverCore/Cursor/AgentCursorView.swiftlibs/cua-driver/Sources/CuaDriverCore/Cursor/Bezier.swiftlibs/cua-driver/Sources/CuaDriverCore/Cursor/CursorMotionPath.swiftlibs/cua-driver/Sources/CuaDriverCore/Focus/AXEnablementAssertion.swiftlibs/cua-driver/Sources/CuaDriverCore/Focus/FocusGuard.swiftlibs/cua-driver/Sources/CuaDriverCore/Focus/SyntheticAppFocusEnforcer.swiftlibs/cua-driver/Sources/CuaDriverCore/Focus/SystemFocusStealPreventer.swiftlibs/cua-driver/Sources/CuaDriverCore/Input/AXInput.swiftlibs/cua-driver/Sources/CuaDriverCore/Input/CursorControl.swiftlibs/cua-driver/Sources/CuaDriverCore/Input/CursorPoint.swiftlibs/cua-driver/Sources/CuaDriverCore/Input/FocusWithoutRaise.swiftlibs/cua-driver/Sources/CuaDriverCore/Input/KeyboardInput.swiftlibs/cua-driver/Sources/CuaDriverCore/Input/MouseInput.swiftlibs/cua-driver/Sources/CuaDriverCore/Input/SkyLightEventPost.swiftlibs/cua-driver/Sources/CuaDriverCore/Permissions/Permissions.swiftlibs/cua-driver/Sources/CuaDriverCore/Permissions/PermissionsGate.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/ClickMarkerRenderer.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/CursorSampler.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/RecordingSession.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/Render/FrameTransform.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/Render/RecordingRenderer.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/Render/TrajectoryLoader.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/VideoRecorder.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/Zoom/CursorTelemetry.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/Zoom/ZoomMath.swiftlibs/cua-driver/Sources/CuaDriverCore/Recording/Zoom/ZoomRegion.swiftlibs/cua-driver/Sources/CuaDriverCore/Telemetry/TelemetryClient.swiftlibs/cua-driver/Sources/CuaDriverCore/Windows/SpaceMigrator.swiftlibs/cua-driver/Sources/CuaDriverCore/Windows/WindowCoordinateSpace.swiftlibs/cua-driver/Sources/CuaDriverCore/Windows/WindowEnumerator.swiftlibs/cua-driver/Sources/CuaDriverCore/Windows/WindowInfo.swiftlibs/cua-driver/Sources/CuaDriverServer/AppStateRegistry.swiftlibs/cua-driver/Sources/CuaDriverServer/CuaDriverMCPServer.swiftlibs/cua-driver/Sources/CuaDriverServer/DaemonClient.swiftlibs/cua-driver/Sources/CuaDriverServer/DaemonProtocol.swiftlibs/cua-driver/Sources/CuaDriverServer/DaemonServer.swiftlibs/cua-driver/Sources/CuaDriverServer/ToolRegistry.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/CheckPermissionsTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ClickTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/DoubleClickTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/GetAccessibilityTreeTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/GetAgentCursorStateTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/GetConfigTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/GetCursorPositionTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/GetRecordingStateTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/GetScreenSizeTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/GetWindowStateTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/HotkeyTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ImageResizeRegistry.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/LaunchAppTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ListAppsTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ListWindowsTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/MoveCursorTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/PressKeyTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ReplayTrajectoryTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/RightClickTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ScreenshotTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ScrollTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/SetAgentCursorEnabledTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/SetAgentCursorMotionTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/SetConfigTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/SetRecordingTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/SetValueTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/TypeTextCharsTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/TypeTextTool.swiftlibs/cua-driver/Sources/CuaDriverServer/Tools/ZoomTool.swiftlibs/cua-driver/Tests/FocusMonitorApp/FocusMonitorApp.swiftlibs/cua-driver/Tests/FocusMonitorApp/build.shlibs/cua-driver/Tests/ZoomMathTests/ZoomMathTests.swiftlibs/cua-driver/Tests/integration/driver_client.pylibs/cua-driver/Tests/integration/fixtures/interactive.htmllibs/cua-driver/Tests/integration/test_background_focus.pylibs/cua-driver/Tests/integration/test_blender_background.pylibs/cua-driver/Tests/integration/test_check_permissions_cli.pylibs/cua-driver/Tests/integration/test_chrome_minimized_nav.pylibs/cua-driver/Tests/integration/test_click_pixel_ax.pylibs/cua-driver/Tests/integration/test_double_click_delivery.pylibs/cua-driver/Tests/integration/test_list_windows.pylibs/cua-driver/Tests/integration/test_pixel_click_delivery.pylibs/cua-driver/docs/tool-output-format.mdlibs/cua-driver/scripts/CuaDriver.entitlementslibs/cua-driver/scripts/build-app.shlibs/cua-driver/scripts/install-local.shlibs/cua-driver/scripts/install.shlibs/cua-driver/scripts/test.shlibs/cua-driver/scripts/uninstall.sh
| private static let managementSubcommands: Set<String> = [ | ||
| "mcp", | ||
| "mcp-config", | ||
| "call", | ||
| "list-tools", | ||
| "describe", | ||
| "serve", | ||
| "stop", | ||
| "status", | ||
| "recording", | ||
| "config", | ||
| "diagnose", | ||
| "help", | ||
| ] |
There was a problem hiding this comment.
Keep update out of implicit tool-call rewriting.
UpdateCommand is registered, but "update" is missing from managementSubcommands, so cua-driver update rewrites to cua-driver call update and fails as an unknown tool.
Proposed fix
"recording",
"config",
+ "update",
"diagnose",
"help",
@@
case "recording": return TelemetryEvent.recording
case "config": return TelemetryEvent.config
+ case "update": return "cua_driver_update"
default: return TelemetryEvent.guiLaunchAlso applies to: 182-190, 211-222
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libs/cua-driver/Sources/CuaDriverCLI/CuaDriverCommand.swift` around lines 88
- 101, The managementSubcommands set is missing the "update" entry which causes
the CLI to rewrite "cua-driver update" into an implicit tool call (via the
implicit tool-call rewriting logic) and fail; add "update" to the
managementSubcommands Set definition (the same change should be mirrored in the
other managementSubcommands occurrences referenced around the UpdateCommand
registration) so that UpdateCommand is treated as a management subcommand rather
than rewritten to a tool call.
| public static func mainScreenSize() -> ScreenSize? { | ||
| guard let screen = NSScreen.main else { return nil } | ||
| let frame = screen.frame | ||
| return ScreenSize( | ||
| width: Int(frame.width), | ||
| height: Int(frame.height), | ||
| scaleFactor: screen.backingScaleFactor | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Apple Developer Documentation NSScreen main Swift MainActor isolation
💡 Result:
NSScreen.main is a class computed property in AppKit that returns the NSScreen? containing the window with keyboard focus (the screen the user is most likely interacting with). It is not the primary screen (menu bar screen), which is NSScreen.screens[0]. In Swift, access it as NSScreen.main (optional NSScreen?). Official Apple documentation does not mark NSScreen or its methods as @MainActor isolated. AppKit APIs are traditionally main-thread-only, so call from the main actor/queue: use @MainActor, DispatchQueue.main.sync/async, or MainActor.run. Example usage: if let mainScreen = NSScreen.main { let frame = mainScreen.frame // CGRect bounds let visibleFrame = mainScreen.visibleFrame // excluding dock/menu bar print("Main screen frame: (frame)") } Requires NSApplication.shared to exist (auto-created in Xcode apps). NSScreen.screens returns all screens; do not cache as configuration changes trigger NSApplication.didChangeScreenParametersNotification. No evidence of explicit MainActor isolation in docs or sources; safe on main thread per AppKit conventions. For concurrency, wrap nonisolated calls in Task { @MainActor in ... } if needed.
Citations:
- 1: https://developer.apple.com/documentation/appkit/nsscreen/main?language=objc
- 2: https://developer.apple.com/documentation/AppKit/NSScreen
- 3: https://developer.apple.com/documentation/appkit/nsscreen
- 4: https://developer.apple.com/documentation/appkit/nsscreen/screens
- 5: https://developer.apple.com/documentation/appkit/nsscreen/main?changes=_1_4_3
🏁 Script executed:
# Check the actual file content
cat -n "libs/cua-driver/Sources/CuaDriverCore/Capture/ScreenInfo.swift"Repository: trycua/cua
Length of output: 1317
🏁 Script executed:
# Find all call sites of mainScreenSize
rg "mainScreenSize" --type swift -B 2 -A 2Repository: trycua/cua
Length of output: 1190
🏁 Script executed:
# Check for any swift concurrency settings in the project
find . -name "*.swift" -o -name "package.swift" -o -name "*.xcconfig" | head -20 | xargs grep -l "strict" 2>/dev/null || echo "No strict concurrency found in first pass"Repository: trycua/cua
Length of output: 180
🏁 Script executed:
# Check the GetScreenSizeTool.swift file to see the context of the invoke closure
cat -n "libs/cua-driver/Sources/CuaDriverServer/Tools/GetScreenSizeTool.swift"Repository: trycua/cua
Length of output: 1903
🏁 Script executed:
# Check if strict concurrency is configured in those files
grep -n "strict" "libs/cua-driver/Sources/CuaDriverCore/AppState/AppState.swift" -B 2 -A 2Repository: trycua/cua
Length of output: 427
🏁 Script executed:
# Look for Package.swift or build configuration
find . -maxdepth 3 -name "Package.swift" -o -name "*.xcconfig" | xargs cat 2>/dev/null | head -100Repository: trycua/cua
Length of output: 3517
🏁 Script executed:
# Check MCP ToolHandler definition to understand invoke signature
rg "ToolHandler" --type swift -A 5 | head -50Repository: trycua/cua
Length of output: 4691
🏁 Script executed:
# Look for any swiftSettings that enable/disable strict concurrency
find . -name "Package.swift" -exec grep -A 20 "swiftSettings\|unsafeFlags" {} \;Repository: trycua/cua
Length of output: 36
🏁 Script executed:
# Check if there are any `@MainActor` annotations on invoke handlers elsewhere in the codebase
rg "invoke.*{" --type swift -B 5 -A 3 | grep -A 8 "MainActor\|async"Repository: trycua/cua
Length of output: 135
🏁 Script executed:
# Check the actual GetScreenSizeTool.swift structure to see if invoke is declared as async
cat -n "libs/cua-driver/Sources/CuaDriverServer/Tools/GetScreenSizeTool.swift" | head -30Repository: trycua/cua
Length of output: 1278
🏁 Script executed:
# Verify ToolHandler's invoke type signature completely
rg "invoke.*Sendable" --type swift -A 1Repository: trycua/cua
Length of output: 472
Make AppKit screen access main-actor isolated.
NSScreen access requires the main actor; in Swift 6 strict concurrency, accessing main-thread-only AppKit APIs from a nonisolated context will fail type checking and cause runtime issues.
Proposed actor-isolation fix
- public static func mainScreenSize() -> ScreenSize? {
+ `@MainActor` public static func mainScreenSize() -> ScreenSize? {
guard let screen = NSScreen.main else { return nil }
let frame = screen.frame
return ScreenSize(Then update the tool call site:
- guard let size = ScreenInfo.mainScreenSize() else {
+ guard let size = await ScreenInfo.mainScreenSize() else {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libs/cua-driver/Sources/CuaDriverCore/Capture/ScreenInfo.swift` around lines
24 - 31, The function mainScreenSize() accesses AppKit's NSScreen and must be
main-actor isolated; annotate the function with `@MainActor` (or move its body
into a `@MainActor-isolated` helper) so that ScreenInfo.mainScreenSize() is
executed on the main actor, and then update all call sites of
ScreenInfo.mainScreenSize() to call it from the main actor (e.g., mark callers
`@MainActor` or use await/Task { `@MainActor` in ... } as appropriate) to satisfy
Swift 6 strict concurrency rules.
| // Simple blocking connect — we only wait ~250ms so a dead socket | ||
| // doesn't hang the CLI. | ||
| let addrLen = socklen_t(MemoryLayout<sockaddr_un>.size) | ||
| // Set send/recv timeouts so we don't block forever on a broken daemon. | ||
| var tv = timeval( | ||
| tv_sec: Int(timeout), | ||
| tv_usec: __darwin_suseconds_t((timeout - Double(Int(timeout))) * 1_000_000) | ||
| ) | ||
| _ = setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &tv, socklen_t(MemoryLayout<timeval>.size)) | ||
|
|
||
| let result = withUnsafePointer(to: &addr) { addrPtr in | ||
| addrPtr.withMemoryRebound(to: sockaddr.self, capacity: 1) { sockPtr in | ||
| Darwin.connect(fd, sockPtr, addrLen) | ||
| } | ||
| } | ||
| if result != 0 { | ||
| close(fd) | ||
| return nil | ||
| } | ||
| return fd | ||
| } | ||
|
|
||
| private static func writeAll(fd: Int32, data: Data) -> Bool { | ||
| return data.withUnsafeBytes { raw -> Bool in | ||
| guard let base = raw.baseAddress else { return false } | ||
| var remaining = raw.count | ||
| var offset = 0 | ||
| while remaining > 0 { | ||
| let n = write(fd, base.advanced(by: offset), remaining) | ||
| if n <= 0 { return false } | ||
| offset += n | ||
| remaining -= n | ||
| } | ||
| return true | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Confirm no global SIGPIPE handler is installed elsewhere in the driver
# that would already mitigate this, and confirm SO_NOSIGPIPE is available
# (it is on Darwin).
rg -nP --type=swift -C2 '\bSIGPIPE\b|\bSO_NOSIGPIPE\b|\bMSG_NOSIGNAL\b|\bsignal\s*\(\s*SIGPIPE'
rg -nP --type=swift -C2 '\bsocket\s*\(\s*AF_UNIX'Repository: trycua/cua
Length of output: 1470
🏁 Script executed:
# Check the target structure to understand which code is the CLI client
fd -e swift . libs/cua-driver/Sources/CuaDriverServer | head -20
# Check if DaemonClient is used by a CLI target
rg -l 'DaemonClient' --type=swift
# Look at the main CLI entry point to see how DaemonClient is used
find . -name 'main.swift' -o -name '*CLI*' -o -name '*cli*' | grep -i swiftRepository: trycua/cua
Length of output: 1812
🏁 Script executed:
# Check how DaemonClient is used in CLI code
rg -A5 'DaemonClient' libs/cua-driver/Sources/CuaDriverCLI/ --type=swift | head -50Repository: trycua/cua
Length of output: 4165
Set SO_NOSIGPIPE on the client socket to prevent SIGPIPE crash on write().
Two issues in this region:
-
SIGPIPE can crash the CLI (major). If the daemon dies or closes between
connect()at line 93–97 andwrite()at line 111, writing to the UDS raises SIGPIPE and terminates the CLI foreground process. The daemon has a SIGPIPE handler (DaemonServer.swift line 193), but that does not protect the client process. On Darwin, setSO_NOSIGPIPEon the fd immediately after socket creation sowrite()returnsEPIPEinstead:let fd = socket(AF_UNIX, SOCK_STREAM, 0) guard fd >= 0 else { return nil } + +var on: Int32 = 1 +_ = setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &on, socklen_t(MemoryLayout<Int32>.size)) -
SO_SNDTIMEOdoes not boundconnect()(minor). The setsockopt at line 91 applies only to subsequent send/write operations, not toconnect()on a blocking socket. UDS connect is typically instant-or-fail, so this is low risk; update the comment to avoid future misreading.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libs/cua-driver/Sources/CuaDriverServer/DaemonClient.swift` around lines 83 -
118, The client must set SO_NOSIGPIPE on the socket right after creating the fd
so that subsequent write() calls (used by writeAll(fd:data:)) return EPIPE
instead of raising SIGPIPE; add a setsockopt call for SO_NOSIGPIPE immediately
after socket() returns and before connect(fd, …) and keep the existing error
handling (close(fd)/return nil) on connect failure. Also update the comment that
the existing setsockopt(SO_SNDTIMEO) only affects send/write operations and does
not bound connect(), to avoid confusion. Reference: the socket creation site
where fd is returned/used, the connect(fd, sockPtr, addrLen) call, and the
writeAll(fd:data:) helper.
| private func installSignalHandlers() { | ||
| let handler: @convention(c) (Int32) -> Void = { _ in | ||
| // Signal handlers can't call into the actor safely. Hand off | ||
| // to a Task so the actual shutdown runs in Swift-concurrency | ||
| // land, where close(listenFD) is safe. | ||
| Task.detached { await DaemonSignal.fireShutdown() } | ||
| } | ||
| signal(SIGINT, handler) | ||
| signal(SIGTERM, handler) | ||
| signal(SIGPIPE, SIG_IGN) | ||
| DaemonSignal.register(self) | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Are Swift runtime calls such as Task.detached safe to call from a POSIX signal handler on Darwin?
💡 Result:
No, Swift runtime calls such as Task.detached are not safe to call from a POSIX signal handler on Darwin. POSIX signal handlers must only call async-signal-safe functions, which are limited to a specific list of low-level C functions like write(2) and sigaction(2). Higher-level Swift APIs, including those in the concurrency runtime like Task.detached, involve heap allocations, locks, and other non-async-signal-safe operations. Swift concurrency runtime is "intimately tied to threads and memory" and not designed for signal handler contexts (9). Experimental attributes like @_noLocks and @_noAllocation exist for no-lock/no-allocation code in real-time or signal-safe scenarios, but even these do not cover async runtime calls, and Task.detached creates a new top-level task requiring runtime support (2). Signal handlers in Swift should set atomic flags (using async-safe atomics if available) or use Dispatch signal sources for async handling, avoiding direct Swift runtime calls (1,13,9). Apple's Swift runtime crash handler itself uses raw POSIX signal handling without concurrency APIs (26).
Citations:
- 1: https://blog.smittytone.net/2021/07/19/tackle-async-signal-safety-in-swift/
- 2: https://developer.apple.com/documentation/swift/task/detached(priority:operation:)-1g00u
- 3: https://forums.swift.org/t/signal-safe-swift-concurrency/75236
- 4: https://sincereflattery.blog/2021/07/13/signal-handling-in-swift/
- 5: Add support for
@_noLocksand@_noAllocationSwift attributes apple/swift-atomics#63 - 6: https://github.com/swiftlang/swift/blob/main/stdlib/public/runtime/CrashHandlerMacOS.cpp
🌐 Web query:
DispatchSourceSignal Darwin Swift async signal handling best practices
💡 Result:
DispatchSourceSignal is the recommended way for async UNIX signal handling in Swift on Darwin (macOS/iOS). It avoids async-signal-safety issues of synchronous handlers by queuing signals for dispatch queue execution, allowing full Swift code in handlers. Best practices: 1. Ignore the signal first with signal(SIG, SIG_IGN) to prevent default termination. 2. Create source: DispatchSource.makeSignalSource(signal: SIGINT, queue: someQueue) 3. Set handlers: setEventHandler, setCancelHandler. Activate with activate or resume. 4. Use dedicated queue, not main if possible, to avoid blocking. 5. Cannot monitor SIGILL, SIGBUS, SIGSEGV. Not a replacement for sigaction if needing to prevent termination. 6. For Swift concurrency integration, use AsyncStream wrapping DispatchSource (like swift-service-lifecycle/UnixSignals does) or withCheckedContinuation. Example (SIGINT graceful shutdown): import Dispatch signal(SIGINT, SIG_IGN) let source = DispatchSource.makeSignalSource(signal: SIGINT, queue: .global) source.setEventHandler { print("SIGINT received, cleaning up...") // Perform async-safe cleanup exit(0) } source.setCancelHandler { // Cleanup on cancel } source.resume // Keep runloop alive, e.g., dispatchMain or loop For modern async/await servers, use swift-server/swift-service-lifecycle which wraps DispatchSourceSignal into UnixSignalsSequence: AsyncSequence. Example: import UnixSignals Task { for await signal in UnixSignalsSequence(trapping: .sigint, .sigterm) { print("Got (signal), shutting down") break // or cleanup } }
Citations:
- 1: https://developer.apple.com/documentation/dispatch/dispatchsource
- 2: https://blog.smittytone.net/2021/07/19/tackle-async-signal-safety-in-swift/
- 3: https://github.com/swift-server/swift-service-lifecycle/blob/main/Sources/UnixSignals/UnixSignalsSequence.swift
- 4: https://developer.apple.com/documentation/dispatch/dispatchsourcesignal
- 5: https://developer.apple.com/documentation/dispatch/dispatch_source_type_signal
- 6: https://developer.apple.com/documentation/dispatch/dispatchsourceprotocol
- 7: https://developer.apple.com/library/archive/documentation/General/Conceptual/ConcurrencyProgrammingGuide/GCDWorkQueues/GCDWorkQueues.html
- 8: https://github.com/swift-server/swift-service-lifecycle
- 9: https://swift-server.github.io/swift-service-lifecycle/docs/current/Lifecycle/Structs/ServiceLifecycle.html
- 10: https://stackoverflow.com/questions/45713819/trapping-signals-in-a-swift-command-line-application
- 11: https://zenn.dev/moutend/articles/7f9a97f0f54b6a
- 12: https://alejandromp.com/development/blog/swift-catch-os-signals-trap
🏁 Script executed:
# Search the codebase for any existing signal handling patterns or DispatchSource usage
rg -i "dispatchsource" --type swift
rg -i "signal.*handler" --type swift -A 3 -B 1
rg "async-signal-safe|signal.*safe" --type swiftRepository: trycua/cua
Length of output: 2599
Replace the C signal handler with DispatchSourceSignal for async-signal-safe shutdown.
Calling Task.detached from a POSIX signal handler violates async-signal-safety requirements on Darwin. The Swift runtime is not async-signal-safe; only a limited set of C functions can be called from signal handlers. Use DispatchSource.makeSignalSource() instead:
- Call
signal(SIGINT, SIG_IGN)andsignal(SIGTERM, SIG_IGN)to suppress default termination - Create signal sources with
DispatchSource.makeSignalSource(signal:queue:) - Set handlers via
setEventHandler {}to safely trigger shutdown in the dispatch queue context - Alternatively, use
swift-service-lifecycle'sUnixSignalsSequencefor native async/await integration
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libs/cua-driver/Sources/CuaDriverServer/DaemonServer.swift` around lines 184
- 195, installSignalHandlers currently installs a C signal handler that calls
Task.detached (unsafe from a POSIX signal handler); replace it by first
suppressing default handlers with signal(SIGINT, SIG_IGN) and signal(SIGTERM,
SIG_IGN), then create DispatchSource signal sources via
DispatchSource.makeSignalSource(signal:queue:) for SIGINT and SIGTERM, call
setEventHandler { Task { await DaemonSignal.fireShutdown() } } (or dispatch to a
serial queue) to perform the async shutdown safely, keep SIGPIPE ignored with
signal(SIGPIPE, SIG_IGN), and continue to call DaemonSignal.register(self) as
before; update references in installSignalHandlers and ensure the dispatch
sources are retained so they remain active.
| guard let rawPid = arguments?["pid"]?.intValue else { | ||
| return errorResult("Missing required integer field pid.") | ||
| } | ||
| let pid = Int32(rawPid) |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Swift FixedWidthInteger initializer from another integer out of range trap runtime documentation
💡 Result:
In Swift, FixedWidthInteger types (like Int8, UInt16, etc.) conform to BinaryInteger. The initializer init<T: BinaryInteger>(_ source: T) creates a new instance from another binary integer. If the source value is not representable in the target type's range (e.g., converting Int(500) to Int8), it triggers a runtime trap (crash) with an overflow error. This behavior is documented in the official Apple Swift documentation for BinaryInteger: - "If the value passed as source is not representable in this type, a runtime error may occur." - Example: Int16(500) succeeds, but Int8(500) fails, "triggering a runtime error." The source code in Swift standard library (Integers.swift) includes the same documentation comments, confirming: "If the value passed as source is not representable in this type, a runtime error may occur." with examples like UInt32(-500 as Int) erroring. Alternatives to avoid trapping: - init?(exactly:): Returns nil if out of range. - init(clamping:): Clamps to min/max. - init(truncatingIfNeeded:): Truncates or sign-extends bits, no range check. The runtime trap is a deliberate safety feature, appearing as EXC_BREAKPOINT (SIGTRAP) or EXC_BAD_INSTRUCTION (SIGILL) in crash logs, as per Swift runtime error docs. This applies to FixedWidthInteger since they are fixed-size binary integers implementing these init methods with range checks.
Citations:
- 1: https://developer.apple.com/documentation/swift/binaryinteger
- 2: https://sosumi.ai/documentation/swift/binaryinteger
- 3: https://swiftdoc.org/v5.1/protocol/binaryinteger/
- 4: https://sosumi.ai/documentation/swift/binaryinteger/init(_:)-2ln0u
- 5: https://github.com/apple/swift/blob/main/stdlib/public/core/Integers.swift
- 6: https://developer.apple.com/documentation/swift/int
- 7: https://developer.apple.com/documentation/swift/fixedwidthinteger
- 8: https://developer.apple.com/documentation/swift/numericcast(_:)
- 9: https://developer.apple.com/documentation/xcode/addressing-crashes-from-swift-runtime-errors
🏁 Script executed:
# First, let's find and examine the DoubleClickTool.swift file
fd -t f "DoubleClickTool.swift" --exec cat -n {} \;Repository: trycua/cua
Length of output: 15776
Validate integer ranges before narrowing conversions to prevent daemon crashes.
Int32(rawPid) at line 109 and UInt32(rawWindowId) at lines 143 and 147 will trap on out-of-range values, causing the daemon to crash instead of returning isError. Malformed MCP input (negative window_id or out-of-range pid) triggers runtime traps rather than graceful error handling.
🛡️ Proposed validation
guard let rawPid = arguments?["pid"]?.intValue else {
return errorResult("Missing required integer field pid.")
}
+ guard rawPid >= Int(Int32.min), rawPid <= Int(Int32.max) else {
+ return errorResult("pid is out of Int32 range.")
+ }
let pid = Int32(rawPid)
@@
if elementIndex != nil && rawWindowId == nil {
return errorResult(
"window_id is required when element_index is used — the "
+ "element_index cache is scoped per (pid, window_id). Pass "
+ "the same window_id you used in `get_window_state`.")
}
+ if let rawWindowId,
+ rawWindowId < 0 || rawWindowId > Int(UInt32.max)
+ {
+ return errorResult("window_id is out of UInt32 range.")
+ }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libs/cua-driver/Sources/CuaDriverServer/Tools/DoubleClickTool.swift` around
lines 106 - 109, The code narrows integers unsafely (Int32(rawPid),
UInt32(rawWindowId)) which can trap; before converting, validate that
arguments?["pid"]?.intValue and arguments?["window_id"]?.intValue are within the
target ranges (Int32.min...Int32.max for pid and 0...UInt32.max for window_id)
and return errorResult(...) if they are out of range; replace direct
Int32(rawPid) and UInt32(rawWindowId) conversions with guarded checks that only
perform the narrowing when the value is in-range (using the same symbols rawPid,
pid, rawWindowId, windowId and the existing errorResult helper).
| guard let rawPid = arguments?["pid"]?.intValue else { | ||
| return errorResult("Missing required integer field pid.") | ||
| } | ||
| guard let rawWindowId = arguments?["window_id"]?.intValue else { | ||
| return errorResult( | ||
| "Missing required integer field window_id. Use `list_windows` " | ||
| + "to enumerate the target app's windows, or read `launch_app`'s " | ||
| + "`windows` array.") | ||
| } | ||
| let pid = Int32(rawPid) | ||
| let windowId = UInt32(rawWindowId) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Read-only check for similar unchecked narrowing conversions in tool entry points.
rg -nP '\b(Int32|UInt32)\(\s*raw[A-Za-z0-9_]+\s*\)' -C2Repository: trycua/cua
Length of output: 10448
🏁 Script executed:
cat -n libs/cua-driver/Sources/CuaDriverServer/Tools/GetWindowStateTool.swift | sed -n '100,125p'Repository: trycua/cua
Length of output: 1546
Use bounds-checked conversions for numeric narrowing.
The unchecked conversions Int32(rawPid) and UInt32(rawWindowId) will trap on out-of-range input, crashing the daemon instead of returning an error. This pattern is repeated across 10+ tool entry points.
Bounds-safe conversion
guard let rawWindowId = arguments?["window_id"]?.intValue else {
return errorResult(
"Missing required integer field window_id. Use `list_windows` "
+ "to enumerate the target app's windows, or read `launch_app`'s "
+ "`windows` array.")
}
- let pid = Int32(rawPid)
- let windowId = UInt32(rawWindowId)
+ guard let pid = Int32(exactly: rawPid) else {
+ return errorResult("pid \(rawPid) is outside the supported Int32 range.")
+ }
+ guard let windowId = UInt32(exactly: rawWindowId) else {
+ return errorResult("window_id \(rawWindowId) is outside the supported UInt32 range.")
+ }Similar conversions occur in TypeTextCharsTool, RightClickTool, PressKeyTool, HotkeyTool, TypeTextTool, ZoomTool, ScrollTool, ClickTool, DoubleClickTool, SetValueTool, and AppEnumerator.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libs/cua-driver/Sources/CuaDriverServer/Tools/GetWindowStateTool.swift`
around lines 106 - 116, The conversion from rawPid/rawWindowId using
Int32(rawPid) and UInt32(rawWindowId) can trap on out-of-range values; update
GetWindowStateTool to perform bounds-checked narrowing by either using
Int32(exact: rawPid)/UInt32(exact: rawWindowId) or explicitly checking
rawPid/rawWindowId against Int32.min..Int32.max and UInt32.min..UInt32.max
before assigning pid/windowId, and if out of range return errorResult with a
clear message; apply the same pattern to the other tools listed
(TypeTextCharsTool, RightClickTool, PressKeyTool, HotkeyTool, TypeTextTool,
ZoomTool, ScrollTool, ClickTool, DoubleClickTool, SetValueTool, AppEnumerator)
for all conversions from raw numeric arguments to narrower types, referencing
their respective argument keys and using their existing errorResult helper for
failures.
macOS computer-use driver that speaks the Model Context Protocol
over stdio. Drop-in backend for MCP clients (Claude Code, Cursor) or
as part of the cua-computer-server stack; also usable standalone via
a CLI where every MCP tool is a top-level subcommand.
Designed around a strict no-foreground contract: the user's frontmost
app never changes, the real cursor never warps, and the target never
raises or switches Space. You drive a backgrounded macOS app in one
window while your foreground editor keeps typing in another.
Highlights:
- Element-indexed AX actions that work on hidden, off-Space, or
occluded targets — `get_window_state(pid, window_id)` returns a
per-window AX tree (filtered correctly on multi-window apps),
`click({pid, window_id, element_index})` fires the AX action
without cursor move or focus steal.
- Backgrounded pixel clicks via auth-signed SLEventPostToPid with a
yabai-style focus-without-raise primer — same (x, y) addressing
space as the returned screenshot, supports modifiers and count.
- Three capture modes — `vision` (PNG only; default), `ax` (tree
only, no screen-capture hit), `som` (both).
- Chromium / Electron AX support via the private
`_AXObserverAddNotificationAndCheckRemote` SPI so the tree stays
populated without activating the target.
- Pid-mandatory keyboard — every `press_key` / `type_text` routes
through `CGEvent.postToPid` so keys can't leak into the user's
foreground app.
- Agent-cursor overlay that glides to each target before dispatch,
press-in/ripple on landing, idle-hides. Uniform across AX clicks
and pixel clicks.
- Trajectory recording + replay — per-turn folders with app state,
screenshot, action, click marker. Optional video capture with
zoom-on-click render for demos.
- ScreenCaptureKit screenshots defaulting to a 1568-long-side cap
that matches Anthropic's multimodal input limit, so model-picked
pixel coords match the tool's coordinate space.
See libs/cua-driver/README.md for the full feature list plus
comparison against Codex Computer Use and Claude Computer Use, and
libs/cua-driver/Skills/cua-driver/SKILL.md for the canonical action
loop.
Co-Authored-By: Sarina Li <sarinajin.li@gmail.com>
Co-Authored-By: Dillon DuPont <ddupont@mit.edu>
Co-Authored-By: Claude <noreply@anthropic.com>
Replaces the POSIX signal handler that called Task.detached (not async-signal-safe per Apple's Swift concurrency docs) with a DispatchSource-based pattern. SIGINT/SIGTERM routed through dispatch sources on a user-initiated queue; SIGPIPE continues to be SIG_IGN'd so write() returns EPIPE cleanly. Addresses CodeRabbit #4 on PR #1359. Co-Authored-By: Claude <noreply@anthropic.com>
som returns tree + screenshot, so element_index clicks — the driver's primary addressing mode — work on the first get_window_state call without any configuration. vision (PNG only) stays available as opt-in for vision-first VLM pipelines that specifically don't want the AX walk. The previous vision default meant a user who installed the driver and ran the canonical snapshot-then-click loop ended up on the pixel-click fallback path instead of the element-indexed primary path, which made the driver's distinguishing feature invisible out of the box. Co-Authored-By: Claude <noreply@anthropic.com>
The bounds-checked-narrowing sweep caught the 11 tools CodeRabbit called out but missed `ListWindowsTool.swift`, which had the same `Int32($0)` trap in its `pidFilter` path (filters the window list against a caller-supplied pid). A malformed `pid` over Int32.max would crash the daemon instead of returning a structured error. Adds a `private static func errorResult` alongside the existing `summary` helper so the same pattern the other tools use is available here. Co-Authored-By: Claude <noreply@anthropic.com>
Adds the cua-driver docs subtree to docs/content/docs/cua-driver/,
matching the lume structure (guide + reference sections, no
examples for v0.1). Registers cua-driver in the root docs
meta.json alongside cua/cuabench/cuabot/lume.
- guide/getting-started/{introduction,installation,quickstart,comparison,faq}.mdx
- reference/{cli-reference,mcp-tools,limits}.mdx
- Fumadocs format: frontmatter + Callout imports matching the
lume pages; no VersionHeader yet (only one shipped version).
Co-Authored-By: Claude <noreply@anthropic.com>
Matches the cuabench / lume pattern: bare `/cua-driver`, the `/guide` section root, and the `/reference` section root each redirect to their first content page. Before this, hitting `/docs/cua-driver` returned a 404 — the docs tree existed but the section landing redirects in middleware.ts hadn't been added alongside the new content. Co-Authored-By: Claude <noreply@anthropic.com>
…e-speed render (#1360) **Cursor** - AgentCursorView/AgentCursorRenderer: rewrite agent cursor rendering with SwiftUI Canvas + Dubins-path motion engine (arc → straight → arc) - AgentCursor.animate(): always arrive at 45° (upper-left tip), approaching targets from the lower-right — consistent visual signature on every click **Recording** - ToolRegistry: capture CLOCK_UPTIME_RAW before handler.invoke() so the recorded span brackets the full animation time (t_start_ms_from_session_start) - RecordingSession: add lastAutoRenderURL + auto-render to recording_rendered.mp4 on stop; embed display_scale_factor in session.json - TrajectoryLoader: add loadActionSpans() — walks all turn-*/action.json and extracts ActionSpan{startMs, endMs, windowBounds, clickPoint}; read displayScaleFactor from session.json - ActionSpan (new): ClickPoint, FocusWaypoint; ActionSpanGenerator with padMs=500, fastSpeed=8×, mergeGapMs=5000; per-span focus waypoints for smooth camera pan on merged spans **Video post-processing** - RecordingRenderer: window-bbox zoom (ZoomRegion from windowBounds, letterbox aspect ratio, 400ms eased); variable-speed PTS remapping (1× inside spans, 8× outside); Metal-backed CIContext for GPU frame processing; falls back to legacy click-zoom when no action spans present - SetRecordingTool: report rendered path in stop confirmation message Co-authored-by: cua <cua@cua.localdomain> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds cua-driver alongside Cua / Cua Bench / Lume / Cua-Bot in the docs header nav. Ships Guide + Reference tabs (no Examples in v0.1). Icon is a koala-astronaut mascot provided by Francesco, tinted into black and white PNG variants at build time so both light and dark themes have a readable contrast. SVG equivalents would be nicer (crisper at high DPI, smaller payload) — left as a design follow-up. Co-Authored-By: Claude <noreply@anthropic.com>
"Background computer-use" reads punchier and matches the length of the other product descriptions in the dropdown (Benchmarking toolkit, macOS VM CLI and Framework, etc.). Co-Authored-By: Claude <noreply@anthropic.com>
Add two entries to .lycheeignore to unbreak the Check External Links (lychee) job on PR #1359: `https://openai.com/*` (openai.com returns 403 to automated checkers on URLs like /codex but loads fine in a browser — flagged from docs/content/docs/cua-driver/guide/getting-started/comparison.mdx) and the self-referential install.sh raw-content URL used by the CLI reference page (docs/content/docs/cua-driver/reference/cli-reference.mdx), which 404s only because the install script doesn't exist on main yet and will resolve once this PR merges. The internal link check failure (`/cua/guide/get-started/self-hosted-sandboxes` in docs/content/docs/cua/guide/sandbox/lifecycle.mdx, introduced by PR #1228) is pre-existing on main and out of scope for this PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Swift 6.1 on CI flags `NSApplication.shared`, `setActivationPolicy`, and `.run()` as main-actor-isolated calls from a nonisolated context. Our local Swift 6.3 toolchain was more permissive here, so the errors only surfaced on CI. Compiler-suggested fix: annotate the helper itself with `@MainActor`. All callers are already on the main actor (this is the AppKit bootstrap path), so isolation is tightened without behavior change. Co-Authored-By: Claude <noreply@anthropic.com>
2 participants
Summary
cua-driver - background computer-use driver for any agents.
A macOS driver that lets any agent (Claude, GPT, Gemini, Codex, OpenCode, custom loops) drive a real Mac app in the background while the user keeps working. Speaks MCP over stdio as a drop-in for Claude Code, Cursor, or the cua-computer-server stack, and ships a CLI where every MCP tool is a top-level subcommand.
Built around a strict no-foreground contract: the user's frontmost app never changes, the real cursor never warps, and the target never raises or switches Space. The agent drives one window; the user keeps typing in another.
This PR lands the entire
libs/cua-driversubtree in a single squashed commit. Full changelog lives in the commit body.Highlights
click({pid, window_id, element_index})fires the AX action without cursor move or focus steal.SLEventPostToPidwith a yabai-style focus-without-raise primer. Same (x, y) space as the returned screenshot; supports modifiers and click count.vision(PNG only, default),ax(tree only, no screen-capture cost),som(both)._AXObserverAddNotificationAndCheckRemoteSPI so the tree stays populated without activating the target.press_key/type_textroutes throughCGEvent.postToPid, so keys can't leak into the user's foreground app.Action-loop conventions live in
libs/cua-driver/Skills/cua-driver/SKILL.md. Full technical notes (SkyLight SPIs, yabai recipe, comparison vs Codex Computer Use / Claude Computer Use) ship in the release blog post.Try it
Or point your MCP client at
cua-driver mcp.Test plan
libs/cua-driver/scripts/install.shon a clean Mac: builds without warnings, TCC dialogs appear for Accessibility + Screen Recordingcua-driver check_permissionsreturns both grantstruescripts/test.sh(Python integration suite against the built binary) passes end-to-endcua-driver recording start <dir>→ drive an app →recording stopproducessession.json+ per-turn folders + optional zoom-renderedrecording.mp4🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Chores