feat(computer-use): precise AX-geometry element bounds via Swift AX helper

[nathansmithopenclaw-alt]

What does this PR do?

cua-driver's get_window_state returns AX elements whose bounds are frequently zero or imprecise, so the set-of-marks (SOM) index it produces can't be trusted for coordinate clicking. This adds a Swift Accessibility-API helper (ax_geometry_helper.swift) that resolves real on-screen element geometry and merges/synthesizes coordinate-backed bounds into the element list, with graceful fallback when geometry is unavailable. The result is a SOM index whose coordinates are reliable enough to click.

Why this approach: the cua-driver text/AX-tree bounds are the wrong source of truth for clicking (they're derived, not measured). Querying the macOS AX API directly for the target window's element rects gives measured geometry, and merging it onto the existing element indices keeps the SOM stable while making its coordinates trustworthy. It degrades cleanly (elements are tagged with a geometry_status/source, and "bounds unavailable" is surfaced) so nothing breaks when the helper can't run.

Related Issue

No prior issue — happy to open one if you'd prefer to track it there.

Type of Change

• ✨ New feature (non-breaking change that adds functionality)

Changes Made

• tools/computer_use/ax_geometry_helper.swift — new Swift helper querying the macOS AX API for element geometry (role, label, bounds), depth/node-capped and timeout-bounded; compiled on demand.
• tools/computer_use/cua_backend.py — parse structured elements/ui_elements with geometry, normalize screen→window bounds, and when bounds are missing/zero run the AX helper to fill (_merge_ax_geometry) or synthesize (_elements_from_ax_payload) coordinate-backed elements; each element is tagged with geometry status/source. Also memoizes the backend cache only after a successful start(), and routes clicks through synthesized helper elements.
• tools/computer_use/tool.py — surface the geometry source and a "bounds unavailable" marker in the element summary.
• tests/tools/test_computer_use_geometry_contract.py — contract tests for the parse / merge / synthesize / fallback paths (no live GUI required; uses fake sessions + unit helpers).
• scripts/computer_use_geometry_benchmark.py — small benchmark to measure geometry coverage across apps.

How to Test

1. pytest tests/tools/test_computer_use_geometry_contract.py tests/tools/test_computer_use.py -q → all pass.
2. On macOS with cua-driver installed, run a SOM capture against a GUI app (e.g. TextEdit / iTerm2) and confirm element bounds are populated (source cua_driver.* or the AX helper) instead of (0,0,0,0). python scripts/computer_use_geometry_benchmark.py reports per-app element/bounds coverage.

Platforms tested: macOS (Apple Silicon). The helper is macOS-only, consistent with the existing cua-driver backend — no behavior change on Linux/Windows (geometry simply reports unavailable).

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (feat(computer-use): …)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this feature
• I've run the relevant tests (test_computer_use_geometry_contract.py, test_computer_use.py) and they pass
• I've added tests for my changes

[teknium1]

Superseded by #50552 — a comprehensive cross-platform cua-driver implementation (macOS/Windows/Linux) now merged to main. It covers the platform this PR targeted. Thanks for the contribution; closing as superseded.