trycua · f-trycua · May 1, 2026 · May 1, 2026 · May 1, 2026 · May 1, 2026
diff --git a/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx b/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx
@@ -90,7 +90,43 @@ Tools appear prefixed as `mcp_cua-driver_*`.
 cua-driver mcp-config --client opencode
 ```
 
-Paste the output into your `opencode.json`.
+Paste the output into `~/.config/opencode/config.json` (global) or `opencode.json` at the project root.
+
+<Callout type="warn">
+**Always configure cua-driver as an MCP server — never rely on the CLI fallback.** If MCP is not wired up, OpenCode calls `cua-driver` as a shell subprocess. The `get_window_state` response no longer includes base64 by default, but the screenshot image block is silently dropped — the model receives only the AX tree with no visual context. Use `--screenshot-out-file` or the `screenshot_out_file` param to preserve the image when using the CLI path.
+</Callout>
+
+### Local vision models (Ollama)
+
+If you are using a vision-capable model via Ollama, you must also declare its input modalities in `config.json` — otherwise OpenCode strips images before they reach the model:
+
+```json
+{
+  "mcp": {
+    "cua-driver": {
+      "type": "local",
+      "command": ["/Users/you/.local/bin/cua-driver", "mcp"],
+      "enabled": true
+    }
+  },
+  "provider": {
+    "ollama": {
+      "npm": "@ai-sdk/openai-compatible",
+      "options": { "baseURL": "http://localhost:11434/v1" },
+      "models": {
+        "gemma4:26b": {
+          "modalities": {
+            "input": ["text", "image"],
+            "output": ["text"]
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+The `modalities` field is required because OpenCode's `@ai-sdk/openai-compatible` provider defaults to text-only when no capabilities are declared. Without it, screenshots are replaced with an error string and never reach the model.
 
 ## Hermes (NousResearch)
 

diff --git a/docs/content/docs/cua-driver/reference/cli-reference.mdx b/docs/content/docs/cua-driver/reference/cli-reference.mdx
@@ -182,7 +182,7 @@ Pipe to `pbcopy` to put any output on the clipboard, or pass the printed `claude
 
 ### cua-driver recording start
 
-Enable the trajectory recorder. Every subsequent action-tool call (`click`, `right_click`, `scroll`, `type_text`, `type_text_chars`, `press_key`, `hotkey`, `set_value`) writes a numbered turn folder under `<output-dir>`.
+Enable the trajectory recorder. Every subsequent action-tool call (`click`, `right_click`, `scroll`, `type_text`, `press_key`, `hotkey`, `set_value`) writes a numbered turn folder under `<output-dir>`.
 
 ```bash
 cua-driver recording start ~/cua-trajectories/demo1
@@ -346,7 +346,6 @@ The following MCP tools are callable via `cua-driver <tool>`. For input schemas
 - `check_permissions` — Accessibility + Screen Recording TCC status.
 - `get_screen_size` — main display size in points + scale factor.
 - `get_cursor_position` — current mouse cursor position.
-- `get_accessibility_tree` — lightweight desktop snapshot (apps + visible windows).
 
 **App lifecycle**
 
@@ -368,8 +367,7 @@ The following MCP tools are callable via `cua-driver <tool>`. For input schemas
 
 **Keyboard**
 
-- `type_text` — insert text via `AXSelectedText`. Pid-scoped.
-- `type_text_chars` — character-by-character via `CGEvent.postToPid`. Reaches Chromium/Electron inputs.
+- `type_text` — insert text via `AXSelectedText` with automatic CGEvent fallback for Chromium/Electron inputs. Pid-scoped.
 - `press_key` — single key press. Pid-scoped.
 - `hotkey` — modifier combo (e.g. `["cmd","c"]`). Pid-scoped.
 

diff --git a/docs/content/docs/cua-driver/reference/mcp-tools.mdx b/docs/content/docs/cua-driver/reference/mcp-tools.mdx
@@ -58,16 +58,6 @@ Return the current mouse cursor position in screen points (top-left origin).
 {}
 ```
 
-### get_accessibility_tree
-
-Lightweight desktop snapshot: running regular apps and on-screen visible windows with bounds, z-order, and owner pid. For a single window's internal UI, use `get_window_state`.
-
-**Arguments:** none.
-
-```json
-{}
-```
-
 ### screenshot
 
 Raw ScreenCaptureKit capture. Full main display, or a single window when `window_id` is set. Returns an image content block plus a text summary listing on-screen windows.
@@ -271,30 +261,18 @@ Modifier combo as a single array, e.g. `["cmd", "c"]`. Requires at least two ent
 
 ### type_text
 
-Insert text at the target's current cursor via `AXSelectedText`. Fast (single AX write) but skipped by apps with custom text layers; for Chromium / Electron inputs use `type_text_chars`.
+Insert text at the target's current cursor. Attempts `AXSelectedText` write first (fast, single AX call); automatically falls back to character-by-character `CGEvent.postToPid` synthesis when the target input doesn't implement `AXSelectedText` — this makes it work for Chromium and Electron inputs without any manual switching.
 
 **Arguments:**
 
 - `pid` (integer, required): Target process ID.
 - `text` (string, required): Text to insert at the target's cursor.
 - `element_index` (integer, optional): When present, the element is focused before the write. Requires `window_id`.
 - `window_id` (integer, optional): Required when `element_index` is used.
+- `delay_ms` (integer, optional): Milliseconds between characters when the CGEvent fallback path is used, 0-200. Default 30.
 
 ```json
 {"pid": 844, "window_id": 10725, "element_index": 12, "text": "hello"}
-```
-
-### type_text_chars
-
-Character-by-character input via `CGEvent.postToPid`. Slower than `type_text` but reaches Chromium and Electron inputs that ignore AX writes.
-
-**Arguments:**
-
-- `pid` (integer, required): Target process ID.
-- `text` (string, required): Text to type into the target's focused element.
-- `delay_ms` (integer, optional): Milliseconds between characters, 0-200. Default 30.
-
-```json
 {"pid": 844, "text": "hello world", "delay_ms": 40}
 ```
 
@@ -450,7 +428,7 @@ Supported keys and ranges: see the [CLI reference](/cua-driver/reference/cli-ref
 
 ## Recording and replay
 
-The trajectory recorder captures every action-tool call (`click`, `right_click`, `scroll`, `type_text`, `type_text_chars`, `press_key`, `hotkey`, `set_value`) into numbered turn folders. Recordings can be replayed turn-by-turn.
+The trajectory recorder captures every action-tool call (`click`, `right_click`, `scroll`, `type_text`, `press_key`, `hotkey`, `set_value`) into numbered turn folders. Recordings can be replayed turn-by-turn.
 
 ### get_recording_state
 

@@ -6,7 +6,7 @@ user explicitly asks to record — the skill does not auto-enable this.
 
 `set_recording` turns on a session-scoped trajectory recorder. While
 enabled, every action-tool call (`click`, `right_click`, `scroll`,
-`type_text`, `type_text_chars`, `press_key`, `hotkey`, `set_value`)
+`type_text`, `press_key`, `hotkey`, `set_value`)
 writes a numbered turn folder under a caller-chosen output
 directory. Read-only tools (`get_window_state`, `list_windows`,
 `screenshot`, `list_apps`, permission probes, agent-cursor getters /
@@ -101,8 +101,7 @@ keyed on `(pid, window_id)`, so a recorded
 resolve today — the pid is usually different, the window_id always
 is. The call returns `Invalid element_index` or `No cached AX
 state`. Pixel clicks (`click({pid, x, y})`) and keyboard tools
-(`press_key`, `type_text_chars`, `hotkey`, `type_text` without
-element_index) replay cleanly; element-indexed actions require a
+(`press_key`, `hotkey`, `type_text` without element_index) replay cleanly; element-indexed actions require a
 live snapshot that replay doesn't currently re-emit (read-only tools
 like `get_window_state` aren't recorded). For a reliable replay, either
 compose the trajectory from pixel + keyboard primitives, or capture

@@ -420,8 +420,7 @@ you're interacting with a long-lived process). In the default
 get_window_state → reason over PNG → pixel click`. When you need
 `element_index` dispatch (AX-addressable elements, backgrounded
 clicks), flip to `som` first: `cua-driver set_config '{"key":
-"capture_mode", "value": "som"}'`, or call `get_accessibility_tree`
-directly. The rest of this section walks through `som` mode, which
+"capture_mode", "value": "som"}'`. The rest of this section walks through `som` mode, which
 is what you want once you've decided element-indexed addressing is
 required.
 
@@ -432,33 +431,26 @@ In `som` mode the response carries:
   ~1600 elements, ~190 KB); when it exceeds token limits the MCP
   harness saves it to a file and returns the path. Use `Bash` +
   `jq -r '.tree_markdown'` + `grep` to pull the section you need.
-- `screenshot_png_b64` + `screenshot_width` / `_height` /
-  `_scale_factor` — the window screenshot (actually JPEG-85 despite
-  the `_png_` field name, hard-coded in
-  `WindowCapture.captureFrontmostWindow`). Present in `som` mode
-  (spliced into the structured JSON alongside the tree). In `vision`
-  mode the image arrives as a native MCP image content block with no
-  structured wrapper. Omitted when the target has no on-screen
-  window.
-- `has_screenshot: bool` — **gate on this before piping the PNG**.
-  Otherwise `jq -r '.screenshot_png_b64'` emits the literal
-  `"null"`, base64-decodes into 3 bytes of garbage, and downstream
-  vision APIs reject it with an opaque "Could not process image"
-  error.
-
-```
-# canonical, works in every capture mode — writes the image bytes
-# wherever you point, stdout stays readable (tree in som, summary
-# in vision). stderr warns (exit 0) if the response had no image.
-cua-driver get_window_state '{"pid":N,"window_id":W}' --image-out /tmp/shot.png
-
-# som-only legacy path: pull the spliced base64 out of structuredContent.
-# Prefer --image-out above — it's one flag vs a probe + pipe.
-if [ "$(cua-driver get_window_state '{"pid":N,"window_id":W}' | jq -r '.has_screenshot')" = "true" ]; then
-  cua-driver get_window_state '{"pid":N,"window_id":W}' | jq -r '.screenshot_png_b64' | base64 -d > shot.png
-fi
+- `screenshot_file_path` — absolute path to the saved screenshot when
+  `screenshot_out_file` was passed. Absent otherwise.
+- `screenshot_width` / `_height` / `_scale_factor` — dimensions of the
+  captured image. Present whenever a screenshot was taken.
+**Getting the screenshot as a file (CLI and context-constrained agents):**
+
+```bash
+# write to file — stdout stays readable (AX tree / summary only, no base64)
+cua-driver get_window_state '{"pid":N,"window_id":W,"screenshot_out_file":"/tmp/shot.jpg"}'
+
+# CLI --screenshot-out-file flag is equivalent and works for all capture modes
+cua-driver get_window_state '{"pid":N,"window_id":W}' --screenshot-out-file /tmp/shot.jpg
 ```
 
+Pass `screenshot_out_file` when using `get_window_state` via CLI or from an
+agent whose context window can't absorb ~31 KB of inline base64 (e.g.
+OpenCode with a local Ollama model). The MCP image content block is omitted
+from the response when this param is set — the model receives only the AX
+tree and `screenshot_file_path`, then reads the image from disk.
+
 **Reason over both the tree AND the screenshot — they're
 complementary, not redundant.** In `som` mode every
 turn's `get_window_state` gives you both halves and you should pull
@@ -512,7 +504,7 @@ anchor the conversion against a specific window):
 | Focus + send key | `press_key({pid, key, window_id, element_index, modifiers})` | element_index sets AXFocused, then posts key |
 | Send key to pid | `press_key({pid, key, modifiers})` | no focus change; key goes to pid's current focus |
 | Modifier combo | `hotkey({pid, keys})` | e.g. `["cmd","c"]`; posted per-pid, not HID tap |
-| Unicode keystrokes | `type_text_chars({pid, text, delay_ms})` | CGEvent-to-pid; reaches Chromium/Electron inputs |
+| Unicode keystrokes | `type_text({pid, text, delay_ms})` | AX write with automatic CGEvent fallback; reaches Chromium/Electron inputs |
 
 **All keyboard/text primitives require `pid`.** There is no
 frontmost-routed variant — every key goes to the named target via
@@ -556,7 +548,7 @@ below against the full-resolution file in that case.
 1. `get_window_state({pid, window_id})` returns an image capped
    at 1568 long-side (default) plus its dimensions
    (`screenshot_width` / `screenshot_height`). Write the bytes to
-   disk with `--image-out <path>` in any capture mode — works
+   disk with `--screenshot-out-file <path>` in any capture mode — works
    identically in `vision` (where it's the only way) and `som`
    (where it sidesteps the jq + base64 dance on the spliced
    `screenshot_png_b64` field).

@@ -116,11 +116,11 @@ These are where CuaDriver's AX-activation trio matters:
 ### 8. Slack — sparse Electron, hotkey fallback
 **Prompt:** `In Slack, jump to the #pr-reviews channel using the quick switcher (⌘K).`
 
-**Exercises:** "retry snapshot once, then `hotkey` + `type_text_chars`" fallback path.
+**Exercises:** "retry snapshot once, then `hotkey` + `type_text`" fallback path.
 
 **Success:**
 - Re-snapshot shows Slack's channel title area (AX role `AXStaticText` or similar) contains `pr-reviews`.
-- Claude uses `hotkey` and `type_text_chars` — NOT `simulate_click` / pixel coords.
+- Claude uses `hotkey` and `type_text` — NOT `simulate_click` / pixel coords.
 
 **Fail signals:** Claude drops to `simulate_click` (guardrail violation — pixel fallback is not allowed on sparse AX trees), wrong channel joined, typed text echoes into the message composer instead of the switcher.
 
@@ -168,7 +168,7 @@ These are where CuaDriver's AX-activation trio matters:
 ### 12. Chrome proper — omnibox
 **Prompt:** `In Google Chrome, open a new tab and navigate to https://trycua.com.`
 
-**Exercises:** `hotkey(["cmd","t"])` + `type_text_chars` + Return. Chrome's omnibox typically isn't AX-exposed even after activation.
+**Exercises:** `hotkey(["cmd","t"])` + `type_text` + Return. Chrome's omnibox typically isn't AX-exposed even after activation; `type_text` automatically falls back to CGEvent synthesis for Chromium/Electron inputs.
 
 **Success:**
 - A new Chrome tab whose AX title contains `Cua` or `trycua` is present.

@@ -46,10 +46,10 @@ pixels:
    `hotkey({pid, keys: ["cmd", "enter"]})`, `hotkey({pid, keys:
    ["cmd", "k"]})`, etc. Posted via `CGEvent.postToPid`, reaches the
    target regardless of AX state, no activation required.
-3. For typing into web inputs where `type_text` silently drops
-   (input doesn't implement `AXSelectedText`), use `type_text_chars`
-   — pure CGEvent keystrokes reach any focused keyboard receiver,
-   including Unicode / emoji.
+3. For typing into web inputs, use `type_text` — it automatically
+   falls back to CGEvent synthesis when the input doesn't implement
+   `AXSelectedText`, reaching any focused keyboard receiver including
+   Unicode / emoji.
 4. If none of the above reaches the target, tell the user this
    interaction isn't reachable from the driver today and ask for
    guidance.
@@ -86,7 +86,7 @@ documented only as historical context:
 ```
 # DON'T DO THIS — ⌘L steals focus. Use launch_app above.
 hotkey({pid, keys: ["cmd", "l"]})
-type_text_chars({pid, text: "https://cua.ai", delay_ms: 30})
+type_text({pid, text: "https://cua.ai", delay_ms: 30})
 get_window_state({pid, window_id})
 click({pid, window_id, element_index: <suggestion>})
 ```
@@ -157,7 +157,7 @@ When the target window is **minimized** (genie'd into the Dock):
   `AXFocused=true` on a minimized window's descendants doesn't
   propagate to real keyboard focus). Symptom: macOS system-alert
   beep, or silent no-op. Example: `hotkey cmd+L` +
-  `type_text_chars URL` + `press_key return` on minimized Chrome —
+  `type_text URL` + `press_key return` on minimized Chrome —
   the URL lands in the omnibox AX value but Return doesn't commit
   the navigation.
 - **Primary workaround — use `set_value` to commit directly**: For
@@ -465,6 +465,7 @@ type_text({pid, window_id, element_index: <input_field>, text: "…"})
 ```
 
 If it silently drops (some web inputs don't implement
-`AXSelectedText`), click the field first, then use
-`type_text_chars({pid, text})` — pure CGEvent keystrokes delivered
-to the pid, reaching any focused keyboard receiver.
+`AXSelectedText`), `type_text` automatically falls back to CGEvent
+synthesis — pure CGEvent keystrokes delivered to the pid, reaching
+any focused keyboard receiver. You can also click the field first
+to ensure focus before typing.