refactor(cua-driver): test taxonomy + protocol split/dedup + transport coverage (Phases 3-6)

[f-trycua]

Final phase of the test-suite cleanup (Phases 1-2 = #2039, #2040). Bundles the rename, protocol split+dedup, the transport-coverage test, and the docs.

Phase 3 — 4-family taxonomy (git mv, blame preserved)

protocol_ / transport_ / harness_ / modality_ / guard_. So ls tests/ reads as a map. Renames: harness_web_windows→harness_web, harness_lo_vcl→harness_libreoffice, harness_bg_modality→modality_background, harness_desktop_scope→modality_desktop_scope, focus_check→modality_focus, e2e_windows_bg_input→modality_input_e2e, ux_guard→guard_ux, element_token→protocol_element_token. No CI/scripts referenced the old names (they just run cargo test); the two docs are updated in Phase 6.

Phase 5 — split + dedup the protocol monolith

mcp_protocol_test.rs (3,412 lines / 65 tests) → 5 protocol_*_test.rs files (handshake / tools_call / schema / media / session) on a new testkit RawDriver (raw send/recv, no auto-init — these tests drive the handshake themselves). 24 macOS↔Windows mirror pairs deduped into single cfg!-branching tests.

Running the deduped suite surfaced 5 never-run stale Windows assertions (the old Windows mirror had never actually executed):

• serverInfo.name asserted "cua-driver-rs"; the server returns "cua-driver".
• tool roster expected type_text_chars + screenshot; Windows registers neither.
• type_text_chars schema test assumed Windows exposes the tool; it's hidden on both platforms.
• set_config_screenshot_resize asserted unconditionally; now skips when a screenshot is unavailable (Session 0 / no display) on both platforms.
• multi_cursor assumed get_agent_cursor_state filters to one cursor; on Windows it returns all — verify each cursor's state within the list.

All fixed to match real behavior. Result: 37 protocol tests pass on macOS, 28 on Windows — the first time the Windows protocol tests have actually passed.

Phase 4 — transport coverage

New transport_config_persistence_test: set_config persists to disk across stateless CLI invocations (#2034) vs visible within an MCP session — the one behavior only observable across both transports.

Phase 6 — docs

Rewrote TEST_SUITE.md / TEST_HARNESS_STRUCTURE.md to the new names + a transport × modality × platform coverage matrix.

Verification

• ✅ macOS: full compile clean; all protocol_* tests run + pass (37).
• ✅ Windows VM: full compile clean (0 warnings); all protocol_* tests run + pass (28).
• #[ignore] suites (harness/modality/guard/transport) compile on both; live --ignored behavior unchanged.

Deferred (noted, not done)

• Merging modality_input_e2e ↔ modality_background.
• Extracting the focus-steal sentinel into a shared testkit assert (3 timing-sensitive Windows oracles — risk > reward here).

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Expanded driver protocol coverage for handshakes, tool calls, media capture, recording/replay, and session behavior.
  • Added support checks for raw protocol interaction and configuration persistence across runs and sessions.

• Bug Fixes

  • Improved validation around tool schemas, error handling, cursor isolation, and media response formats.

• Documentation

  • Updated test harness and suite documentation to reflect the current behavior-test structure and coverage.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Ignored		Jun 26, 2026 7:08am

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Adds RawDriver to the testkit and expands Rust integration coverage for MCP handshake, tools/list schema, tools/call behavior, screenshot and recording flows, cursor-session isolation, config persistence, and harness documentation.

Changes

CUA driver protocol coverage

Layer / File(s)	Summary
RawDriver API and process bridge libs/cua-driver/rust/crates/cua-driver-testkit/src/lib.rs, libs/cua-driver/rust/crates/cua-driver-testkit/src/raw.rs	RawDriver is added to the testkit API and implements raw stdin/stdout JSON-RPC spawning, send/recv, and child cleanup.
Handshake and schema checks libs/cua-driver/rust/crates/cua-driver/tests/protocol_handshake_test.rs, libs/cua-driver/rust/crates/cua-driver/tests/protocol_schema_test.rs	Adds initialize and tools/list assertions, tool capability/version checks, unknown-tool and unknown-action errors, and schema-shape checks for advertised inputs.
General tool-call coverage libs/cua-driver/rust/crates/cua-driver/tests/protocol_tools_call_test.rs	Adds end-to-end tools/call tests for multiple tools with platform-specific assertions and skips.
Screenshot and zoom flows libs/cua-driver/rust/crates/cua-driver/tests/protocol_media_test.rs	Adds screenshot assertions, JPEG capture, zoom capture, and zoom/click round-trip checks on macOS and Windows.
Recording and replay flows libs/cua-driver/rust/crates/cua-driver/tests/protocol_media_test.rs	Adds recording lifecycle checks, screenshot file capture, record_video flag handling, and replay counters.
Debug image and resize checks libs/cua-driver/rust/crates/cua-driver/tests/protocol_media_test.rs	Adds click debug_image_out validation and screenshot resize assertions against max_image_dimension.
Session isolation and cursor state libs/cua-driver/rust/crates/cua-driver/tests/protocol_session_test.rs	Adds multi-process protocol checks, cursor isolation, liveness probes, overlay move loops, and Bezier motion knob assertions.
Config persistence across transports libs/cua-driver/rust/crates/cua-driver/tests/transport_config_persistence_test.rs	Adds ignored CLI and MCP tests for capture_scope persistence and restoration.
Harness taxonomy docs libs/cua-driver/test-harness/TEST_SUITE.md, libs/cua-driver/test-harness/TEST_HARNESS_STRUCTURE.md	Rewrites the test-harness overview, inventory, coverage matrix, and harness-app mapping to match the current file taxonomy.

Sequence Diagram(s)

sequenceDiagram
  participant protocol_handshake_test
  participant RawDriver
  participant "cua-driver binary" as cua_driver_binary
  protocol_handshake_test->>RawDriver: spawn()
  RawDriver->>cua_driver_binary: start child with piped stdin/stdout
  protocol_handshake_test->>RawDriver: send initialize request
  RawDriver->>cua_driver_binary: write newline-delimited JSON-RPC frame
  cua_driver_binary-->>RawDriver: stdout response line
  RawDriver-->>protocol_handshake_test: parsed JSON Value

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• trycua/cua#1666: Updates the page tool contract and unknown-action path that the new protocol tests assert.
• trycua/cua#1703: Changes type_text_chars registration, matching the new tools/list schema assertions.
• trycua/cua#2038: Revises the same test-harness documentation files and coverage mapping updated here.

Poem

I hopped through JSON in the pale moon glow,
With RawDriver whiskers and a tidy flow.
I nibbled tools, then watched tests bloom,
And thumped the moon with a happy “vroom!”
🐇✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: test-suite refactor with protocol split/dedup and added transport coverage.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch refactor/test-suite-phases-3-6

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Linux visual regression artifacts

Matrix jobs now run independently. Download visual artifacts from this workflow run.
Each background-GUI job uploads a .gif of the interaction plus two annotated PNGs (<app>.png raw, <app>-atspi.png with AT-SPI element boxes); the cua-driver-linux-som-overlays artifact adds <app>-som.png cua Set-of-Marks overlays:

• cua-driver-linux-cursor-click-gif
• cua-driver-linux-background-terminal-gif
• cua-driver-linux-parallel-drag-xserver
• cua-driver-linux-background-gui-chromium
• cua-driver-linux-background-gui-tk
• cua-driver-linux-background-gui-gtk3-gedit
• cua-driver-linux-background-gui-gtk3-mousepad
• cua-driver-linux-background-gui-gtk3-scite
• cua-driver-linux-background-gui-gtk4-characters
• cua-driver-linux-background-gui-qt5-manuskript
• cua-driver-linux-background-gui-qt5-klog
• cua-driver-linux-background-gui-qt5-openambit
• cua-driver-linux-background-gui-qt6-kate
• cua-driver-linux-background-gui-qt6-kcalc
• cua-driver-linux-background-gui-qt6-okular
• cua-driver-linux-background-gui-qt6-qownnotes
• cua-driver-linux-background-gui-electron-zettlr
• cua-driver-linux-background-gui-electron-joplin
• cua-driver-linux-background-gui-electron-logseq
• cua-driver-linux-som-overlays

Open workflow run and download artifacts