Skip to content

feat(cua-driver): add stable health_report MCP tool - #1905

Closed
f-trycua wants to merge 1 commit into
mainfrom
feat/cua-driver-health-report
Closed

feat(cua-driver): add stable health_report MCP tool#1905
f-trycua wants to merge 1 commit into
mainfrom
feat/cua-driver-health-report

Conversation

@f-trycua

@f-trycua f-trycua commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Downstream consumers (Hermes Agent and similar) want an end-to-end health check for cua-driver but should not have to know cua-driver internals: specific MCP tool names, TCC field names, bundle IDs, per-platform check matrix. Today they stitch together check_permissions, doctor, version probes, bundle-attribution checks, and a screenshot smoke test — which means every diagnostic refactor inside cua-driver breaks them.

The right shape is for cua-driver to own the health model entirely and expose it as one stable MCP tool. Consumers stay thin. cua-driver evolves freely.

This PR adds that tool: health_report.

Schema (stable contract, schema_version: "1")

{
  "schema_version": "1",
  "platform": "darwin" | "win32" | "linux",
  "driver_version": "<semver>",
  "overall": "ok" | "degraded" | "failed",
  "checks": [
    {
      "name": "binary_version" | "platform_supported" | "tcc_accessibility" | "tcc_screen_recording" | "bundle_identity" | "ax_capability" | "screen_capture_capability" | "session_active",
      "status": "pass" | "fail" | "skip",
      "message": "<one-line summary, always present>",
      "hint": "<remediation step — present when status == fail>",
      "data": { /* check-specific structured data — optional */ }
    }
  ]
}

overall rules

  • ok — every non-skipped check passes
  • degraded — at least one non-core check fails (binary still usable)
  • failed — any core check fails (binary_version, platform_supported, session_active)

Input (all optional)

{
  "include": ["<check_name>", ...],
  "skip":    ["<check_name>", ...]
}

Both use canonical check names from the output enum. If both are given, include wins. Unknown names in include are silently ignored (forward-compat — a consumer may know names from a newer driver).

Stability commitment

schema_version: "1" is the contract. Future breaking changes go to "2".

Non-breaking under v1: adding new check names. Consumers must tolerate unknown check names.
Breaking (would bump to v2): removing fields, renaming canonical check names, changing status / overall enum values, changing JSON-key casing.

What runs

macOS check matrix (everything in this PR):

  • binary_version — always pass; surfaces CuaDriverCore.version.
  • platform_supported — pass with data.os_version, data.architecture.
  • session_active — pass by construction.
  • bundle_identity — pass iff CFBundleIdentifier == com.trycua.driver; fails with data.bundle_identifier populated for downstream attribution-drift detection.
  • tcc_accessibility — reuses Permissions.currentStatus().
  • tcc_screen_recording — reuses Permissions.currentStatus().
  • ax_capabilityAXIsProcessTrusted() probe.
  • screen_capture_capability — live SCShareableContent.excludingDesktopWindows probe with data.display_count. Nothing hits disk. No stream is started.

TCC checks delegate to Permissions.currentStatus() — no duplication.

Test strategy

  • Unit (HealthReportTests, 15 tests) — registry advertisement, schema_version commitment in tool description, selectChecks/computeOverall truth tables, parseStringSet edge cases, end-to-end Codable round-trip asserting documented top-level + per-check keys, skip filter honored, and the documented bundle_identity fail-mode shape (status=fail + hint + data.bundle_identifier).
  • Integration (test_health_report_mcp.py, 6 tests) — spawns the real cua-driver mcp stdio server, performs JSON-RPC handshake, asserts tools/list advertises health_report, calls it with no args / skip filter / include filter, validates the schema shape, asserts isError is never set, and exercises the documented fail mode end-to-end.
  • Full Swift test suite (swift test) — 51 tests, all green.

Scope notes — Rust follow-up

libs/cua-driver-rs (and the libs/cua-driver/rust workspace) exists with macOS + Windows + Linux ports. Adding health_report there means landing it in three platform-* crates plus extending the parity suite — too much for one pass. Filing as a follow-up. Swift is shipped here; the schema contract above will drive the Rust port verbatim.

Constraints honored

  • No existing MCP tool schemas were modified — new tool only.
  • No external dependencies added.
  • No code-signing / entitlements / notarization changes.
  • All tests pass on the branch.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Introduced a new health report tool that performs comprehensive driver system diagnostics with support for filtering specific checks and providing results in both structured and human-readable formats
  • Tests

    • Added comprehensive unit test suite and integration tests validating health report functionality, schema structure, filtering behavior, and diagnostic output format

…iver diagnostics

Downstream consumers (Hermes Agent and similar) want one stable diagnostic
call against cua-driver, not a tangle of check_permissions, doctor, version,
bundle-attribution, and screenshot probes glued together. This new MCP tool
gives them that: a single call with a stable, versioned JSON output shape.

The tool owns the health model entirely. Consumers stay thin and cua-driver
evolves freely.

Schema:
  {
    "schema_version": "1",
    "platform": "darwin",
    "driver_version": "<semver>",
    "overall": "ok" | "degraded" | "failed",
    "checks": [
      {
        "name": "<canonical check name>",
        "status": "pass" | "fail" | "skip",
        "message": "<one-line summary>",
        "hint": "<remediation, present on fail>",
        "data": { ... optional per-check structured fields ... }
      },
      ...
    ]
  }

schema_version="1" is the stability commitment. Adding new check names
under the same version is non-breaking; consumers must tolerate unknown
names. Breaking shape changes go to "2".

Reuses Permissions.currentStatus() for the TCC checks rather than
duplicating the probe logic. Adds a ScreenCaptureKit reachability probe
(in-memory, nothing hits disk) and a CFBundleIdentifier check that
surfaces the runtime bundle id so consumers can detect TCC attribution
drift without parsing message text.

Tests: 15 unit + 6 stdio MCP integration tests, all green.

Rust port (libs/cua-driver-rs / libs/cua-driver/rust) is left as a
follow-up — adding the same tool to three platform crates plus parity
tests is too much for one pass.
@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Jun 16, 2026 6:09am

Request Review

@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Adds a new health_report MCP tool to the cua-driver Swift package. The tool runs eight macOS driver diagnostics (binary version, platform, session, bundle identity, TCC permissions, AX trust, ScreenCaptureKit) with include/skip filtering and returns a structured JSON report plus human-readable text. Swift XCTest unit tests and Python stdio integration tests are included.

Changes

health_report MCP Tool

Layer / File(s) Summary
Report data model, tool handler, and filtering
libs/cua-driver/swift/Sources/CuaDriverServer/Tools/HealthReportTool.swift
Defines Codable output types (Report, Overall, Status, CheckEntry, CheckData), the health_report MCP tool handler with include/skip input schema, check selection logic (parseStringSet, selectChecks), textSummary renderer, and uname_m() architecture helper.
Check implementations and overall rollup
libs/cua-driver/swift/Sources/CuaDriverServer/Tools/HealthReportTool.swift
Implements runCheck dispatch and all eight individual check functions. Implements computeOverall which maps core-check failures to failed, non-core failures to degraded, and no failures to ok.
Package target and ToolRegistry registration
libs/cua-driver/swift/Package.swift, libs/cua-driver/swift/Sources/CuaDriverServer/ToolRegistry.swift
Declares the HealthReportTests test target with CuaDriverCore, CuaDriverServer, and MCP dependencies. Registers HealthReportTool.handler in ToolRegistry.default.
Swift XCTest unit and end-to-end tests
libs/cua-driver/swift/Tests/HealthReportTests/HealthReportTests.swift
Covers registry/description presence, selectChecks filter behavior, computeOverall rollup logic, async end-to-end JSON shape validation, skip-filter invocation, checkBundleIdentity fail-mode, parseStringSet edge cases, and private buildReport helpers.
Python stdio integration tests
libs/cua-driver/swift/Tests/integration/test_health_report_mcp.py
Boots the real MCP server via DriverClient and validates tool discovery, schema shape, skip/include filter behavior, bundle_identity stdio fail-mode contract, and that isError is never set.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • trycua/cua#1418: The docs auto-generation PR serializes ToolRegistry.default.allTools to produce MCP reference docs, and this PR adds health_report to ToolRegistry.default, so the new tool will appear in the generated reference output.

Poem

🐇 Hop, hop, a new report in sight,
Eight checks run through the macOS night.
Bundle, AX, and ScreenCapture too—
ok, degraded, or failed for you.
The rabbit inspects each permission with care,
JSON and text summaries float through the air! 🌟

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding a new health_report MCP tool with a stable schema to the cua-driver.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/cua-driver-health-report

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@f-trycua

Copy link
Copy Markdown
Collaborator Author

Closing — wrong target. The Swift implementation under libs/cua-driver/swift/ is deprecated; the canonical surface now lives in the Rust workspace at libs/cua-driver/rust/ (9 crates, cross-platform via platform-macos / platform-windows / platform-linux). The Rust workspace is what cua-driver ships and what downstream consumers (Hermes Agent and friends) end up depending on.

Re-implementing in the Rust workspace in a follow-up PR. The schema design from this PR (schema_version: "1" envelope, 8 check names, status/message/hint/data shape) is the authoritative contract for the re-implementation — the Rust impl should match it byte-for-byte. Test patterns transfer: 15 unit tests + 6 stdio MCP integration tests covering filter modes, fail mode, and the isError contract.

The branch feat/cua-driver-health-report and worktree at /Users/francesco/cua-health-report are kept locally as reference; deleting after the Rust port lands.

Apologies for the noise — wrong framing on my end when dispatching the original work.

@f-trycua f-trycua closed this Jun 16, 2026
@f-trycua
f-trycua deleted the feat/cua-driver-health-report branch June 21, 2026 02:44
f-trycua added a commit that referenced this pull request Jun 21, 2026
…iver diagnostics

Single-call diagnostic tool so downstream consumers (Hermes Agent's
`hermes computer-use doctor`, NousResearch/hermes-agent#47065) get a
stable health-check contract without knowing cua-driver internals —
specific MCP tool names, TCC field names, bundle IDs, per-platform
check matrix. cua-driver owns the health model entirely; consumers stay
thin and the driver evolves freely.

This is the canonical Rust port of the now-closed Swift PR #1905. The
Swift implementation landed by mistake in the deprecated swift/ tree;
this lands the same schema + test contract in the canonical Rust
workspace (libs/cua-driver/rust/).

Architecture:
- cua-driver-core/src/health_report.rs — cross-platform skeleton:
  Report / CheckEntry / CheckStatus / CheckData types, the
  HealthCheckProvider trait, select_checks / compute_overall /
  parse_string_set logic, and the HealthReportTool MCP wrapper.
- platform-macos/src/tools/health_report.rs — macOS provider
  (binary, platform, session, bundle_identity via CFBundleGetMainBundle,
  TCC reuse via existing permissions::status plumbing, live
  SCShareableContent probe for screen_capture_capability).
- platform-windows/src/health_report.rs — Windows provider (UIA via
  the existing diagnostics::ui_automation_available, D3D11CreateDevice
  probe for screen capture, tcc_* + bundle_identity skipped with
  "not applicable on Windows").
- platform-linux/src/health_report.rs — Linux provider (X11 connect
  probe shared between ax_capability and screen_capture_capability,
  tcc_* + bundle_identity skipped with "not applicable on Linux").

Wire-format contract is stable: schema_version="1" is the commitment,
future breaking changes go to "2". Adding new check names is
non-breaking; consumers must tolerate unknown names. health_report
NEVER itself sets isError — that's the whole point of a "what's
broken?" probe.

Tests:
- cua-driver-core: 16 unit tests for selectChecks, computeOverall,
  parseStringSet, end-to-end dispatcher round-trip, fail-mode contract,
  schema_version=1 description commitment.
- platform-macos: 5 tests including a documented fail-mode fixture
  (the test host's bundle id is not com.trycua.driver, so
  bundle_identity reliably fails with the full message+hint+data
  shape consumers see in the wild).
- platform-windows: 4 tests covering the canonical skip("not
  applicable on Windows") shape for TCC/bundle.
- platform-linux: 4 tests covering the canonical skip("not
  applicable on Linux") shape for TCC/bundle.
- tests/integration/test_health_report_mcp.py: 6 stdio MCP tests
  (tools/list discovery + schema_version commitment, no-args round
  trip, include filter, skip filter, fail-mode shape, never-isError).

Refs:
- Closed Swift PR: #1905 (deprecated swift/ tree)
- Downstream consumer: NousResearch/hermes-agent#47065

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
f-trycua added a commit that referenced this pull request Jun 21, 2026
…iver diagnostics

Single-call diagnostic tool so downstream consumers (Hermes Agent's
`hermes computer-use doctor`, NousResearch/hermes-agent#47065) get a
stable health-check contract without knowing cua-driver internals —
specific MCP tool names, TCC field names, bundle IDs, per-platform
check matrix. cua-driver owns the health model entirely; consumers stay
thin and the driver evolves freely.

This is the canonical Rust port of the now-closed Swift PR #1905. The
Swift implementation landed by mistake in the deprecated swift/ tree;
this lands the same schema + test contract in the canonical Rust
workspace (libs/cua-driver/rust/).

Architecture:
- cua-driver-core/src/health_report.rs — cross-platform skeleton:
  Report / CheckEntry / CheckStatus / CheckData types, the
  HealthCheckProvider trait, select_checks / compute_overall /
  parse_string_set logic, and the HealthReportTool MCP wrapper.
- platform-macos/src/tools/health_report.rs — macOS provider
  (binary, platform, session, bundle_identity via CFBundleGetMainBundle,
  TCC reuse via existing permissions::status plumbing, live
  SCShareableContent probe for screen_capture_capability).
- platform-windows/src/health_report.rs — Windows provider (UIA via
  the existing diagnostics::ui_automation_available, D3D11CreateDevice
  probe for screen capture, tcc_* + bundle_identity skipped with
  "not applicable on Windows").
- platform-linux/src/health_report.rs — Linux provider (X11 connect
  probe shared between ax_capability and screen_capture_capability,
  tcc_* + bundle_identity skipped with "not applicable on Linux").

Wire-format contract is stable: schema_version="1" is the commitment,
future breaking changes go to "2". Adding new check names is
non-breaking; consumers must tolerate unknown names. health_report
NEVER itself sets isError — that's the whole point of a "what's
broken?" probe.

Tests:
- cua-driver-core: 16 unit tests for selectChecks, computeOverall,
  parseStringSet, end-to-end dispatcher round-trip, fail-mode contract,
  schema_version=1 description commitment.
- platform-macos: 5 tests including a documented fail-mode fixture
  (the test host's bundle id is not com.trycua.driver, so
  bundle_identity reliably fails with the full message+hint+data
  shape consumers see in the wild).
- platform-windows: 4 tests covering the canonical skip("not
  applicable on Windows") shape for TCC/bundle.
- platform-linux: 4 tests covering the canonical skip("not
  applicable on Linux") shape for TCC/bundle.
- tests/integration/test_health_report_mcp.py: 6 stdio MCP tests
  (tools/list discovery + schema_version commitment, no-args round
  trip, include filter, skip filter, fail-mode shape, never-isError).

Refs:
- Closed Swift PR: #1905 (deprecated swift/ tree)
- Downstream consumer: NousResearch/hermes-agent#47065

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant