Skip to content

fix(cua-driver): verify set_value writes with an AXValue read-back - #2621

Merged
f-trycua merged 3 commits into
trycua:mainfrom
hqhq1025:codex/set-value-readback
Jul 28, 2026
Merged

fix(cua-driver): verify set_value writes with an AXValue read-back#2621
f-trycua merged 3 commits into
trycua:mainfrom
hqhq1025:codex/set-value-readback

Conversation

@hqhq1025

@hqhq1025 hqhq1025 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

set_value treated a successful AX write as proof that the value reached the application. Chromium, WebKit, and Electron can echo an accessibility write even when the renderer did not accept it, so that could produce a false verified: true.

Change

  • Read AXValue before and after the write.
  • Trust a matching read-back only on native surfaces.
  • Downgrade AXWebArea results to unverified and provide the appropriate browser or pixel escalation.
  • Return the standard structured action contract: path, verified, effect, and optional escalation.
  • Use the unverified message marker instead of a success checkmark when confirmation is unavailable.
  • Compare values numerically only for numeric controls, using a bounded relative tolerance.
  • Preserve the original before-value when verifying the slider stepping fallback.

The AXPopUpButton path remains unverified because it selects a menu item instead of writing AXValue directly.

Validation

  • cargo test -p platform-macos --locked: 193 passed
  • cargo fmt --all -- --check
  • Live macOS native and web-surface checks are required before merge.

This is split from #2210 so the verification change can be reviewed independently.

`set_value` reported success on the `AXUIElementSetAttributeValue` return
code alone:

    if err == kAXErrorSuccess {
        Ok(format!("✅ Set AXValue on [{element_index}] {role}."))
    }

A success code is not evidence that the value landed. `type_text`'s own
docs already describe the trap: web content behind an `AXWebArea` "accepts
a write and echoes it back through AXValue while the renderer/DOM never
observes it". `set_value` had no equivalent guard, so an agent driving a
web input got a green checkmark for a write the page never took.

This reads AXValue before and after the write and classifies the result:

- `verified: true`  — read-back equals the requested value
- `verified: false` — read-back ran but does not match; the message tells
  the caller to confirm via screenshot
- `verified` absent — AXValue is not readable, so the write can be neither
  confirmed nor denied
- `changed: false`  — the value did not move, which combined with
  `verified` separates "already had this value" from "the write did not
  take"

Numeric controls are compared numerically, so a slider reporting `"25.0"`
verifies a requested `"25"`.

Both flags are emitted in `structuredContent` so clients can branch on
them instead of parsing the human-readable message. The AXPopUpButton path
drives menu items rather than writing AXValue, so it reports neither flag
rather than claiming an unverified write.

This follows the same honest-verification direction as the 0.7.0 work: a
tool should not report success it cannot substantiate.

Verified:
- `cargo test -p platform-macos`: 189 passed, 0 failed (6 new unit tests
  covering the echo, idempotent, unreadable, and numeric cases)
- `cargo fmt -p platform-macos -- --check`: clean
@hqhq1025
hqhq1025 requested a review from f-trycua as a code owner July 28, 2026 07:05
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

Context for this one is in #2608 (comment) — it is part of a three-way split of #2210, and each piece is independent of the other two. Reviewable on its own; no dependency on the others landing.

Reuse the existing AXWebArea ancestor proof so Chromium, WebKit, and Electron accessibility echoes never become verified renderer writes. Align set_value responses with the path, verified, effect, and escalation contract, and compare numeric values only for numeric controls with a bounded relative tolerance.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>
@f-trycua
f-trycua requested a review from injaneity as a code owner July 28, 2026 11:10

@f-trycua f-trycua left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed after surface-aware verification hardening. Web AX echoes are untrusted, native numeric comparison is bounded, and responses follow the action effect contract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants