Skip to content

fix(accessibility): expose focused terminal text area - #14587

Open
austinywang wants to merge 16 commits into
mainfrom
issue-9563-axfocuseduielement-returns-the-window
Open

austinywang wants to merge 16 commits into
mainfrom
issue-9563-axfocuseduielement-returns-the-window

Conversation

@austinywang

@austinywang austinywang commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Closes #9563

When a terminal pane owns keyboard focus, cmux now reports its text area as AXFocusedUIElement and exposes the rendered screen through AXValue, character-range, and attributed-text APIs. Focus changes notify both the window and application accessibility clients, so selection readers can follow pane switches.

Regression coverage first reproduces empty terminal AX content and window-level focus, followed by the fix. Local validation passed Swift syntax parsing, localization parity across all nine macOS locales, package policy, and diff checks. Native tests and builds require the tagged fleet workflow.

— SableLoom9563 pending (run: run_issue9563_14d2618fe095; session: 01a0d838-5a14-7a41-a0be-040700eacaf5)


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Makes AXFocusedUIElement resolve to the focused terminal view so macOS AX clients can read the rendered screen when a terminal pane has keyboard focus instead of stopping at the window element. Closes #9563.

  • AXFocusedUIElement returns the terminal view when it owns keyboard focus; previously it returned the window.
  • AXValue, character-range, line, and attributed-text APIs now expose the rendered screen; AXValue previously returned only selected text or an empty string.
  • Focus changes post .focusedUIElementChanged notifications to both the window and the app so selection readers can follow pane switches.
  • Accessibility help text is now localized across all nine macOS locales.
  • Regression tests reproduce the old empty-content and window-level focus behavior, then verify the fix.

Written for commit 1ff69fa. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Accessibility
    • Screen readers can now access the terminal’s visible contents and identify the focused terminal view.
    • Focus changes are announced to assistive technologies.
  • Localization
    • Added translated accessibility help text in English, Arabic, German, Spanish, French, Japanese, Korean, Simplified Chinese, and Traditional Chinese.

@cursor

cursor Bot commented Sep 25, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

Next included review available in 7 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 33494524-6218-4554-8607-57fd892b2e57

📥 Commits

Reviewing files that changed from the base of the PR and between 522da7b and 1ff69fa.

📒 Files selected for processing (4)
  • Resources/Localizable.xcstrings
  • Sources/App/CmuxMainWindow.swift
  • Sources/GhosttyTerminalView.swift
  • cmuxTests/TerminalAndGhosttyTests.swift
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@cursor

cursor Bot commented Sep 25, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@cursor

cursor Bot commented Sep 25, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@cursor

cursor Bot commented Sep 26, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@cursor

cursor Bot commented Sep 26, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@teamleaderleo

Copy link
Copy Markdown
Collaborator

Review notes:

  1. Compile admission is red from cmuxTests/TerminalAndGhosttyTests.swift:1026: guard let surfaceView = hostedView.surfaceView but surfaceView is non-optional; a plain let fixes it.
  2. The AXValue read uses GHOSTTY_POINT_SCREEN (GhosttyTerminalView.swift:6385/6391), which is the full scrollback, on the main thread up to every 0.5 s under the Ghostty lock whenever an AX client polls. GHOSTTY_POINT_VIEWPORT would match the "rendered screen" intent and be cheaper.
  3. Voice input tools use this text area. AXValue was empty or the selection before; now it's the screen. A dictation tool that reads value, inserts, then writes the whole value back via setAccessibilityValue would inject the screen into the shell. Worth a dogfood pass with macOS Dictation plus Wispr Flow or SuperWhisper before merge.
  4. accessibilitySelectedTextRange offsets aren't reconciled with the new AXValue.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AXFocusedUIElement returns the window, not the terminal text area — AX selection tools read nothing

2 participants