Skip to content

Fix iOS keyboard focus ownership after photo picker - #9371

Merged
azooz2003-bit merged 13 commits into
mainfrom
task-ios-keyboard-bottom-bar-pinning
Aug 4, 2026
Merged

azooz2003-bit merged 13 commits into
mainfrom
task-ios-keyboard-bottom-bar-pinning

Conversation

@azooz2003-bit

@azooz2003-bit azooz2003-bit commented Aug 1, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes first-tap keyboard recovery and bottom-dock coordination in workspace detail.

The terminal, composer, photo picker, and scene lifecycle now share one input-session reducer. Ordinary terminal taps focus synchronously before asynchronous artifact/click work. UIKit responder results remain the source of actual ownership.

Tests:

  • 11 TerminalInputSessionReducer tests
  • iOS Simulator SwiftPM builds for CmuxMobileTerminal and CmuxMobileShellUI
  • Terminal dock transition and letterbox geometry suites
  • Focused XCUITests for terminal and composer recovery after picker cancellation

Hosted XCUITests are currently blocked before execution by the unrelated WorkspaceListTableCoordinatorDropTests compile error already present on main.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Fixes iOS keyboard focus after the Photos picker by centralizing terminal/composer input ownership so the first tap reliably restores the keyboard and keeps the bottom dock aligned using UIKit’s keyboard guide. Also restricts foreground-only recovery by requiring a live client for secondary aggregation and skipping it during redial/validation.

  • Bug Fixes

    • First-tap keyboard recovery after cancelling PhotosPicker for both terminal and composer; no stale owner after dismissal.
    • Dock pinned via UIKit’s keyboard guide; deterministic align/letterbox and full-height return when the keyboard hides.
    • Foreground recovery: secondary Mac aggregation runs only when connected with a live client and not redialing/validating.
  • Refactors

    • Centralized input ownership with TerminalInputSessionReducer and TerminalInputSessionCoordinator; UIKit/SwiftUI first-responder changes flow through one owner.
    • Added GhosttySurfaceViewDelegate.inputPolicyForTap… and TerminalInputTapIntent; immediate focus only with a fresh artifact snapshot, otherwise deferred until classification.
    • TerminalComposerView routes focus and PhotosPicker lifecycle through the input session; SwiftUI @FocusState mirrors UIKit; TerminalInputTextView reports first-responder changes.
    • Cached visible-text snapshot now returns columns and generation for artifact hit testing; async clicks are invalidated by a click generation counter.
    • Removed custom dock transition planner and tests (TerminalDockKeyboardTransition*, TerminalKeyboardHeightAnimation).

Written for commit 91b1677. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • New Features

    • Improved terminal and composer focus coordination across scene changes, modal presentations, and keyboard interactions.
    • Added more reliable handling for terminal taps on clickable artifacts.
    • Improved photo-picker presentation, dismissal, and focus restoration behavior.
    • Added richer accessibility diagnostics for input ownership and keyboard state.
  • Bug Fixes

    • Fixed stale focus and keyboard states after photo-picker cancellation.
    • Improved focus recovery after terminal and composer handoffs.
    • Prevented disconnected recovery attempts from triggering duplicate background reconnect activity.
  • Tests

    • Expanded coverage for focus transitions, artifact taps, modal cancellation, recovery, and keyboard restoration.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026 •

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR adds a terminal input-session state machine and UIKit coordinator. GhosttySurfaceView now owns terminal and composer focus, lifecycle handling, responder reconciliation, and artifact-aware tap decisions. Cached artifact snapshots and click generations support deferred tap validation. Foreground recovery now prevents concurrent secondary aggregation while disconnected.

Changes

iOS input session and tap coordination

Layer / File(s) Summary
Input session state machine and UIKit adapter
Packages/iOS/CmuxMobileTerminalKit/..., Packages/iOS/CmuxMobileTerminal/..., Packages/iOS/CmuxMobileTerminalKit/Tests/...
Adds reducer types for input ownership, lifecycle, modal state, tap intent, deferred decisions, and focus commands. The UIKit coordinator executes commands and reports responder transitions. Tests cover retries, handoffs, modal dismissal, scene changes, detachment, and stale decisions.
Surface-owned focus and lifecycle integration
Packages/iOS/CmuxMobileTerminal/Sources/...
Routes terminal and composer focus, responder changes, scene transitions, modal boundaries, attachment, detachment, and active-input resignation through the coordinator.
Composer focus and photo-picker callbacks
Packages/iOS/CmuxMobileShell/..., Packages/iOS/CmuxMobileShellUI/..., ios/cmuxUITests/cmuxUITests.swift
Forwards composer focus and responder events to the surface. Photo-picker presentation and dismissal emit lifecycle callbacks. UI tests verify terminal and composer recovery after cancellation.
Artifact-aware tap policy and stale-click handling
Packages/iOS/CmuxMobileShellUI/..., Packages/iOS/CmuxMobileTerminal/...
Adds cached snapshot metadata and delegate tap policies. Deferred artifact validation uses click generations to reject stale work.

Foreground connection recovery

Layer / File(s) Summary
Connected-state recovery scheduling and validation
Packages/iOS/CmuxMobileShell/..., Packages/iOS/CmuxMobileShell/Tests/...
Secondary aggregation is scheduled only while connected. Foreground recovery tests verify foreground-only behavior and exactly one attach-ticket request.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

  • manaflow-ai/cmux-dev-artifacts#7655: Updates the same photo-picker cancellation coverage for terminal focus and keyboard restoration.

Possibly related PRs


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error, 2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Cmux Algorithmic Complexity ❌ Error GhosttySurfaceCoordinator+Artifacts.swift:384 adds a synchronous TerminalArtifactTapHitTester.path call on every tap; its nested row/token and continuation scans are then repeated by async click ha... Cache an indexed hit-test result per settled snapshot, or pass the synchronous classification/path into click handling, so each tap does not rescan the visible grid.
Docstring Coverage ⚠️ Warning Docstring coverage is 19.05% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description explains the focus-recovery change and testing, but it omits the required Demo Video, Review Trigger, and Checklist sections. Add the missing template sections, include a demo link or attachment, paste the review trigger, and complete every checklist item.
Cmux Full Internationalization ❓ Inconclusive The checked-out diff does not match the supplied PR summary: only one recovery file is changed, while the summary lists many Swift files. Provide the PR base and head revision or a checkout that contains the full PR diff.
✅ Passed checks (21 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Swift Actor Isolation ✅ Passed The pure reducer models are explicit Sendable values, UIKit coordination and the delegate are @MainActor, and the diff adds no shared Sendable reference types or background UI-store access.
Cmux Swift Blocking Runtime ✅ Passed The PR adds no semaphores, blocking waits, sleeps, delayed dispatch, polling, main-queue sync, or locks; focus commands execute synchronously and deferred taps use an async artifact callback.
Cmux Browser Automation Off-Main ✅ Passed The PR diff contains only iOS files; it does not modify browser automation targets or add browser/WebKit/socket-worker references.
Cmux Expensive Synchronous Load ✅ Passed The main...HEAD diff adds no agent-history loaders, JSON/JSONL reads, directory scans, or synchronous file APIs to production Swift; changes are focus state, UIKit, and cached terminal snapshots.
Cmux Cache Substitution Correctness ✅ Passed The cache only drives transient tap-focus classification; cold or generation-stale snapshots defer, while async click handling still performs a fresh snapshot read.
Cmux No Hacky Sleeps ✅ Passed The full diff changes only Swift files; this check covers only non-Swift TypeScript, JavaScript, shell, and build/runtime scripts, so it is not applicable.
Cmux Swift Concurrency ✅ Passed Targeted PR diff adds no DispatchQueue, DispatchGroup, Combine, or completion-handler patterns; the new coordinator is synchronous @MainActor, and existing Tasks remain stored/cancelled or are unch...
Cmux Swift @Concurrent ✅ Passed The diff adds no @concurrent or new nonisolated async UI work; the changed async method is actor-isolated, and the heavy image helper is unchanged in isolation and call sites.
Cmux Swift Package Boundaries ✅ Passed The pure input-session reducer and tests are in CmuxMobileTerminalKit; remaining changes are UIKit/Ghostty/SwiftUI glue or app-lifecycle recovery composition.
Cmux Swiftpm Lockfiles ✅ Passed The PR diff changes no Package.swift, Package.resolved, .gitignore, workflow, or Xcode project package-reference files, so the lockfile policy has no violation.
Cmux Swift Logging ✅ Passed The PR diff adds no print, debugPrint, dump, NSLog, file/stdout logging, or secret logging; new accessibility diagnostics are DEBUG-only and existing Logger/debug logs are unchanged.
Cmux User-Facing Error Privacy ✅ Passed The production diff adds no user-facing error, alert, command/API error, or recovery copy. New diagnostics are DEBUG/UI-test instrumentation, and tests/comments are allowed.
Cmux Swiftui State Layout ✅ Passed The diff adds no new ObservableObject/@published, GeometryReader, lazy-row store reference, or render-time state write; existing TerminalComposerView state is only touched incidentally via callbacks.
Cmux Architecture Rethink ✅ Passed The reducer is the single owner of requested/actual input, scene, modal, and tap state; the coordinator is the UIKit bridge. The old delayed focus Task was removed, and no new sleep, polling, lock,...
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed The PR changes only iOS terminal/input and recovery code; no standalone NSWindow, NSPanel, NSWindowController, Window, or WindowGroup was added or changed. The auxiliary-window lint also passes.
Cmux Source Artifacts ✅ Passed All 14 changed paths are Swift source or test files under Packages/ or ios/; the diff adds no logs, screenshots, recordings, caches, build output, temp folders, or forbidden artifact directories.
Cmux No Test Or Debug Seam In Production Source ✅ Passed The PR adds no test/debug-named seam or test-build guard in production Sources; new public accessors have production callers, and the only new guard is canImport(UIKit).
Cmux No Ambient Global State ✅ Passed The production Swift diff adds constructable input-session types and instance state only; no new file-scope API, mutable global, namespace-only type, or shared/standard/default singleton was found.
Title check ✅ Passed The title clearly summarizes the main change: fixing iOS keyboard focus ownership after photo picker dismissal.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch task-ios-keyboard-bottom-bar-pinning

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ios/cmuxUITests/cmuxUITests.swift`:
- Around line 6588-6591: Update the composer-field setup in cmuxUITests to wait
for the field to be hittable, not merely existent, before calling field.tap().
Reuse the existing predicate helper and keep the interaction as a single user
tap before the reducer assertions.

In
`@Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/GhosttySurfaceCoordinator`+Artifacts.swift:
- Around line 373-390: Replace the cached-snapshot focus authority in
ghosttySurfaceView(_:inputPolicyForTapAtCol:row:) with the shared input-session
decision flow using a freshness token. Treat
cachedVisibleTextForArtifactHitTesting() only as guidance for artifact
detection; when the snapshot is missing or stale, defer input rather than
returning immediate focus, while preserving reliable artifact decisions. Add a
regression test covering a path added after the cached snapshot was created.

In
`@Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/GhosttySurfaceRepresentable.swift`:
- Around line 194-197: Remove the unused clickGeneration property from
GhosttySurfaceRepresentable and delete its increment in stopMountedTasks(),
leaving the surrounding click and focus task handling unchanged.

In
`@Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/TerminalComposerView.swift`:
- Around line 344-348: Gate the simultaneous tap gesture in TerminalComposerView
on the same enabled-state condition used by the composer field’s .disabled
modifier, so requestInputFocus() is not called when .locksComposerField is true.
Preserve the existing focus request behavior when the field is enabled.

In
`@Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceView.swift`:
- Line 2871: Replace the `.sceneWillResignActive` emission in
`prepareForReuseAfterDetach()` with a dedicated `TerminalInputSessionEvent` such
as `.surfaceDetached`. Handle this event in the session reducer by clearing
input intent and resigning the surface without changing the scene phase, so an
active scene remains active and `inputScene` is not reported as inactive.
- Around line 2816-2822: Remove the conditional inputActualOwnerDidChange(nil)
call from resignCurrentInput. Keep synchronizeActualInputOwner and
inputSession.send(.releaseFocus) unchanged, relying on
TerminalInputSessionCoordinator’s actualOwnerDidChange publication as the sole
writer of active input ownership state.

In
`@Packages/iOS/CmuxMobileTerminalKit/Tests/CmuxMobileTerminalKitTests/TerminalInputSessionReducerTests.swift`:
- Around line 4-201: Add a test in TerminalInputSessionReducerTests covering
lifecycleBoundary: retain a composer request after an initial failed focus
completion, then verify handling .lifecycleBoundary emits [.focus(.composer)].
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: de11d7fe-df3b-4dac-9f4b-96d89399b888

📥 Commits

Reviewing files that changed from the base of the PR and between 89c52b8 and 71856de.

📒 Files selected for processing (13)
  • Packages/iOS/CmuxMobileShell/Sources/CmuxMobileShell/MobileShellComposite.swift
  • Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/GhosttySurfaceCoordinator+Artifacts.swift
  • Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/GhosttySurfaceRepresentable.swift
  • Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/TerminalComposerView.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceTapDisposition.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceView+Artifacts.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceView.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceViewDelegate.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/TerminalInputSessionCoordinator.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/TerminalInputTextView.swift
  • Packages/iOS/CmuxMobileTerminalKit/Sources/CmuxMobileTerminalKit/TerminalInputSessionReducer.swift
  • Packages/iOS/CmuxMobileTerminalKit/Tests/CmuxMobileTerminalKitTests/TerminalInputSessionReducerTests.swift
  • ios/cmuxUITests/cmuxUITests.swift

Comment thread ios/cmuxUITests/cmuxUITests.swift

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
ios/cmuxUITests/cmuxUITests.swift (1)

6610-6610: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Wait for the composer field to become hittable after picker dismissal.

waitForKeyboardDismissal confirms keyboard removal. It does not confirm that PhotosPicker restored the field hit target. Wait for field to become hittable before this first recovery tap.

Proposed fix
+        XCTAssertTrue(waitForHittable(field, timeout: 4))
         field.tap()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ios/cmuxUITests/cmuxUITests.swift` at line 6610, Before the first recovery
tap on field in the picker-dismissal flow, explicitly wait until field is
hittable; keep waitForKeyboardDismissal for keyboard state, then perform
field.tap() only after the element’s hit-test readiness is confirmed.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@ios/cmuxUITests/cmuxUITests.swift`:
- Line 6610: Before the first recovery tap on field in the picker-dismissal
flow, explicitly wait until field is hittable; keep waitForKeyboardDismissal for
keyboard state, then perform field.tap() only after the element’s hit-test
readiness is confirmed.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d7ab0889-23f5-46dc-b202-952d294bb01b

📥 Commits

Reviewing files that changed from the base of the PR and between 71856de and a8bc02d.

📒 Files selected for processing (8)
  • Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/GhosttySurfaceCoordinator+Artifacts.swift
  • Packages/iOS/CmuxMobileShellUI/Sources/CmuxMobileShellUI/TerminalComposerView.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceView+Artifacts.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceView.swift
  • Packages/iOS/CmuxMobileTerminal/Sources/CmuxMobileTerminal/GhosttySurfaceViewDelegate.swift
  • Packages/iOS/CmuxMobileTerminalKit/Sources/CmuxMobileTerminalKit/TerminalInputSessionReducer.swift
  • Packages/iOS/CmuxMobileTerminalKit/Tests/CmuxMobileTerminalKitTests/TerminalInputSessionReducerTests.swift
  • ios/cmuxUITests/cmuxUITests.swift

@azooz2003-bit

Copy link
Copy Markdown
Collaborator Author

Verification at a8bc02d

Passed: reducer 14/14, transition planner 6/6, letterbox geometry 24/24, iOS SwiftPM builds, iOS 26.5 and 18.4 Photos Cancel first-retap recovery, iOS 26.5 photo-selection first-retap recovery, exact or sub-point dock pinning, continuous rotation/foreground evidence, and a continuous 20-terminal-tap plus 10-owner-alternation stress recording.

Blocked: hosted XCUITests fail before selection on the unchanged WorkspaceListTableCoordinatorDropTests compile error from main; Instruments cannot produce an exportable simulator trace; live presence attachment returns HTTP 404; Aziz is unavailable, so the signed final build remains queued.

Formal verify-implementation approval remains incomplete. The PR is ready for phone dogfood and has not been merged.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@Packages/iOS/CmuxMobileShell/Sources/CmuxMobileShell/MobileShellComposite`+ConnectionRecovery.swift:
- Around line 100-105: Update the secondary aggregation condition in the
connection recovery flow to also require a non-nil remoteClient alongside
multiMacAggregationEnabled, trigger.reschedulesSecondaryAggregation, and
connectionState == .connected. This must prevent scheduling aggregation when the
connection is marked connected but no live remote client exists.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c1066168-948e-433f-a928-4ea2b8959901

📥 Commits

Reviewing files that changed from the base of the PR and between 1fe820a and 18ef9b9.

📒 Files selected for processing (2)
  • Packages/iOS/CmuxMobileShell/Sources/CmuxMobileShell/MobileShellComposite+ConnectionRecovery.swift
  • Packages/iOS/CmuxMobileShell/Tests/CmuxMobileShellTests/MobileShellForegroundResumeTests.swift

@azooz2003-bit

Copy link
Copy Markdown
Collaborator Author

Verification update at ec983f9:\n\n- The phone traces exposed a foreground-recovery versus secondary-aggregation route race. Foreground recovery now stays the sole route owner while disconnected, and aggregation requires both connected state and a live foreground client.\n- The new clientless-foreground regression failed before the guard and passes after it. The seven-test foreground recovery suite passed 20 consecutive iterations, 140 behavior tests total.\n- Computer Use passed the isolated iOS 26.5 Photos Cancel paths for the first terminal tap and first composer tap. The composer accepted input, and both bottom bars stayed attached immediately above the keyboard.\n- Fresh macOS build: https://github.com/manaflow-ai/cmux/actions/runs/30727950319\n- Final focused hosted run: https://github.com/manaflow-ai/cmux/actions/runs/30728806181\n- The signed iPhone app embeds ec983f9, bundle dev.cmux.ios.kbpin, team 7WLXT3NR37. Aziz is currently unavailable, so the exact app is queued for automatic install, launch, sign-in, and pairing.\n\nRemaining verification: observe the final recovery build attached on Aziz for at least 60 seconds. No merge requested.

@azooz2003-bit
azooz2003-bit merged commit 145b60e into main Aug 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant