Skip to content

test: find the onboarding window the test presented, not a leftover - #15015

Merged
teamleaderleo merged 2 commits into
mainfrom
test/computer-use-stale-window
Sep 27, 2026
Merged

teamleaderleo merged 2 commits into
mainfrom
test/computer-use-stale-window

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Run 36316398822 (PR #14974), app-host shard 3 on cmux13s, failed two ComputerUseOnboardingWindowTests:

  • offscreenCompanionReturnsWithoutMovingTheOverview
  • permissionCompanionRemainsWhenAnotherApplicationActivates

Both failed on .isVisible → false, and both read the same window, 0xc10ac7200. Each test presents its own controller, so a window shared by the two can only be a leftover. The tests looked up the main onboarding window by identifier alone. An earlier test's onboarding window can still be in NSApp.windows, closed but retained or left open, and the lookup found that one. This is a test isolation bug, not the machine: the hook's console gate admitted the job on an unlocked session, and the other GUI tests in the shard passed.

What

newWindow(identified:_:) snapshots NSApp.windows, runs the action and returns only a window with that identifier the action opened. It serves the two main-window lookups after present() and the five permission-companion lookups after configureForPermissionCompanion, since the companion is also kept when closed (subagent review). defer { controller.dismiss() } now comes before the #require, so a failed lookup still dismisses.

Verification

Per the no-local-tests rule on Air Blue, this runs in the app-host shards on CI.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Improved the reliability of automated onboarding and permission-window tests by ensuring they check windows created by the action under test, rather than previously retained windows. This helps test results more accurately reflect the behavior being checked.

ComputerUseOnboardingWindowTests looked the main onboarding window up by
identifier alone. An earlier test's onboarding window can still be in
NSApp.windows, so offscreenCompanionReturnsWithoutMovingTheOverview and
permissionCompanionRemainsWhenAnotherApplicationActivates both read the same
stale, invisible window (0xc10ac7200) and failed on .isVisible in run
36316398822 shard 3 on cmux13s. The lookup now takes only a window that
present() opened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 8b1efada-9c21-4147-a321-279be7a87d62

📥 Commits

Reviewing files that changed from the base of the PR and between 5e19a98 and 288ef5c.

📒 Files selected for processing (1)
  • cmuxTests/ComputerUseOnboardingWindowTests.swift

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

Window tests now select onboarding and permission-companion windows created by the current test action. A helper compares window identities before and after the action to avoid selecting retained windows with matching identifiers.

Changes

Window test selection

Layer / File(s) Summary
Capture windows created by test actions
cmuxTests/ComputerUseOnboardingWindowTests.swift
A new helper identifies windows created during an action. Onboarding and permission-companion tests use it to select their windows.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: austinywang

Merge Risk: ⚪ Minimal · up to 288ef

This test-only change selects windows created by the current action rather than stale retained windows, with cleanup registered before required lookups. Static inspection finds no concrete merge-blocking risk; normal app-host CI remains the final validation.

🚥 Pre-merge checks | ✅ 23 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description explains the failure, implementation, and intended CI verification. However, it does not use the required Summary, Testing, Changelog, Demo Video, and Checklist sections, and it does n… Rewrite the description using the repository template. Add Summary, Testing, Changelog, Demo Video, and Checklist sections. State the exact CI job or command used for verification, use none for the changelog if appropriate, explain why no…
✅ Passed checks (23 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input ✅ Passed PASS. The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. It adds a test-only newWindow helper and updates onboarding/companion window assertions. It does not change Cl…
Cmux Swift Actor Isolation ✅ Passed PASS: The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The added newWindow helper and updated calls are test code, and the changed test methods are MainActor-isolate…
Cmux Swift Blocking Runtime ✅ Passed PASS. The PR changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The added newWindow helper snapshots NSApp.windows, runs the test action, and filters by ObjectIdentifier; it adds n…
Cmux Browser Automation Off-Main ✅ Passed PASS: The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The diff adds a @MainActor test helper for AppKit window lookup and updates onboarding window tests. It contai…
Cmux Expensive Synchronous Load ✅ Passed PASS. The reviewed diff changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. It adds a newWindow test helper that snapshots NSApp.windows, runs a synchronous window action, and filters…
Cmux Cache Substitution Correctness ✅ Passed PASS. The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift, which is test code, not production Swift. The newWindow helper tracks newly created NSWindow identities for …
Cmux No Hacky Sleeps ✅ Passed PASS — The PR changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The diff adds a deterministic test-only newWindow helper and updates window lookups; it adds no TypeScript, JavaScript,…
Cmux Algorithmic Complexity ✅ Passed PASS. The PR changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The added newWindow helper scans NSApp.windows, but it is test-only scaffolding. The algorithmic-complexity rule expli…
Cmux Swift Concurrency ✅ Passed The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The added newWindow helper uses a synchronous () -> Void action closure and @MainActor for AppKit access. The di…
Cmux Swift @Concurrent ✅ Passed PASS: The diff adds only a synchronous @MainActor helper, newWindow(identified:_:), which snapshots and queries NSApp.windows while executing synchronous UI actions. The changed call sites are i…
Cmux Swift Package Boundaries ✅ Passed PASS. The PR changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The diff adds a private test helper and updates AppKit test setup. It does not change production Swift, app-target domain …
Cmux Swiftpm Lockfiles ✅ Passed PASS: The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. It does not change a SwiftPM package, Package.swift, Package.resolved, .gitignore, workflow, Xcode project…
Cmux Swift Logging ✅ Passed PASS: The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift, which is test code. The diff adds newWindow and updates window lookup logic; it adds no print, debugPrint,…
Cmux User-Facing Error Privacy ✅ Passed PASS — The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The diff adds test-only window tracking and updates test setup; it does not change production user-facing error…
Cmux Full Internationalization ✅ Passed PASS: The authoritative diff changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The changes are test-only window lookup and cleanup logic, and the custom check explicitly passes tests. N…
Cmux Swiftui State Layout ✅ Passed The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The diff adds an AppKit NSWindow snapshot helper and updates AppKit window-test actions. It adds no `ObservableObjec…
Cmux Architecture Rethink ✅ Passed PASS. The PR changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. It adds a newWindow test helper that snapshots NSApp.windows identities and selects only a newly created window. It do…
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed PASS. The PR changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. It updates test fixture window lookup and adds a private test helper; it does not add or materially change production `NSW…
Cmux Source Artifacts ✅ Passed The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. The diff contains hand-written Swift test code and a test helper that isolates newly opened windows. It adds no logs, …
Cmux No Test Or Debug Seam In Production Source ✅ Passed The pull request changes only cmuxTests/ComputerUseOnboardingWindowTests.swift. It adds a private newWindow test helper in the test target and changes test window lookup there. It does not modify …
Title check ✅ Passed The title clearly describes the main change: tests now select the onboarding window presented by the current test instead of a leftover window.
Full details: Description check

Explanation

The description explains the failure, implementation, and intended CI verification. However, it does not use the required Summary, Testing, Changelog, Demo Video, and Checklist sections, and it does not provide the required changelog or checklist information.

Resolution

Rewrite the description using the repository template. Add Summary, Testing, Changelog, Demo Video, and Checklist sections. State the exact CI job or command used for verification, use none for the changelog if appropriate, explain why no demo video applies to this test-only change, and complete the applicable checklist items.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

Review: the permission companion is also kept when closed, so the five bare
companion lookups could read an earlier test's panel the same way. One helper,
newWindow(identified:_:), now serves both identifiers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo
teamleaderleo merged commit 0758c9f into main Sep 27, 2026
50 of 51 checks passed
@teamleaderleo
teamleaderleo deleted the test/computer-use-stale-window branch September 27, 2026 14:20
@github-actions

Copy link
Copy Markdown
Contributor

Merge receipt for 288ef5cedb: every check was green at merge (16 verified; 14 skipped by policy). Full suite runs on main after merge.

rustybret pushed a commit to rustybret/bmux that referenced this pull request Sep 27, 2026
648d5c1 Add a Paste Last Screenshot action with an unbound shortcut (manaflow-ai#14955)
ff61677 ci: avoid partial blobs in catch-up merges (manaflow-ai#15023)
4d0d112 ci: retry transient catch-up GraphQL failures (manaflow-ai#15021)
212e808 ci: attribution scores a lone suspect and reports app-host crashes apart (manaflow-ai#14952)
4cabdf4 test: settle the window before measuring the unread sidebar-row invalidation (manaflow-ai#14568)
12ec99b Add a release-media capture tool for changelog screenshots and clips (manaflow-ai#15010)
ee2cda0 Backfill Unreleased changelog and draft next release cards (manaflow-ai#14999)
be4adf8 Show a brief notice when Cmd+V fails on an oversized image or a timeout (manaflow-ai#14953)
23d22d7 ci: an owned pool the run starts on now beats an earlier one it queues on (manaflow-ai#14993)
05d0190 ci: catch-up posts once per head, says less, and merges inserted declarations (manaflow-ai#15018)
4ee4b21 ci: fail stalled Swift package tests instead of waiting out the job timeout (manaflow-ai#14997)
9ce512a merge-main: run local guards only when asked (manaflow-ai#15016)
d60108a ci: clear test-e2e's fixed DerivedData with clear-dirs.sh (manaflow-ai#14994)
1d7895e ci: run the shell and CLI no-socket lanes in parallel (manaflow-ai#14990)
6e7d25f Honor macOS Differentiate Without Color, Increase Contrast and Reduce Transparency (manaflow-ai#14991)
966b355 Stop interrupting focused work: sidebar jumps, Computer Use focus steal, quit dialog on logout (manaflow-ai#14961)
e1f1cb2 Strip control characters from feedback attachment filenames (manaflow-ai#14783)
0758c9f test: find the onboarding window the test presented, not a leftover (manaflow-ai#15015)
b35c540 fix(spm): resolve GhosttyKit/GhosttyRuntimeTestStubs target name collisions (manaflow-ai#10569)
ef33bed Map .purs artifacts to the Haskell highlight.js grammar (manaflow-ai#14202)
e2a167a Highlight Elixir and Erlang files in the file editor (manaflow-ai#13732)
972c449 fix: wrap Linux browser download card label (manaflow-ai#11157)
f563884 Add Aside to browser data import detection (manaflow-ai#13379)
091d0ea Add cmux send --paste and hint at it for large multi-line sends (manaflow-ai#14937)
3ffcdbb test(ios): keep folder-tap stat tests off the real 2 s deadline (manaflow-ai#15017)
68d3936 test: keep CmuxTerminal pasteboard tests off the cooperative pool (manaflow-ai#15006)
teamleaderleo added a commit that referenced this pull request Sep 27, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant