Skip to content

Restore Ghostty overlay CI coverage - #4913

Merged
lawrencecchen merged 1 commit into
mainfrom
issue-4524-ghostty-overlay-tests
May 28, 2026
Merged

lawrencecchen merged 1 commit into
mainfrom
issue-4524-ghostty-overlay-tests

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented May 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • deflake GhosttySurfaceOverlayTests cleanup and search overlay focus assertions
  • remove the CI skip for GhosttySurfaceOverlayTests

Fixes #4524.

Verification

  • git diff --check
  • xcodebuild -project cmux.xcodeproj -scheme cmux-unit -configuration Debug -destination 'platform=macOS' -derivedDataPath /tmp/cmux-ghostty-overlay-build CMUX_SKIP_ZIG_BUILD=1 build -quiet\n- AWS: cmuxTests/GhosttySurfaceOverlayTests passed, result bundle /tmp/cmux-ghostov-fix3-1779927163.xcresult

View with Codesmith Autofix with Codesmith
Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.


Note

Low Risk
Changes are limited to XCTest helpers and CI test selection; no production terminal or overlay behavior is modified.

Overview
Re-enables GhosttySurfaceOverlayTests in CI by dropping the xcodebuild -skip-testing entry for that class in the unit-test job.

The overlay test suite is hardened for flaky app-host runs: tearDown clears debug key observers and releases tracked TerminalSurface instances via makeTrackedTerminalSurface, waitUntil polls on the main run loop instead of XCTNSPredicateExpectation, and find-overlay focus tests use a fake key window plus AppDelegate / TabManager test registration. Scroller-style expectations are loosened so legacy vs overlay width checks are less brittle on runners, and the retention test explicitly releases the surface before asserting deallocation.

Reviewed by Cursor Bugbot for commit c2d15b1. Bugbot is set up for automated code reviews on this repo. Configure here.


Summary by cubic

Restores CI coverage for Ghostty overlay by deflaking GhosttySurfaceOverlayTests and removing the CI skip. Fixes #4524.

  • Bug Fixes
    • Added a deterministic wait helper and a KeyStatusTestWindow to stabilize search overlay focus and mount timing.
    • Tracked and released TerminalSurface instances in tearDown to prevent leaks; added a retain-cycle check.
    • Mounted hosted views in a real window for focus-related tests; adjusted scroller style assertions and synchronization.

Written for commit c2d15b1. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

Tests

  • Re-enabled previously excluded test suite in automated CI pipeline, expanding automated test coverage
  • Refactored test infrastructure with enhanced cleanup mechanisms to prevent test interference and improve test isolation
  • Modernized async waiting patterns for more reliable and deterministic test execution across the suite

Review Change Stack

@vercel

vercel Bot commented May 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Canceled Canceled May 28, 2026 12:29am
cmux-staging Building Building Preview, Comment May 28, 2026 12:29am

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented May 28, 2026 •

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f525e54f-fb21-47da-8461-768d97b40ee5

📥 Commits

Reviewing files that changed from the base of the PR and between 610fd97 and c2d15b1.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml
  • cmuxTests/TerminalAndGhosttyTests.swift
💤 Files with no reviewable changes (1)
  • .github/workflows/ci.yml

📝 Walkthrough

Walkthrough

CI re-enables GhosttySurfaceOverlayTests by removing a broad skip flag. The test class is hardened with deterministic surface lifecycle tracking, modern deadline-based async polling, and systematic refactoring of dependent tests to use the new infrastructure for reliable cleanup.

Changes

GhosttySurfaceOverlayTests Stability and Re-enablement

Layer / File(s) Summary
CI re-enables GhosttySurfaceOverlayTests
.github/workflows/ci.yml
Removes -skip-testing cmuxTests/GhosttySurfaceOverlayTests from xcodebuild unit test invocation.
Surface tracking and deterministic cleanup infrastructure
cmuxTests/TerminalAndGhosttyTests.swift
Adds private trackedSurfaces array, KeyStatusTestWindow helper for focus state, tearDown() override, and makeTrackedTerminalSurface(tabId:) factory to ensure all surfaces are released in reverse order after each test.
Modernized async waiting with deadline polling
cmuxTests/TerminalAndGhosttyTests.swift
Rewrites waitUntil helper from XCTNSPredicateExpectation/XCTWaiter to a deadline-based loop that repeatedly runs the main run loop and evaluates conditions, explicitly failing via XCTFail on timeout.
Refactor primary overlay tests to use tracked surfaces
cmuxTests/TerminalAndGhosttyTests.swift
Updates preferred scroller style test with new grid-width assertions across style transitions, and refactors search overlay mount/unmount and rapid toggle tests to use makeTrackedTerminalSurface().
Deferred search overlay focus test with window context
cmuxTests/TerminalAndGhosttyTests.swift
Establishes main-window test context via KeyStatusTestWindow, mounts search overlay, and polls until find text field is present and owned by window's first responder, with proper app delegate restoration.
Refactor remaining overlay and portal tests to use tracked surfaces
cmuxTests/TerminalAndGhosttyTests.swift
Updates escape-to-dismiss overlay, keyboard copy mode indicator, forceRefresh regression, search overlay non-retention, portal rebind survivability, and portal visibility toggle tests to use makeTrackedTerminalSurface(). The non-retention test explicitly verifies overlays do not retain surfaces via weak references.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • manaflow-ai/cmux#4882: Removes a different -skip-testing entry (BrowserPanelWebViewLifecycleTests) from the same xcodebuild unit-test invocation as part of the same CI quarantine replacement initiative.
  • manaflow-ai/cmux#4874: Also modifies .github/workflows/ci.yml to change which test cases are excluded via -skip-testing, narrowing AppDelegate shortcut routing test quarantine as part of the same effort.
  • manaflow-ai/cmux#4562: Wires cmuxTests/*.swift source files into the build target, ensuring tests are discoverable and executable in CI alongside this PR's work to actually run the re-enabled test suite.

Suggested reviewers

  • jesstelford

Poem

🐰 A scroller slides, a surface tracked,
No leaks escape, no tests hacked.
With async polls and teardown care,
GhosttySurface tests run fair. 🎯✨

🚥 Pre-merge checks | ✅ 17 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (17 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Restore Ghostty overlay CI coverage' accurately reflects the main objective: re-enabling GhosttySurfaceOverlayTests in CI by removing the skip.
Description check ✅ Passed The PR description addresses the template sections adequately: Summary explains what changed and why, Verification details local testing and AWS results, though Demo Video section is appropriately omitted for non-UI changes.
Linked Issues check ✅ Passed The PR directly addresses issue #4524's acceptance criteria by removing the broad class-level skip for GhosttySurfaceOverlayTests and replacing it with deterministic test isolation through improved cleanup and focus handling.
Out of Scope Changes check ✅ Passed All changes are in-scope: CI workflow modification to remove GhosttySurfaceOverlayTests skip and test file improvements to deflake the suite. No unrelated changes were introduced.
Cmux Swift Actor Isolation ✅ Passed PR changes only test file (cmuxTests/TerminalAndGhosttyTests.swift) and CI config; custom check explicitly passes for tests. No production Swift actor isolation issues introduced.
Cmux Swift Blocking Runtime ✅ Passed PR modifies only test code (cmuxTests/) with new waitUntil helper using RunLoop polling for deterministic test scaffolding—no Task.sleep, Thread.sleep, or semaphores.
Cmux No Hacky Sleeps ✅ Passed PR changes are out of scope: only GitHub Actions YAML (CI orchestration) and Swift test code (covered by swift-blocking-runtime.md) were modified, not production TypeScript/JavaScript/shell code.
Cmux Algorithmic Complexity ✅ Passed Changes are to CI workflow configuration and test-only code (cmuxTests directory), both outside the algorithmic complexity rule's scope for production code.
Cmux Swift Concurrency ✅ Passed PR changes only affect test infrastructure and CI configuration; no legacy async patterns introduced in cmux production code, only test-only cleanup and synchronization refactoring.
Cmux Swift @Concurrent ✅ Passed All Swift changes are test-only with no async functions, @concurrent annotations, or concurrent isolation violations. Test class is properly @MainActor-annotated.
Cmux Swift File And Package Boundaries ✅ Passed PR modifies only test code and CI workflow with no production Swift changes, exempting it under the "test fixtures" allowed case.
Cmux Swift Logging ✅ Passed PR contains only test file changes (cmuxTests/TerminalAndGhosttyTests.swift) and YAML CI config; no logging violations in test-only code which is explicitly allowed.
Cmux User-Facing Error Privacy ✅ Passed PR modifies only test code (cmuxTests/TerminalAndGhosttyTests.swift) and CI configuration (.github/workflows/ci.yml), which are explicitly allowed by user-facing-errors.md rule.
Cmux Full Internationalization ✅ Passed The PR modifies only test files and CI configuration (ci.yml and TerminalAndGhosttyTests.swift), which are explicitly exempt from internationalization requirements per the check rules.
Cmux Swiftui State Layout ✅ Passed PR modifies only CI workflow YAML and test code; no SwiftUI production code changes detected. No violations of @Published/@observable state, GeometryReader layout, or render-time mutations found.
Cmux Architecture Rethink ✅ Passed All changes are test-only: surface tracking in tearDown, KeyStatusTestWindow for focus tests, and manual RunLoop waitUntil. These fall under allowed test-only synchronization per rules.
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed Changes are test-only: two new NSWindow subclasses (KeyStatusTestWindow, ContentViewCountingWindow) are private nested test helpers explicitly allowed by the rule.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-4524-ghostty-overlay-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented May 28, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR deflakes GhosttySurfaceOverlayTests and removes its CI skip, enabling the Ghostty overlay test suite to run in CI again.

  • Introduces a surfacesToRelease array with a tearDown() that explicitly calls releaseSurfaceForTesting() in reverse order, replacing ad-hoc inline surface lifecycle management across seven test functions.
  • Replaces XCTNSPredicateExpectation/XCTWaiter with a main-thread run-loop polling loop in waitUntil, and wires testSearchOverlayFocusesSearchFieldAfterDeferredAttach through a real TabManager/AppDelegate context (with a KeyStatusTestWindow stub) to get a reliable first-responder signal without a fixed-duration sleep.

Confidence Score: 4/5

Changes are entirely test-scoped; production behavior is unchanged. The teardown lifecycle and focus-test AppDelegate wiring are careful but have a subtle ordering question worth checking before merging.

The testPreferredScrollerStyleChangeRestoresOverlayScrollbarWidth behavioral assertion was dropped in favour of a precondition check, quietly reducing what the test actually verifies. In testSearchOverlayFocusesSearchFieldAfterDeferredAttach, the surface is both appended to surfacesToRelease and indirectly owned by the tab manager that is restored before tearDown() runs, creating a potential double-release ordering question. Neither issue affects production code, but they could let regressions slip through undetected or introduce new test instability.

cmuxTests/TerminalAndGhosttyTests.swift — the weakened scrollbar assertion and the surface-lifecycle ordering in the focus test both warrant a second look.

Important Files Changed

Filename Overview
.github/workflows/ci.yml Removes the -skip-testing:cmuxTests/GhosttySurfaceOverlayTests CI skip flag so the overlay test suite runs again in CI.
cmuxTests/TerminalAndGhosttyTests.swift Adds tracked surface teardown, a KeyStatusTestWindow helper, a polling-based waitUntil, and wires testSearchOverlayFocusesSearchFieldAfterDeferredAttach through TabManager/AppDelegate; weakens one behavioral assertion and leaves a potential surface double-release path.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Test starts] --> B[makeTrackedTerminalSurface]
    B --> C[surface added to surfacesToRelease]
    C --> D[Test body runs]
    D --> E{Special case: focus test?}
    E -- yes --> F[Register TabManager + AppDelegate.shared]
    F --> G[Obtain surface from terminalPanel\nappend to surfacesToRelease]
    G --> H[Test assertions / waitUntil loop]
    H --> I[defer: restore TabManager + AppDelegate.shared]
    I --> J[tearDown runs]
    E -- no --> H
    H --> J
    J --> K[releaseSurfaceForTesting for each in surfacesToRelease reversed]
    K --> L[surfacesToRelease cleared]
    L --> M[super.tearDown]
Loading

Comments Outside Diff (1)

  1. cmuxTests/TerminalAndGhosttyTests.swift, line 3543-3547 (link)

    P2 Potential surface double-release in testSearchOverlayFocusesSearchFieldAfterDeferredAttach

    surface is obtained from terminalPanel.surface and appended to surfacesToRelease. tearDown() will call surface.releaseSurfaceForTesting() after the test body's defer block, which restores appDelegate.tabManager = originalTabManager. If restoring the original tab manager also releases (or tears down) the surfaces it no longer owns, tearDown() will then call releaseSurfaceForTesting() on an already-released surface. Whether this is safe depends on the idempotency of releaseSurfaceForTesting(). If a second release is not a no-op, this could cause a crash or inconsistent state in subsequent tests in the suite.

Reviews (1): Last reviewed commit: "Deflake Ghostty overlay CI coverage" | Re-trigger Greptile

Comment on lines +3385 to 3389
XCTAssertEqual(scrollView.scrollerStyle, .legacy)
assertPendingSurfaceWidth(
initialSurfaceSize.width,
"Changing the scroll view style alone should leave the terminal grid stale until the scroller-style observer runs"
"Changing the scroll view style alone should leave the terminal grid unchanged until the scroller-style observer runs"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The assertion that verifies the behavior of legacy scrollbars (that they reduce the scroll view's content width) was replaced with a trivial precondition check: XCTAssertEqual(scrollView.scrollerStyle, .legacy) only confirms that the assignment took effect, not that the terminal content area was actually affected. If the scrollerStyle-to-content-width mapping regresses, this assertion won't catch it. The assertion below (restoredContentWidth >= legacyContentWidth) covers the recovery direction but not the initial contraction. If the flake was caused by the assertion failing on headless CI runners that report zero-size scroll views, a better deflake guard would be to skip the assertion when initialContentWidth == 0 rather than dropping the check entirely.

Suggested change
XCTAssertEqual(scrollView.scrollerStyle, .legacy)
assertPendingSurfaceWidth(
initialSurfaceSize.width,
"Changing the scroll view style alone should leave the terminal grid stale until the scroller-style observer runs"
"Changing the scroll view style alone should leave the terminal grid unchanged until the scroller-style observer runs"
)
XCTAssertEqual(scrollView.scrollerStyle, .legacy)
if initialContentWidth > 0 {
XCTAssertLessThan(
legacyContentWidth,
initialContentWidth,
"Legacy scrollbars should reserve width in the scroll view content area"
)
}
assertPendingSurfaceWidth(
initialSurfaceSize.width,
"Changing the scroll view style alone should leave the terminal grid unchanged until the scroller-style observer runs"
)

This branch was successfully deployed

1 active deployment
Preview – cmux — c2d15b17 Deployed May 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace broad macOS app-host CI quarantines with gated coverage

1 participant