Skip to content

Add split cycle terminal rendering regression test - #3789

Closed
lawrencecchen wants to merge 1 commit into
mainfrom
task-ghostty-render-split-cycle-regression
Closed

lawrencecchen wants to merge 1 commit into
mainfrom
task-ghostty-render-split-cycle-regression

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented May 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Adds an XCUITest that repeats Cmd+D, Cmd+D, Ctrl+D, Ctrl+D ten times.
  • Verifies the original leftmost Ghostty terminal remains the only surviving terminal and keeps rendering new marker output after every cycle.

Testing

  • xcodebuild -project GhosttyTabs.xcodeproj -scheme cmux -configuration Debug -destination 'platform=macOS' -derivedDataPath /tmp/cmux-splitrender build-for-testing -only-testing:cmuxUITests/BrowserPaneNavigationKeybindUITests/testRepeatedCmdDSplitAndCtrlDExitKeepsLeftmostTerminalRendering passed.\n- test-e2e.yml passed for BrowserPaneNavigationKeybindUITests/testRepeatedCmdDSplitAndCtrlDExitKeepsLeftmostTerminalRendering: https://github.com/manaflow-ai/cmux/actions/runs/25593148882\n\n## Task\n- Regression coverage for repeated split and terminal exit churn making the leftmost Ghostty terminal fail to render.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@vercel

vercel Bot commented May 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment May 9, 2026 5:41am
cmux-staging Building Building Preview, Comment May 9, 2026 5:41am

@coderabbitai

coderabbitai Bot commented May 9, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This pull request adds a new regression test that verifies terminal surface stability during repeated split/unsplit operations. The test launches the app, repeatedly creates and removes terminal splits via keyboard shortcuts, and verifies that the original leftmost terminal surface remains alive, maintains its identity, and continues to re-render after each cycle.

Changes

Terminal Surface Render Verification Regression

Layer / File(s) Summary
Data Models and Types
cmuxUITests/BrowserPaneNavigationKeybindUITests.swift
Introduces TerminalSurfaceInfo (id, isFocused), TerminalRenderStats (renderCount, updateCount), and PanelSnapshotInfo (baseline/after-marker hashes) as CustomStringConvertible types.
Regression Test Implementation
cmuxUITests/BrowserPaneNavigationKeybindUITests.swift
testRepeatedCmdDSplitAndCtrlDExitKeepsLeftmostTerminalRendering launches app with socket enabled, records initial terminal surface, loops 10 times performing Cmd+D/Cmd+D/Ctrl+D/Ctrl+D cycles, asserts terminal surface counts at each step, verifies leftmost surface ID never changes, and confirms render advancement after each marker.
Socket Helper Functions
cmuxUITests/BrowserPaneNavigationKeybindUITests.swift
Private socket-driven helpers: listTerminalSurfaces, terminalRenderStats, captureAndComparePanelSnapshot (baseline + after-marker), sendMarkerToSurface, readTerminalScrollbackText, surface.send_text, debug.panel_snapshot, surface.read_text wrappers, and JSON response parsing (socketResult, intValue, boolValue).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • manaflow-ai/cmux#1145: Both PRs modify BrowserPaneNavigationKeybindUITests.swift and add terminal/portal inspection APIs (surface debug getters and panel snapshots) for rendering verification and split/zoom tests.
  • manaflow-ai/cmux#2101: The regression test directly exercises TerminalSurface liveness and ownership guards (liveSurfaceForGhosttyAccess, cmuxSurfacePointerAppearsLive) introduced in this PR through repeated split/close cycles.
  • manaflow-ai/cmux#808: Both PRs address terminal-surface portal binding lifecycle issues and include regression tests that repeatedly split/close terminals to prevent ghost/rebind problems.

Poem

🐰 A rabbit splits the panes with might,
With Cmd+D shining bright,
Yet leftmost lives, re-renders true,
Through cycles old and markers new,
No ghosts shall haunt these terminals right!


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 2 warnings)

Check name Status Explanation Resolution
Cmux Swiftui State Layout ❌ Error PR adds 39 new ObservableObject classes using legacy @Published instead of modern @Observable. Violates swiftui-state-layout.md rule requiring @Observable for new cmux-owned state. Migrate 39 ObservableObject classes (AuthManager, CmuxConfigStore, FileExplorerState, etc.) to @Observable macro per modern SwiftUI patterns.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description covers Summary and Testing sections with clear details, but lacks Demo Video, Review Trigger block, and Checklist items required by the template. Add the missing Demo Video section, include the Review Trigger comment block, and complete the Checklist with checkboxes for local testing, tests added, docs updates, bot reviews, and comment resolution.
✅ Passed checks (12 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Swift Actor Isolation ✅ Passed PR adds only test code (XCUITest) with private value structs and helpers. No @MainActor annotations, no implicit MainActor coupling, no Sendable violations. Rules explicitly pass tests.
Cmux Swift Blocking Runtime ✅ Passed All blocking patterns (poll() for socket I/O, RunLoop UI animation delays) are test-only scaffolding in cmuxUITests directory and explicitly allowed per swift-blocking-runtime.md rules.
Cmux No Hacky Sleeps ✅ Passed PR adds only Swift UI test code. The check excludes Swift timing (covered by separate check) and test scaffolding. No production non-Swift code changes present.
Cmux Swift Concurrency ✅ Passed Test adds synchronous socket I/O helpers. No DispatchQueue, Combine, completion handlers, or fire-and-forget Tasks. RunLoop delays are test-only (allowed). XCTest boundary code.
Cmux Swift @Concurrent ✅ Passed No concurrent annotation violations. All new helper functions are synchronous with no async work or @concurrent annotations. Socket I/O is synchronous, acceptable in test code.
Cmux Swift File And Package Boundaries ✅ Passed Adds 304 lines to UI test file (1602→1906 lines). Test code with helper structs and socket-driven test utilities is allowed. No mixed responsibilities.
Cmux Swift Logging ✅ Passed Test file has no logging violations. CLI files use print() which is allowed for "CLI command output that is the intended user-facing result" per swift-logging.md rules.
Cmux Architecture Rethink ✅ Passed Test-only regression code. No production changes. Uses XCTest synchronization patterns. Complies with allowed test-only exception in architecture rules.
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed Test-only fixture (exempt per rule). All production windows have stable cmux.* identifiers registered in cmuxAuxiliaryWindowIdentifiers with proper close-shortcut handling.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding a regression test for terminal rendering during split/exit cycles.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch task-ghostty-render-split-cycle-regression

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented May 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds an XCUITest (testRepeatedCmdDSplitAndCtrlDExitKeepsLeftmostTerminalRendering) that stress-tests 10 consecutive Cmd+D/Cmd+D/Ctrl+D/Ctrl+D cycles and asserts the original leftmost Ghostty terminal surface survives and continues to render new output after every round. It introduces three private structs (TerminalSurfaceInfo, TerminalRenderStats, PanelSnapshotInfo) and a set of socket-helper methods to support the new assertions.

  • The core regression loop correctly captures a baseline surface ID, runs 10 split/close cycles via keyboard events, and re-verifies the same surface ID survives each round.
  • assertSurfaceRendersNewMarker validates rendering via three complementary checks (terminal text model, GPU render counters, pixel-change count), but the debug.panel_snapshot.reset result is silently discarded, which can cause accumulated changedPixels from prior cycles to inflate the pixel-change assertion in cycles 2\u201310.
  • The presentCount || drawCount OR guard fires as soon as any render event occurs, making the 6-second timeout effectively dead code once waitForSurfaceText has already confirmed text delivery.

Confidence Score: 3/5

The test-only change is safe to merge but has a correctness gap in its own assertion logic that could hide the very rendering regression it was written to catch.

The silent discard of debug.panel_snapshot.reset means that from cycle 2 onward the changedPixels > 20 guard is comparing against a never-zeroed accumulator, so a terminal that stops rendering new pixels would still pass if enough stale changes carried over from previous cycles. This makes the regression coverage weaker than it appears in later iterations of the loop.

cmuxUITests/BrowserPaneNavigationKeybindUITests.swift — specifically assertSurfaceRendersNewMarker and the panel snapshot reset path.

Important Files Changed

Filename Overview
cmuxUITests/BrowserPaneNavigationKeybindUITests.swift Adds a 10-cycle Cmd+D/Ctrl+D regression test with helper structs and socket helpers; debug.panel_snapshot.reset result silently discarded may inflate changedPixels across cycles, and the render-counter OR guard is weaker than it appears.

Sequence Diagram

sequenceDiagram
    participant Test as XCUITest
    participant App as cmux App
    participant Socket as Control Socket
    participant Terminal as Ghostty Terminal

    Test->>App: "launch (CMUX_UI_TEST_MODE=1)"
    Test->>Socket: ping → PONG
    Test->>Socket: surface.list → initialSurfaceId

    Test->>Socket: debug.terminal.render_stats (baseline)
    Test->>Socket: debug.panel_snapshot.reset
    Test->>Socket: debug.panel_snapshot (baseline snapshot)
    Test->>Socket: surface.send_text (marker command)
    Test->>Socket: surface.read_text (wait for marker text)
    Test->>Socket: debug.terminal.render_stats (wait counter advance)
    Test->>Socket: debug.panel_snapshot (after snapshot)
    Note over Test: Assert changedPixels > 20

    loop 10 cycles
        Test->>App: Cmd+D (split)
        Test->>Socket: "surface.list (wait count=2)"
        Test->>App: Cmd+D (split)
        Test->>Socket: "surface.list (wait count=3)"
        Test->>App: Ctrl+D (close)
        Test->>Socket: "surface.list (wait count=2)"
        Test->>App: Ctrl+D (close)
        Test->>Socket: "surface.list (wait count=1)"
        Test->>Socket: "surface.list → remaining[0].id == initialSurfaceId"
        Test->>Terminal: assertSurfaceRendersNewMarker(cycle N)
    end
Loading

Reviews (1): Last reviewed commit: "Add split cycle terminal rendering regre..." | Re-trigger Greptile

"\(context): expected left terminal window to be visible. stats=\(baselineStats)"
)

_ = socketResult(method: "debug.panel_snapshot.reset", params: ["surface_id": surfaceId])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Silent reset failure can inflate changedPixels across cycles

debug.panel_snapshot.reset discards its result, so a socket error or a missing implementation won't surface during the test. If the reset silently fails, the snapshot accumulator is never zeroed: every subsequent debug.panel_snapshot call for that surface will report changedPixels that include accumulated changes from all previous cycles. In cycle 2 onward, the changedPixels > 20 assertion can pass even if the terminal renders nothing new for the current marker, giving false confidence that the regression is caught.

Comment on lines +1491 to +1493
}
return stats.presentCount > baselineStats.presentCount
|| stats.drawCount > baselineStats.drawCount

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 OR between presentCount and drawCount weakens the render-advance assertion

The condition fires as soon as either counter moves for any reason (cursor blink, focus change, window resize) — it does not require both the draw and the present to have happened for this specific frame. Since waitForSurfaceText has already confirmed the text is in the terminal model before this block runs, the render counters will almost always be satisfied immediately on the first poll iteration, making the 6-second guard near-vacuous. Using AND (&&) would be a stricter signal that a full draw→present cycle completed for the marked frame.

@lawrencecchen lawrencecchen added the stale-revisit Closed after 30+ days without activity; preserved for possible revisit or reopening. label Sep 23, 2026
@github-project-automation github-project-automation Bot moved this from Todo to Done in cmux backlog Sep 23, 2026

This branch was successfully deployed

1 active deployment
Preview – cmux — f7f6a971 Deployed May 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale-revisit Closed after 30+ days without activity; preserved for possible revisit or reopening.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants