Skip to content

fix: typing/CLI race on terminal surfaces (#3339) - #3341

Closed
austinywang wants to merge 2 commits into
mainfrom
issue-3339-typing-cli-race
Closed

austinywang wants to merge 2 commits into
mainfrom
issue-3339-typing-cli-race

Conversation

@austinywang

@austinywang austinywang commented Apr 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #3339 by serializing Ghostty surface operations that can be reached from both local terminal typing and cmux CLI commands.

Reproduction

Before changing code, I reproduced the corruption against an existing tagged DEBUG build using the debug socket and /tmp/cmux-cli:

  • Workspace: workspace:13
  • Typing surface: surface:13
  • Concurrent CLI surface: surface:14
  • Started cat on the typing surface.
  • Typed this expected line into the terminal while high-frequency cmux read-screen, cmux tree, and sibling cmux send commands ran against the same workspace:
you need to reproduce it first and then write a failing test and then

Observed corrupted rendered line under CLI pressure:

you need tuce it first and then write a failing test

Root Cause

The local keyboard path and socket/CLI paths both reached Ghostty surface APIs without a single serialization boundary. Main-queue scheduling ordered some Swift work, but Ghostty surface reads, writes, refreshes, focus/geometry changes, and teardown could still interleave through separate call sites, allowing CLI-side stale refresh/read activity to race with local input echo/rendering.

Fix

  • Added a GhosttySurfaceOperationGate backed by NSRecursiveLock.
  • Routed Ghostty surface read/write/refresh/focus/geometry/teardown calls through that gate.
  • Added a regression test that simulates concurrent typing and stale CLI refresh writes and asserts the rendered line remains the typed text.

Verification

  • git diff --check
  • ./scripts/reload.sh --tag issue-3339-typing-cli-race --launch

Local XCTest runs were intentionally not run because repository instructions for this task said not to run local tests.


Note

Medium Risk
Touches many call sites around the Ghostty surface C-API and changes their synchronization behavior; bugs could manifest as deadlocks or latency regressions on input/render paths if the lock is misused.

Overview
Prevents a typing corruption race by introducing GhosttySurfaceOperationGate (an NSRecursiveLock) and routing Ghostty surface operations through it.

This wraps surface lifecycle (create/free), geometry/display updates, focus/occlusion changes, text/selection reads, refreshes, and key/text input calls in both GhosttyTerminalView and TerminalController so CLI/socket commands can’t interleave with local typing on the same runtime surface.

Adds a regression test (TerminalSurfaceTypingCLIRaceTests) that simulates concurrent typing and a CLI refresh and asserts the final rendered line remains intact.

Reviewed by Cursor Bugbot for commit 1feab55. Bugbot is set up for automated code reviews on this repo. Configure here.


Summary by cubic

Serialize Ghostty surface operations to prevent typed input from being corrupted when CLI commands (read-screen/tree/send) run at the same time. Fixes #3339.

  • Bug Fixes
    • Added GhosttySurfaceOperationGate using NSRecursiveLock to serialize native ghostty_surface_* reads/writes/refresh/focus/geometry/free.
    • Routed relevant surface calls in GhosttyTerminalView, GhosttyNSView, and TerminalController through the gate.
    • Added a regression test that simulates concurrent typing and CLI refresh; asserts the rendered line matches the typed text.

Written for commit 1feab55. Summary will update on new commits. Review in cubic

Summary by CodeRabbit

Release Notes

  • Bug Fixes
    • Improved terminal stability by synchronizing concurrent rendering and input operations to prevent race conditions.
    • Enhanced reliability of terminal display updates and keyboard input processing during simultaneous interactions.

The repro shows local typed input losing characters while read-screen, tree, and sibling send commands pressure the same workspace. This regression adds a simulated terminal surface that models stale CLI refresh state overwriting a concurrently typed line, with the operation gate intentionally still a pass-through in this commit.

Constraint: Local tests are not run per task and repo instructions; this commit is the failing-test half of the required two-commit structure.

Confidence: medium

Scope-risk: narrow

Tested: Not run locally by instruction

Not-tested: XCTest execution; final verification is deferred until after the fix/build path
Local keyboard input, socket send/read-screen, refresh, focus, geometry, and teardown all cross the same native Ghostty surface boundary. The repro shows a stale CLI-side surface operation can overlap a typed line and drop rendered characters, so this adds a recursive operation gate and routes the relevant surface API calls through it.

Constraint: Do not use sleeps/retries; preserve socket command behavior and keep the change inside the native surface access boundary.

Rejected: Debounce or slow cmux read-screen/tree/send loops | hides the race and changes automation responsiveness

Rejected: Only lock cmux send | read-screen/refresh and local typing still share the native surface state

Confidence: medium

Scope-risk: moderate

Tested: git diff --check

Not-tested: Local XCTest by instruction; final tagged reload build still pending
@vercel

vercel Bot commented Apr 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment Apr 30, 2026 2:55am
cmux-staging Building Building Preview, Comment Apr 30, 2026 2:55am

@coderabbitai

coderabbitai Bot commented Apr 30, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Introduces GhosttySurfaceOperationGate, a global serialized lock using NSRecursiveLock, that gates nearly all Ghostty surface C-API invocations across lifecycle, render, input, and selection operations to eliminate concurrent access race conditions between local input rendering and external CLI operations.

Changes

Cohort / File(s) Summary
Ghostty Surface Synchronization Gate
Sources/GhosttyTerminalView.swift
Adds GhosttySurfaceOperationGate global lock and wraps all Ghostty C-API calls—lifecycle (ghostty_surface_free), render/topology setters (ghostty_surface_set_display_id, ghostty_surface_set_size, etc.), input delivery (ghostty_surface_key, ghostty_surface_text), and selection/accessibility reads—within synchronized blocks to serialize surface access.
Terminal Controller Input Routing
Sources/TerminalController.swift
Routes terminal text-read and key-event injection paths through GhosttySurfaceOperationGate.sync to align with synchronization requirements, preserving existing nil/empty-string handling and deferred cleanup via ghostty_surface_free_text.
Concurrency Race Test
cmuxTests/TerminalAndGhosttyTests.swift
Adds new test case simulating concurrent typing and CLI surface refresh using semaphore coordination, validating that gated operations prevent buffer corruption when render and refresh operations execute in parallel.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

Poem

🐰 A lock to tame the racing keys,
Where typing danced with send and read,
Now Gate keeps order, calm as breeze,
No brackets lost, no garbled thread.
At last, the terminal text runs free! 🎯

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 9.30% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed Title clearly summarizes the main change: fixing a typing/CLI race on terminal surfaces with issue reference.
Description check ✅ Passed Description follows the template with Summary, Reproduction, Root Cause, Fix, and Verification sections; all key information provided.
Linked Issues check ✅ Passed Changes fully address issue #3339 requirements: serialized Ghostty surface operations prevent typing corruption from concurrent CLI commands.
Out of Scope Changes check ✅ Passed All changes directly support the core objective: adding GhosttySurfaceOperationGate, serializing surface operations, and adding regression test.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-3339-typing-cli-race

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 1/8 review remaining, refill in 50 minutes and 36 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented Apr 30, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

Fixes the typing/CLI race (#3339) by introducing GhosttySurfaceOperationGate, an NSRecursiveLock-backed gate that serializes all Ghostty surface reads, writes, refresh, focus, geometry, and teardown calls across both the local keyboard path and the socket/CLI path. The regression test correctly demonstrates that the gate prevents a read-modify-write corruption between a typing thread and a concurrent CLI refresh thread.

Two design concerns stand out:

  • The gate is process-global rather than per-surface, so a CLI operation on any surface blocks keyboard events on every other unrelated surface simultaneously.
  • Lock acquisition is now on the keystroke hot path (sendGhosttyKey / keyDown), which CLAUDE.md explicitly marks as latency-sensitive; under high-frequency CLI load the full CLI read hold time becomes visible typing stall.

Confidence Score: 4/5

Safe to merge as a correctness fix; the global-lock scope and keystroke-path latency are design trade-offs worth revisiting but not blockers.

All findings are P2. The gate correctly eliminates the reported data corruption. The global lock scope and typing-path latency are real concerns in multi-surface workloads and under heavy CLI load, but neither is an immediate correctness regression. The missing two-commit structure is a process violation only.

Sources/GhosttyTerminalView.swift — global lock scope and keystroke-path lock acquisition deserve follow-up before this pattern is extended further.

Important Files Changed

Filename Overview
Sources/GhosttyTerminalView.swift Adds process-global GhosttySurfaceOperationGate (NSRecursiveLock) and wraps ~25 Ghostty surface call sites; the global scope serializes unrelated surfaces and adds lock acquisition to every keystroke on a latency-sensitive path
Sources/TerminalController.swift Routes CLI-side ghostty_surface_read_text and ghostty_surface_key calls through the gate; correctly serializes the CLI path against local typing
cmuxTests/TerminalAndGhosttyTests.swift Adds TerminalSurfaceTypingCLIRaceTests; test correctly demonstrates serialization prevents read-modify-write corruption but was not split into a failing-first commit per the repo policy

Sequence Diagram

sequenceDiagram
    participant KB as Local Keyboard (Main Thread)
    participant Gate as GhosttySurfaceOperationGate
    participant CLI as CLI Thread (cmux read-screen)
    participant GS as Ghostty Surface C API

    CLI->>Gate: sync { ghostty_surface_read_text }
    Gate-->>CLI: lock acquired
    CLI->>GS: ghostty_surface_read_text(surface, ...)
    Note over CLI,GS: CLI holds lock for duration of read

    KB->>Gate: sendGhosttyKey → sync { ghostty_surface_key }
    Note over KB,Gate: ⚠️ Blocked — waits for CLI to release
    CLI->>GS: ghostty_surface_free_text(surface, ...)
    Gate-->>CLI: lock released
    KB->>Gate: sync { ghostty_surface_key }
    Gate-->>KB: lock acquired
    KB->>GS: ghostty_surface_key(surface, keyEvent)
    Gate-->>KB: lock released
Loading

Comments Outside Diff (2)

  1. Sources/GhosttyTerminalView.swift, line 9-21 (link)

    P2 Process-global lock serializes unrelated terminal surfaces

    GhosttySurfaceOperationGate holds a single NSRecursiveLock for the entire process. A cmux read-screen on surface:14 will now block keyboard events on an unrelated surface:1 (different workspace, different pane) until the read completes. The race described in Typing into terminal corrupted while cmux CLI command runs concurrently #3339 is between a typing surface and a CLI surface operating on the same surface; a per-surface lock would prevent the actual race without coupling unrelated surfaces.

    Consider holding the lock as a property of TerminalSurface and passing a sync closure through that instance, or keying the gate on the ghostty_surface_t pointer.

  2. cmuxTests/TerminalAndGhosttyTests.swift, line 631-693 (link)

    P2 Regression test commit policy: test and fix committed together

    Per CLAUDE.md's regression test commit policy, a two-commit structure is required so CI can prove the test catches the bug:

    • Commit 1: failing test only → CI goes red
    • Commit 2: fix → CI goes green

    The test and fix appear in a single commit, so there is no CI evidence that the test would have failed without the gate. The test verifies the gate's serialization contract (correctly), but it can't serve as a proven regression guard without the red-first step visible in the PR's commit history.

    Context Used: CLAUDE.md (source)

Reviews (1): Last reviewed commit: "Serialize Ghostty surface operations acr..." | Re-trigger Greptile

Comment on lines 7678 to +7683
#if DEBUG
Self.debugGhosttySurfaceKeyEventObserver?(keyEvent)
#endif
return ghostty_surface_key(surface, keyEvent)
return GhosttySurfaceOperationGate.sync {
ghostty_surface_key(surface, keyEvent)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Lock acquisition on every keystroke on typing-latency-sensitive path

sendGhosttyKey — and by extension the entire keyDown / keyUp pipeline — now acquires GhosttySurfaceOperationGate on every keystroke. Per CLAUDE.md, this path is explicitly flagged as latency-sensitive: "Do not add allocations, file I/O, or formatting here." Under the exact high-frequency-CLI scenario described in the PR, a cmux read-screen holding the lock while reading a large buffer will block every pending keystroke for the full duration of that read.

The PR's own test demonstrates the CLI hold time can reach 100 ms (Thread.sleep(forTimeInterval: 0.10)). With a global lock this directly translates to 100 ms of input stall per read cycle.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@Sources/GhosttyTerminalView.swift`:
- Around line 21-33: The current gate allows callers to capture self.surface
before locking, risking use-after-free when teardownSurface/deinit nils and
frees the pointer; add a new helper on GhosttySurfaceOperationGate (e.g.
withLockedSurface<T>(_ body: (OpaquePointer) throws -> T) rethrows -> T?) that
calls sync and, inside the lock, reads self.surface, validates it's non-nil and
not freed, then passes the safe snapshot into the body; update call sites that
currently do guard let surface = surface before locking (sendText, sendNamedKey,
updateSize, setFocus and the GhosttyNSView send/read helpers) to use
withLockedSurface so the surface is captured and validated inside
GhosttySurfaceOperationGate.sync, and ensure teardownSurface/free paths still
nil and free the pointer as before.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4f65287c-e27b-45c6-aca6-de953efa8c24

📥 Commits

Reviewing files that changed from the base of the PR and between 9713725 and 1feab55.

📒 Files selected for processing (3)
  • Sources/GhosttyTerminalView.swift
  • Sources/TerminalController.swift
  • cmuxTests/TerminalAndGhosttyTests.swift

Comment on lines +21 to +33
enum GhosttySurfaceOperationGate {
private static let lock: NSRecursiveLock = {
let lock = NSRecursiveLock()
lock.name = "com.cmux.ghostty-surface-operation-gate"
return lock
}()

static func sync<T>(_ operation: () throws -> T) rethrows -> T {
lock.lock()
defer { lock.unlock() }
return try operation()
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | 🏗️ Heavy lift

Serialize surface lookup with the operation, not just the FFI call.

The new gate still lets callers capture self.surface before locking. That leaves a use-after-free window: teardownSurface()/deinit nil out self.surface, then free the old pointer under this gate later; a background sender can grab the old pointer first, block behind ghostty_surface_free, and then call ghostty_surface_text/ghostty_surface_key on freed memory once the lock opens. Please add a helper that snapshots and validates the live surface inside GhosttySurfaceOperationGate.sync and route the send/size/focus paths through it.

Suggested direction
 enum GhosttySurfaceOperationGate {
     private static let lock: NSRecursiveLock = {
         let lock = NSRecursiveLock()
         lock.name = "com.cmux.ghostty-surface-operation-gate"
         return lock
     }()

     static func sync<T>(_ operation: () throws -> T) rethrows -> T {
         lock.lock()
         defer { lock.unlock() }
         return try operation()
     }
 }
+
+extension TerminalSurface {
+    private func withLockedSurface<T>(_ body: (ghostty_surface_t) throws -> T) rethrows -> T? {
+        try GhosttySurfaceOperationGate.sync {
+            guard let surface = self.surface else { return nil }
+            return try body(surface)
+        }
+    }
+}

Then convert paths that currently do guard let surface = surface before locking, e.g. sendText, sendNamedKey, updateSize, setFocus, and the corresponding GhosttyNSView send/read helpers, to use withLockedSurface(...).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Sources/GhosttyTerminalView.swift` around lines 21 - 33, The current gate
allows callers to capture self.surface before locking, risking use-after-free
when teardownSurface/deinit nils and frees the pointer; add a new helper on
GhosttySurfaceOperationGate (e.g. withLockedSurface<T>(_ body: (OpaquePointer)
throws -> T) rethrows -> T?) that calls sync and, inside the lock, reads
self.surface, validates it's non-nil and not freed, then passes the safe
snapshot into the body; update call sites that currently do guard let surface =
surface before locking (sendText, sendNamedKey, updateSize, setFocus and the
GhosttyNSView send/read helpers) to use withLockedSurface so the surface is
captured and validated inside GhosttySurfaceOperationGate.sync, and ensure
teardownSurface/free paths still nil and free the pointer as before.

This branch was successfully deployed

1 active deployment
Preview – cmux — 1feab558 Deployed Apr 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Typing into terminal corrupted while cmux CLI command runs concurrently

1 participant