cmux-next: keystroke echo no longer waits for vsync (20.5 → 5.4 ms); typing probe

[teamleaderleo]

In cmux-next, a typed character took a median 20.5 ms from the key event until the frame showing it was handed to Core Animation (60 Hz). About 18 ms of that was Ghostty's renderer waiting for the next display-link tick. This PR bumps GhosttyNextKit to xcframework-2199dc93…-apple-v6 (manaflow-ai/ghostty-next#28), which draws the first frame after idle immediately. That brings the median to 5.4 ms with window-vsync still on. The PR also adds the opt-in probe and bench that measured it.

Before and after

Same host as hq#1373: a capture Mac (M4, macOS 26.4, 60 Hz), Debug fleet builds. Each arm is 61 real key-downs posted by CuaDriver to a zsh prompt, 300 ms apart. Times are in ms after NSEvent.timestamp.

hop	before eb550eec (median / p95)	after 2d003cfc (median / p95)
echo parsed on the output lane	2.36 / 3.27	2.74 / 4.26
frame set as IOSurfaceLayer.contents	20.46 / 28.16 (max 30.38)	5.39 / 7.14 (max 7.59)

Everything before the frame (AppKit dispatch, Ghostty io_write, attach socket, cmux-tui daemon, terminal host, shell echo, main-actor hop, parse) totals about 2.5 ms. The photon follows at the compositor's next vsync, at most one more refresh (16.7 ms). So the worst case on screen is now under about 25 ms, against 30 + 16.7 ms before. For comparison, window-vsync = false on the old build gave 4.37 / 6.85 ms; the fix gets nearly all of that while frames stay vsync-paced during streaming output.

hq#1373's method (terminal-render-soak.py: CLI write, then CLI screen read polling every 50 ms), on the fixed build: ack median 23.1 ms, visible median 39.2 ms (p95 67.4, range 33.6–71.8), from 64 samples. 12 more samples were spaces, which this harness can't detect, because screen reads trim trailing blanks. That number is the harness's floor: a baseline read, the ~23 ms CLI write and at least one poll. It isn't a frame time. Classic's 146 ms from 4 samples has the same structure.

Classic shares the cause. Its Ghostty fork (324c027) keeps the display link running while a surface is focused and defers every draw to vsync (Thread.zig:1788). Classic's fix would be the same idea, in that fork.

What's here

• Pin bump: Packages/Shared/CmuxGhosttyKit/Package.swift (checksum 8ef39066…, attested release) and the ghostty-next gitlink, with notices regenerated by ghosttykit_repin.py.
• TypingLatencyProbe (CMUX_NEXT_TYPING_PROBE=<csv>): one row per key across 8 hops. When unset, each hop costs one load. TypingFrameProbe adds a setContents: override to Ghostty's layer class only while the probe is on.
• scripts/cmux-next/bench-typing.py: posts keys with CuaDriver press_key and prints the per-hop and per-segment median, p95 and range.
• TypingLatencyProbeTests: hop ordering, and the probe is a no-op when disabled.

Verification

• The numbers above come from fleet builds cbfbe5e17ef2 (before) and 9896631dd64f (after). Per-arm summaries are in the PR comments.
• ghostty-next#28 was reviewed by a fresh subagent (verdict: ship). The residual risk it named: Ghostty's macOS 14.4+ warning about out-of-sync rendering. This change draws at most one unpaced frame per refresh and only from idle. I measured only on macOS 26.4.
• I didn't run the unit tests locally (Linux host). CI's cmux-next swift test runs them.

Changelog

• Fixed: terminal keystrokes reach the screen up to one display refresh sooner (median 20.5 ms → 5.4 ms to the presented layer at 60 Hz).

🤖 Generated with Claude Code

[teamleaderleo]

Raw probe data from fleet build cbfbe5e17ef2 (eb550eec), a capture Mac, 61 measured keys per arm.

probe-v2.summary.json

{
  "rows": 66,
  "complete": 66,
  "measured": 61,
  "io_write_on_main": 61,
  "hops": {
    "dispatch_start_ms": {
      "n": 61,
      "median": 0.59,
      "p95": 1.13,
      "min": 0.19,
      "max": 1.55
    },
    "dispatch_end_ms": {
      "n": 61,
      "median": 1.55,
      "p95": 2.25,
      "min": 0.55,
      "max": 2.55
    },
    "io_write_ms": {
      "n": 61,
      "median": 1.47,
      "p95": 2.15,
      "min": 0.53,
      "max": 2.49
    },
    "socket_submit_ms": {
      "n": 61,
      "median": 1.73,
      "p95": 2.42,
      "min": 0.63,
      "max": 2.78
    },
    "output_decoded_ms": {
      "n": 61,
      "median": 2.2,
      "p95": 3.08,
      "min": 0.85,
      "max": 3.23
    },
    "output_main_ms": {
      "n": 61,
      "median": 2.33,
      "p95": 3.24,
      "min": 0.88,
      "max": 3.45
    },
    "output_parsed_ms": {
      "n": 61,
      "median": 2.36,
      "p95": 3.27,
      "min": 0.9,
      "max": 3.5
    },
    "contents_ms": {
      "n": 61,
      "median": 20.46,
      "p95": 28.16,
      "min": 12.34,
      "max": 30.38
    }
  },
  "segments": {
    "event to dispatch": {
      "n": 61,
      "median": 0.59,
      "p95": 1.13,
      "min": 0.19,
      "max": 1.55
    },
    "dispatch to io_write": {
      "n": 61,
      "median": 0.71,
      "p95": 1.09,
      "min": 0.34,
      "max": 1.47
    },
    "io_write to socket": {
      "n": 61,
      "median": 0.22,
      "p95": 0.35,
      "min": 0.1,
      "max": 0.42
    },
    "socket to echo decoded (daemon, host, shell)": {
      "n": 61,
      "median": 0.44,
      "p95": 0.79,
      "min": 0.21,
      "max": 0.81
    },
    "decoded to main actor": {
      "n": 61,
      "median": 0.11,
      "p95": 0.19,
      "min": 0.04,
      "max": 0.25
    },
    "main actor to parsed": {
      "n": 61,
      "median": 0.04,
      "p95": 0.05,
      "min": 0.02,
      "max": 0.07
    },
    "parsed to frame contents": {
      "n": 61,
      "median": 18.34,
      "p95": 25.65,
      "min": 9.41,
      "max": 26.89
    }
  }
}

probe-v2-novsync.summary.json

{
  "rows": 66,
  "complete": 66,
  "measured": 61,
  "io_write_on_main": 61,
  "hops": {
    "dispatch_start_ms": {
      "n": 61,
      "median": 0.56,
      "p95": 1.57,
      "min": 0.34,
      "max": 2.29
    },
    "dispatch_end_ms": {
      "n": 61,
      "median": 1.41,
      "p95": 2.22,
      "min": 0.87,
      "max": 3.53
    },
    "io_write_ms": {
      "n": 61,
      "median": 1.34,
      "p95": 2.15,
      "min": 0.83,
      "max": 3.44
    },
    "socket_submit_ms": {
      "n": 61,
      "median": 1.59,
      "p95": 2.58,
      "min": 0.99,
      "max": 3.83
    },
    "output_decoded_ms": {
      "n": 61,
      "median": 2.2,
      "p95": 3.49,
      "min": 1.3,
      "max": 5.71
    },
    "output_main_ms": {
      "n": 61,
      "median": 2.29,
      "p95": 3.6,
      "min": 1.38,
      "max": 5.89
    },
    "output_parsed_ms": {
      "n": 61,
      "median": 2.34,
      "p95": 3.64,
      "min": 1.4,
      "max": 5.97
    },
    "contents_ms": {
      "n": 61,
      "median": 4.37,
      "p95": 6.85,
      "min": 3.02,
      "max": 7.8
    }
  },
  "segments": {
    "event to dispatch": {
      "n": 61,
      "median": 0.56,
      "p95": 1.57,
      "min": 0.34,
      "max": 2.29
    },
    "dispatch to io_write": {
      "n": 61,
      "median": 0.77,
      "p95": 1.22,
      "min": 0.48,
      "max": 1.38
    },
    "io_write to socket": {
      "n": 61,
      "median": 0.26,
      "p95": 0.41,
      "min": 0.15,
      "max": 0.47
    },
    "socket to echo decoded (daemon, host, shell)": {
      "n": 61,
      "median": 0.53,
      "p95": 0.95,
      "min": 0.28,
      "max": 1.87
    },
    "decoded to main actor": {
      "n": 61,
      "median": 0.1,
      "p95": 0.23,
      "min": 0.06,
      "max": 0.47
    },
    "main actor to parsed": {
      "n": 61,
      "median": 0.04,
      "p95": 0.06,
      "min": 0.02,
      "max": 0.08
    },
    "parsed to frame contents": {
      "n": 61,
      "median": 2.0,
      "p95": 3.95,
      "min": 0.72,
      "max": 4.18
    }
  }
}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: 2ab0f737-188d-4b25-a187-0a3230214670

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Not verified: macOS jobs did not run: compile and app tests were skipped on 819c86704a; ci-status is green without them.

CI passes on 819c86704a (run 37409141798 attempt 1).
macOS jobs did not run: compile and app tests were skipped: Fast static checks skipped (the macos job needs both to succeed). Re-run CI once they pass, before merging.

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way.

[teamleaderleo]

After the fix: fleet build 9896631dd64f (2d003cfc), same host, 61 measured keys.

probe-v3.summary.json

{
  "rows": 66,
  "complete": 66,
  "measured": 61,
  "io_write_on_main": 61,
  "hops": {
    "dispatch_start_ms": {
      "n": 61,
      "median": 0.64,
      "p95": 1.68,
      "min": 0.38,
      "max": 2.43
    },
    "dispatch_end_ms": {
      "n": 61,
      "median": 1.67,
      "p95": 2.85,
      "min": 0.98,
      "max": 3.73
    },
    "io_write_ms": {
      "n": 61,
      "median": 1.59,
      "p95": 2.8,
      "min": 0.94,
      "max": 3.65
    },
    "socket_submit_ms": {
      "n": 61,
      "median": 1.9,
      "p95": 2.96,
      "min": 1.1,
      "max": 4.03
    },
    "output_decoded_ms": {
      "n": 61,
      "median": 2.51,
      "p95": 4.08,
      "min": 1.48,
      "max": 6.88
    },
    "output_main_ms": {
      "n": 61,
      "median": 2.68,
      "p95": 4.22,
      "min": 1.56,
      "max": 6.99
    },
    "output_parsed_ms": {
      "n": 61,
      "median": 2.74,
      "p95": 4.26,
      "min": 1.6,
      "max": 7.03
    },
    "contents_ms": {
      "n": 61,
      "median": 5.39,
      "p95": 7.14,
      "min": 3.24,
      "max": 7.59
    }
  },
  "segments": {
    "event to dispatch": {
      "n": 61,
      "median": 0.64,
      "p95": 1.68,
      "min": 0.38,
      "max": 2.43
    },
    "dispatch to io_write": {
      "n": 61,
      "median": 0.93,
      "p95": 1.22,
      "min": 0.54,
      "max": 1.44
    },
    "io_write to socket": {
      "n": 61,
      "median": 0.31,
      "p95": 0.4,
      "min": 0.15,
      "max": 0.56
    },
    "socket to echo decoded (daemon, host, shell)": {
      "n": 61,
      "median": 0.54,
      "p95": 1.11,
      "min": 0.32,
      "max": 2.85
    },
    "decoded to main actor": {
      "n": 61,
      "median": 0.12,
      "p95": 0.21,
      "min": 0.07,
      "max": 0.26
    },
    "main actor to parsed": {
      "n": 61,
      "median": 0.04,
      "p95": 0.08,
      "min": 0.02,
      "max": 0.1
    },
    "parsed to frame contents": {
      "n": 61,
      "median": 2.23,
      "p95": 4.13,
      "min": 0.56,
      "max": 4.28
    }
  }
}

[teamleaderleo]

ios-simulator-build failed on a stale explicit-module cache on the persistent runner, not on code: ghostty.h has been modified since the module file GhosttyNextKit-….pcm was built (size changed from expected 52726 to 67538). The PCM is from the previous GhosttyKit pin. The fleet iOS build of this SHA (job 2c4987f6b499) passed. I've rerun the failed jobs.

cmux-b2 (unregistered)

[teamleaderleo]

Pre-merge review (subagent, correctness first): ship with nits, nothing blocking. With the probe off, nothing changes: the hot paths do no allocation, locking or environment reads, and the layer hook is never installed.

Fixed in 819c867:

• The setContents: block is now built in a nonisolated helper, so it carries no main-actor isolation check.
• bench-typing.py ignores rows left in the same CSV by an earlier run.

Left as is (probe-on only):

• The hook calls CALayer's IMP directly, not the superclass's.
• The sample struct is copied once per mark while the lock is held.
• disabledWithoutPathWritesNothing only checks that disabled calls don't crash.

The merge is still blocked on archive: vt-link-graph.json is still for ghostty-next 3320bd06 and needs regenerating on a Blacksmith Testbox, which I can't reach from here.

cmux-b2 (unregistered)

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.