Skip to content

fix(switch-frames): a switch to a hidden terminal shows its newest output in the first frame - #32588

Closed
lawrencecchen wants to merge 9 commits into
feat-cmux-nextfrom
fix/switch-preshow-hidden-output
Closed

lawrencecchen wants to merge 9 commits into
feat-cmux-nextfrom
fix/switch-preshow-hidden-output

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

SWITCH FRAMES SF-5, pre-show (lead decision on #32446). With attach-visibility-v1, a hidden terminal's output waits in the daemon (it is flushed once a second). When the terminal was shown again, that output came one round trip later. So a tab or workspace switch showed the older screen first, then the newest one a frame or more later.

Now the first frame of the switch shows the newest screen:

  • Daemon (attach-visibility-shown-v1):
    • A snapshot viewer that is shown again gets a shown event when its stream holds every byte printed before the show. shown carries shows, the number of shows answered, so the app can drop a marker from an older show.
    • The first output of each hidden period goes out at once, so the app knows whether the terminal printed while it was hidden. Any older batch goes out before it.
  • App:
    • The reveal sends visibility(true) on the switch's own turn, before the view shows.
    • The output lane counts every byte it queues. A terminal that got bytes after it was hidden waits for shown.
    • The waiting output is parsed on the output lane, off the main thread. The wait ends when a frame drawn after that parse presents.
    • The pane (tab switch) or the window (workspace switch) keeps what it showed until then. This uses PanePaintHold, the same hold that waits for a page's first paint.
    • If the parse is slow, the last frame stays; the older screen never shows. The limit is 500 ms, the same as every other hold.
    • A terminal that printed nothing while hidden shows at once.
  • Other changes:
    • GhosttyDiagnosticsModel moves from CmuxNextApp to CmuxNextTerminal (CmuxNextApp has fewer lines).
    • The hidden-step plan in TerminalAttachment moves to its own file (file cap).

Changelog

Fixed: Switching to a terminal that printed while it was hidden now shows its newest output in the first frame, not the older screen first.

Testing

Red commit f480352 (tests only, plus an inert caughtUp event):

Test Red (f480352, base behavior) Green (head)
Rust a_viewer_shown_again_gets_shown_after_every_byte_printed_while_hidden step 8bd98166: FAILED, "no shown within 500 ms of the show" (the hidden output arrived, but no marker) step 2c6c36f9 at 8cc932a: 53 of 53 hidden tests pass (the new one and both SF-5 batch tests)
Swift TerminalSwitchCatchUpTests x5 (tab switch, workspace switch) step 2920d077: failed on every run; the pane showed [old, new, new, ...] and [old, old, new, ...] step eb25d67e at 8cc932a: 5 of 5 runs pass (3 tests each); all six neighbor suites pass
clippy (-D warnings) step 1b595a2a at 8cc932a: pass

The green run also includes the neighbor suites TabSwitchNoBlankFrameTests, WorkspaceSwitchRedrawHoldTests, TerminalRevealFrameTests, EvictedTerminalRevealFrameTests, TerminalAttachHiddenBatchTests and TerminalAttachVisibilityWireTests.

Review (subagent, correctness first): three rounds. Round 1 FIX: off-main output did not mark the wait, and the workspace resize hold regressed. Round 2 FIX: a hidden period's first output could get ahead of an older batch. Both are fixed. Round 3: SHIP at 8cc932a.

Host switch-latency A/B (lead order; same builds and same mini for both arms)

Builds: base 314d542f (feat 25c84f3, which has the #32533 daemon fix) and fix c0496d4f (this PR merged into the same tip). Both have the bench configuration. Every hidden terminal prints all the time (beacons at 240 Hz). Times are in ms from the HID key-down, filmed at 120 Hz. The start tier is the core tier of the app's main thread, read per 1 ms with proc_pidinfo.

W1 LIGHT (9 workspaces x 2 beacon panes), cmux23, 3 rounds, 78 switches per arm:

measure base p50 / p95 / p99 fix p50 / p95 / p99
key -> first changed frame 41.7 / 53.8 / 59.0 43.1 / 54.5 / 55.5
key -> first complete target frame 61.3 / 75.6 / 84.0 60.7 / 72.2 / 80.3
key -> first frame with the newest output 83.7 / 96.7 / 105.1 80.7 / 97.0 / 100.4
refreshes from complete to newest 3 / 4 / 4 3 / 4 / 4
app intent -> CA commit (app timer) 23.2 / 33.7 / 36.1 23.5 / 32.3 / 35.5
main thread starts on Super / Perf / Eff 55 / 8 / 15 52 / 6 / 20

W1 HEAVY (9 workspaces x 12 beacon panes, 4 of them full-pane redraws, 40 workspaces), cmux23, 3 rounds, 54 switches per arm:

measure base p50 / p95 / p99 fix p50 / p95 / p99
key -> first changed frame 53.6 / 60.7 / 70.8 53.5 / 59.5 / 62.4
key -> first complete target frame 263 / 292 / 320 268 / 303 / 312
key -> first frame with the newest output 269 / 313 / 328 271 / 303 / 312
app intent -> CA commit 34.4 / 39.5 / 50.3 34.5 / 39.8 / 43.9

Result. The hold does not make a switch slower: every measure is the same within noise, under light and heavy load. It does not make the host-measured switch faster either:

Observed / Not run

  • Observed: the fleet red and green results above, and the compile check of the head (job 17dda4a4).
  • Not run:
    • The overflow reattach case. A burst at the show that overflows the stream reattaches without a status change, so it holds up to 500 ms. It is still correct and never shows an older frame. Follow-up: end the wait on the new link's first screen.
    • The show/hide race section of the Rust test is not forced; it only guards offset order when timing allows.
    • Known limit: if the program first prints while hidden and the switch comes back before that first byte reaches the app (one-way latency), the switch shows at once and the byte draws one frame later.

…utput in the first frame (red)

SWITCH FRAMES SF-5, pre-show (lead decision on #32446). Red:
- Rust: a viewer shown again gets `shown` after every byte printed
  while it was hidden, and none when it never hid.
- Swift: a tab switch and a workspace switch to a hidden terminal whose
  output waits in the daemon show only the newest screen, never the older
  screen first.
TerminalIOEvent.caughtUp is added inert so the test compiles.
…e until the terminal drew its waiting output

SWITCH FRAMES SF-5, pre-show (lead decision on #32446). With
attach-visibility-v1 a hidden terminal's output waits in the daemon (once a
second); shown again, it came one round trip after the show, so a switch
showed the older screen first and the newest one a frame or more later.

- Daemon (attach-visibility-shown-v1): a snapshot viewer shown again gets
  `shown` once its stream holds every byte printed before the show.
- App: the reveal, on the switch's own turn, already sends visibility(true)
  before the view shows. The session now asks its IO whether the daemon held
  output (before the IO hears the show) and waits for `shown`; the waiting
  output parses on the output lane (off main), and the session waits until a
  frame drawn after that parse presents (as TerminalFrameHold).
- The pane (tab switch) and the window (workspace switch) keep what they
  showed until then through PanePaintHold, as for a page's first paint: the
  first shown frame is the newest screen; a slow parse keeps the last frame,
  never shows the older screen. Bound: 500 ms, as every hold.
- TerminalAttachment's pure hidden-step plan moves to its own file (file cap).
… GhosttyDiagnosticsModel moves next to GhosttyRuntime

Keeps CmuxNextApp from growing: the catch-up finder is CmuxNextTerminal's,
and the Ghostty config diagnostics model (no App dependency) moves out of
the App with it.
…othing new ends the wait at the marker

- The daemon compares the output offset only (it runs on across grid
  generations): a resize while hidden no longer leaves the marker unsent.
- A terminal that got no output since it hid shows its current screen at
  the marker; it no longer waits for a new presented frame (the renderer
  may present none, which held the switch for the 500 ms bound).
…nted while hidden, by the lane's byte count

Review FIX at 1acfafd (must-fix):
- Output of a terminal with content goes to its lane off main and never
  reached the catch-up flags, so the wait ended at the marker before the
  owed bytes were parsed (older frame first). The lane now counts every
  byte queued; the catch-up compares it with its count at the hide.
- A workspace shown again at another size no longer asked its pages to
  redraw (the size check ran after the window set the new frame): the
  workspace keeps its parked size and compares that.
Should-fix:
- An idle hidden terminal no longer waits a daemon round trip: the daemon
  sends a hidden period's first output at once, so the app knows whether
  the terminal printed while hidden; one that did not shows at once.
- A new link's first READY or replay ends the wait (its attach starts
  shown, no marker comes); a link that leaves connected ends it.
- Only terminals inside the workspace's bounds hold a workspace switch.
- The catch-up reads the session's current surface (a replay swap).
Tests: the test daemon sends the first hidden bytes at once; a switch to
an idle hidden terminal shows it in its own turn.
…ver overtakes an older batch; shown names its show

- Must-fix: a show and a hide before the worker looped left the older
  period's batch in place, and the new period's first output went out
  ahead of it. The batch now goes first.
- `shown` carries the daemon's count of the viewer's shows; the app
  counts the shows it sent and drops an older show's marker.
- A READY or replay no longer counts as the marker (one in flight across
  the show could end the wait early); a new link ends the wait through
  its status, else the 500 ms bound.
- Rust test: the first hidden output arrives at once (< 300 ms), offsets
  stay contiguous through `shown`, and through a show-hide race.
@lawrencecchen lawrencecchen added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 2, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (subagent, correctness first): SHIP at 8cc932a

Three rounds. Round 1 (FIX at 1acfafd): output from the off-main fast path never marked the wait, and the workspace resize hold regressed. Round 2 (FIX at dbee88c): a hidden period's first output could get ahead of an older batch. Round 3: both fixed, and the shows counting matches one for one on the daemon and app sides. Left open, not blocking: an overflow reattach during the wait holds up to 500 ms (still correct); the Rust show/hide race section is not forced.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 8cc932a: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr32588 -f ref=fix/switch-preshow-hidden-output -f platform=macos -f backend_mode=local

@lawrencecchen lawrencecchen added ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds and removed ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds labels Nov 2, 2026
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 2, 2026 22:27 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 2, 2026 22:27 Inactive
@teamleaderleo

Copy link
Copy Markdown
Collaborator

Review (next-review patch-id; carried): SHIP at f3f0e11

The independent SHIP at 8cc932a carries to this head: the PR's own change (git diff $(git merge-base feat-cmux-next HEAD) HEAD) has the same git patch-id --verbatim (27bc9a2a4ba3f118) at both heads, and no merge between them was resolved by hand. Nothing new to review. A newer BLOCK, or any change to the PR's own diff, ends the carry.

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (subagent, correctness first): SHIP at f3f0e11

The head is cmux-bot's feat merge. The PR's own 28-file change is the same as at 8cc932a, which passed 3 review rounds: same patch ID, and no conflict was resolved in those files. Compile check of this head: job 651dcf22 passed (CmuxNext and app). Host A/B (lead order) is in the PR body: the change is not slower than base on any measure, LIGHT 78 and HEAVY 54 switches per arm on one mini.

@lawrencecchen lawrencecchen added ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds and removed ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds labels Nov 3, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Lead hold (perf lead): the title says a switch to a hidden terminal shows its newest output in the first frame. The host A/B in this body (STATUS 8093) shows newest output 3 refreshes after the complete frame on BOTH arms (base 83.7/96.7 ms, fix 80.7/97.0 ms p50/p95) and no other gain. So the host measure does not show the behavior the title claims, and +883 lines with no measured effect is not a root-cause fix. Before ready-to-land again, show one of these: (1) a host measure or a film where, with the fix, the first complete frame already has the newest output and on base it does not; or (2) why the host measure cannot see it, plus the in-process test that proves it, and a title that says what the PR proves. If the fix does nothing on the current base, close it and keep the follow-ups.

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Closing (switch-frames, on the lead's hold). The host data cannot back the title, and I will not spend more fleet time on rounds of the same builds.

What the host data says. In the LIGHT A/B (78 switches per arm, one mini, the same feat base with #32533), the newest output comes 3 refreshes after the first complete frame on BOTH arms (p50 3, p95 4). If base showed output that waited up to 1 s in the daemon, base would be later than the fix. It is not.

Most likely reason. On this base a switch takes about 23 ms to commit and about 60 ms to its first complete frame. The daemon sends the waiting output on the show, and one local round trip is far shorter than that. So on base the waiting bytes have already arrived before the target's first frame, and the defect this PR fixes does not happen on the host today. The remaining 3 refreshes are most likely the normal output lag of a busy terminal that is shown. #32638 measures that lag for a terminal that stays shown, with no switch.

What the PR proves only in process. TerminalSwitchCatchUpTests (fleet steps 2920d077 red, eb25d67e green) shows the race with a 5 ms daemon round trip and a pane that shows in the switch's own turn. The real app's switch is much slower than that, which hides the race.

What stays.

This branch was previously deployed

3 inactive deployments
ci-routing-agent — f3f0e116 Deployed Nov 2, 2026 by github-actions[bot] via cmux-next macOS placement #63893
artifacts — f3f0e116 Deployed Nov 2, 2026 by cmux-bot[bot] via publish tree-addressed cmux-tui #102795
sentry-dsyms — f3f0e116 Deployed Nov 2, 2026 by cmux-bot[bot] via upload cmux-tui dSYMs to Sentry #102795
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants