Skip to content

cmux-next agent pane: render at full rate until scrolls miss frames - #16487

Merged
teamleaderleo merged 8 commits into
feat-cmux-nextfrom
nx-pane-adaptive-rate
Oct 1, 2026
Merged

teamleaderleo merged 8 commits into
feat-cmux-nextfrom
nx-pane-adaptive-rate

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Follows #16471, per #16424. The agent pane renders at the display's full rate (160 Hz on a 160 Hz display) while scrolls keep up, and drops to WebKit's 80 Hz cap while the machine is loaded. The aim is 160 Hz on an idle machine and a steady 80 Hz under load, with no flag to dogfood.

How it works.

  • Page. ScrollPacing samples rAF intervals while the transcript scrolls. Once the scroll has been still for 250 ms, it posts them as pane.framePacing. That moment is an idle point, so a rate change never lands mid-scroll.
  • Rate choice. AgentPaneFramePacing decides the rate from those intervals:
    • Full rate drops to the cap when more than 20% of a scroll's frames are late (over 1.5× the expected interval).
    • After a 10 s backoff, a capped scroll with at most 5% late frames brings full rate back.
    • The backoff doubles, up to 160 s, if full rate fails again within it.
    • Displays already near 60 Hz are left alone.
  • Native. AgentPaneView switches WebKit's PreferPageRenderingUpdatesNear60FPSEnabled on the live page's preferences.
  • Defaults. AgentTabStore makes every pane adaptive. Debug builds can fix the rate with CMUX_NEXT_AGENT_PANE_FULL_RATE (1 full, 0 capped).
  • Debug tool. debug.agent_pane gains full_rate, which reads or switches the live rate. Every action now turns off the pane's window-occlusion detection, so a tagged build can be measured behind other windows.

When the live switch takes effect (unverified on hardware).

  • WebKit recomputes a page's rendering rate only on a screen, visual-idle, low-power, thermal or animation-rate change. Changing the preference itself doesn't trigger a recompute.
  • On macOS, visual idle flips once the app's windows stop updating and again on the next interaction, so a rate set when a scroll settles should apply at that edge.
  • A reload keeps the same Page, so there is no reload fallback.
  • No awake display above 60 Hz was available to measure this. The fleet minis are 60 Hz, where both rates are the same.
  • Worst case, if the switch never applies live: an adaptive pane stays at the full rate it was created with, which is the same as cmux-next agent pane: a flag to render at the full display rate (off by default) #16471's switch on. Details are in the PR comments.

Testing

  • AgentPaneFramePacingTests (swift test lane): red at 25af4f2f99a, where a stub always renders at full rate; five tests fail, including aScrollThatMissesFramesDropsToTheCappedRate. Green on the head.
  • AgentPaneRenderingTests.anAdaptivePaneCapsItsRateAfterAScrollThatMissesFrames covers the wiring from the model through the view to the preference.
  • AgentPaneHandshakeTests covers decoding pane.framePacing.
  • pacing.test.ts covers settle, report and stop.
  • bun test src/agent-session/ and tsc are clean.
  • check-concurrency, check-crash-safety and build-agent-pane-web.sh --check pass.

Changelog

  • Changed: The agent pane scrolls at the display's full refresh rate, and drops to 80 Hz while the Mac is too busy to keep up.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD

teamleaderleo and others added 3 commits October 1, 2026 16:00
AgentPaneView.rendersAtFullRate reads and sets WebKit's near-60 fps
feature on the live web view's preferences, and debug.agent_pane gets a
full_rate action to drive it, so the fling bench can measure whether a
live switch takes effect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD
Red first, with a stub that always renders at full rate. A scroll that
misses frames at full rate should drop the pane to WebKit's capped
rate. A clean capped scroll after a backoff should restore full rate,
and the backoff should double on a quick relapse. A display already
near 60 Hz should be left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD
WebKit pauses rendering in a web view whose window is covered, so a
fling in a tagged build behind the user's windows got no frames. Each
debug.agent_pane action now turns off the pane's window-occlusion
detection (_setWindowOcclusionDetectionEnabled:, when present).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD
@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 2a54786b-ed18-40ec-913b-08989e973e2e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

teamleaderleo and others added 2 commits October 1, 2026 16:27
#expect can't call the mutating record(...), so the frame pacing tests
didn't compile. Each decision is now taken first and then checked. The
stub still always renders at full rate, so the tests stay red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD
AgentPaneFramePacing picks the rate from each settled scroll:
- Full rate drops to WebKit's cap when more than 20% of a scroll's
  frames are late (over 1.5x the expected interval).
- After a 10 s backoff, a capped scroll with at most 5% late frames
  brings full rate back.
- The backoff doubles, up to 160 s, when full rate fails again within
  it.
- A display already near 60 Hz is left alone.

The page samples rAF intervals while the transcript scrolls
(ScrollPacing). It sends them as pane.framePacing once the scroll has
been still for 250 ms, an idle point. The model hands them to the view,
which switches the live page's preference. AgentTabStore makes every
pane adaptive. Debug builds can still fix the rate with
CMUX_NEXT_AGENT_PANE_FULL_RATE (1 full, 0 capped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Red at 25af4f2f99a, where the stub always renders at full rate. Five AgentPaneFramePacingTests fail, all the cases that expect a drop to the cap, for example aScrollThatMissesFramesDropsToTheCappedRate: Expectation failed: !(decision2). The earlier red at 129253bae2e was a compile error (#expect can't call a mutating method); 25af4f2f99a fixes that. The implementation is now 742521f78f1.

On the live switch, WebKit main's source says:

  • PreferPageRenderingUpdatesNear60FPSEnabled has no webcoreOnChange.
  • Page re-runs adjustRenderingUpdateFrequency / renderingUpdateFramesPerSecondChanged only on a screen change (with an early return if unchanged), a visual-idle change, a low-power change, a thermal change or an animation-rate change.
  • On macOS, IsVisuallyIdle flips when WindowServer reports the app's window modifications have stopped, and flips back on the next interaction. A hide/show of the pane also changes it.

So a rate set when a scroll settles should take effect at the next visual-idle edge. A reload keeps the same Page and its scheduler, so a reload fallback wouldn't help, and I left it out.

This is unverified on hardware: no awake display above 60 Hz is available. The fleet minis run at 60 Hz, where capped and full are the same rate; a probe on cmux-mac-mini counted 120 frames per 2 s in every mode. Worst case, if the switch never applies live, an adaptive pane stays at the full rate it was created with. That is the same as #16471's switch on.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Requires human review: Auto-approval blocked because this review re-detected 2 unresolved issues already reported by Cubic.

Re-trigger cubic

teamleaderleo and others added 2 commits October 1, 2026 16:35
…ptive-rate

# Conflicts:
#	Packages/macOS/CmuxNext/Sources/CmuxNextAgentPane/Resources/agent-pane/index.html
The render-rate declarations had landed between close() and its doc
comment. displayFramesPerSecond's doc now says what it is: the fallback
when the pane has no window screen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Subagent review at 742521f, with checks run on 45fe16b (a base merge that changes no PR logic): LGTM.

  • The governor matches all seven tests (traced by hand), and cappedInterval gives 160 Hz → 12.5 ms, 120 → 16.67, 144 → 13.89, and 60 → the same rate, so it's left alone.
  • count(where:) is in the Swift 6.0 standard library. onFramePacing is MainActor with a weak self.
  • The DEBUG SPI is guarded. pane.framePacing passes the bridge's trusted-frame check and fails closed on non-numbers.
  • The page's rAF loop stops on unmount, and Release is adaptive.

Risk, accepted and noted in the body: if the live switch lags a scroll, the governor judges that scroll against the wrong rate, which could push backoffs toward 160 s capped. Hardening (infer the active rate from the median interval) is a follow-up once someone can dogfood at more than 60 Hz.

Nits: a1a51ce1673 fixes the two doc nits (close()'s comment, the displayFramesPerSecond doc). Left as they are: idle frames in the 250 ms settle window dilute the late share slightly, and the rendering tests return without checking anything on a WebKit without the feature.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

CI failure attribution

CI passes on 4d20a974b6 (run 36923408339 attempt 1).

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Subagent review at 4d20a97: LGTM.

  • Since the last review at 742521f, the only changes are the close() doc comment move (a1a51ce) and the lint fix in pacing.test.ts (4d20a97, Array.from(timers), which still takes a snapshot before deleting). The base merge only moves hunk offsets.
  • Re-checked: the governor's late shares, backoff (doubles up to 160 s, resets on success) and no-decision cases (60 Hz, fewer than 30 frames); pacing.ts sampling and reporting.
  • bun test src/agent-session/ 142/142, tsc clean, oxlint --deny-warnings clean, bundle current.
  • Optional nit: the renderRate init doc could say it defaults to .capped.
  • Still unverified: the live 60 fps switch on 120 Hz hardware (see PR body).

@teamleaderleo
teamleaderleo merged commit c7b4635 into feat-cmux-next Oct 1, 2026
52 checks passed
@teamleaderleo
teamleaderleo deleted the nx-pane-adaptive-rate branch October 1, 2026 20:50
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Live 120 Hz verification of the render-rate switch, run after merge on a fleet mini.

Setup:

  • cmux-mac-mini (M4, macOS 26.3.1). Its physical display is 60 Hz, so I added a 1920×1080 virtual display at 120 Hz through the private CGVirtualDisplay API, the same one BetterDisplay uses.
  • The virtual display becomes the main display, and CVDisplayLink on it ticks at p50 8.33 ms.
  • I tore it down after each run, and the mini is back to its 60 Hz display.
  • Build: fleet nx-measured-v3 at 58ecdae (feat-cmux-next plus cmux-next agent pane: place drawn rows by their drawn height #16491), mock pane, 5000 rows, the fling debug action.
mode (CMUX_NEXT_AGENT_PANE_FULL_RATE) frames/fling p50 p95 dropped
0 (capped) 181 17 ms 18 ms 0
1 (full) 360–361 8 ms 9 ms 1–2
unset (adaptive) 360–361 8 ms 9–10 ms 1–2
  • The capped setting works: on a 120 Hz screen, capped renders at 60 Hz, and full and adaptive render at 120 Hz.
  • Adaptive under CPU load: I ran 4 flings with yes on every core, then again with 4× oversubscription (40 processes). The pane stayed at 120 Hz with p95 10 ms and 0–2 dropped, so the governor correctly stayed at full rate.
  • Not exercised live: the drop to capped and the backoff. CPU saturation doesn't make WebContent frames late on an M4. Triggering it would need a render-bound load, such as a debug action that stalls frames. Those decisions are covered by AgentPaneFramePacingTests.
  • Still unverified: the live toggle mid-session (full_rate debug action) at the next visual-idle edge. These runs set the mode at launch.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

On the same 120 Hz virtual display, toggling the rate mid-session does nothing.

I launched capped (CMUX_NEXT_AGENT_PANE_FULL_RATE=0) with 5000 rows and toggled full rate with the full_rate debug action:

step frames/fling p50
capped at launch 181 17 ms
full_rate → true, flings 1–3 181 / 181 / 181 17 ms
full_rate → false, flings 1–2 182 / 181 17 ms
  • The preference applies only when the page is created. Setting PreferPageRenderingUpdatesNear60FPSEnabled on a live page doesn't change the rate within at least three flings (about 20 s), so flinging doesn't reach a visual-idle edge.
  • What adaptive actually does today: it renders at full rate for the life of the pane, which is what both adaptive runs above showed. The governor's decisions to drop to capped never reach WebKit. That's the worst case the PR body called acceptable (the same as switch-on), but the drop to capped under load isn't functional.
  • Options for a follow-up:
    • Force a rendering-frequency re-evaluation after the switch, for example a brief hide and show of the web view, after checking it doesn't flicker.
    • Rebuild the page on a switch (the reload fallback we skipped; a reload keeps the same Page, so it would need a new web view).
    • Drop adaptive and pick one rate per launch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant