Skip to content

perf(cmux-next): hidden agent chat pages freeze like browser tabs; background-opened tabs start their hidden clock - #27047

Merged
teamleaderleo merged 10 commits into
feat-cmux-nextfrom
perf/agent-page-freeze
Nov 1, 2026
Merged

teamleaderleo merged 10 commits into
feat-cmux-nextfrom
perf/agent-page-freeze

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Leo, 2026-10-28: "There's just loads of nonsense in my sidebar that I'm not even looking at. So how could rendering be this fing slow? I'm at, what, 40% CPU, 78% RAM? You have room. So you're just not properly fing using it."

What

Hidden cmux pages now freeze the way hidden browser tabs already do.

  1. Hidden agent chat pages freeze (AgentPageHibernation). A hidden chat's Chromium page freezes after browser.freezeHiddenTabs seconds (30 by default). It uses the tab's own BrowserFreezing control, the same CEFTab.setFrozen that browser tabs use. It thaws:

    • when it shows,
    • when its turn starts or waits on the person,
    • right before a release (the release asks the page for its state and frame).

    The page keeps everything it holds; nothing reloads.

    Before this, a hidden chat page kept running every push, timer and CSS animation until its whole workspace hibernated after 60 idle minutes. The 30 s freeze only covered cache.browsers (BrowserHibernation.swift:436-437).

    Freezing has its own eligibility (keepsAwake), not the release rules:

    • These stay awake: a turn that works or waits on the person, a down turn watch, a pinned tab (unless pinned tabs hibernate), a New Tab page, a Chief chat, a crashed page, and a page still in its window.
    • These still freeze: a shell command run in the chat and another machine's chat. A freeze drops nothing, so neither needs to stay awake.

    browser.hibernation: off turns it off, as it does for browser tabs.

  2. Pages built while their tab is not drawn start their hidden clock. This covers chats restored or opened in the background, which never "hid" and so never froze.

  3. ContentLifecycle .adopted conceals. A browser tab whose page was made in the background was recorded .mountedHidden but got no conceal effect (ContentLifecycle.swift:238-244). So hiddenAt was never set and it never froze until it had been shown once. It now gets the same conceal as any tab that stops showing:

    • setContentVisible(false), so Chromium counts it hidden and the fork's freeze accepts it;
    • tabDidHide, which starts the hidden clock.

    The conceal takes no snapshot here, because the page was never shown.

No default changes and nothing is removed. browser.freezeHiddenTabs, browser.hibernation and the discard rules are unchanged.

How this differs from #26975

#26975 (draft) hides the parked New Tab spare and the pooled Settings host. It is in NewTabSparePool, PageHostPool, AgentPageRedraw and AgentPaneView+Warmup.

This PR freezes real hidden chat tabs and background-opened tabs. It touches only AgentPageHibernation and ContentLifecycle, and it excludes New Tab pages so it never meets the spare pool. The two PRs share no files and stack cleanly.

Evidence (before)

The before numbers are from a heavily loaded dogfood Mac, measured read-only. It had 134 workspaces, about 3,000 acpmux sessions and 38 streaming, and the person was idle:

  • The Chromium renderer of agent chat pages ran at 23–50 % CPU and 647 MB, with its main thread 37–50 % busy.
  • Two other cmux-page renderers ran at up to 10–11 % CPU each.
  • The GPU process ran at 15–20 % CPU.

Performance

Harness: scripts/measure/perf-10x-idle.sh (cmux-ci run --class exclusive --measurement, both runs on the same fleet Mac, fresh home), counts=1 agents=20 measure=120: 20 idle agent chats on the fake ACP agent, one workspace each, so 19 hidden; 20 s settle, then 120 s measured. Rows are the idle-20 sample. Before is 9d2e8cc8 (the base with #28433's harness fix, the first commit where this bench opens its chats); after is the same commit plus this PR's diff (7b5c3e85, a measure-only branch). One run each.

metric before after delta harness
chat renderers (cef-renderer, n=24) CPU % 5.012 4.703 -0.31 pt (-6 %) perf-10x-idle.sh agents=20; steps 5d283c1b76d4e8112ee62c50 / 6c79baacf25bee1ba3bfc08e
chat renderers wakeups/s 10.65 10.12 -0.53 (-5 %) same steps
app wakeups/s 45.23 30.60 -14.63 (-32 %) same steps
app CPU % 1.213 1.204 -0.01 pt (~0) same steps
chat renderers footprint 1840.0 MB 1858.2 MB +18.2 MB (+1 %, noise; a freeze keeps the page in memory) same steps
CEF GPU footprint 237.2 MB 238.1 MB +0.9 MB (~0) same steps

These chats are idle, so their pages had little to stop: the saving here is the timers and wakeups of hidden idle pages. The case from the Evidence section (hidden chats re-rendering on every update of 38 streaming sessions) is not reproduced by this bench. Before this, the bench measured one workspace instead of 20 on every fleet step (#28433).

Tests

  • AgentPageFreezeTests (new):
    • a hidden chat freezes after freezeHiddenTabs and thaws when shown;
    • a busy chat stays awake, and a frozen chat thaws when its turn starts;
    • off, pinned and unknown turns stay awake;
    • a refused freeze is asked again later;
    • a chat built in the background freezes;
    • a frozen chat thaws before release.
  • WorkspaceHibernationPlannerTests: .adopted now returns one conceal with a current token.

No capture: no visible change; a hidden chat page's renderer is paused and resumes when shown, and the page looks the same.

Timers

One new DemandTimer (AgentPageFreeze): one deadline for the next hidden page to come due. Its stop condition: it is cancelled whenever no hidden page is due, the setting is off, or freezeHiddenTabs is unset. Planning stays event-driven (presentation changes, turn-state changes, setting and pressure changes); nothing polls.

Risk

Medium.

  • Frozen pages and queued work: a frozen page runs no script, so a push queued to it waits until it thaws. The frame pacer already holds nonurgent pushes for hidden panes, and every path that needs the page (show, turn start, release) thaws it first.
  • Chief: Chief chats are excluded.

🤖 Generated with Claude Code

Agent session: agent session

…ckground-opened tabs start their hidden clock

Leo, 2026-10-28: "There's just loads of bullshit in my sidebar that I'm not
even looking at. So how could rendering be this fucking slow?"

A hidden chat's Chromium page now freezes after browser.freezeHiddenTabs
(30 s), as a hidden browser tab does, with its own eligibility: a turn that
works or waits on the person, an unknown turn state, a pinned tab (unless
pinned tabs hibernate), a New Tab page, a Chief chat and a crashed page stay
awake. It thaws when shown, when its turn starts, and before a release. A
page built while its tab does not draw starts its hidden clock too.

ContentLifecycle's .adopted (a tab whose content was made in the background)
now conceals it, so its hidden clock starts and it freezes; before, it never
froze until shown once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-6c7d0900cab519ec0cb2
@teamleaderleo

teamleaderleo commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-2v6r11 created for this PR by the cmux coordinator.

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 28, 2026
…page arms its own freeze

Review findings on the hidden chat freeze:
- The frame pacer sends a hidden page the pushes it waits on (a reply, a
  turn's end, a backlog). AgentPaneTransport.willDeliver runs right before a
  push; a frozen page's hibernation thaws it there and freezes it again
  freezeHiddenTabs after that if it stays hidden and idle.
- A page built while its tab does not draw got no event, so its clock waited
  for an unrelated replan. AgentTabStore.onPageBuilt now starts its clock and
  arms the deadline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-6c7d0900cab519ec0cb2
… is there; the quit's relaunch state thaws frozen chats

Second review round:
- A Chromium chat page gets its browser after its view, so at build time it
  had no freeze control and was never planned. It is now planned
  (AgentPaneView.waitsForChromium) and looked at again, backing off, until
  its browser is there.
- The quit asks every chat page for its relaunch state, and the page's draft
  ack waits on a JavaScript timer that a frozen page never runs.
  AgentTabStore.relaunchWakeStates now thaws frozen pages first and keeps
  every page awake until it ends (AgentPageHibernation.holdAwake).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-6c7d0900cab519ec0cb2
… instead of being lost frozen

Third review round: a provisional chat tab can freeze before its creation
answers; AgentTabStore.rekey then moves its page to the store's key, and the
next plan dropped the frozen entry without thawing it, so no turn start, push
or quit collection reached it again. Frozen pages are now kept with the page
and the control they froze through: one no longer under its key thaws, starts
its hidden clock under the new key and freezes again later; a push finds its
page by the page itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-6c7d0900cab519ec0cb2
…r key its tab has now

Fourth review round: a workspace release right after a provisional tab's
rekey (no plan between) thawed by the new key and missed the entry under the
old one, so the release asked a frozen page for its state. thaw(key) now also
finds the entry by the page the key shows; tests cover push and release after
a rekey with no plan between.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-6c7d0900cab519ec0cb2
… a page released under one key never refreezes under another

Fifth review round:
- Debug and automation actions (DebugAgentPane chat_state, send_prompt, ...)
  call the page's scripts directly. AgentPaneTransport.willDeliver becomes
  beforePageWork and also runs at the start of AgentPaneView.callPageScript
  and pageSnapshot, so a frozen page thaws before any of them reaches it.
- With a provisional key's alias still resolving, a page is under two keys;
  the freeze plan now skips a page being released or already frozen under
  either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-6c7d0900cab519ec0cb2
Sixth review round: with a provisional tab's alias kept, one page is under two
keys with two hidden clocks; work for it thawed it under one key and the plan
that followed froze it again under the other. The freeze plan now groups the
hidden keys by page (the latest hide of any key counts), and the wake cooldown
is kept per page, so one page is planned, frozen and woken once whatever its
keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-6c7d0900cab519ec0cb2
swift-frontend crashed (signal 6) type-checking an inline call on a
#require result (AgentPageFreezeTests.swift:225); bind it to a constant
first, as aPushForAFrozenChatThawsItFirst already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Agent-Session: https://cmux.com/deeplink/agent?id=aid-d5a1ae14beefc4eef784
@blacksmith-sh

blacksmith-sh Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt
epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)/
test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt
epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)
View Logs
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)/
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review ba18722: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr27047 -f ref=perf/agent-page-freeze -f platform=macos -f backend_mode=local

web and size checks: green

Shipped in:

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high, router-review): SHIP at ba18722

SHIP
Visual verdict: No visual or layout changes; screenshots are not applicable.

  1. No blocking findings.

The focused Swift tests could not run because Swift/Xcode is unavailable in the review environment.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra, router-review): SHIP at ba18722

GraphQL: API rate limit already exceeded for user ID 13091533.
SHIP at ba18722

Visual verdict: no images supplied (range review); the visual verdict must come separately.

No correctness findings.

Follow-ups: none

exit=0

@teamleaderleo teamleaderleo removed the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

SW-11 performance review — hidden working chats and repeated page wakes

Read at ba187229173f42d66d47a5b9933c2729e599c0d3. I would keep this PR's idle-page freeze and its ownership; it has a valuable CPU hypothesis and solid lifecycle tests. The missing performance evidence is the workload with many actively streaming background agents, which motivated the issue.

Two specific paths to measure:

  1. keepsAwake L379–395 exempts .working and .needsInput chats from freezing, even when hidden. The cited dogfood Mac had 38 streaming agents and a chat renderer consuming 23–50% CPU. Freezing idle pages may deliver a large win, while those 38 active hidden chats remain awake. That distinction belongs in the A/B result.
  2. AgentPaneTransport.flush L382–420 invokes beforePageWork for a delivery; pageWorkNeeds L441–447 thaws and calls replan(). replan L183–214 walks the live view set, while replanFreeze L331–367 rebuilds a per-page map from every hidden clock. A background page with continuing deliveries can therefore spend much of its time awake and make each thaw an O(number of retained pages) main-actor pass. The existing test covers a single push followed by silence, rather than sustained traffic.

Suggested acceptance experiment (hypothesis; no live A/B run from this review): Extend scripts/measure/perf-10x-idle.sh or its bench_memory.py agent mode on one exclusive Mac. Compare the same workload at the PR's base a9e3148 and head ba18722: 1 / 20 / 100 agent pages, 95% hidden where possible, with quiet idle, active working turns, and sustained hidden-page deliveries (e.g. 2/s for 120 s). Add a 3,000-session daemon fan-out condition without allocating 3,000 Chromium pages. Report process-kind CPU, physical footprint, wakeups/s, renderer freeze/thaw duty, main-thread stalls >50 ms, input p99, and lost/late turn/permission UI events. Keep idle and busy results separate. The PR's performance table currently says pending.

If busy-hidden CPU stays high: the systemic follow-up is to detach the hidden chat's browser projection from live message processing. acpmux already owns session/turn/transcript state (layer-ownership rules 1–3). Keep an offscreen Rust-owned subscription and revision/sequence watermark, update native turn/permission indicators, and hydrate a visible page from a bounded snapshot plus ordered deltas. Validate attach/reveal, draft preservation, permissions, remote sessions, crash/reconnect, and no duplicates before deleting the hidden-page push path. Run behind a rollback switch. This could lower CPU for the very workload whose turns currently exempt those pages. It is a separate scoped project, with the current PR owner retaining this slice.

I am recording the broader ranking and other ownership-sensitive findings on HQ #5599 under RESULT — SW-11.

@teamleaderleo
teamleaderleo marked this pull request as ready for review November 1, 2026 20:22
@teamleaderleo
teamleaderleo merged commit adc50b8 into feat-cmux-next Nov 1, 2026
225 of 233 checks passed
@teamleaderleo
teamleaderleo deleted the perf/agent-page-freeze branch November 1, 2026 20:23
lawrencecchen added a commit that referenced this pull request Nov 1, 2026
… to CmuxNextHibernation (#31302)

Base red: arch-ratchet [app_lines] CmuxNextApp +209 over e716128..b2ff685,
from #27047 (adc50b8), which added the hidden chat page freeze to
CmuxNextApp's AgentPageHibernation.

The policy and its clocks (hidden times, shown and released pages, the
workspace-due set, frozen pages, the wake cooldown, the hold during the
quit's collection, the release and freeze plans and their two deadlines)
are now HiddenChatPages in CmuxNextHibernation, beside HibernationPlanner,
generic over the App's page and freeze control. AgentPageHibernation keeps
the App's side: the seams, the tab store's own eligibility (this Mac's chat,
New Tab, Chief, crashed, shell runs), the release itself (wake state, frame,
displace, tab store) and the wiring. Behavior-neutral: every rule and its
order is the same; the tests (AgentPageHibernationTests,
AgentPageFreezeTests) call the same surface.

No new package edge: CmuxNextHibernation keeps CmuxNextCompat and
CmuxNextSettings; the deadlines are WorkspaceHibernationDeadline (the App
passes DemandTimer, as for workspace hibernation).

Co-authored-by: cmux-bot[bot] <348531757+cmux-bot[bot]@users.noreply.github.com>

This branch was previously deployed

1 inactive deployment
ci-routing-agent — ba187229 Deployed Nov 1, 2026 by teamleaderleo via namespace autofix #11600
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant