cmux NEXT agent engine bench: WebKit modes stay WebKit, and --shots captures each engine over the window glass by teamleaderleo · Pull Request #24160 · manaflow-ai/cmux

teamleaderleo · 2026-10-22T03:58:46Z

Leo: "lawrence wants to do chromium... well how could we do glass through chromium?"

Scope now: the engine bench only. This PR first added a Debug toggle (agent.chromiumGlass, off) that made the Chromium agent chat see-through. That part was dropped from the branch (force-push at 04:56Z, before e397eec → d583e5d) because 8e36a5b (lawrencecchen, "cmux pages in Chromium are see-through like their WebKit pages") landed the same fork binding (cmux_tab_set_transparent_background, API 17) for every cmux page, with no toggle. No Swift, no tunable and no default change is left here.

What it does (`scripts/cmux-next/agent-engine-bench.py`, +27/-2)

WebKit modes stay WebKit. agent.chromium is on by default since 69a51bf, so the bench's own and page baselines were loading the chat in Chromium. They now pin agent.chromium: false; chromium mode pins it true.
--shots DIR: light and dark screen captures (screencapture, one file per display) of the open chat in each mode, plus debug.window_list, restoring debug.appearance to system afterwards.
With --shots, the scratch Ghostty config turns the window glass on (background-opacity = 0.7, background-blur = macos-glass-regular); an empty config is an opaque window, which hides what a page does over the glass. Without --shots the config stays empty, as before.

Bench numbers (same harness, cmux-mac-mini-cua, window glass on, 10 chats; taken before the Swift part was dropped)

	WebKit (shipping)	Chromium (opaque)	Chromium see-through
Total memory with 1 chat	640 MB	944 MB	956 MB
Total memory with 10 chats	1010 MB	1157 MB	1156 MB
First chat, first content	3112 ms (cold outlier; 318 ms in the opaque run)	70 ms	100 ms
Next chats, ready (median)	37 ms	42 ms	43 ms
Typing to frame p50 / p95	4 / 7 ms	7.3 / 10.3 ms	9.7 / 11.3 ms
Typing to paint p95	8 ms	15 ms	18 ms
Fling: dropped frames / p95 frame, 3 runs	28, 15, 27 / 33–40 ms	2, 1, 2 / 17–19 ms	3, 2, 2 / 16.8 ms

See-through costs nothing measurable over opaque Chromium on these numbers.

No capture: the diff is a bench script on the fleet, with no app UI change. The private side-by-side is in the design references repo as cmux-next-chromium-glass-side-by-side.

Agent session: cosmic-gray
Agent session: Air-Blue

🤖 Generated with Claude Code

github-actions · 2026-10-22T03:59:08Z

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review ce66008: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr24160 -f ref=cef-chat-glass -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: red check: Checks job structure (+1 more) is not red on the base bb8bfdf; do not merge until green; see lane-rules

blacksmith-sh · 2026-10-22T04:01:48Z

Found 11 test failures on Blacksmith runners:

Failures

Test	View Logs
`/tmp/tmp2ocsado0/summary-missing.json (live):/summary-missing.json (live):`	View Logs
`/tmp/tmp_vfoxtes/summary-fail.json (live):/summary-fail.json (live):`	View Logs
`[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/` `[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows`	View Logs
`[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames`	View Logs
`[tabs] debug.tab_switch timed 0 of 40 switches/` `[tabs] debug.tab_switch timed 0 of 40 switches`	View Logs
`[tabs] the window draws no frames/[tabs] the window draws no frames`	View Logs
`convention violations found (ERROR lines above)./` `convention violations found (ERROR lines above).`	View Logs
`nightly cmux-tui resolve/nightly cmux-tui resolve`	View Logs
`test_every_submodule_free_checkout_on_an_owned_runner_resets_submodules (main.Reuse` `dWorkspaceSubmodules.test_every_submodule_free_checkout_on_an_owned_runner_resets_submo` `dules)/` `test_every_submodule_free_checkout_on_an_owned_runner_resets_submodules (main.Reuse` `dWorkspaceSubmodules.test_every_submodule_free_checkout_on_an_owned_runner_resets_submo` `dules)`	View Logs
`test_every_submodule_free_checkout_on_an_owned_runner_resets_submodules (main.Reuse` `dWorkspaceSubmodules.test_every_submodule_free_checkout_on_an_owned_runner_resets_submo` `dules) (job='web-bundles')/` `test_every_submodule_free_checkout_on_an_owned_runner_resets_submodules (main.Reuse` `dWorkspaceSubmodules.test_every_submodule_free_checkout_on_an_owned_runner_resets_submo` `dules) (job='web-bundles')`	View Logs

...and 1 more test failures. View all on Blacksmith

^{Need help on this PR? Tag @codesmith-bot with what you need.}

teamleaderleo · 2026-10-22T04:21:24Z

Bead cx-sgyio created for this PR by the cmux coordinator.

…dow glass; WebKit modes stay WebKit `--shots DIR` takes light and dark screen captures of the open chat in each mode with the window glass on (background-opacity 0.7, macos-glass-regular): an empty Ghostty config is an opaque window, which hides what a page does over the glass. The screen, not the app's snapshot, since only the window server composites a Chromium page window over the glass. The own and page modes pin agent.chromium off: it is on by default since 69a51bf, so the WebKit baseline was loading in Chromium. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

teamleaderleo · 2026-10-22T07:20:05Z

Owned by orphan-sweep on air-blue · session

Agent-Session: orphan-sweep lander (air-blue, session 01a15cfc-663f-7650-b9f7-64d198c6b88b)

teamleaderleo · 2026-10-22T08:30:07Z

Review (codex xhigh via acpmux on cmux16, correctness first): SHIP at ce66008

Reviewer verdict, quoted: no blocking findings. The own/page modes pin agent.chromium off and chromium pins it on, which matches the base tunables and their defaults. The tunables file format matches the app's reader. debug.appearance (light/dark/system) and debug.window_list exist. screencapture with one file per display is valid. The glass config is written only under --shots. The diff is script-only (+27/-2): no Swift, and no app default or component changes.

Non-blocking follow-up: restore the appearance in a finally, and record appearance-RPC and capture failures explicitly.

Compile: cmux-ci compile-check --ref ce66008ccabb984ee64038827e681aa0e82643f4 --app passed (job aaebec2322a3f96d0d1048ce).
Reds at this head are base reds: the tree is base bb8bfdf plus only scripts/cmux-next/agent-engine-bench.py, which nothing compiles or tests.

fast tier: SideOpenStripTests.swift does not compile (from cmux NEXT: a tab opened to the side starts its open on the right, never in the left pane #23969).
cmux-next checks: namespace-type lint in AgentPaneNeedsYouSetting.swift and AttentionHighlightLook.swift.
release rails: migration-runner tests.
release-notary and Guard status.

#24269) * docs(next): a Chromium page is see-through, and the gap is backdrop-filter The "Known gap" paragraph said a Chromium page is an opaque child window that cannot show a see-through backdrop. 8e36a5b fixed that the same day the chat moved to Chromium, by binding the fork's cmux_tab_set_transparent_background through the shim, and PageWebView+Chromium applies it to every cmux page; the paragraph was never updated. Leo asked today whether we can show the background and style pages like AppKit, and this is the line someone reads to answer that. The real remaining gap goes in its place: backdrop-filter does not draw over a see-through Chromium page, so a glass panel over one reads as tint without the frost it has over WebKit (#24160 measured it). Leo-ask: ask-665f9ad8ba Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15cb8-bbf0-7010-8d04-f9f2c0cb55df&host=big-red * docs(next): drop the backdrop-filter gap claim, which was backwards An independent review blocked the gap sentence this PR had added: "backdrop-filter does not draw over a see-through Chromium page, so a glass panel over one reads as tint without the frost it has over WebKit." The repo's only probe of that says the opposite. glass-lens-DESIGN.md (macOS 27, Playwright WebKit, 2026-10-20): WKWebView "parses `backdrop-filter: url(#svg-filter)` but renders nothing for it ... where Chromium bent them." #24024 separately frosts a glass card over the Chromium agent chat. Replacing one false claim with another unverified one is the thing this PR exists to stop, so the sentence goes rather than gets narrowed. What stays is what the review verified hop by hop: the shim binding (`shim_process.mm`, `shim_background.mm`), `CEFTab.pageSeeThrough`, the single unconditional `setSeeThrough(true)` on the one Chromium start path in `PageWebView+Chromium`, and a browser site staying opaque because `pageSeeThrough` defaults false. Two additions, both cited in the tree: a browser tab on the New Tab or blank page is see-through in both engines, Chromium's by masking the page out (`PageBackground`, `CEFTab.applyOcclusion`, Lawrence 2026-10-10); and the engine-agnostic reason a `backdrop-filter` has nothing to bend over a transparent region, which is why the lens hands each page a still of the wallpaper (glass-lens-DESIGN.md). "web pages" became "a site in a browser tab": the One backdrop rule twenty lines up uses "web pages" for cmux's own surfaces, so the two read as a contradiction. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15cb8-bbf0-7010-8d04-f9f2c0cb55df&host=big-red * docs(next): make the see-through paragraph say exactly what each engine does Three precision findings from the second review, all of them wording this doc exists to get right. The New Tab sentence said "see-through in both engines too", which the source it paraphrases (`PageBackground`) also says unqualified — but the two mechanisms are not the same and they diverge. WebKit re-reads the URL every time (`WebKitTab+LoadingBackground.pageDidFinish`: `guard !PageBackground.isBlank`), while Chromium holes the page out only until the first real page and latches it (`CEFTab+LoadingBackground.reachedFirstRealPage`: `guard !pastFirstRealPage`, then `pastFirstRealPage = true`, with no reset in Sources). So a tab that visits a real page and returns to the New Tab page is see-through in WebKit and not in Chromium. The paragraph now names both mechanisms and that consequence. The wallpaper still is the host's, not the lens's: glass-lens-DESIGN.md says "the host gives each page a still of the wallpaper as it is drawn, and the page lenses its own copy of it", and the lens is the in-page canvas that consumes it. The see-through hop is gated on fork API 17 (`CEFShimLibrary`: "0 when the fork lacks it"), which the text stated flatly; it now names the API, as the sentence this PR deleted once named the fork. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15cb8-bbf0-7010-8d04-f9f2c0cb55df&host=big-red --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

github-actions · 2026-10-23T07:51:03Z

Shipped in:

cmux NEXT nightly 1.0.0-nightly.3991367032401 (b9c3d5aabc)
cmux NEXT nightly 1.0.0-nightly.3991575404801 (a62351daaa)

github-actions · 2026-10-23T08:04:41Z

Shipped in:

cmux NEXT nightly 1.0.0-nightly.3991503232101 (cd84bc62d1)

teamleaderleo temporarily deployed to ci-routing-agent October 22, 2026 03:58 — with GitHub Actions Inactive

github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 22, 2026

teamleaderleo temporarily deployed to ci-routing-agent October 22, 2026 04:49 — with GitHub Actions Inactive

teamleaderleo force-pushed the cef-chat-glass branch from e397eec to d583e5d Compare October 22, 2026 04:56

teamleaderleo mentioned this pull request Oct 22, 2026

docs(next): a Chromium page is see-through; the gap is backdrop-filter #24269

Merged

Merge feat-cmux-next into cef-chat-glass

ce66008

Agent-Session: orphan-sweep lander (air-blue, session 01a15cfc-663f-7650-b9f7-64d198c6b88b)

teamleaderleo changed the title ~~cmux NEXT agent pane: window glass through the Chromium chat (Debug toggle, off)~~ cmux NEXT agent engine bench: WebKit modes stay WebKit, and --shots captures each engine over the window glass Oct 22, 2026

teamleaderleo merged commit 958f2e9 into feat-cmux-next Oct 22, 2026

teamleaderleo deleted the cef-chat-glass branch October 22, 2026 08:30

teamleaderleo mentioned this pull request Oct 22, 2026

cmux NEXT glass: in a see-through Chromium page, menus and the composer no longer show the chat text sharp through the glass #24358

Merged

Navigation Menu

Repository navigation

cmux NEXT agent engine bench: WebKit modes stay WebKit, and --shots captures each engine over the window glass - #24160

cmux NEXT agent engine bench: WebKit modes stay WebKit, and --shots captures each engine over the window glass#24160
teamleaderleo merged 2 commits into
feat-cmux-nextfrom
cef-chat-glass

teamleaderleo commented Oct 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 22, 2026 •

edited

Loading

Uh oh!

blacksmith-sh Bot commented Oct 22, 2026 •

edited

Loading

Uh oh!

teamleaderleo commented Oct 22, 2026

Uh oh!

teamleaderleo commented Oct 22, 2026

Uh oh!

teamleaderleo commented Oct 22, 2026

Uh oh!

github-actions Bot commented Oct 23, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

teamleaderleo commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does (scripts/cmux-next/agent-engine-bench.py, +27/-2)

Bench numbers (same harness, cmux-mac-mini-cua, window glass on, 10 chats; taken before the Swift part was dropped)

Uh oh!

github-actions Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blacksmith-sh Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Failures

Uh oh!

teamleaderleo commented Oct 22, 2026

Uh oh!

teamleaderleo commented Oct 22, 2026

Uh oh!

teamleaderleo commented Oct 22, 2026

Uh oh!

github-actions Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

teamleaderleo commented Oct 22, 2026 •

edited

Loading

What it does (`scripts/cmux-next/agent-engine-bench.py`, +27/-2)

github-actions Bot commented Oct 22, 2026 •

edited

Loading

blacksmith-sh Bot commented Oct 22, 2026 •

edited

Loading

github-actions Bot commented Oct 23, 2026 •

edited

Loading