Skip to content

cmux NEXT: no Chromium browser is made once quit began to stop Chromium (crash on quit) - #23383

Merged
teamleaderleo merged 6 commits into
feat-cmux-nextfrom
cef-quit-no-create
Oct 21, 2026
Merged

teamleaderleo merged 6 commits into
feat-cmux-nextfrom
cef-quit-no-create

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Leo's NIGHTLY aborted on quit (2026-10-20 22:00, binary UUID 10cda577…). The report names 1.0.0-nightly.3953614980201, but the UUID matches the dSYM of feat-cmux-next c78e7b2 (artifact nightly-next-dsyms-c78e7b2…, run 39529676482); 3953614980201 was only installed by the update this quit was for. Crash summary: leo-workspace briefs/2026-10-20-gui-push/crashes/2026-10-20-2200-quit-cef-create-window.txt.

What happened

Symbolicated with that dSYM:

NSApplication terminate: → _closeForTermination → NSWindow _finishClosingWindow
→ WindowController.windowWillClose → WindowManager.windowWillClose → WindowController.teardown
→ WorkspaceContentController.teardown → PaneController.teardown
→ PaneContentView.detachContent(parks:) (PaneContentView.swift:296, shown.removeFromSuperview())
→ -[NSView _recursiveLostHiddenAncestorBySwitchingWindow:] ×3
→ CEFTabContentView.viewDidUnhide → CEFTab.contentDidAppear → CEFPaneHost.present
→ CEFPaneHost.ensureCreated (CEFPaneHost.swift:212, shim.createWindow) → cmux_shim_create_window
→ CefBrowserHost::CreateBrowser → "Check failed: !IsCefShutdown()" (libcef_dll/shutdown_checker.cc) → abort

The sequence:

  1. Quit stops Chromium in applicationShouldTerminate: it closes every browser, waits for them, then calls CefShutdown.
  2. AppKit then closes the windows, and each window's teardown detaches its panes' content.
  3. A pane in a parked (hidden) workspace that selects a Chromium tab whose browser was never made loses its hidden ancestor while it is still in the window. AppKit sends that tab's view viewDidUnhide.
  4. The pane makes the browser. ensureCreated only checked that the shim was loaded.

Fix

  • CEFRuntime.acceptsNewBrowsers is false once shutdown has started (the browsers are closed and Chromium is being waited on) or after CefShutdown. It is also false when quit came while the library was still loading.
  • CEFPaneHost.ensureCreated and duplicate make no browser unless it is true. A tab whose view appears at that point simply stays empty, since the app is exiting.
  • The shim checks too: after cmux_shim_close_all, cmux_shim_create_window, cmux_shim_tab_add and cmux_shim_tab_duplicate return 0. This covers any path that reaches Chromium without going through the pane. The public header is unchanged, so the shim ABI identity is unchanged.

Overlap check:

Neither touches this path.

Tests

  • CEFQuitCreationTests/aParkedTabUnhiddenAfterChromiumStoppedMakesNoBrowser: the crash's path, against a spy shim.
    • CEFShimLibrary.resolving builds the shim from stand-in functions, and CEFRuntime(shim:storage:) runs panes against it without Chromium.
    • A parked tab's view unhides in its window, and the test counts the cmux_shim_create_window calls.
    • Expected: 1 while Chromium runs, 0 after shutdown.
  • CEFQuitCreationTests/noBrowserIsMadeOnceChromiumBeganToStop: checks acceptsNewBrowsers in each state (committed red first, e528869).
  • Live: scripts/cmux-next/quit-hidden-chromium-live.py.
    • Setup: 2 windows, each with a parked workspace of 3 panes whose selected Chromium tabs were never made (18 never made, 2 hibernated).
    • It quits through the Quit Apple event and checks the exit status, the logs and the crash reports.

Changelog

  • Fixed: cmux NEXT no longer crashes on quit when windows hold hidden Chromium tabs.

No capture: nothing on screen changes; the app quits without aborting.

Observed / Not run

  • Red/green, focused fleet runs (cmux-ci run --class light --script scripts/ci/package-test-lane.sh --arg suite --arg Packages/macOS/CmuxNext --arg CEFQuitCreationTests,...). Both refs included fix(next tests): PaletteMenuVocabularyTests compiles again #23386, which base needed for its test target to compile; it has since landed.
    • Red: base 0284c98 plus the spy-shim test (f7b6872, a throwaway branch). The test fails on try unhideParkedTab(runtime) == 0: base makes a browser after CefShutdown. The check that a browser is made while Chromium runs passes.
    • Green: the fix (2423bae): CEFQuitCreationTests, CEFRuntimeSourceTests and CEFShimABITests pass.
  • Live, on capture mini cmux-mac-mini-cua-3, fleet DEV builds (cmux-ci build cmux --backend-mode local):
    • Base 0d7d9cb: 5 of 5 runs aborted. Each had wait status 6 (SIGABRT), Check failed: !IsCefShutdown() in app.log, CEF shutdown complete logged before the abort, and Leo's stack (crash report cmux DEV-2026-10-21-053300.ips; macOS held back reports for the repeats).
    • Fix e28bb94: 5 of 5 exited 0, with no FATAL and no crash report.
  • Compile at the exact head f467071: the fleet DEV app build passed (cmux-ci build cmux --ref f4670712048, job aa7af3dc31d4; it builds the shim from source). GitHub's compile jobs were still queued for runners. Review: SHIP at f467071 (comment 6316538458). Reds are base reds: fast gates (webviews check:text-focus) and defaults-snapshot (menu rows) also fail on base pushes 0a4548e and 0b1a93e.
  • Not run:
    • Sentry count: no machine here has a Sentry read credential (only CI's upload-only token). Someone with a Sentry login can search project:cmuxterm-macos for IsCefShutdown or cmux_shim_create_window.
    • The refusal is not logged, so the live runs show it only through the clean exit.

Agent session: cosmic-gray

🤖 Generated with Claude Code

teamleaderleo and others added 2 commits October 21, 2026 05:08
…op Chromium (red)

Crash on quit in NIGHTLY 1.0.0-nightly.3953614980201: a hidden Chromium tab's content view was
unhidden while AppKit closed the windows after CefShutdown, and the pane created its browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray
…p Chromium

A pane created its browser for a hidden Chromium tab whose content view AppKit unhid while it
closed the windows for termination, after CefShutdown; CEF aborts on that (crash on quit,
NIGHTLY 1.0.0-nightly.3953614980201). Panes now make or duplicate a browser only while
CEFRuntime.acceptsNewBrowsers (not shut down, no shutdown under way), and the shim refuses
create_window, tab_add and tab_duplicate after close_all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray
@teamleaderleo teamleaderleo added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 21, 2026
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review f467071: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr23383 -f ref=cef-quit-no-create -f platform=macos -f backend_mode=local

Not verified: macOS jobs did not run: compile and app tests were skipped on f467071204; ci-status is green without them.

CI passes on f467071204 (run 39602531977 attempt 1).
macOS jobs did not run: compile and app tests were skipped: Fast static checks skipped (the macos job needs both to succeed). Re-run CI once they pass, before merging.

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way, seen on other PRs that it failed on another pull request's run lately.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by tab-hibernation on cosmic-gray · session

teamleaderleo and others added 2 commits October 21, 2026 05:18
…pped makes no browser, against a spy shim

CEFShimLibrary.resolving builds the shim from stand-in functions and CEFRuntime(shim:storage:)
runs panes against it without Chromium, so the test drives the crash's path: viewDidUnhide,
contentDidAppear, present, ensureCreated, cmux_shim_create_window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-cx2ho created for this PR by the cmux coordinator.

teamleaderleo and others added 2 commits October 21, 2026 05:34
…Chromium tabs exits cleanly

scripts/cmux-next/quit-hidden-chromium-live.py parks workspaces whose panes select Chromium tabs
never made, quits through the Quit Apple event and checks the exit status, cef/app logs and crash
reports. On a capture mini: base 0d7d9cb aborted 5 of 5 (CEF FATAL !IsCefShutdown, SIGABRT,
Leo's stack); the fix e28bb94 exited 0 in 5 of 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent, correctness first): SHIP at f467071

I read the diff with git diff origin/feat-cmux-next...HEAD and the code around it with git show. I did not build or run anything.

Quit cannot be cancelled after Chromium starts stopping. CEFRuntime.shutdown is the only thing that sets shutdownSequence, and nothing sets it back to nil. Its only caller is CEFEngine.shutdown, called from the stopBrowserEngines step in QuitCoordinator.complete. QuitCompletion.run runs that step last, after the unsaved-documents step, the quit sheet and the Retry / Quit Anyway loop. complete then always calls reply(toApplicationShouldTerminate: true), and QuitDeadline is armed before the step. The paths that cancel (resolveUnsaved returning false, .cancel on the sheet) all reply false before complete is ever reached. So acceptsNewBrowsers turning false cannot strand a running app with a dead Chromium.

cmux_shim_close_all is only called from quit. Its one caller is shim.closeAll() in CEFRuntime.shutdown, right after shutdownSequence is set. Nothing inside the shim calls it either. The shim's stopping() flag is one-way and is set at the same moment as the Swift gate, so it can't block creation in a session that keeps running. All access is on the main thread (CEF's UI thread here).

Other ways a browser could be made after shutdown:

  • Every Swift caller of ensureCreated now goes through the gate: present, CEFAgentRelay.createBrowser, CEFSpareWindow.prepare, and the popup placeholder in ensureOwnWindow. The .creating and .live (addToWindow) branches are behind the gate too.
  • windowCreated → addToWindow for tabs waiting on a creating window has no Swift gate. In shutdown the shim's tab_add refusal covers it, and this case can only happen while CEF is still up.
  • DevTools (ShowDevTools in shim_devtools.mm) and extension options need a live BrowserById. After CefShutdown none exist, because readyToShutdown requires tabsByBrowser to be empty and windowCount() == 0. That leaves no crash path.
  • Popups adopt browsers that Chromium itself made, and popupWindowAttach attaches a window that already exists.

A refused creation leaves no stuck state. The gate returns before isCreationPending, windowTab, window = .creating or pendingWindows are set. The shim-refusal path (create_window returns 0) clears pendingWindows[request], resets window and calls creationFailed(). shutdownProgressed only looks at tabsByBrowser and windowCount(), so a placeholder or never-made tab doesn't hold it up.

Spy shim: I checked every @convention(c) field in CEFShimLibrary. Each returns Void, Int32/UInt32/Int, or an optional pointer. A () -> Int stand-in returning 0 gives 0, or nil for the pointer fields. That is safe on arm64: extra arguments, including the Double in setZoomLevel, are simply ignored. No struct, float or must-free return exists. resolving skips checkABI, so the nil abiIDFn is never read.

The test is reliable:

  • Only this suite uses SpyShim.
  • It is @MainActor and .serialized.
  • It uses its own temp storage root.
  • CEFStartSteps.shared is not asserted by any test.
  • The first == 1 expectation is a positive control, proving viewDidUnhide really reaches cmux_shim_create_window. The second fails on base.

Skipping the OffTheRecordProfiles observer in the test init is fine, and better than registering it. It only releases incognito contexts. Registering it from every test runtime would pile observers onto the process-wide singleton.

Live script:

  • It identifies the process by the exact bundle executable path, after end_tag() has cleared the tag.
  • It kills only exact PIDs: the run's own pid, and orphaned __terminal-host processes matched by --owner <bundle id>:<tag>@<dead daemon pid>.
  • TagTeardown is installed with a finally, and the HTTP server shuts down there too.
  • Pages come from a localhost server. I found no tokens, user paths or private data, only the tag's own logs under --out.

CI: the three reds are not this PR's. Fast gates fails on the webviews check:text-focus lint, and this PR touches no webviews. defaults-snapshot fails on menu rows and inline selectors, and this PR has no menu changes.

Non-blocking notes:

  1. addToWindow (reached from windowCreated during the shutdown wait) has no Swift gate. A tab waiting on a window that Chromium finishes making after close_all gets the shim's refusal, so it goes through creationFailed() and shows a load failure, unlike the silent refusal in ensureCreated. This is rare and harmless at quit. A runtime.acceptsNewBrowsers check there would make the two behave the same.
  2. Not new in this PR: a browser that create_window started just before close_all isn't in browsers() yet, so close_all doesn't close it. Its OnAfterCreated comes later, and the wait may time out and skip CefShutdown. This could be a follow-up: close the browser in OnAfterCreated when stopping() is set.
  3. CEFAgentRelay.browserCreated() waiters are not resumed when ensureCreated refuses in shutdown. This matches the existing shim == nil behavior, and the process is exiting anyway.
  4. REVIEW.md section 8 says no test seams in production source. CEFShimLibrary.resolving and CEFRuntime(shim:storage:) are internal injection points, used through @testable. I'd accept them here, because they let the test drive the real crash path.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

cmux-next fast gates / fast gates is red on 15 of 22 fresh feat-cmux-next PR heads in the last 2:00:00, so it is most likely a base red, not this PR's. The base-reds lane has it; a base merge picks up the fix once it lands.

@teamleaderleo
teamleaderleo merged commit 001b266 into feat-cmux-next Oct 21, 2026
@teamleaderleo
teamleaderleo deleted the cef-quit-no-create branch October 21, 2026 13:00
teamleaderleo added a commit that referenced this pull request Oct 21, 2026
…browser is closed too (red)

A creation under way at cmux_shim_close_all ends in AFTER_CREATED during the shutdown wait; the
runtime registered the browser and never closed it, so the wait ran out and CefShutdown was
skipped (review finding on #23383).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

teamleaderleo added a commit that referenced this pull request Oct 21, 2026
… browser, so CefShutdown runs (#23445)

* test(next browser): a browser Chromium makes after quit closed every browser is closed too (red)

A creation under way at cmux_shim_close_all ends in AFTER_CREATED during the shutdown wait; the
runtime registered the browser and never closed it, so the wait ran out and CefShutdown was
skipped (review finding on #23383).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray

* fix(next browser): quit closes a browser Chromium makes after it closed every browser

A creation under way at cmux_shim_close_all reaches AFTER_CREATED during the shutdown wait. The
runtime now closes that browser instead of registering it, so Chromium destroys its window, the
wait ends and CefShutdown runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray

* test(next browser): live in-flight case: a Chromium browser still being made at quit is closed, so CefShutdown runs

quit-hidden-chromium-live.py --case in-flight opens front Chromium tabs in fresh profiles (their
first browser waits on its request context) every 200 ms and sends the Quit Apple event into the
burst, so a creation is under way when quit closes every browser; it checks the shutdown lines.
On a capture mini: base 001b266 logged "CEF shutdown timed out" in 5 of 5 (a browser
registered after close_all each time), the fix d2cf6de "CEF shutdown complete" in 5 of 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d2c-7e44-7fc1-abd1-eae4a427ac80&host=cosmic-gray

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant