diff --git a/cmux-tui/README.md b/cmux-tui/README.md index 515db365fd19..bc53ea6ced03 100644 --- a/cmux-tui/README.md +++ b/cmux-tui/README.md @@ -36,6 +36,14 @@ The default session is `main`. Default sockets live at `$TMPDIR/cmux-tui-/< Use `--term ` to set `TERM` for child PTYs. Without it, children get `xterm-256color`; `CMUX_TUI_TERM` can override the process default in the surface layer, with `CMUX_MUX_TERM` retained as a legacy fallback. +## Browser Realism + +By default, browser panes launch your real Google Chrome or another Chrome-family binary in `browser.mode: "headful"` with a visible window and a persistent per-session profile. Log into Google or other sites once in that visible window; cookies and logins persist across sessions. Set `browser.mode: "headless"` to hide the launched Chrome window. Both modes keep the anti-throttle flags, `--disable-blink-features=AutomationControlled`, the persistent `--user-data-dir`, and `about:blank` startup. + +Chrome 136 and newer reject CDP remote debugging on the OS-default profile directory, and a running normal Chrome owns its profile `SingletonLock`. Use the mux profile, set `browser.user_data_dir` to a copy or a dedicated directory after quitting normal Chrome, or attach to a Chrome you started with `--remote-debugging-port`. + +To attach instead of launching, set `browser.cdp_url`, `CMUX_MUX_CDP_URL`, or enable discovery. Agent Browser works the same way: run `agent-browser get cdp-url` and use the returned `ws://` URL. This build supports `ws://` and `http://` CDP endpoints; `wss://` is not supported. + ## Development ```bash diff --git a/cmux-tui/crates/cmux-tui-cdp/src/chrome.rs b/cmux-tui/crates/cmux-tui-cdp/src/chrome.rs index 108442dd406e..24e1854731fa 100644 --- a/cmux-tui/crates/cmux-tui-cdp/src/chrome.rs +++ b/cmux-tui/crates/cmux-tui-cdp/src/chrome.rs @@ -1,6 +1,6 @@ use std::ffi::OsString; use std::io::{BufRead, BufReader}; -use std::path::PathBuf; +use std::path::{Path, PathBuf}; use std::process::{Child, Command, Stdio}; use std::sync::Mutex; use std::sync::atomic::{AtomicU64, Ordering}; @@ -12,10 +12,18 @@ static PROFILE_SEQ: AtomicU64 = AtomicU64::new(1); #[derive(Debug, Clone)] pub struct ChromeLaunchOptions { pub binary: PathBuf, + pub mode: BrowserMode, pub user_data_dir: Option, pub ephemeral: bool, } +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub enum BrowserMode { + #[default] + Headful, + Headless, +} + /// A launched Chrome/Chromium process plus its profile dir. pub struct Chrome { child: Mutex>, @@ -25,25 +33,24 @@ pub struct Chrome { } impl Chrome { - /// Launch Chrome in headless mode and wait for the browser CDP - /// endpoint printed on stderr. + /// Launch Chrome (headful by default, headless when the launch + /// options request it) and wait for the browser CDP endpoint printed + /// on stderr. pub fn launch(binary: PathBuf) -> anyhow::Result { - Chrome::launch_with(&ChromeLaunchOptions { binary, user_data_dir: None, ephemeral: true }) + Chrome::launch_with(&ChromeLaunchOptions { + binary, + mode: BrowserMode::default(), + user_data_dir: None, + ephemeral: true, + }) } pub fn launch_with(options: &ChromeLaunchOptions) -> anyhow::Result { let (profile_dir, profile_ephemeral) = profile_dir_for(options)?; std::fs::create_dir_all(&profile_dir)?; - let mut child = Command::new(&options.binary) - .arg("--headless=new") - .arg("--remote-debugging-port=0") - .arg("--no-first-run") - .arg("--no-default-browser-check") - .arg("--disable-background-timer-throttling") - .arg("--disable-backgrounding-occluded-windows") - .arg("--disable-renderer-backgrounding") - .arg(format!("--user-data-dir={}", profile_dir.display())) - .arg("about:blank") + let mut command = Command::new(&options.binary); + command.args(chrome_args_for(&profile_dir, options.mode)); + let mut child = command .stdin(Stdio::null()) .stdout(Stdio::null()) .stderr(Stdio::piped()) @@ -133,6 +140,28 @@ fn make_profile_dir() -> anyhow::Result { Ok(dir) } +fn chrome_args_for(profile_dir: &Path, mode: BrowserMode) -> Vec { + let mut args = Vec::new(); + if mode == BrowserMode::Headless { + args.push("--headless=new".to_string()); + } + args.extend([ + "--remote-debugging-port=0".to_string(), + "--no-first-run".to_string(), + "--no-default-browser-check".to_string(), + "--disable-background-timer-throttling".to_string(), + "--disable-backgrounding-occluded-windows".to_string(), + "--disable-renderer-backgrounding".to_string(), + "--disable-blink-features=AutomationControlled".to_string(), + format!("--user-data-dir={}", profile_dir.display()), + ]); + if mode == BrowserMode::Headful { + args.push("--window-size=1280,900".to_string()); + } + args.push("about:blank".to_string()); + args +} + fn profile_dir_for(options: &ChromeLaunchOptions) -> anyhow::Result<(PathBuf, bool)> { if options.ephemeral { return Ok((make_profile_dir()?, true)); @@ -173,6 +202,7 @@ mod tests { let options = ChromeLaunchOptions { binary: PathBuf::from("chrome"), + mode: BrowserMode::Headful, user_data_dir: Some(explicit_dir.clone()), ephemeral: true, }; @@ -191,6 +221,7 @@ mod tests { std::env::temp_dir().join(format!("cmux-tui-cdp-verbatim-{}", std::process::id())); let options = ChromeLaunchOptions { binary: PathBuf::from("chrome"), + mode: BrowserMode::Headful, user_data_dir: Some(explicit_dir.clone()), ephemeral: false, }; @@ -199,4 +230,34 @@ mod tests { assert!(!ephemeral); assert_eq!(selected, explicit_dir); } + + #[test] + fn headful_args_omit_headless_and_keep_stealth_throttle_profile_window() { + let profile = PathBuf::from("/tmp/cmux profile"); + let args = chrome_args_for(&profile, BrowserMode::Headful); + + assert!(!args.iter().any(|arg| arg == "--headless=new")); + assert!(args.iter().any(|arg| arg == "--remote-debugging-port=0")); + assert!(args.iter().any(|arg| arg == "--no-first-run")); + assert!(args.iter().any(|arg| arg == "--no-default-browser-check")); + assert!(args.iter().any(|arg| arg == "--disable-background-timer-throttling")); + assert!(args.iter().any(|arg| arg == "--disable-backgrounding-occluded-windows")); + assert!(args.iter().any(|arg| arg == "--disable-renderer-backgrounding")); + assert!(args.iter().any(|arg| arg == "--disable-blink-features=AutomationControlled")); + assert!(args.iter().any(|arg| arg == "--user-data-dir=/tmp/cmux profile")); + assert!(args.iter().any(|arg| arg == "--window-size=1280,900")); + assert_eq!(args.last().map(String::as_str), Some("about:blank")); + } + + #[test] + fn headless_args_add_headless_and_omit_window_size() { + let profile = PathBuf::from("/tmp/cmux-profile"); + let args = chrome_args_for(&profile, BrowserMode::Headless); + + assert!(args.iter().any(|arg| arg == "--headless=new")); + assert!(args.iter().any(|arg| arg == "--disable-blink-features=AutomationControlled")); + assert!(args.iter().any(|arg| arg == "--user-data-dir=/tmp/cmux-profile")); + assert!(!args.iter().any(|arg| arg == "--window-size=1280,900")); + assert_eq!(args.last().map(String::as_str), Some("about:blank")); + } } diff --git a/cmux-tui/crates/cmux-tui-cdp/src/client.rs b/cmux-tui/crates/cmux-tui-cdp/src/client.rs index 54cdebaac875..2cba804a3c2f 100644 --- a/cmux-tui/crates/cmux-tui-cdp/src/client.rs +++ b/cmux-tui/crates/cmux-tui-cdp/src/client.rs @@ -172,6 +172,15 @@ impl CdpClient { self.call("Target.setDiscoverTargets", json!({ "discover": discover }), None).map(|_| ()) } + pub fn browser_version(&self) -> anyhow::Result { + let result = self.call("Browser.getVersion", json!({}), None)?; + result + .get("userAgent") + .and_then(|value| value.as_str()) + .map(str::to_string) + .ok_or_else(|| anyhow::anyhow!("Browser.getVersion response missing userAgent")) + } + pub fn create_target(&self, url: &str) -> anyhow::Result { let result = self.call("Target.createTarget", json!({ "url": url }), None)?; result @@ -198,10 +207,29 @@ impl CdpClient { self.call("Target.closeTarget", json!({ "targetId": target_id }), None).map(|_| ()) } + pub fn close_target_detached(&self, target_id: &str) -> anyhow::Result<()> { + let id = self.inner.next_id.fetch_add(1, Ordering::Relaxed); + let msg = json!({ + "id": id, + "method": "Target.closeTarget", + "params": { "targetId": target_id }, + }); + self.send_value(&msg) + } + pub fn page_enable(&self, session_id: &str) -> anyhow::Result<()> { self.call("Page.enable", json!({}), Some(session_id)).map(|_| ()) } + pub fn set_user_agent(&self, session_id: &str, user_agent: &str) -> anyhow::Result<()> { + self.call( + "Emulation.setUserAgentOverride", + json!({ "userAgent": user_agent }), + Some(session_id), + ) + .map(|_| ()) + } + pub fn start_screencast( &self, session_id: &str, diff --git a/cmux-tui/crates/cmux-tui-cdp/src/lib.rs b/cmux-tui/crates/cmux-tui-cdp/src/lib.rs index 7f6fc453808c..bf1c16664acb 100644 --- a/cmux-tui/crates/cmux-tui-cdp/src/lib.rs +++ b/cmux-tui/crates/cmux-tui-cdp/src/lib.rs @@ -7,7 +7,7 @@ mod chrome; mod client; -pub use chrome::{Chrome, ChromeLaunchOptions}; +pub use chrome::{BrowserMode, Chrome, ChromeLaunchOptions}; pub use client::{ CdpClient, CdpEvent, CdpKeyEvent, NavigationEntry, NavigationHistory, ScreencastFrame, TargetCreated, TargetInfo, discover_browser_ws_url, resolve_browser_ws_url, diff --git a/cmux-tui/crates/cmux-tui-core/src/browser.rs b/cmux-tui/crates/cmux-tui-core/src/browser.rs index 68b104d45255..03eea9a8dda9 100644 --- a/cmux-tui/crates/cmux-tui-core/src/browser.rs +++ b/cmux-tui/crates/cmux-tui-core/src/browser.rs @@ -1,7 +1,7 @@ use std::collections::HashMap; use std::path::PathBuf; use std::sync::atomic::{AtomicBool, Ordering}; -use std::sync::mpsc::{Receiver, Sender, SyncSender, TrySendError}; +use std::sync::mpsc::{Receiver, Sender, SyncSender, TrySendError, sync_channel}; use std::sync::{Arc, Mutex, Weak}; use std::time::{Duration, Instant}; @@ -114,12 +114,69 @@ struct BrowserState { live_since: Option, last_frame_at: Option, stall_nudged: bool, + not_responding_reported: bool, +} + +enum BrowserCommand { + WakeLatest, + Mouse { + event_type: String, + x: f64, + y: f64, + button: Option, + click_count: Option, + }, + Wheel { + x: f64, + y: f64, + delta_y: f64, + }, + Key { + event_type: String, + key: String, + code: String, + windows_virtual_key_code: u32, + modifiers: u32, + text: Option, + }, + InsertText(String), + Navigate(String), + Back, + Forward, + Reload, + Activate, + Reconfigure { + width: u32, + height: u32, + }, +} + +impl BrowserCommand { + fn is_input(&self) -> bool { + matches!( + self, + BrowserCommand::Mouse { .. } + | BrowserCommand::Wheel { .. } + | BrowserCommand::Key { .. } + | BrowserCommand::InsertText(_) + ) + } + + fn is_mouse_move(&self) -> bool { + matches!(self, BrowserCommand::Mouse { event_type, .. } if event_type == "mouseMoved") + } +} + +#[derive(Default)] +struct BrowserWorkerErrorState { + consecutive_timeouts: u8, } pub struct BrowserRuntime { client: CdpClient, chrome: Option, source: BrowserSource, + stealth_user_agent: Option, routes: Mutex, closed: AtomicBool, } @@ -138,6 +195,11 @@ pub struct BrowserSurface { dead: AtomicBool, cell_pixels: Mutex<(u16, u16)>, capture_options: BrowserCaptureOptions, + command_tx: Mutex>>, + latest_reconfigure: Arc>>, + latest_nav: Arc>>, + #[cfg(test)] + worker_done: Mutex>>, } #[derive(Debug, Clone, Copy)] @@ -148,17 +210,33 @@ struct BrowserCaptureOptions { const DEFAULT_CAPTURE_MEGAPIXELS: f64 = 2.0; const STALL_THRESHOLD: Duration = Duration::from_secs(2); +const BROWSER_COMMAND_QUEUE_CAPACITY: usize = 64; +const BROWSER_NOT_RESPONDING_MESSAGE: &str = "browser is not responding"; impl BrowserRuntime { pub fn connect(opts: &SurfaceOptions) -> anyhow::Result> { let (web_socket_url, chrome, source) = runtime_endpoint(opts)?; + Self::connect_to_endpoint(&web_socket_url, chrome, source) + } + + fn connect_to_endpoint( + web_socket_url: &str, + chrome: Option, + source: BrowserSource, + ) -> anyhow::Result> { let (event_tx, event_rx) = std::sync::mpsc::channel(); - let client = CdpClient::connect(&web_socket_url, event_tx)?; + let client = CdpClient::connect(web_socket_url, event_tx)?; client.set_discover_targets(true)?; + let stealth_user_agent = if source == BrowserSource::Launched { + client.browser_version().ok().and_then(|ua| clean_headless_user_agent(&ua)) + } else { + None + }; let runtime = Arc::new(BrowserRuntime { client, chrome, source, + stealth_user_agent, routes: Mutex::new(Routes::default()), closed: AtomicBool::new(false), }); @@ -220,6 +298,9 @@ impl BrowserRuntime { if browser.is_dead() { anyhow::bail!("browser surface was closed before it started"); } + if let Some(user_agent) = self.stealth_user_agent.as_deref() { + let _ = self.client.set_user_agent(session_id, user_agent); + } self.client.page_enable(session_id)?; let (pixel_w, pixel_h) = browser.pixel_size(); self.client.set_device_metrics(session_id, pixel_w, pixel_h)?; @@ -248,10 +329,10 @@ impl BrowserRuntime { routes.by_target.remove(target_id); } - fn close_surface(&self, target_id: &str, session_id: &str) { + fn close_surface_detached(&self, target_id: &str, session_id: &str) { self.unregister(target_id, session_id); if !self.is_closed() { - let _ = self.client.close_target(target_id); + let _ = self.client.close_target_detached(target_id); } } @@ -274,6 +355,7 @@ pub(crate) fn new_surface( size: (u16, u16), cell_pixels: (u16, u16), opts: &SurfaceOptions, + mux: Weak, ) -> Arc { let normalized_url = normalize_url(&url); let (cols, rows) = (size.0.max(1), size.1.max(1)); @@ -283,7 +365,16 @@ pub(crate) fn new_surface( let capture_options = BrowserCaptureOptions::from_options(opts); let capture_scale = capture_scale_for(pixel_w, pixel_h, capture_options); let capture_pixels = scaled_pixels(pixel_w, pixel_h, capture_scale); - Arc::new(Surface::Browser(BrowserSurface { + let (command_tx, command_rx) = sync_channel(BROWSER_COMMAND_QUEUE_CAPACITY); + let latest_reconfigure = Arc::new(Mutex::new(None)); + let latest_nav = Arc::new(Mutex::new(None)); + #[cfg(test)] + let (worker_done_tx, worker_done_rx) = std::sync::mpsc::channel(); + #[cfg(test)] + let worker_done_tx = Some(worker_done_tx); + #[cfg(not(test))] + let worker_done_tx = None; + let surface = Arc::new(Surface::Browser(BrowserSurface { meta: SurfaceMeta { id, name: Mutex::new(None), selection: Mutex::new(None) }, session: Mutex::new(None), state: Mutex::new(BrowserState { @@ -302,12 +393,27 @@ pub(crate) fn new_surface( live_since: None, last_frame_at: None, stall_nudged: false, + not_responding_reported: false, }), dirty: AtomicBool::new(true), dead: AtomicBool::new(false), cell_pixels: Mutex::new((cell_w, cell_h)), capture_options, - })) + command_tx: Mutex::new(Some(command_tx)), + latest_reconfigure: latest_reconfigure.clone(), + latest_nav: latest_nav.clone(), + #[cfg(test)] + worker_done: Mutex::new(Some(worker_done_rx)), + })); + start_browser_worker( + surface.clone(), + command_rx, + latest_reconfigure, + latest_nav, + mux, + worker_done_tx, + ); + surface } impl BrowserCaptureOptions { @@ -380,6 +486,7 @@ fn runtime_endpoint( }; let chrome = Chrome::launch_with(&ChromeLaunchOptions { binary: chrome_binary, + mode: opts.browser_mode, user_data_dir, ephemeral: opts.browser_ephemeral, })?; @@ -387,6 +494,10 @@ fn runtime_endpoint( Ok((web_socket_url, Some(chrome), BrowserSource::Launched)) } +fn clean_headless_user_agent(user_agent: &str) -> Option { + user_agent.contains("HeadlessChrome").then(|| user_agent.replace("HeadlessChrome", "Chrome")) +} + fn resolve_chrome_binary(explicit: Option<&str>) -> anyhow::Result { if let Some(path) = explicit.filter(|s| !s.trim().is_empty()) { let path = PathBuf::from(path); @@ -568,6 +679,178 @@ fn start_surface_thread( Ok(()) } +fn start_browser_worker( + surface: Arc, + rx: Receiver, + latest_reconfigure: Arc>>, + latest_nav: Arc>>, + mux: Weak, + done_tx: Option>, +) { + let id = surface.id; + let _ = + std::thread::Builder::new().name(format!("browser-surface-{id}-worker")).spawn(move || { + let mut failures = BrowserWorkerErrorState::default(); + while let Ok(first) = rx.recv() { + let mut batch = vec![first]; + while let Ok(next) = rx.try_recv() { + batch.push(next); + } + coalesce_worker_mouse_moves(&mut batch); + for command in batch { + if matches!(command, BrowserCommand::WakeLatest) { + for command in take_latest_worker_commands(&latest_reconfigure, &latest_nav) + { + run_browser_worker_command(&surface, command, &mux, id, &mut failures); + } + } else { + run_browser_worker_command(&surface, command, &mux, id, &mut failures); + } + } + for command in take_latest_worker_commands(&latest_reconfigure, &latest_nav) { + run_browser_worker_command(&surface, command, &mux, id, &mut failures); + } + } + if let Some(done_tx) = done_tx { + let _ = done_tx.send(()); + } + }); +} + +fn take_latest_worker_commands( + latest_reconfigure: &Arc>>, + latest_nav: &Arc>>, +) -> Vec { + let reconfigure = latest_reconfigure.lock().unwrap().take(); + let nav = latest_nav.lock().unwrap().take(); + reconfigure.into_iter().chain(nav).collect() +} + +fn coalesce_worker_mouse_moves(batch: &mut Vec) { + let mut index = 0; + while index + 1 < batch.len() { + if batch[index].is_mouse_move() && batch[index + 1].is_mouse_move() { + batch.remove(index); + } else { + index += 1; + } + } +} + +fn run_browser_worker_command( + surface: &Surface, + command: BrowserCommand, + mux: &Weak, + id: SurfaceId, + failures: &mut BrowserWorkerErrorState, +) { + let is_input = command.is_input(); + let result = { + let Some(browser) = surface.as_browser() else { + return; + }; + match command { + BrowserCommand::WakeLatest => Ok(()), + BrowserCommand::Mouse { event_type, x, y, button, click_count } => { + browser.mouse_event_blocking(&event_type, x, y, button.as_deref(), click_count) + } + BrowserCommand::Wheel { x, y, delta_y } => browser.wheel_blocking(x, y, delta_y), + BrowserCommand::Key { + event_type, + key, + code, + windows_virtual_key_code, + modifiers, + text, + } => browser.key_event_blocking( + &event_type, + &key, + &code, + windows_virtual_key_code, + modifiers, + text.as_deref(), + ), + BrowserCommand::InsertText(text) => browser.insert_text_blocking(&text), + BrowserCommand::Navigate(url) => browser.navigate_blocking(&url), + BrowserCommand::Back => browser.back_blocking(), + BrowserCommand::Forward => browser.forward_blocking(), + BrowserCommand::Reload => browser.reload_blocking(), + BrowserCommand::Activate => browser.activate_blocking(), + BrowserCommand::Reconfigure { width, height } => { + browser.reconfigure_blocking(width, height) + } + } + }; + record_browser_worker_result(surface, mux, id, is_input, result, failures); +} + +fn record_browser_worker_result( + surface: &Surface, + mux: &Weak, + id: SurfaceId, + is_input: bool, + result: anyhow::Result<()>, + failures: &mut BrowserWorkerErrorState, +) { + match result { + Ok(()) => { + failures.consecutive_timeouts = 0; + if !is_input { + emit_browser_dirty(mux, id); + } + } + Err(err) => { + let message = err.to_string(); + let timeout = is_cdp_timeout_error(&message); + if timeout { + failures.consecutive_timeouts = failures.consecutive_timeouts.saturating_add(1); + if failures.consecutive_timeouts >= 2 { + let should_report = surface + .as_browser() + .is_some_and(BrowserSurface::claim_not_responding_report); + if should_report { + if let Some(browser) = surface.as_browser() { + browser.mark_failed(BROWSER_NOT_RESPONDING_MESSAGE.to_string()); + } + emit_browser_failure(mux, id, BROWSER_NOT_RESPONDING_MESSAGE.to_string()); + } + } + } else { + failures.consecutive_timeouts = 0; + } + if !(is_input || timeout && failures.consecutive_timeouts >= 2) { + emit_browser_status(mux, message); + emit_browser_dirty(mux, id); + } + } + } +} + +fn is_cdp_timeout_error(message: &str) -> bool { + message.contains("CDP call ") && message.contains(" timed out") +} + +fn emit_browser_status(mux: &Weak, message: String) { + if let Some(mux) = mux.upgrade() { + mux.emit(MuxEvent::Status(message)); + } +} + +fn emit_browser_dirty(mux: &Weak, id: SurfaceId) { + if let Some(mux) = mux.upgrade() { + mux.emit(MuxEvent::TitleChanged(id)); + mux.emit(MuxEvent::SurfaceOutput(id)); + } +} + +fn emit_browser_failure(mux: &Weak, id: SurfaceId, message: String) { + if let Some(mux) = mux.upgrade() { + mux.emit(MuxEvent::Status(message)); + mux.emit(MuxEvent::TitleChanged(id)); + mux.emit(MuxEvent::SurfaceOutput(id)); + } +} + impl BrowserSurface { pub fn latest_frame(&self) -> Option { let state = self.state.lock().unwrap(); @@ -614,18 +897,29 @@ impl BrowserSurface { self.dirty.swap(false, Ordering::AcqRel) } + #[cfg(test)] + pub(crate) fn take_worker_done_for_test(&self) -> Receiver<()> { + self.worker_done.lock().unwrap().take().expect("worker done receiver already taken") + } + pub fn kill(&self) { if self.dead.swap(true, Ordering::AcqRel) { return; } self.close_taps(); if let Some(session) = self.session.lock().unwrap().take() { - session.runtime.close_surface(&session.target_id, &session.session_id); + session.runtime.close_surface_detached(&session.target_id, &session.session_id); } + self.close_command_sender(); } pub fn resize(&self, cols: u16, rows: u16) { - if let Err(e) = self.try_resize(cols, rows) { + let Some((width, height)) = self.update_resize_state(cols, rows) else { + return; + }; + if let Err(e) = + self.enqueue_latest_reconfigure(BrowserCommand::Reconfigure { width, height }) + { eprintln!("cmux-tui: browser resize failed for surface {}: {e}", self.meta.id); } } @@ -643,7 +937,7 @@ impl BrowserSurface { self.resize(cols, rows); } - fn try_resize(&self, cols: u16, rows: u16) -> anyhow::Result<()> { + fn update_resize_state(&self, cols: u16, rows: u16) -> Option<(u32, u32)> { let (cols, rows) = (cols.max(1), rows.max(1)); let cell = *self.cell_pixels.lock().unwrap(); let pixel_w = cols as u32 * cell.0.max(1) as u32; @@ -667,17 +961,21 @@ impl BrowserSurface { (unchanged, capture_pixels.0, capture_pixels.1) }; if unchanged { - return Ok(()); + return None; } + Some((capture_w, capture_h)) + } + + fn reconfigure_blocking(&self, width: u32, height: u32) -> anyhow::Result<()> { let Some(session) = self.live_session()? else { return Ok(()) }; - session.runtime.client.set_device_metrics(&session.session_id, capture_w, capture_h)?; + session.runtime.client.set_device_metrics(&session.session_id, width, height)?; let _ = session.runtime.client.stop_screencast(&session.session_id); - session.runtime.client.start_screencast(&session.session_id, capture_w, capture_h)?; + session.runtime.client.start_screencast(&session.session_id, width, height)?; Ok(()) } pub fn attach_frames(&self) -> (BrowserAttachState, BrowserFrameStream) { - let (tx, rx) = std::sync::mpsc::sync_channel(1); + let (tx, rx) = sync_channel(1); let slot = Arc::new(Mutex::new(BrowserAttachUpdate::default())); let mut state = self.state.lock().unwrap(); let snapshot = browser_attach_state_locked(&state, Instant::now(), self.is_dead(), true); @@ -690,11 +988,33 @@ impl BrowserSurface { fn store_frame(&self, mut frame: BrowserFrame) { let mut state = self.state.lock().unwrap(); // Screencast frames keep streaming the previous page after a - // failed navigation; they must not clear the Failed status. A - // successful navigate/reload clears it (set_url_title, - // clear_error), same as mark_live. - if !matches!(state.status, BrowserStatus::Failed(_)) { + // failed navigation; they must not mask that failure. A fresh + // frame does prove Chrome recovered from the worker's + // not-responding state, so clear only that class here. + let clears_not_responding = matches!( + state.status, + BrowserStatus::Failed(ref error) if error == BROWSER_NOT_RESPONDING_MESSAGE + ); + if !matches!(state.status, BrowserStatus::Failed(_)) || clears_not_responding { state.status = BrowserStatus::Live; + if clears_not_responding { + state.not_responding_reported = false; + // `mark_failed` overwrote the title with "browser failed: ..." + // and broadcast the failure to attach clients. Recovering only + // in-memory would leave remote TUIs stuck on the failed + // status/title even as fresh frames arrive. Restore a non-failed + // title from the retained URL (the next CDP title event refines + // it) and broadcast the recovered state to attach clients the + // same way the failure was broadcast. + // + // Do NOT set `self.dirty` here: the caller that delivers this + // frame emits `SurfaceOutput` via `if !dirty.swap(true)`, which + // is what redraws the local TUI. Pre-setting `dirty` would + // consume that transition and suppress the local recovery + // redraw, leaving the local status line stuck on the failure. + state.title = state.url.clone(); + Self::mark_state_dirty_locked(&mut state); + } } frame.seq = state.next_frame_seq; state.next_frame_seq = state.next_frame_seq.saturating_add(1); @@ -719,6 +1039,7 @@ impl BrowserSurface { self.dead.store(true, Ordering::Release); self.close_taps(); let _ = self.session.lock().unwrap().take(); + self.close_command_sender(); } fn mark_live(&self, session: BrowserSession) -> anyhow::Result<()> { @@ -859,6 +1180,96 @@ impl BrowserSurface { } } + // Bounded, in-order delivery for disposable pointer/key input. Input events + // are high-frequency and individually expendable, so under backpressure the + // worker queue drops the newest event rather than blocking or replacing an + // unrelated queued one. Callers are intentionally told `ok` even on drop: + // losing one mouse-move or keystroke frame is not a reported failure. + fn enqueue_bounded(&self, command: BrowserCommand) -> anyhow::Result<()> { + if self.is_dead() { + anyhow::bail!("browser surface is closed"); + } + let tx = self.command_sender()?; + match tx.try_send(command) { + Ok(()) | Err(TrySendError::Full(_)) => Ok(()), + Err(TrySendError::Disconnected(_)) => anyhow::bail!("browser command worker is closed"), + } + } + + // Bounded, in-order delivery for discrete control actions + // (back/forward/reload/activate). These stay in FIFO order so a `Back` can + // never be swallowed by a later `Forward` (unlike the latest-wins nav slot), + // but unlike disposable input they must not be silently dropped: losing a + // control action the caller asked for is a user-visible action that + // vanished. When the queue is full (a wedged/unresponsive worker) report + // backpressure as an error instead of a false `ok` so the caller learns the + // command was rejected. `try_send` never blocks, so this preserves the + // non-blocking contract. URL navigation uses the latest-wins slot instead + // (see `enqueue_latest_nav`), where only the final destination matters. + fn enqueue_control(&self, command: BrowserCommand) -> anyhow::Result<()> { + if self.is_dead() { + anyhow::bail!("browser surface is closed"); + } + let tx = self.command_sender()?; + match tx.try_send(command) { + Ok(()) => Ok(()), + Err(TrySendError::Full(_)) => { + anyhow::bail!("browser command queue is full; browser may be unresponsive") + } + Err(TrySendError::Disconnected(_)) => anyhow::bail!("browser command worker is closed"), + } + } + + fn enqueue_latest_reconfigure(&self, command: BrowserCommand) -> anyhow::Result<()> { + if self.is_dead() { + anyhow::bail!("browser surface is closed"); + } + *self.latest_reconfigure.lock().unwrap() = Some(command); + self.wake_worker() + } + + fn enqueue_latest_nav(&self, command: BrowserCommand) -> anyhow::Result<()> { + if self.is_dead() { + anyhow::bail!("browser surface is closed"); + } + self.enqueue_latest_nav_ignoring_dead(command) + } + + fn enqueue_latest_nav_ignoring_dead(&self, command: BrowserCommand) -> anyhow::Result<()> { + *self.latest_nav.lock().unwrap() = Some(command); + self.wake_worker() + } + + fn wake_worker(&self) -> anyhow::Result<()> { + let tx = self.command_sender()?; + match tx.try_send(BrowserCommand::WakeLatest) { + Ok(()) | Err(TrySendError::Full(_)) => Ok(()), + Err(TrySendError::Disconnected(_)) => anyhow::bail!("browser command worker is closed"), + } + } + + fn command_sender(&self) -> anyhow::Result> { + self.command_tx + .lock() + .unwrap() + .clone() + .ok_or_else(|| anyhow::anyhow!("browser command worker is closed")) + } + + fn close_command_sender(&self) { + let _ = self.command_tx.lock().unwrap().take(); + } + + fn claim_not_responding_report(&self) -> bool { + let mut state = self.state.lock().unwrap(); + if state.not_responding_reported { + false + } else { + state.not_responding_reported = true; + true + } + } + pub fn mouse_event( &self, event_type: &str, @@ -866,6 +1277,23 @@ impl BrowserSurface { y: f64, button: Option<&str>, click_count: Option, + ) -> anyhow::Result<()> { + self.enqueue_bounded(BrowserCommand::Mouse { + event_type: event_type.to_string(), + x, + y, + button: button.map(ToOwned::to_owned), + click_count, + }) + } + + fn mouse_event_blocking( + &self, + event_type: &str, + x: f64, + y: f64, + button: Option<&str>, + click_count: Option, ) -> anyhow::Result<()> { let session = self.require_live_session()?; if event_type == "mousePressed" { @@ -883,6 +1311,10 @@ impl BrowserSurface { } pub fn wheel(&self, x: f64, y: f64, delta_y: f64) -> anyhow::Result<()> { + self.enqueue_bounded(BrowserCommand::Wheel { x, y, delta_y }) + } + + fn wheel_blocking(&self, x: f64, y: f64, delta_y: f64) -> anyhow::Result<()> { let session = self.require_live_session()?; self.maybe_nudge_stalled_external(&session); let (x, y) = self.scale_input_point(x, y); @@ -898,6 +1330,25 @@ impl BrowserSurface { windows_virtual_key_code: u32, modifiers: u32, text: Option<&str>, + ) -> anyhow::Result<()> { + self.enqueue_bounded(BrowserCommand::Key { + event_type: event_type.to_string(), + key: key.to_string(), + code: code.to_string(), + windows_virtual_key_code, + modifiers, + text: text.map(ToOwned::to_owned), + }) + } + + fn key_event_blocking( + &self, + event_type: &str, + key: &str, + code: &str, + windows_virtual_key_code: u32, + modifiers: u32, + text: Option<&str>, ) -> anyhow::Result<()> { let session = self.require_live_session()?; self.maybe_nudge_stalled_external(&session); @@ -908,12 +1359,20 @@ impl BrowserSurface { } pub fn insert_text(&self, text: &str) -> anyhow::Result<()> { + self.enqueue_bounded(BrowserCommand::InsertText(text.to_string())) + } + + fn insert_text_blocking(&self, text: &str) -> anyhow::Result<()> { let session = self.require_live_session()?; self.maybe_nudge_stalled_external(&session); session.runtime.client.insert_text(&session.session_id, text) } pub fn navigate(&self, url: &str) -> anyhow::Result<()> { + self.enqueue_latest_nav(BrowserCommand::Navigate(url.to_string())) + } + + fn navigate_blocking(&self, url: &str) -> anyhow::Result<()> { let session = self.require_live_session()?; let normalized = normalize_url(url); if let Some(error) = session.runtime.client.navigate(&session.session_id, &normalized)? { @@ -926,14 +1385,22 @@ impl BrowserSurface { } pub fn back(&self) -> anyhow::Result<()> { - self.navigate_history(-1) + self.enqueue_control(BrowserCommand::Back) } pub fn forward(&self) -> anyhow::Result<()> { - self.navigate_history(1) + self.enqueue_control(BrowserCommand::Forward) + } + + fn back_blocking(&self) -> anyhow::Result<()> { + self.navigate_history_blocking(-1) } - fn navigate_history(&self, delta: isize) -> anyhow::Result<()> { + fn forward_blocking(&self) -> anyhow::Result<()> { + self.navigate_history_blocking(1) + } + + fn navigate_history_blocking(&self, delta: isize) -> anyhow::Result<()> { let session = self.require_live_session()?; let history = session.runtime.client.navigation_history(&session.session_id)?; let next = history.current_index as isize + delta; @@ -950,6 +1417,10 @@ impl BrowserSurface { } pub fn reload(&self) -> anyhow::Result<()> { + self.enqueue_control(BrowserCommand::Reload) + } + + fn reload_blocking(&self) -> anyhow::Result<()> { let session = self.require_live_session()?; session.runtime.client.reload(&session.session_id)?; self.clear_error(); @@ -957,6 +1428,10 @@ impl BrowserSurface { } pub fn activate(&self) -> anyhow::Result<()> { + self.enqueue_control(BrowserCommand::Activate) + } + + fn activate_blocking(&self) -> anyhow::Result<()> { let session = self.require_live_session()?; session.runtime.client.activate_target(&session.target_id, &session.session_id) } @@ -1133,15 +1608,18 @@ fn percent_encode_query(input: &str) -> String { #[cfg(test)] mod tests { use super::{ - BrowserCaptureOptions, BrowserFrame, BrowserSource, BrowserStatus, capture_scale_for, - new_surface, normalize_url, runtime_endpoint, scaled_pixels, + BrowserCaptureOptions, BrowserCommand, BrowserFrame, BrowserSource, BrowserStatus, + capture_scale_for, new_surface, normalize_url, runtime_endpoint, scaled_pixels, + take_latest_worker_commands, }; - use crate::SurfaceOptions; + use crate::{Mux, Surface, SurfaceOptions}; + use serde_json::{Value, json}; use std::io::{Read, Write}; use std::net::{TcpListener, TcpStream}; - use std::sync::mpsc; + use std::sync::{Arc, Mutex, Weak, mpsc}; use std::thread; use std::time::{Duration, Instant}; + use tungstenite::{Message, accept}; fn test_frame(seq: u64) -> BrowserFrame { BrowserFrame { @@ -1224,10 +1702,28 @@ mod tests { runtime_endpoint(opts).map_err(|err| last_err.unwrap_or(err)) } + fn test_surface() -> Arc { + let opts = SurfaceOptions::default(); + new_surface(1, "https://example.test".into(), (10, 5), (8, 16), &opts, Weak::new()) + } + + fn read_ws_json(ws: &mut tungstenite::WebSocket) -> Value { + loop { + match ws.read().unwrap() { + Message::Text(text) => return serde_json::from_str(&text).unwrap(), + Message::Binary(bytes) => return serde_json::from_slice(&bytes).unwrap(), + _ => {} + } + } + } + + fn write_ws_json(ws: &mut tungstenite::WebSocket, value: Value) { + ws.send(Message::Text(value.to_string().into())).unwrap(); + } + #[test] fn frames_do_not_clear_failed_status() { - let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (10, 5), (8, 16), &opts); + let surface = test_surface(); let browser = surface.as_browser().expect("browser surface"); browser.store_frame(test_frame(1)); assert_eq!(browser.status(), BrowserStatus::Live); @@ -1264,6 +1760,403 @@ mod tests { assert_eq!(scaled_pixels(800, 600, 0.5), (400, 300)); } + #[test] + fn launched_runtime_cleans_headless_user_agent_once_and_replays_per_surface() { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let (seen_tx, seen_rx) = mpsc::channel(); + + let server = thread::Builder::new() + .name("browser-stealth-ua-fake-cdp".into()) + .spawn(move || { + let (stream, _) = listener.accept().unwrap(); + let mut ws = accept(stream).unwrap(); + let mut start_count = 0; + loop { + let request = read_ws_json(&mut ws); + let id = request["id"].clone(); + let method = request["method"].as_str().unwrap().to_string(); + seen_tx.send(request.clone()).unwrap(); + match method.as_str() { + "Target.setDiscoverTargets" => { + write_ws_json(&mut ws, json!({"id": id, "result": {}})); + } + "Browser.getVersion" => { + write_ws_json( + &mut ws, + json!({ + "id": id, + "result": { + "userAgent": "Mozilla/5.0 HeadlessChrome/136.0 HeadlessChrome/136.0 Safari/537.36" + } + }), + ); + } + "Emulation.setUserAgentOverride" => { + assert_eq!( + request["params"]["userAgent"], + "Mozilla/5.0 Chrome/136.0 Chrome/136.0 Safari/537.36" + ); + write_ws_json(&mut ws, json!({"id": id, "result": {}})); + } + "Page.enable" + | "Emulation.setDeviceMetricsOverride" + | "Page.startScreencast" => { + write_ws_json(&mut ws, json!({"id": id, "result": {}})); + if method == "Page.startScreencast" { + start_count += 1; + if start_count == 2 { + break; + } + } + } + method => panic!("unexpected CDP method {method}"), + } + } + }) + .unwrap(); + + let runtime = super::BrowserRuntime::connect_to_endpoint( + &format!("ws://{addr}/devtools/browser/fake"), + None, + BrowserSource::Launched, + ) + .unwrap(); + let opts = SurfaceOptions::default(); + let first = + new_surface(11, "https://one.test".into(), (10, 5), (8, 16), &opts, Weak::new()); + runtime + .setup_attached_surface(&first, "target-1", "session-1", "https://one.test") + .unwrap(); + let second = + new_surface(12, "https://two.test".into(), (10, 5), (8, 16), &opts, Weak::new()); + runtime + .setup_attached_surface(&second, "target-2", "session-2", "https://two.test") + .unwrap(); + + server.join().unwrap(); + let methods = seen_rx + .try_iter() + .map(|value| value["method"].as_str().unwrap().to_string()) + .collect::>(); + assert_eq!( + methods.iter().filter(|method| method.as_str() == "Browser.getVersion").count(), + 1 + ); + assert_eq!( + methods + .iter() + .filter(|method| method.as_str() == "Emulation.setUserAgentOverride") + .count(), + 2 + ); + runtime.shutdown(); + } + + #[test] + fn launched_runtime_continues_when_browser_version_fails() { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let (seen_tx, seen_rx) = mpsc::channel(); + + let server = thread::Builder::new() + .name("browser-stealth-version-failure-fake-cdp".into()) + .spawn(move || { + let (stream, _) = listener.accept().unwrap(); + let mut ws = accept(stream).unwrap(); + loop { + let request = read_ws_json(&mut ws); + let id = request["id"].clone(); + let method = request["method"].as_str().unwrap().to_string(); + seen_tx.send(request.clone()).unwrap(); + match method.as_str() { + "Target.setDiscoverTargets" => { + write_ws_json(&mut ws, json!({"id": id, "result": {}})); + } + "Browser.getVersion" => { + write_ws_json( + &mut ws, + json!({"id": id, "error": {"code": -32000, "message": "unavailable"}}), + ); + } + "Page.enable" + | "Emulation.setDeviceMetricsOverride" + | "Page.startScreencast" => { + write_ws_json(&mut ws, json!({"id": id, "result": {}})); + if method == "Page.startScreencast" { + break; + } + } + "Emulation.setUserAgentOverride" => { + panic!("user agent override should be skipped after getVersion failure") + } + method => panic!("unexpected CDP method {method}"), + } + } + }) + .unwrap(); + + let runtime = super::BrowserRuntime::connect_to_endpoint( + &format!("ws://{addr}/devtools/browser/fake"), + None, + BrowserSource::Launched, + ) + .unwrap(); + let surface = test_surface(); + runtime + .setup_attached_surface(&surface, "target-1", "session-1", "https://example.test") + .unwrap(); + + server.join().unwrap(); + let methods = seen_rx + .try_iter() + .map(|value| value["method"].as_str().unwrap().to_string()) + .collect::>(); + assert!(methods.iter().any(|method| method == "Browser.getVersion")); + assert!(!methods.iter().any(|method| method == "Emulation.setUserAgentOverride")); + runtime.shutdown(); + } + + #[test] + fn external_runtime_does_not_query_or_override_user_agent() { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + + let server = thread::Builder::new() + .name("browser-external-stealth-negative-fake-cdp".into()) + .spawn(move || { + let (stream, _) = listener.accept().unwrap(); + let mut ws = accept(stream).unwrap(); + loop { + let request = read_ws_json(&mut ws); + let id = request["id"].clone(); + let method = request["method"].as_str().unwrap().to_string(); + match method.as_str() { + "Target.setDiscoverTargets" => { + write_ws_json(&mut ws, json!({"id": id, "result": {}})); + } + "Page.enable" + | "Emulation.setDeviceMetricsOverride" + | "Page.startScreencast" => { + write_ws_json(&mut ws, json!({"id": id, "result": {}})); + if method == "Page.startScreencast" { + break; + } + } + "Browser.getVersion" | "Emulation.setUserAgentOverride" => { + panic!( + "external runtimes must not receive launched-runtime stealth calls" + ) + } + method => panic!("unexpected CDP method {method}"), + } + } + }) + .unwrap(); + + let runtime = super::BrowserRuntime::connect_to_endpoint( + &format!("ws://{addr}/devtools/browser/fake"), + None, + BrowserSource::External, + ) + .unwrap(); + let surface = test_surface(); + runtime + .setup_attached_surface(&surface, "target-1", "session-1", "https://example.test") + .unwrap(); + + server.join().unwrap(); + runtime.shutdown(); + } + + #[test] + fn latest_reconfigure_and_nav_slots_do_not_clobber_each_other() { + let latest_reconfigure = + Arc::new(Mutex::new(Some(BrowserCommand::Reconfigure { width: 111, height: 222 }))); + let latest_nav = + Arc::new(Mutex::new(Some(BrowserCommand::Navigate("https://next.test".to_string())))); + + let commands = take_latest_worker_commands(&latest_reconfigure, &latest_nav); + assert_eq!(commands.len(), 2); + match &commands[0] { + BrowserCommand::Reconfigure { width, height } => { + assert_eq!((*width, *height), (111, 222)); + } + _ => panic!("reconfigure must drain before nav"), + } + match &commands[1] { + BrowserCommand::Navigate(url) => assert_eq!(url, "https://next.test"), + _ => panic!("nav command was lost"), + } + assert!(latest_reconfigure.lock().unwrap().is_none()); + assert!(latest_nav.lock().unwrap().is_none()); + } + + #[test] + fn kill_drops_sender_and_worker_exits() { + let surface = test_surface(); + let browser = surface.as_browser().expect("browser surface"); + let done = browser.take_worker_done_for_test(); + + browser.kill(); + assert!(browser.navigate("after-close.test").is_err()); + done.recv_timeout(Duration::from_secs(1)).expect("browser worker exited after kill"); + } + + #[test] + fn timeout_failed_status_notice_is_emitted_once_per_stall_episode() { + let surface = test_surface(); + let mux = Mux::new("timeout-latch-test", SurfaceOptions::default()); + let events = mux.subscribe(); + let weak = Arc::downgrade(&mux); + let mut failures = super::BrowserWorkerErrorState::default(); + + super::record_browser_worker_result( + &surface, + &weak, + surface.id, + false, + Err(anyhow::anyhow!("CDP call Page.navigate timed out")), + &mut failures, + ); + assert!(matches!( + events.recv_timeout(Duration::from_secs(1)).unwrap(), + crate::MuxEvent::Status(message) if message == "CDP call Page.navigate timed out" + )); + while events.try_recv().is_ok() {} + + super::record_browser_worker_result( + &surface, + &weak, + surface.id, + false, + Err(anyhow::anyhow!("CDP call Page.navigate timed out")), + &mut failures, + ); + assert!(matches!( + events.recv_timeout(Duration::from_secs(1)).unwrap(), + crate::MuxEvent::Status(message) if message == super::BROWSER_NOT_RESPONDING_MESSAGE + )); + while events.try_recv().is_ok() {} + + super::record_browser_worker_result( + &surface, + &weak, + surface.id, + false, + Err(anyhow::anyhow!("CDP call Page.navigate timed out")), + &mut failures, + ); + assert!(events.recv_timeout(Duration::from_millis(100)).is_err()); + } + + #[test] + fn frame_clearing_not_responding_rearms_timeout_notice() { + let surface = test_surface(); + let browser = surface.as_browser().expect("browser surface"); + let mux = Mux::new("timeout-frame-reset-test", SurfaceOptions::default()); + let events = mux.subscribe(); + let weak = Arc::downgrade(&mux); + let mut failures = super::BrowserWorkerErrorState::default(); + + super::record_browser_worker_result( + &surface, + &weak, + surface.id, + false, + Err(anyhow::anyhow!("CDP call Page.navigate timed out")), + &mut failures, + ); + while events.try_recv().is_ok() {} + + super::record_browser_worker_result( + &surface, + &weak, + surface.id, + false, + Err(anyhow::anyhow!("CDP call Page.navigate timed out")), + &mut failures, + ); + assert!(matches!( + events.recv_timeout(Duration::from_secs(1)).unwrap(), + crate::MuxEvent::Status(message) if message == super::BROWSER_NOT_RESPONDING_MESSAGE + )); + assert_eq!( + browser.status(), + BrowserStatus::Failed(super::BROWSER_NOT_RESPONDING_MESSAGE.to_string()) + ); + while events.try_recv().is_ok() {} + + browser.store_frame(test_frame(1)); + assert_eq!(browser.status(), BrowserStatus::Live); + + super::record_browser_worker_result( + &surface, + &weak, + surface.id, + false, + Err(anyhow::anyhow!("CDP call Page.navigate timed out")), + &mut failures, + ); + assert!(matches!( + events.recv_timeout(Duration::from_secs(1)).unwrap(), + crate::MuxEvent::Status(message) if message == super::BROWSER_NOT_RESPONDING_MESSAGE + )); + assert_eq!( + browser.status(), + BrowserStatus::Failed(super::BROWSER_NOT_RESPONDING_MESSAGE.to_string()) + ); + } + + // Regression: when a fresh frame clears the worker's not-responding + // failure, the recovery must be broadcast to attach clients (remote TUIs), + // not just flipped in memory. Before the fix `store_frame` set status back + // to Live but left the "browser failed: ..." title `mark_failed` had + // written and never marked the state dirty, so attached clients stayed + // stuck on the failed status/title even as frames streamed in. + #[test] + fn recovery_from_not_responding_broadcasts_live_state_to_attach_clients() { + let surface = test_surface(); + let browser = surface.as_browser().expect("browser surface"); + // Give the surface a known URL so the recovered title is derived from it. + browser.set_url_title("https://recovered.test".to_string(), "recovered".to_string()); + // Attach before the failure so the tap observes both the failure and the recovery. + let (_snapshot, stream) = browser.attach_frames(); + + let failed_title = format!("browser failed: {}", super::BROWSER_NOT_RESPONDING_MESSAGE); + browser.mark_failed(super::BROWSER_NOT_RESPONDING_MESSAGE.to_string()); + let failed = stream.slot.lock().unwrap().state.clone().expect("failure was broadcast"); + assert_eq!( + failed.status, + BrowserStatus::Failed(super::BROWSER_NOT_RESPONDING_MESSAGE.to_string()) + ); + assert_eq!(failed.title, failed_title); + // Simulate the event thread drawing the failure and consuming the dirty + // flag, so the recovery below starts from a clean flag like it would in + // production. + assert!(browser.take_dirty(), "mark_failed must mark the surface dirty"); + + // A fresh frame proves Chrome recovered. + browser.store_frame(test_frame(1)); + assert_eq!(browser.status(), BrowserStatus::Live); + // The event thread that delivers this frame emits the local TUI redraw + // via `if !dirty.swap(true)`. store_frame must leave that transition + // available (dirty still clear) instead of pre-consuming it, or the + // local status line stays stuck on the failure. + assert!( + !browser.take_dirty(), + "recovery must not pre-consume the dirty transition the event thread emits on" + ); + let recovered = + stream.slot.lock().unwrap().state.clone().expect("recovery must be broadcast too"); + assert_eq!(recovered.status, BrowserStatus::Live); + assert_ne!( + recovered.title, failed_title, + "recovered attach state still shows the stale failure title" + ); + assert_eq!(recovered.title, "https://recovered.test"); + } + #[test] fn browser_discovery_is_explicit_opt_in() { let listener = TcpListener::bind("127.0.0.1:0").unwrap(); @@ -1279,6 +2172,15 @@ mod tests { browser_discover_ports: vec![port], ..Default::default() }; + let explicit_opts = SurfaceOptions { + cdp_url: Some("ws://127.0.0.1:9/devtools/browser/explicit".to_string()), + ..opts.clone() + }; + let (url, chrome, source) = runtime_endpoint(&explicit_opts).unwrap(); + assert_eq!(url, "ws://127.0.0.1:9/devtools/browser/explicit"); + assert!(chrome.is_none()); + assert_eq!(source, BrowserSource::External); + let err = match runtime_endpoint(&opts) { Ok((url, _, source)) => { panic!("default config should launch, not discover; got {source:?} {url}") @@ -1303,7 +2205,8 @@ mod tests { #[test] fn input_mapping_uses_latest_frame_viewport() { let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (476, 182), (10, 14), &opts); + let surface = + new_surface(1, "https://example.test".into(), (476, 182), (10, 14), &opts, Weak::new()); let browser = surface.as_browser().expect("browser surface"); { let state = browser.state.lock().unwrap(); @@ -1322,7 +2225,8 @@ mod tests { #[test] fn input_mapping_falls_back_to_capture_pixels_before_first_frame() { let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (476, 182), (10, 14), &opts); + let surface = + new_surface(1, "https://example.test".into(), (476, 182), (10, 14), &opts, Weak::new()); let browser = surface.as_browser().expect("browser surface"); assert_eq!(browser.scale_input_point(2380.0, 1274.0), (966.5, 517.5)); @@ -1332,8 +2236,7 @@ mod tests { #[test] fn input_mapping_clamps_to_page_viewport() { - let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (10, 5), (8, 16), &opts); + let surface = test_surface(); let browser = surface.as_browser().expect("browser surface"); browser.store_frame(test_frame(1)); @@ -1342,8 +2245,7 @@ mod tests { #[test] fn frames_stalled_requires_live_surface_over_threshold() { - let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (10, 5), (8, 16), &opts); + let surface = test_surface(); let browser = surface.as_browser().expect("browser surface"); let now = Instant::now(); { @@ -1367,8 +2269,7 @@ mod tests { #[test] fn same_size_resize_does_not_reset_stall_state() { - let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (10, 5), (8, 16), &opts); + let surface = test_surface(); let browser = surface.as_browser().expect("browser surface"); let now = Instant::now(); { @@ -1397,8 +2298,7 @@ mod tests { #[test] fn attach_frames_are_latest_wins_and_close_detaches() { - let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (10, 5), (8, 16), &opts); + let surface = test_surface(); let browser = surface.as_browser().expect("browser surface"); let (_state, stream) = browser.attach_frames(); @@ -1422,8 +2322,7 @@ mod tests { #[test] fn launched_surfaces_never_report_frame_stalls() { - let opts = SurfaceOptions::default(); - let surface = new_surface(1, "https://example.test".into(), (10, 5), (8, 16), &opts); + let surface = test_surface(); let browser = surface.as_browser().expect("browser surface"); let now = Instant::now(); { @@ -1442,6 +2341,38 @@ mod tests { assert!(browser.frames_stalled_at(now)); } + #[test] + fn worker_double_timeout_marks_browser_not_responding_without_waiting() { + let surface = test_surface(); + let mut failures = super::BrowserWorkerErrorState::default(); + + super::record_browser_worker_result( + &surface, + &Weak::new(), + surface.id, + true, + Err(anyhow::anyhow!("CDP call Input.dispatchMouseEvent timed out")), + &mut failures, + ); + assert_ne!( + surface.as_browser().unwrap().status(), + BrowserStatus::Failed(super::BROWSER_NOT_RESPONDING_MESSAGE.to_string()) + ); + + super::record_browser_worker_result( + &surface, + &Weak::new(), + surface.id, + true, + Err(anyhow::anyhow!("CDP call Input.dispatchMouseEvent timed out")), + &mut failures, + ); + assert_eq!( + surface.as_browser().unwrap().status(), + BrowserStatus::Failed(super::BROWSER_NOT_RESPONDING_MESSAGE.to_string()) + ); + } + #[test] fn normalizes_browser_urls() { assert_eq!(normalize_url("example.com"), "https://example.com"); diff --git a/cmux-tui/crates/cmux-tui-core/src/lib.rs b/cmux-tui/crates/cmux-tui-core/src/lib.rs index 94d0e1c89f14..85c99d459ad4 100644 --- a/cmux-tui/crates/cmux-tui-core/src/lib.rs +++ b/cmux-tui/crates/cmux-tui-core/src/lib.rs @@ -35,6 +35,7 @@ pub use surface::{ BrowserStatus, DefaultColors, Surface, SurfaceKind, SurfaceOptions, }; +pub use cmux_tui_cdp::BrowserMode; pub use ghostty_vt::Rgb; pub type SurfaceId = u64; diff --git a/cmux-tui/crates/cmux-tui-core/src/mux.rs b/cmux-tui/crates/cmux-tui-core/src/mux.rs index 58d5cb490d99..b3f6b88d7c1d 100644 --- a/cmux-tui/crates/cmux-tui-core/src/mux.rs +++ b/cmux-tui/crates/cmux-tui-core/src/mux.rs @@ -390,7 +390,8 @@ impl Mux { let opts = self.surface_options.lock().unwrap().clone(); let size = size.unwrap_or((opts.cols, opts.rows)); let cell_pixels = *self.cell_pixels.lock().unwrap(); - let surface = browser::new_surface(id, url.clone(), size, cell_pixels, &opts); + let surface = + browser::new_surface(id, url.clone(), size, cell_pixels, &opts, Arc::downgrade(self)); self.state.lock().unwrap().surfaces.insert(id, surface.clone()); self.start_browser_bootstrap(surface.clone(), BrowserBootstrap::Create { url }, None); surface @@ -1060,21 +1061,10 @@ impl Mux { let opts = self.surface_options.lock().unwrap().clone(); let size = size.unwrap_or((opts.cols, opts.rows)); let cell_pixels = *self.cell_pixels.lock().unwrap(); - let surface = browser::new_surface(id, url.clone(), size, cell_pixels, &opts); + let surface = + browser::new_surface(id, url.clone(), size, cell_pixels, &opts, Arc::downgrade(self)); let active_at = self.next_active_at(); - let attached = { - let mut state = self.state.lock().unwrap(); - let Some(pane) = state.panes.get_mut(&pane_id) else { - return false; - }; - pane.tabs.push(surface.id); - pane.active_tab = pane.tabs.len() - 1; - pane.active_at = active_at; - state.surfaces.insert(surface.id, surface.clone()); - true - }; - if !attached { - surface.kill(); + if !self.attach_browser_surface_to_pane_or_kill(pane_id, &surface, active_at) { return false; } self.emit(MuxEvent::TreeChanged); @@ -1086,6 +1076,31 @@ impl Mux { true } + fn attach_browser_surface_to_pane_or_kill( + &self, + pane_id: PaneId, + surface: &Arc, + active_at: u64, + ) -> bool { + let attached = { + let mut state = self.state.lock().unwrap(); + match state.panes.get_mut(&pane_id) { + Some(pane) => { + pane.tabs.push(surface.id); + pane.active_tab = pane.tabs.len() - 1; + pane.active_at = active_at; + state.surfaces.insert(surface.id, surface.clone()); + true + } + None => false, + } + }; + if !attached { + surface.kill(); + } + attached + } + /// Working directory of a pane's active surface, if reported. fn pane_cwd(&self, pane: PaneId) -> Option { let surface = { @@ -2059,6 +2074,27 @@ mod tests { assert!(mux.with_state(|state| state.surfaces.contains_key(&second.id))); } + #[test] + fn failed_browser_surface_attach_kills_worker() { + let mux = test_mux(); + let opts = mux.surface_options.lock().unwrap().clone(); + let surface = browser::new_surface( + 999, + "https://example.test".to_string(), + (10, 5), + (8, 16), + &opts, + Arc::downgrade(&mux), + ); + let browser = surface.as_browser().expect("browser surface"); + let done = browser.take_worker_done_for_test(); + + assert!(!mux.attach_browser_surface_to_pane_or_kill(123_456, &surface, 1)); + assert!(browser.is_dead()); + done.recv_timeout(Duration::from_secs(1)) + .expect("browser worker exited after failed attach"); + } + #[test] fn notification_sets_unread_and_clears_when_tab_is_viewed() { let mux = test_mux(); diff --git a/cmux-tui/crates/cmux-tui-core/src/surface.rs b/cmux-tui/crates/cmux-tui-core/src/surface.rs index 6f745a2cf235..b68c3db245c8 100644 --- a/cmux-tui/crates/cmux-tui-core/src/surface.rs +++ b/cmux-tui/crates/cmux-tui-core/src/surface.rs @@ -20,6 +20,7 @@ use crate::browser::BrowserSurface; pub use crate::browser::{ BrowserAttachState, BrowserFrame, BrowserFrameStream, BrowserSource, BrowserStatus, }; +use cmux_tui_cdp::BrowserMode; /// How to spawn surface children. #[derive(Debug, Clone)] @@ -45,6 +46,8 @@ pub struct SurfaceOptions { pub browser_discover_ports: Vec, /// Optional Chrome user data directory for launched browser runtime. pub browser_user_data_dir: Option, + /// Whether launched Chrome should show a visible window or run headless. + pub browser_mode: BrowserMode, /// Session component for the default launched Chrome profile path. pub browser_session_name: String, /// Use a temporary launched Chrome profile and delete it on shutdown. @@ -72,6 +75,7 @@ impl Default for SurfaceOptions { browser_discover: false, browser_discover_ports: vec![9222], browser_user_data_dir: None, + browser_mode: BrowserMode::Headful, browser_session_name: "default".to_string(), browser_ephemeral: false, browser_max_capture_megapixels: 2.0, diff --git a/cmux-tui/crates/cmux-tui-core/tests/browser_runtime.rs b/cmux-tui/crates/cmux-tui-core/tests/browser_runtime.rs index 41b7c12f5761..6bd433520417 100644 --- a/cmux-tui/crates/cmux-tui-core/tests/browser_runtime.rs +++ b/cmux-tui/crates/cmux-tui-core/tests/browser_runtime.rs @@ -470,8 +470,7 @@ fn socket_browser_attach_streams_frames_input_and_cell_pixels() { &socket_path, json!({"id": 9, "cmd": "browser-navigate", "surface": surface, "url": "bad.test"}), ); - assert_eq!(navigate["ok"], false); - assert!(navigate["error"].as_str().unwrap().contains("ERR_NAME_NOT_RESOLVED")); + assert_eq!(navigate["ok"], true, "browser-navigate should ack accepted work: {navigate}"); let navigate_request = recv_method(&seen_rx, "Page.navigate"); assert_eq!(navigate_request["sessionId"], "session-1"); assert_eq!(navigate_request["params"]["url"], "https://bad.test"); @@ -491,6 +490,382 @@ fn socket_browser_attach_streams_frames_input_and_cell_pixels() { server.join().unwrap(); } +#[test] +fn wedged_browser_navigate_does_not_block_same_socket_connection() { + let _guard = TEST_LOCK.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let (seen_tx, seen_rx) = mpsc::channel(); + + let server = thread::spawn(move || { + let (stream, _) = listener.accept().unwrap(); + let mut ws = accept(stream).unwrap(); + loop { + let request = read_json(&mut ws); + let id = request["id"].clone(); + let method = request["method"].as_str().unwrap().to_string(); + seen_tx.send(request.clone()).unwrap(); + match method.as_str() { + "Target.setDiscoverTargets" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Target.createTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"targetId": "target-1"}})); + } + "Target.attachToTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"sessionId": "session-1"}})); + } + "Page.enable" | "Emulation.setDeviceMetricsOverride" | "Page.startScreencast" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Page.navigate" => { + // Deliberately never respond. The browser worker may + // sit in CdpClient::call until timeout, but this mux + // socket connection must remain usable. + } + "Target.closeTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"success": true}})); + break; + } + method => panic!("unexpected CDP method {method}"), + } + } + }); + + let opts = SurfaceOptions { + cdp_url: Some(format!("ws://{addr}/devtools/browser/fake")), + browser_discover: false, + ..Default::default() + }; + let mux = Mux::new("browser-wedged-navigate-test", opts); + let socket_path = std::env::temp_dir() + .join(format!( + "cmux-browser-wedged-navigate-test-{}-{}", + std::process::id(), + SOCKET_SERIAL.fetch_add(1, std::sync::atomic::Ordering::Relaxed) + )) + .join("session.sock"); + server::serve(mux.clone(), Some(socket_path.clone())).unwrap(); + let created = rpc( + &socket_path, + json!({"id": 1, "cmd": "new-browser-tab", "url": "example.test", "cols": 10, "rows": 5}), + ); + assert_eq!(created["ok"], true); + let surface = created["data"]["surface"].as_u64().unwrap(); + wait_for( + || matches!(mux.surface(surface)?.browser_status()?, BrowserStatus::Live).then_some(()), + Duration::from_secs(10), + ) + .expect("browser went live"); + + let mut stream = UnixStream::connect(&socket_path).unwrap(); + stream.set_read_timeout(Some(Duration::from_millis(500))).unwrap(); + let navigate = + json!({"id": 2, "cmd": "browser-navigate", "surface": surface, "url": "wedged.test"}); + stream.write_all(navigate.to_string().as_bytes()).unwrap(); + stream.write_all(b"\n").unwrap(); + + let navigate_request = recv_method(&seen_rx, "Page.navigate"); + assert_eq!(navigate_request["sessionId"], "session-1"); + assert_eq!(navigate_request["params"]["url"], "https://wedged.test"); + + let started = Instant::now(); + let second_navigate = + json!({"id": 3, "cmd": "browser-navigate", "surface": surface, "url": "still-wedged.test"}); + stream.write_all(second_navigate.to_string().as_bytes()).unwrap(); + stream.write_all(b"\n").unwrap(); + let resize = + json!({"id": 4, "cmd": "resize-surface", "surface": surface, "cols": 12, "rows": 6}); + stream.write_all(resize.to_string().as_bytes()).unwrap(); + stream.write_all(b"\n").unwrap(); + let list = json!({"id": 5, "cmd": "list-workspaces"}); + stream.write_all(list.to_string().as_bytes()).unwrap(); + stream.write_all(b"\n").unwrap(); + + let mut reader = BufReader::new(stream); + let mut first = String::new(); + reader.read_line(&mut first).expect("first navigate ack timed out"); + let first: Value = serde_json::from_str(&first).unwrap(); + assert_eq!(first["id"], 2); + assert_eq!(first["ok"], true); + + let mut second = String::new(); + reader.read_line(&mut second).expect("second navigate ack timed out"); + let second: Value = serde_json::from_str(&second).unwrap(); + assert_eq!(second["id"], 3); + assert_eq!(second["ok"], true); + + let mut third = String::new(); + reader.read_line(&mut third).expect("resize-surface ack timed out"); + let third: Value = serde_json::from_str(&third).unwrap(); + assert_eq!(third["id"], 4); + assert_eq!(third["ok"], true); + + let mut fourth = String::new(); + reader.read_line(&mut fourth).expect("list-workspaces response timed out"); + let fourth: Value = serde_json::from_str(&fourth).unwrap(); + assert_eq!(fourth["id"], 5); + assert_eq!(fourth["ok"], true); + assert!( + started.elapsed() < Duration::from_millis(500), + "same socket was blocked behind wedged navigate for {:?}", + started.elapsed() + ); + + let close_started = Instant::now(); + mux.close_surface(surface); + assert!( + close_started.elapsed() < Duration::from_millis(500), + "wedged browser close blocked for {:?}", + close_started.elapsed() + ); + mux.shutdown(); + server::cleanup(&socket_path); + server.join().unwrap(); +} + +// Regression: discrete history/control commands must not collapse into a +// single latest-wins slot. While the worker is blocked inside a slow +// `Page.navigate`, a `browser-back` then a `browser-forward` are both accepted; +// both must reach the worker in order (entry 10 then entry 12). With the old +// shared `latest_nav` slot the forward silently overwrote the back and only one +// `Page.navigateToHistoryEntry` was ever sent. +#[test] +fn queued_back_and_forward_do_not_collapse_while_worker_is_blocked() { + let _guard = TEST_LOCK.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let (seen_tx, seen_rx) = mpsc::channel(); + let (release_tx, release_rx) = mpsc::channel::<()>(); + + let server = thread::spawn(move || { + let (stream, _) = listener.accept().unwrap(); + let mut ws = accept(stream).unwrap(); + loop { + let request = read_json(&mut ws); + let id = request["id"].clone(); + let method = request["method"].as_str().unwrap().to_string(); + seen_tx.send(request.clone()).unwrap(); + match method.as_str() { + "Target.setDiscoverTargets" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Target.createTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"targetId": "target-1"}})); + } + "Target.attachToTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"sessionId": "session-1"}})); + } + "Page.enable" | "Emulation.setDeviceMetricsOverride" | "Page.startScreencast" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Page.navigate" => { + // Hold the worker inside the CDP call until the test has + // queued back+forward behind it, then let it finish. + let _ = release_rx.recv(); + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Page.getNavigationHistory" => { + write_json( + &mut ws, + json!({ + "id": id, + "result": { + "currentIndex": 1, + "entries": [ + {"id": 10, "url": "https://back.test", "title": "back"}, + {"id": 11, "url": "https://current.test", "title": "current"}, + {"id": 12, "url": "https://forward.test", "title": "forward"} + ] + } + }), + ); + } + "Page.navigateToHistoryEntry" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Target.closeTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"success": true}})); + break; + } + method => panic!("unexpected CDP method {method}"), + } + } + }); + + let opts = SurfaceOptions { + cdp_url: Some(format!("ws://{addr}/devtools/browser/fake")), + browser_discover: false, + ..Default::default() + }; + let mux = Mux::new("browser-history-collapse-test", opts); + let socket_path = std::env::temp_dir() + .join(format!( + "cmux-hist-collapse-{}-{}", + std::process::id(), + SOCKET_SERIAL.fetch_add(1, std::sync::atomic::Ordering::Relaxed) + )) + .join("session.sock"); + server::serve(mux.clone(), Some(socket_path.clone())).unwrap(); + let created = rpc( + &socket_path, + json!({"id": 1, "cmd": "new-browser-tab", "url": "example.test", "cols": 10, "rows": 5}), + ); + assert_eq!(created["ok"], true); + let surface = created["data"]["surface"].as_u64().unwrap(); + wait_for( + || matches!(mux.surface(surface)?.browser_status()?, BrowserStatus::Live).then_some(()), + Duration::from_secs(10), + ) + .expect("browser went live"); + + // Block the worker inside Page.navigate. + let navigate = rpc( + &socket_path, + json!({"id": 2, "cmd": "browser-navigate", "surface": surface, "url": "wedged.test"}), + ); + assert_eq!(navigate["ok"], true); + let navigate_request = recv_method(&seen_rx, "Page.navigate"); + assert_eq!(navigate_request["params"]["url"], "https://wedged.test"); + + // Queue back then forward while the worker is stuck on the navigate. Both + // are accepted immediately; neither may drop the other. + let back = rpc(&socket_path, json!({"id": 3, "cmd": "browser-back", "surface": surface})); + assert_eq!(back["ok"], true); + let forward = rpc(&socket_path, json!({"id": 4, "cmd": "browser-forward", "surface": surface})); + assert_eq!(forward["ok"], true); + + // Let the navigate finish; the worker now drains the queued history commands. + release_tx.send(()).unwrap(); + + let back_nav = recv_method(&seen_rx, "Page.navigateToHistoryEntry"); + assert_eq!(back_nav["params"]["entryId"], 10, "back must navigate to entry 10"); + let forward_nav = recv_method(&seen_rx, "Page.navigateToHistoryEntry"); + assert_eq!( + forward_nav["params"]["entryId"], 12, + "forward must not be swallowed by back through a shared latest-wins slot" + ); + + mux.close_surface(surface); + mux.shutdown(); + server::cleanup(&socket_path); + server.join().unwrap(); +} + +// Regression: discrete control commands (back/forward/reload/activate) must +// not be silently dropped when the bounded command queue is full. Disposable +// pointer/key input may drop under backpressure, but a control action the +// caller explicitly asked for is user-visible; dropping it while returning a +// false `ok:true` loses the action with no signal. While the worker is wedged +// inside a never-completing `Page.navigate`, the queue cannot drain, so once it +// saturates further control commands must be reported as `ok:false`. +#[test] +fn control_command_reports_backpressure_when_worker_queue_is_full() { + let _guard = TEST_LOCK.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let (seen_tx, seen_rx) = mpsc::channel(); + + let server = thread::spawn(move || { + let (stream, _) = listener.accept().unwrap(); + let mut ws = accept(stream).unwrap(); + loop { + let request = read_json(&mut ws); + let id = request["id"].clone(); + let method = request["method"].as_str().unwrap().to_string(); + seen_tx.send(request.clone()).unwrap(); + match method.as_str() { + "Target.setDiscoverTargets" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Target.createTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"targetId": "target-1"}})); + } + "Target.attachToTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"sessionId": "session-1"}})); + } + "Page.enable" | "Emulation.setDeviceMetricsOverride" | "Page.startScreencast" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Page.navigate" => { + // Never respond: the worker stays inside the CDP call so the + // bounded command queue cannot drain. + } + "Page.reload" | "Page.navigateToHistoryEntry" => { + write_json(&mut ws, json!({"id": id, "result": {}})); + } + "Target.closeTarget" => { + write_json(&mut ws, json!({"id": id, "result": {"success": true}})); + break; + } + method => panic!("unexpected CDP method {method}"), + } + } + }); + + let opts = SurfaceOptions { + cdp_url: Some(format!("ws://{addr}/devtools/browser/fake")), + browser_discover: false, + ..Default::default() + }; + let mux = Mux::new("browser-control-backpressure-test", opts); + let socket_path = std::env::temp_dir() + .join(format!( + "cmux-control-backpressure-{}-{}", + std::process::id(), + SOCKET_SERIAL.fetch_add(1, std::sync::atomic::Ordering::Relaxed) + )) + .join("session.sock"); + server::serve(mux.clone(), Some(socket_path.clone())).unwrap(); + let created = rpc( + &socket_path, + json!({"id": 1, "cmd": "new-browser-tab", "url": "example.test", "cols": 10, "rows": 5}), + ); + assert_eq!(created["ok"], true); + let surface = created["data"]["surface"].as_u64().unwrap(); + wait_for( + || matches!(mux.surface(surface)?.browser_status()?, BrowserStatus::Live).then_some(()), + Duration::from_secs(10), + ) + .expect("browser went live"); + + // Wedge the worker inside Page.navigate so the queue can never drain. + let navigate = rpc( + &socket_path, + json!({"id": 2, "cmd": "browser-navigate", "surface": surface, "url": "wedged.test"}), + ); + assert_eq!(navigate["ok"], true); + let navigate_request = recv_method(&seen_rx, "Page.navigate"); + assert_eq!(navigate_request["params"]["url"], "https://wedged.test"); + + // Flood control commands. Early ones fit in the 64-slot queue and are + // accepted; once it saturates the surface must report ok:false rather than a + // silent drop with a false ok:true. Bounded loop so a broken (never-full) + // queue fails the test instead of hanging. + let mut saw_accept = false; + let mut saw_rejection = false; + for i in 0..512u64 { + let reload = + rpc(&socket_path, json!({"id": 1000 + i, "cmd": "browser-reload", "surface": surface})); + if reload["ok"] == true { + saw_accept = true; + } else { + saw_rejection = true; + break; + } + } + assert!(saw_accept, "control commands must be accepted before the queue saturates"); + assert!( + saw_rejection, + "a full command queue must be reported as ok:false, not silently dropped with ok:true" + ); + + mux.close_surface(surface); + mux.shutdown(); + server::cleanup(&socket_path); + server.join().unwrap(); +} + #[test] fn browser_capture_scale_applies_to_metrics_screencast_and_input() { let _guard = TEST_LOCK.lock().unwrap_or_else(|poisoned| poisoned.into_inner()); diff --git a/cmux-tui/crates/cmux-tui/src/app.rs b/cmux-tui/crates/cmux-tui/src/app.rs index dea4f374ed9c..35e682cc7656 100644 --- a/cmux-tui/crates/cmux-tui/src/app.rs +++ b/cmux-tui/crates/cmux-tui/src/app.rs @@ -621,6 +621,7 @@ pub fn run( // Crossterm input → app channel. Start this after startup terminal // probes so DA / window-size responses are not consumed as key input. std::thread::Builder::new().name("input".into()).spawn({ + let tx = tx.clone(); move || { while let Ok(event) = crossterm::event::read() { if tx.send(AppEvent::Input(event)).is_err() { @@ -687,7 +688,7 @@ pub fn run( cell_pixels, pointer_shape: false, last_browser_hover: None, - browser_input: BrowserInputDispatcher::spawn()?, + browser_input: BrowserInputDispatcher::spawn(tx)?, drag: None, encoder, encode_buf: Vec::with_capacity(64), @@ -1479,13 +1480,7 @@ impl App { return Ok(RenderAction::Draw); } let url = cmux_tui_core::normalize_url(input); - match self.session.surface(state.surface) { - Some(handle) => { - self.status_message = - handle.browser_navigate(&url).err().map(|e| e.to_string()); - } - None => self.status_message = Some("unknown browser surface".to_string()), - } + self.enqueue_browser_command(state.surface, BrowserInputKind::Navigate(url)); } InputEvent::Changed | InputEvent::None => {} } @@ -1630,18 +1625,15 @@ impl App { Action::ScrollUp => self.scroll_active(-10), Action::ScrollDown => self.scroll_active(10), Action::BrowserBack => { - let result = self.browser_back(); - self.set_status_from_browser_result(result); + self.enqueue_active_browser_command(BrowserInputKind::Back); return Ok(RenderAction::Draw); } Action::BrowserForward => { - let result = self.browser_forward(); - self.set_status_from_browser_result(result); + self.enqueue_active_browser_command(BrowserInputKind::Forward); return Ok(RenderAction::Draw); } Action::BrowserReload => { - let result = self.browser_reload(); - self.set_status_from_browser_result(result); + self.enqueue_active_browser_command(BrowserInputKind::Reload); return Ok(RenderAction::Draw); } Action::BrowserEditUrl => { @@ -1661,10 +1653,6 @@ impl App { Ok(RenderAction::Draw) } - fn set_status_from_browser_result(&mut self, result: anyhow::Result<()>) { - self.status_message = result.err().map(|err| err.to_string()); - } - fn open_rename_tab_prompt(&mut self, pane: Option) { let Some(pane) = pane else { return }; let Some(tab) = self.tree.pane(pane).and_then(|p| p.tabs.get(p.active_tab)) else { @@ -1715,28 +1703,6 @@ impl App { Ok(()) } - fn active_browser_handle(&self) -> anyhow::Result { - let Some(surface) = self.active_surface_handle() else { - anyhow::bail!("no active surface"); - }; - if surface.kind() != SurfaceKind::Browser { - anyhow::bail!("active surface is not a browser"); - } - Ok(surface) - } - - fn browser_back(&mut self) -> anyhow::Result<()> { - self.active_browser_handle()?.browser_back() - } - - fn browser_forward(&mut self) -> anyhow::Result<()> { - self.active_browser_handle()?.browser_forward() - } - - fn browser_reload(&mut self) -> anyhow::Result<()> { - self.active_browser_handle()?.browser_reload() - } - fn focus_omnibar(&mut self, pane: PaneId) { let Some(surface_id) = self.tree.pane(pane).and_then(|pane| pane.active_surface()) else { return; @@ -1773,7 +1739,7 @@ impl App { Some(OmnibarState { pane, surface, input: TextInput::new(buffer), select_all }); } - fn browser_handle_for_pane(&self, pane: PaneId) -> anyhow::Result { + fn browser_surface_for_pane(&self, pane: PaneId) -> anyhow::Result<(SurfaceId, SurfaceHandle)> { let Some(surface_id) = self.tree.pane(pane).and_then(|pane| pane.active_surface()) else { anyhow::bail!("pane has no active surface"); }; @@ -1783,7 +1749,58 @@ impl App { if surface.kind() != SurfaceKind::Browser { anyhow::bail!("active surface is not a browser"); } - Ok(surface) + Ok((surface_id, surface)) + } + + /// Dispatch a discrete browser control command (navigate/back/forward/ + /// reload/activate). Unlike disposable input, a full dispatcher queue + /// (worker wedged in a blocking browser call) must not drop the command + /// silently: surface backpressure through the status line so the user + /// knows the action did not take effect. + fn dispatch_browser_control( + &mut self, + surface_id: SurfaceId, + surface: SurfaceHandle, + kind: BrowserInputKind, + ) { + if self.browser_input.enqueue(BrowserInputEvent { surface_id, surface, kind }) { + self.status_message = None; + } else { + self.status_message = Some("browser is busy; command dropped".to_string()); + } + } + + fn enqueue_active_browser_command(&mut self, kind: BrowserInputKind) { + let Some((surface_id, surface)) = self.active_surface_with_handle() else { + self.status_message = Some("no active surface".to_string()); + return; + }; + if surface.kind() != SurfaceKind::Browser { + self.status_message = Some("active surface is not a browser".to_string()); + return; + } + self.dispatch_browser_control(surface_id, surface, kind); + } + + fn enqueue_browser_command_for_pane(&mut self, pane: PaneId, kind: BrowserInputKind) { + match self.browser_surface_for_pane(pane) { + Ok((surface_id, surface)) => { + self.dispatch_browser_control(surface_id, surface, kind); + } + Err(err) => self.status_message = Some(err.to_string()), + } + } + + fn enqueue_browser_command(&mut self, surface_id: SurfaceId, kind: BrowserInputKind) { + let Some(surface) = self.session.surface(surface_id) else { + self.status_message = Some("unknown browser surface".to_string()); + return; + }; + if surface.kind() != SurfaceKind::Browser { + self.status_message = Some("active surface is not a browser".to_string()); + return; + } + self.dispatch_browser_control(surface_id, surface, kind); } fn browser_copy_url(&mut self, pane: PaneId) { @@ -1832,26 +1849,18 @@ impl App { } MenuAction::CloseScreen(id) => self.session.close_screen(id), MenuAction::BrowserBack(id) => { - let result = - self.browser_handle_for_pane(id).and_then(|handle| handle.browser_back()); - self.set_status_from_browser_result(result); + self.enqueue_browser_command_for_pane(id, BrowserInputKind::Back); } MenuAction::BrowserForward(id) => { - let result = - self.browser_handle_for_pane(id).and_then(|handle| handle.browser_forward()); - self.set_status_from_browser_result(result); + self.enqueue_browser_command_for_pane(id, BrowserInputKind::Forward); } MenuAction::BrowserReload(id) => { - let result = - self.browser_handle_for_pane(id).and_then(|handle| handle.browser_reload()); - self.set_status_from_browser_result(result); + self.enqueue_browser_command_for_pane(id, BrowserInputKind::Reload); } MenuAction::BrowserEditUrl(id) => self.focus_omnibar(id), MenuAction::BrowserCopyUrl(id) => self.browser_copy_url(id), MenuAction::BrowserActivate(id) => { - let result = - self.browser_handle_for_pane(id).and_then(|handle| handle.browser_activate()); - self.set_status_from_browser_result(result); + self.enqueue_browser_command_for_pane(id, BrowserInputKind::Activate); } MenuAction::RenameTab(id) => self.open_rename_tab_prompt(Some(id)), MenuAction::CopyTabId(id) => { @@ -2037,7 +2046,7 @@ impl App { .modifiers .intersects(KeyModifiers::CONTROL | KeyModifiers::ALT | KeyModifiers::SUPER) { - self.browser_input.enqueue(BrowserInputEvent { + let _ = self.browser_input.enqueue(BrowserInputEvent { surface_id, surface, kind: BrowserInputKind::InsertText(c.to_string()), @@ -2055,13 +2064,13 @@ impl App { modifiers, text, }; - self.browser_input.enqueue(BrowserInputEvent { + let _ = self.browser_input.enqueue(BrowserInputEvent { surface_id, surface: surface.clone(), kind: key_event("keyDown", text), }); if key.kind == KeyEventKind::Press { - self.browser_input.enqueue(BrowserInputEvent { + let _ = self.browser_input.enqueue(BrowserInputEvent { surface_id, surface, kind: key_event("keyUp", None), @@ -2072,7 +2081,7 @@ impl App { fn paste(&mut self, text: &str) { let Some((surface_id, surface)) = self.active_surface_with_handle() else { return }; if surface.kind() == SurfaceKind::Browser { - self.browser_input.enqueue(BrowserInputEvent { + let _ = self.browser_input.enqueue(BrowserInputEvent { surface_id, surface, kind: BrowserInputKind::InsertText(text.to_string()), @@ -2399,21 +2408,13 @@ impl App { } match hit { OmnibarHit::Back => { - let result = - self.browser_handle_for_pane(pane).and_then(|handle| handle.browser_back()); - self.set_status_from_browser_result(result); + self.enqueue_browser_command_for_pane(pane, BrowserInputKind::Back); } OmnibarHit::Forward => { - let result = self - .browser_handle_for_pane(pane) - .and_then(|handle| handle.browser_forward()); - self.set_status_from_browser_result(result); + self.enqueue_browser_command_for_pane(pane, BrowserInputKind::Forward); } OmnibarHit::Reload => { - let result = self - .browser_handle_for_pane(pane) - .and_then(|handle| handle.browser_reload()); - self.set_status_from_browser_result(result); + self.enqueue_browser_command_for_pane(pane, BrowserInputKind::Reload); } OmnibarHit::Edit => self.focus_omnibar(pane), } @@ -2929,7 +2930,7 @@ impl App { if area.content.contains(x, y) { let (px, py) = self.browser_point(area.content, x, y); let delta = if down { 3.0 } else { -3.0 } * f64::from(self.cell_pixels.1); - self.browser_input.enqueue(BrowserInputEvent { + let _ = self.browser_input.enqueue(BrowserInputEvent { surface_id, surface, kind: BrowserInputKind::Wheel { x: px, y: py, delta_y: delta }, @@ -2988,7 +2989,7 @@ impl App { ) { let Some(surface) = self.session.surface(surface_id) else { return }; let (px, py) = self.browser_point(content, x, y); - self.browser_input.enqueue(BrowserInputEvent { + let _ = self.browser_input.enqueue(BrowserInputEvent { surface_id, surface, kind: BrowserInputKind::Mouse { @@ -3353,7 +3354,7 @@ mod tests { cell_pixels: (8, 16), pointer_shape: false, last_browser_hover: None, - browser_input: BrowserInputDispatcher::spawn().unwrap(), + browser_input: BrowserInputDispatcher::spawn(std::sync::mpsc::channel().0).unwrap(), drag: None, encoder: KeyEncoder::new().unwrap(), encode_buf: Vec::new(), diff --git a/cmux-tui/crates/cmux-tui/src/browser_input.rs b/cmux-tui/crates/cmux-tui/src/browser_input.rs index d7485cbcabc6..787ea1cba689 100644 --- a/cmux-tui/crates/cmux-tui/src/browser_input.rs +++ b/cmux-tui/crates/cmux-tui/src/browser_input.rs @@ -1,4 +1,4 @@ -//! Off-loop browser input forwarding. +//! Off-loop browser command forwarding. //! //! Forwarding input to a browser surface ultimately performs blocking //! I/O: a CDP request/response on the shared WebSocket for local @@ -16,14 +16,15 @@ //! call), events are dropped instead of blocking the UI. Dropped //! input against a wedged browser was going nowhere anyway. //! -//! Results are intentionally discarded: browser input has no caller -//! that can act on a per-event error, and the surface's own status -//! (`BrowserStatus`) is what the UI reports. +//! Results are intentionally discarded: browser commands report their +//! user-visible errors through the surface's own status (`BrowserStatus`) +//! and status events. -use std::sync::mpsc::{Receiver, SyncSender, sync_channel}; +use std::sync::mpsc::{Receiver, Sender, SyncSender, sync_channel}; -use cmux_tui_core::SurfaceId; +use cmux_tui_core::{MuxEvent, SurfaceId}; +use crate::app::AppEvent; use crate::session::SurfaceHandle; /// Bounded queue depth. Input events are tiny; this is sized so bursts @@ -59,6 +60,11 @@ pub enum BrowserInputKind { text: Option<&'static str>, }, InsertText(String), + Navigate(String), + Back, + Forward, + Reload, + Activate, } impl BrowserInputKind { @@ -67,6 +73,20 @@ impl BrowserInputKind { fn is_mouse_move(&self) -> bool { matches!(self, BrowserInputKind::Mouse { event_type: "mouseMoved", .. }) } + + /// Discrete control actions the user explicitly invoked. Unlike disposable + /// pointer/key input, a control command that fails to reach the browser + /// must surface backpressure instead of vanishing. + fn is_control(&self) -> bool { + matches!( + self, + BrowserInputKind::Navigate(_) + | BrowserInputKind::Back + | BrowserInputKind::Forward + | BrowserInputKind::Reload + | BrowserInputKind::Activate + ) + } } pub struct BrowserInputDispatcher { @@ -74,20 +94,43 @@ pub struct BrowserInputDispatcher { } impl BrowserInputDispatcher { - pub fn spawn() -> anyhow::Result { + /// `feedback` is the app event channel. A discrete control command that + /// fails inside the worker (per-surface queue full, surface closed, or a + /// remote request error) reports back through it as a status event so the + /// user sees the command did not take effect; disposable input errors stay + /// discarded. + pub fn spawn(feedback: Sender) -> anyhow::Result { let (tx, rx) = sync_channel(QUEUE_CAPACITY); - std::thread::Builder::new().name("mux-browser-input".into()).spawn(move || worker(rx))?; + std::thread::Builder::new() + .name("mux-browser-input".into()) + .spawn(move || worker(rx, feedback))?; Ok(BrowserInputDispatcher { tx }) } - /// Queue an event; never blocks. A full queue (worker wedged inside - /// a blocking browser call) drops the event. - pub fn enqueue(&self, event: BrowserInputEvent) { - let _ = self.tx.try_send(event); + /// Queue an event; never blocks. Returns `false` when the queue is + /// full (the worker is wedged inside a blocking browser call) and the + /// event was dropped. Disposable input (mouse/key) may ignore the + /// result, but discrete control commands (navigate/back/forward/ + /// reload/activate) must surface backpressure to the user instead of + /// dropping silently: a reload the user asked for that never runs and + /// gives no feedback is a bug, unlike a coalesced mouse move. + #[must_use = "control commands must surface backpressure instead of dropping silently"] + pub fn enqueue(&self, event: BrowserInputEvent) -> bool { + self.tx.try_send(event).is_ok() } } -fn worker(rx: Receiver) { +#[cfg(test)] +impl BrowserInputDispatcher { + /// Build a dispatcher whose worker never runs, so the caller can hold + /// the receiver and saturate the queue deterministically. + fn without_worker() -> (Self, Receiver) { + let (tx, rx) = sync_channel(QUEUE_CAPACITY); + (BrowserInputDispatcher { tx }, rx) + } +} + +fn worker(rx: Receiver, feedback: Sender) { while let Ok(event) = rx.recv() { // Drain whatever queued behind the first event so mouse moves // can be coalesced across the batch. @@ -97,7 +140,7 @@ fn worker(rx: Receiver) { } coalesce_mouse_moves(&mut batch); for event in batch { - dispatch(&event); + dispatch(&event, &feedback); } } } @@ -119,9 +162,9 @@ fn coalesce_mouse_moves(batch: &mut Vec) { } } -fn dispatch(event: &BrowserInputEvent) { +fn dispatch(event: &BrowserInputEvent, feedback: &Sender) { let surface = &event.surface; - let _ = match &event.kind { + let result = match &event.kind { BrowserInputKind::Mouse { event_type, x, y, button, click_count } => { surface.browser_mouse_event(event_type, *x, *y, *button, *click_count) } @@ -142,7 +185,24 @@ fn dispatch(event: &BrowserInputEvent) { *text, ), BrowserInputKind::InsertText(text) => surface.browser_insert_text(text), + BrowserInputKind::Navigate(url) => surface.browser_navigate(url), + BrowserInputKind::Back => surface.browser_back(), + BrowserInputKind::Forward => surface.browser_forward(), + BrowserInputKind::Reload => surface.browser_reload(), + BrowserInputKind::Activate => surface.browser_activate(), }; + // Disposable input errors are discarded by design (a wedged browser + // surfaces itself via BrowserStatus). A discrete control command the user + // invoked must not fail silently here: the outer queue already accepted it, + // so this inner failure (per-surface queue full, surface closed, remote + // request error) is the only place left to report it. Surface it as a + // status event, matching the outer-queue backpressure path. + if event.kind.is_control() + && let Err(err) = result + { + let _ = feedback + .send(AppEvent::Mux(MuxEvent::Status(format!("browser command failed: {err}")))); + } } #[cfg(test)] @@ -177,6 +237,54 @@ mod tests { } } + fn reload_event(surface: SurfaceId) -> BrowserInputEvent { + BrowserInputEvent { + surface_id: surface, + surface: SurfaceHandle::RemoteBrowserUnsupported, + kind: BrowserInputKind::Reload, + } + } + + // Regression: a full dispatcher queue (worker wedged inside a blocking + // browser call) must report the drop so control commands can surface + // backpressure to the user, instead of the old `let _ = try_send` that + // swallowed the failure and made a dropped reload/navigate look accepted. + #[test] + fn full_queue_reports_drop_instead_of_swallowing_it() { + let (dispatcher, _rx) = BrowserInputDispatcher::without_worker(); + for _ in 0..QUEUE_CAPACITY { + assert!(dispatcher.enqueue(reload_event(1)), "queue should accept until full"); + } + assert!( + !dispatcher.enqueue(reload_event(1)), + "a full queue must report the drop, not swallow it as accepted" + ); + } + + // Regression: a discrete control command that fails inside the worker + // (here: RemoteBrowserUnsupported bails) must report a status event so the + // user learns it did not take effect, instead of the old `let _ = ...` that + // swallowed the inner result even after the outer queue accepted it. + // Disposable input must not report. + #[test] + fn failed_control_command_reports_status_but_input_does_not() { + use std::sync::mpsc::channel; + let (tx, rx) = channel::(); + + dispatch(&reload_event(1), &tx); + match rx.try_recv() { + Ok(AppEvent::Mux(MuxEvent::Status(msg))) => { + assert!(msg.contains("browser command failed"), "unexpected message: {msg}"); + } + Ok(_) => panic!("control failure emitted a non-status event"), + Err(_) => panic!("a failed control command must emit a status event"), + } + + // Disposable input never reports, so the worker stays quiet for it. + dispatch(&move_event(1, 1.0), &tx); + assert!(rx.try_recv().is_err(), "disposable input must not emit status feedback"); + } + fn positions(batch: &[BrowserInputEvent]) -> Vec<(&'static str, SurfaceId)> { batch .iter() diff --git a/cmux-tui/crates/cmux-tui/src/config.rs b/cmux-tui/crates/cmux-tui/src/config.rs index 1c38147c9a74..94f8d963123e 100644 --- a/cmux-tui/crates/cmux-tui/src/config.rs +++ b/cmux-tui/crates/cmux-tui/src/config.rs @@ -37,6 +37,7 @@ //! }, //! "browser": { //! "chrome_binary": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", +//! "mode": "headful", //! "cdp_url": "http://127.0.0.1:9222", //! "discover": false, //! "discover_ports": [9222], @@ -100,6 +101,7 @@ use std::io::Write; use std::path::{Path, PathBuf}; use std::time::{SystemTime, UNIX_EPOCH}; +use cmux_tui_core::BrowserMode; use cmux_tui_core::SidebarPluginOptions; use cmux_tui_core::SurfaceOptions; use cmux_tui_core::platform; @@ -382,6 +384,7 @@ struct RawSidebarPlugin { #[serde(deny_unknown_fields)] struct RawBrowser { chrome_binary: Option, + mode: Option, cdp_url: Option, discover: Option, discover_ports: Option>, @@ -391,6 +394,22 @@ struct RawBrowser { capture_scale: Option, } +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)] +#[serde(rename_all = "kebab-case")] +enum ConfigBrowserMode { + Headful, + Headless, +} + +impl From for BrowserMode { + fn from(mode: ConfigBrowserMode) -> Self { + match mode { + ConfigBrowserMode::Headful => BrowserMode::Headful, + ConfigBrowserMode::Headless => BrowserMode::Headless, + } + } +} + #[derive(Debug, Clone, Default, Deserialize)] #[serde(deny_unknown_fields)] struct RawScrollbar { @@ -542,6 +561,7 @@ fn parse_sidebar_view(value: &str) -> Result { #[derive(Debug, Clone)] pub struct Browser { pub chrome_binary: Option, + pub mode: BrowserMode, pub cdp_url: Option, pub discover: bool, pub discover_ports: Vec, @@ -555,6 +575,7 @@ impl Default for Browser { fn default() -> Self { Browser { chrome_binary: None, + mode: BrowserMode::Headful, cdp_url: None, discover: false, discover_ports: vec![9222], @@ -1118,6 +1139,9 @@ pub fn load() -> Config { } } config.browser.chrome_binary = raw.browser.chrome_binary.filter(|s| !s.trim().is_empty()); + if let Some(mode) = raw.browser.mode { + config.browser.mode = mode.into(); + } config.browser.cdp_url = raw.browser.cdp_url.filter(|s| !s.trim().is_empty()); if let Some(discover) = raw.browser.discover { config.browser.discover = discover; @@ -1158,6 +1182,7 @@ pub fn load() -> Config { pub fn apply_browser_to_surface_options(config: &Config, options: &mut SurfaceOptions) { options.chrome_binary = config.browser.chrome_binary.clone(); + options.browser_mode = config.browser.mode; options.cdp_url = config.browser.cdp_url.clone(); options.browser_discover = config.browser.discover; options.browser_discover_ports = config.browser.discover_ports.clone(); @@ -1610,6 +1635,22 @@ mod tests { assert_eq!(config.theme.border_inactive, Theme::default().border_inactive); } + #[test] + fn browser_mode_defaults_headful_parses_headless_and_rejects_invalid_values() { + let raw: RawConfig = serde_json::from_str(r##"{}"##).unwrap(); + assert!(raw.browser.mode.is_none()); + assert_eq!(Browser::default().mode, BrowserMode::Headful); + + let raw: RawConfig = + serde_json::from_str(r##"{"browser": {"mode": "headless"}}"##).unwrap(); + assert_eq!(raw.browser.mode.map(BrowserMode::from), Some(BrowserMode::Headless)); + + let err = serde_json::from_str::(r##"{"browser": {"mode": "stealth"}}"##) + .unwrap_err() + .to_string(); + assert!(err.contains("unknown variant `stealth`"), "{err}"); + } + #[test] fn config_path_prefers_cmux_tui_json_and_falls_back_to_legacy_mux_json() { let _guard = CONFIG_ENV_LOCK.lock().unwrap(); diff --git a/cmux-tui/docs/browser-panes.md b/cmux-tui/docs/browser-panes.md index af8f3e4af26c..9e5c84d9f54a 100644 --- a/cmux-tui/docs/browser-panes.md +++ b/cmux-tui/docs/browser-panes.md @@ -4,7 +4,7 @@ Browser panes are local Chrome/Chromium targets controlled with the Chrome DevTo ## Requirements -Browser panes need a local CDP endpoint or a launchable Chrome/Chromium-family binary. The TUI can reuse an external endpoint, discover one on configured local ports, or launch Chrome itself in `--headless=new` mode. +Browser panes need a local CDP endpoint or a launchable Chrome/Chromium-family binary. The TUI can reuse an external endpoint, discover one on configured local ports, or launch Chrome itself. Launched Chrome is headful by default; set `browser.mode` to `"headless"` to hide the window. Endpoint selection order: @@ -39,16 +39,22 @@ Printable character keys and paste use CDP insert-text. Enter, Backspace, Tab, E Left click, drag, release, and wheel events inside browser content are forwarded as CDP mouse input. Wheel deltas are scaled by the detected cell height. +Browser input, navigation, activation, and resize reconfiguration are accepted into a per-surface worker queue. Socket responses mean accepted, not completed; later CDP failures are reported through browser status events and status messages. Two consecutive CDP call timeouts mark only that surface failed with `browser is not responding`. + ## Profiles and Lifecycle Browser panes share one browser runtime per mux session. Closing a browser tab closes only its target. Mux shutdown kills Chrome only when cmux launched it. Launched Chrome uses a persistent cmux profile unless `browser.ephemeral` is true. `browser.user_data_dir` overrides the persistent profile path. When ephemeral mode is true, Chrome uses a temporary profile that is deleted on shutdown and ignores `browser.user_data_dir`. -The default launched profile is `~/Library/Application Support/cmux-tui/chrome-profile` on macOS. On non-macOS targets it is `$XDG_DATA_HOME/cmux-tui/chrome-profile` when `XDG_DATA_HOME` is set, then `~/.local/share/cmux-tui/chrome-profile`. +The default launched profile is scoped by session under `~/Library/Application Support/cmux-tui/chrome-profile/` on macOS. On non-macOS targets it is scoped by session under `$XDG_DATA_HOME/cmux-tui/chrome-profile/` when `XDG_DATA_HOME` is set, then `~/.local/share/cmux-tui/chrome-profile/`. + +Chrome 136 and newer reject CDP remote debugging on the OS-default profile directory, and a running normal Chrome owns its profile `SingletonLock`. Use the cmux-tui profile, point `browser.user_data_dir` at a copy or dedicated profile directory after quitting normal Chrome, or attach to a Chrome you launched with `--remote-debugging-port`. + +To attach to an existing runtime, set `browser.cdp_url`, `CMUX_MUX_CDP_URL`, or enable discovery on a local port. Agent Browser can be attached by running `agent-browser get cdp-url` and using the returned `ws://` URL. Only `ws://` and `http://` endpoints are supported in this build; `wss://` is not supported. ## Limitations -Browser panes are local-only as of protocol v6. `attach-surface` returns an error for browser surfaces, attach clients do not receive browser frame streams, and a remote TUI shows a placeholder for browser panes. +Attach clients can stream browser panes as of protocol v6. Older protocol servers show a placeholder for browser panes. -Headful external Chrome can throttle screencast frames when the window or tab is hidden or occluded. Chrome 136 and newer do not allow `--remote-debugging-port` with the default user data directory, so reusable everyday Chrome profiles may not expose CDP. +Headful external Chrome can throttle screencast frames when the window or tab is hidden or occluded. Mux nudges stalled external targets once before interaction with `Target.activateTarget`. diff --git a/cmux-tui/docs/configuration.md b/cmux-tui/docs/configuration.md index 4046e72c88f3..2e7d773f4881 100644 --- a/cmux-tui/docs/configuration.md +++ b/cmux-tui/docs/configuration.md @@ -77,15 +77,20 @@ cmux-tui plugin disable | Key | Type | Default | Effect | | --- | --- | --- | --- | | `browser.chrome_binary` | string | `null` | Chrome/Chromium binary to launch when no external CDP endpoint is used | +| `browser.mode` | `"headful"` or `"headless"` | `"headful"` | Whether launched Chrome shows a visible window or uses `--headless=new` | | `browser.cdp_url` | string | `null` | External CDP endpoint, accepted as `http://host:port` or `ws://...` | -| `browser.discover` | boolean | `true` | Probe discovery ports before launching Chrome | +| `browser.discover` | boolean | `false` | Probe discovery ports before launching Chrome | | `browser.discover_ports` | integer array | `[9222]` | Local ports to probe for `/json/version` | | `browser.user_data_dir` | string | `null` | Persistent profile directory for launched Chrome | | `browser.ephemeral` | boolean | `false` | Use a temporary launched Chrome profile and delete it on shutdown | +| `browser.max_capture_megapixels` | number | `2.0` | Maximum browser capture size before downscaling | +| `browser.capture_scale` | number or null | `null` | Fixed capture scale from 0.0 through 1.0 | When `browser.ephemeral` is true, it takes precedence over `browser.user_data_dir`: launched Chrome uses a fresh temporary profile, and the configured directory is not deleted. -The default launched profile is `~/Library/Application Support/cmux-tui/chrome-profile` on macOS. On non-macOS targets it is `$XDG_DATA_HOME/cmux-tui/chrome-profile` when `XDG_DATA_HOME` is set, then `~/.local/share/cmux-tui/chrome-profile`. +The default launched profile is scoped by session under `~/Library/Application Support/cmux-tui/chrome-profile/` on macOS. On non-macOS targets it is scoped by session under `$XDG_DATA_HOME/cmux-tui/chrome-profile/` when `XDG_DATA_HOME` is set, then `~/.local/share/cmux-tui/chrome-profile/`. + +Chrome 136 and newer reject CDP remote debugging on the OS-default profile directory, and a running normal Chrome owns its profile `SingletonLock`. Use the cmux-tui profile, point `browser.user_data_dir` at a copy or dedicated profile directory after quitting normal Chrome, or attach to a Chrome you launched with `--remote-debugging-port`. Agent Browser can be attached by running `agent-browser get cdp-url` and using the returned `ws://` URL as `browser.cdp_url`. Only `ws://` and `http://` endpoints are supported in this build; `wss://` is not supported. ## Scrollbar @@ -190,11 +195,14 @@ Chord strings can be single characters or a key name with optional `ctrl`, `cont }, "browser": { "chrome_binary": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", + "mode": "headful", "cdp_url": "http://127.0.0.1:9222", - "discover": true, + "discover": false, "discover_ports": [9222, 9223], "user_data_dir": "/Users/me/Library/Application Support/cmux-tui/chrome-profile", - "ephemeral": false + "ephemeral": false, + "max_capture_megapixels": 2.0, + "capture_scale": null }, "scrollbar": { "position": "column" diff --git a/cmux-tui/docs/protocol.md b/cmux-tui/docs/protocol.md index 06313c8cc0bd..f5dfd8b5c1c0 100644 --- a/cmux-tui/docs/protocol.md +++ b/cmux-tui/docs/protocol.md @@ -60,6 +60,15 @@ focus-pane select-tab select-screen select-workspace +browser-mouse +browser-wheel +browser-key +browser-insert-text +browser-navigate +browser-back +browser-forward +browser-reload +browser-activate subscribe attach-surface scroll-surface @@ -101,9 +110,11 @@ Subscribed event lines are: `surface-resized` reports the final clamped cell size and is emitted only when the surface size actually changes. +Browser input, navigation, activation, and browser reconfigure work from `resize-surface` enqueue per-surface CDP work and return `ok:true` after acceptance. Completion or failure is observed later via browser state and status events. Two consecutive CDP call timeouts mark only that browser surface failed with `browser is not responding`. + ## Attach Surface -`attach-surface` streams a PTY surface. Browser surfaces return `browser panes are not supported over attach yet`. +`attach-surface` streams a PTY or browser surface. ```json {"id":30,"cmd":"attach-surface","surface":4} @@ -124,6 +135,8 @@ Then it sends ordered stream frames: The `resized` attach frame carries the new cell size and a fresh VT replay captured at that size. It is delivered in the same attach stream as output frames, so a client can reset its local terminal, apply the replay, and continue consuming later output in order. +For browser surfaces, the server first sends `browser-state` with URL, title, size, status, stalled-frame state, and the latest PNG frame if one exists. Later updates send `browser-state` and `frame` events. Frame payloads are base64 PNG data and slow clients skip older frames rather than buffering unboundedly. + When the stream ends, it sends: ```json @@ -140,4 +153,4 @@ When several attach clients render the same surface at different sizes, sizing f ## Browser Limitations -Browser surfaces appear in `list-workspaces` as `kind: "browser"` with `browser_source: "external"` or `"launched"`. PTY and VT commands against browser surfaces return errors. `attach-surface` does not stream browser pixels as of protocol v6, and the remote TUI shows a placeholder for browser panes. +Browser surfaces appear in `list-workspaces` as `kind: "browser"` with `browser_source: "external"` or `"launched"` once live, plus additive `browser_status`, `browser_error`, and `browser_frames_stalled` fields. PTY and VT commands against browser surfaces return errors.