test(computer-use): cover native Windows and macOS registry dispatch - #109505
Closed
TheSmokeDev wants to merge 55 commits into
Closed
TheSmokeDev wants to merge 55 commits into
TheSmokeDev wants to merge 55 commits into
Conversation
…with human take-over A bot running on a headless Linux gateway now gets its own desktop (TigerVNC Xvnc + a minimal Xfce session, one per profile) that Hermes Desktop streams live. The user can watch the bot work, take over to type a login / 2FA code / CAPTCHA, and hand control back; the bot refuses every computer_use action (screenshots included) while a human holds the screen, then resumes with the session cookies the human just created. Why this shape: - The screen lives on the machine Hermes runs on, not in a vendor cloud browser, so it works for any app the bot drives and keeps the session on the user's host. - Xfce components are launched individually (xfwm4, xfce4-panel, xfdesktop, xfsettingsd) under a private dbus session rather than xfce4-session/the metapackage: no screensaver, power manager or polkit agent to lock or prompt a headless desktop. - Transport is raw RFB over a WebSocket beside /api/ws, authenticated with a one-shot ticket minted through the already-authenticated RPC channel; noVNC runs in the Electron renderer. The bridge parses the RFB client stream and drops keyboard/pointer/clipboard (incl. QEMU Extended KeyEvent, which noVNC switches to once Xvnc advertises it) from any viewer that does not hold the lease, so viewOnly is enforced server-side, not by the client. - One lease per profile (agent | human viewer) is the single truth for the RFB bridge, the computer_use tool and the Desktop UI; taking control evicts other viewers' input with close code 4000 control-taken. - Auto-start happens only at the computer_use tool boundary (headless host, packages present, bot_desktop.auto_start=true); env builders stay pure so status probes and tests never spawn X servers. tests/tools/conftest.py pins the binaries to "missing" for the same reason the browser-use fixture does. Surfaces: Desktop (Bots → right-click → Open Screen; Take over / Hand back), CLI (`hermes computer-use screen status|start|stop|install-deps`), tool (`computer_use` actions request_handoff / wait_for_human), gateway RPCs (display.status/start/stop/observe/lease.acquire/lease.release + display.lease event), docs page user-guide/features/bot-screen.
After a server-side eviction (4000 control-taken) or stream loss noVNC has already torn the client down; detach() then called disconnect() on it and noVNC logged "Tried changing state of a disconnected RFB object". Clear the ref in the disconnect listener so teardown only touches a live client.
The declaration lived in src/types/novnc.d.ts, which .gitignore drops (apps/desktop/src/**/*.d.ts is ignored except for an allowlist), so the pushed tree failed typecheck with TS7016 while the local worktree passed. An ambient 'declare module' needs a script-scoped declaration file, so it joins vite-env.d.ts rather than the module-scoped global.d.ts.
…kage install from Desktop
Three ways in, one install path, no shell required.
- Screen portal box: a compact card ("Screen · Live / Stopped / Not installed on
host", who holds control) rendered at the top of a bot's Scheduled Jobs pane
above the routines, and under each gateway/profile group header in the
Sessions sidebar (new `sidebar.gatewayGroup.header` contribution area, so
the plugin owns the box and core only exposes the slot). Clicking it opens
the Screen pane; it reads the same display.status/lease events as the pane.
- Install from Desktop: `display.install` runs the distro package command on
the gateway host (apt/dnf/pacman) as a supervised child, streams
`display.install.log`, and finishes with `display.install.done`. When sudo
needs a password the host raises `display.install.sudo.request` on the
caller's own WebSocket; the renderer shows the existing masked SudoDialog
(pointed at `display.install.sudo.respond`), so the password never touches
the renderer store as plaintext beyond the field, is redacted from the
gateway trace like `sudo.respond`, and an empty answer cancels without
spawning the package manager. One install per profile at a time. The pane's
"packages missing" state is now an install card with the command shown for
the shell-inclined.
- Opt-in stays intact: nothing installs on `hermes update`; the only triggers
are the Desktop button and `hermes computer-use screen install`.
Why the SudoDialog fallback to the app-level card: the install belongs to the
connection, not to a chat turn, so the request has no session id; the focused
chat's dialog now also shows the session-less card instead of it dying unseen.
Live (headless Electron via CDP, `hermes serve` with the packages hidden):
portal in Scheduled Jobs → click → Screen pane → Install on host → sudo card →
Cancel → "Install cancelled" and the card returns; with the packages present:
portal → Start → live stream, portal flips to "Live · bot in control"; the
same portal renders under the `default` profile in the Sessions sidebar.
…ed Jobs pane The bot's computer is now the first thing in its pane: a big 16:10 box above the title that shows an actual picture of the desktop, refreshed every 4 s while the hero is on screen (paused when scrolled away or the window is hidden). Caption carries who holds control; the chip reads Open live / Start / Install. Clicking the picture expands into the live Screen pane where the user can take over. - `display.thumbnail` RPC: one JPEG grab of the profile's Xvnc display via Pillow's ImageGrab (XAUTHORITY from the launcher's published env), scaled to <=960x600, returned as a data URL. Read-only: it never touches the lease, so a human in control is not disturbed and the bot is not blocked. - `ScreenHero` replaces the small portal row in the routines pane; the compact `ScreenPortal` stays for the Sessions sidebar group header where a 16:10 box would crowd the list. Live: hero "Screen is off" → click → pane → Start → hero shows the Xfce desktop with "Live · bot in control"; an xmessage window opened on the bot's display appeared in the hero on the next refresh; clicking the hero opened the live pane (canvas + Take over).
… theme, curated dock The vendor Xfce panel layout (copied only to silence the first-run dialog) put a light grey bar with dead launchers on every bot screen: File Manager and Text Editor pointed at Thunar/Mousepad we deliberately do not install, Web Browser opened exo's "pick a browser" dialog. It read as an unconfigured VM, and so did the thumbnail in Desktop. - Wallpaper: `tools/bot_desktop/wallpaper.png` (the Nous gradient), seeded via xfconf as a zoomed backdrop over the existing colour fallback. - Dark theme: first dark GTK theme the host ships (Adwaita-dark, Breeze-Dark, Greybird-dark, Arc-Dark, else Adwaita), dark icon theme likewise, xfwm4 Default decorations, DejaVu fonts; both panels dark with slight translucency. - Own panel layout: top bar = menu · task list · tray · clock; bottom dock = only launchers whose program exists on this host, the browser pinned to the one the bot drives (chrome → chromium → firefox) so a human who takes over lands in the bot's own browser profile. Anything the user installs still appears in the Applications menu; the dock is the only curated part. - `launcher.sh` and the wallpaper declared as package data (the launcher was already missing from sealed wheels). - `HERMES_BD_SEED_ONLY=1` stops the launcher after seeding so the config tree is testable on a fake PATH without an X server. Live: fresh screen shows the gradient, dark panels, two dock icons; XTEST clicks on the dock opened xfce4-terminal and Chrome, both dark-themed and listed in the task bar; the Desktop hero and Screen pane show the same picture.
…ight actions, sudo reply pinned to origin, dock Browser is the bot's browser, safe display reuse, install keeps profile scope Independent review of the PR head found the control boundary only held inside one process and several claims the code did not back. Each item below was reproduced, fixed, covered by an invariant test proven red without the fix, and re-verified live on a real Xvnc/Xfce screen. - Lease authority on disk. `lease.json` under an fcntl lock in the profile's bot-desktop dir; every read goes to the file. `hermes serve` (viewer bridge), the messaging gateway, a CLI turn and isolated workers now agree. Live: a takeover in process A made `computer_use capture` in process B return human_has_control; release in C made B work again. - Takeover fences admitted actions. `handle_computer_use` re-checks the lease under the dispatch lock and discards a result produced after the lease epoch changed, so an action admitted before a takeover cannot picture what the human typed during approval / backend start-up waits. - Sudo reply pinned to its origin. `SudoRequest.origin` records the (connection, profile) the card came from; SudoDialog answers through `requestGatewayForAgent` on that socket, never the foreground gateway. A password typed for host A can no longer reach host B. `sudo.expire` and `display.install.sudo.expire` now tear the card down (the Desktop never handled sudo.expire). - Dock Browser IS the bot's browser. `tools/bot_desktop/browser.py` resolves one identity — the Chromium agent-browser drives + a persistent per-profile user-data-dir (`bot-desktop/browser-profile`) — and both sides use it: the agent env gets AGENT_BROWSER_EXECUTABLE_PATH / AGENT_BROWSER_PROFILE, the dock launcher gets the same exe + --user-data-dir. Live: the bot wrote localStorage on http://127.0.0.1:8765 through agent-browser; a dock click and a typed URL on the screen showed BOT-WROTE-THIS in Chrome for Testing. - Safe display reuse. Allocation under a host-wide lock; a recorded number is reused only when no live server holds it; the launcher never unlinks a lock whose pid is alive. Live: A stopped, B took :20, A restarted on :21, B kept running. - Install worker keeps the caller's profile scope (copy_context carries the HERMES_HOME override and the transport); the done event carries the requested profile's status. - Honest scope: `bot_desktop.auto_start` defaults to false (opt-in; Start lives in the Screen pane); request_handoff no longer claims a Telegram/Discord message was sent — the model relays the ask in its reply; docs match.
…ped viewer link keeps exclusion
…ames marked, lease fails closed Second review round (@Julientalbot): - Desktop `display.*` listeners (lease, install log/done) match on (connectionId, profile_key), not the profile path alone: two hosts with the same ~/.hermes path no longer repaint each other's screen pane or install card. One predicate, `isEventForBotScreen`. - Hero thumbnail: three consecutive failed refreshes dim the last frame and caption it "Last seen — screen unreachable"; a dead gateway never keeps looking live. - Lease file present but unparsable reads as HUMAN holds (fail closed); only a missing file is a fresh agent-held profile. The next successful write repairs it. - Taking over after `request_handoff` keeps the agent's reason on the lease and the pane shows it while the human acts, not only before they clicked Take over.
computer_use imports tools.computer_use.handoff (and the lease) on every non-empty action, so the module-level `import fcntl` made every desktop action raise ModuleNotFoundError on native Windows / fcntl-less hosts. The import is now optional; the lease file semantics are unchanged and only the cross-process lock degrades to a no-op where no multi-process Bot Desktop exists. (cherry picked from commit 82ca07dda5d73741a5d6ac60f7b596dd156ec20c)
…ib modules fcntl/pwd/grp/termios/resource/pty/tty fail at import time on Windows and take every importer down with them; the lease regression slipped through because no rule covered this class. Honours the existing `# windows-footgun: ok` marker; scoped to unindented imports so lazy and try/except ImportError forms pass. (cherry picked from commit aa625e7d2649dbe4e4357da712e492cb97f66acc)
…cribed The Event was set before the import and on_change() ran, so a failure there left the flag set and no listener ever installed: every later lease transition would go unbroadcast for the life of the process. (cherry picked from commit c5d43c620fc9ea1428fa546b079ceeca6dbc7ef3)
Real-profile local sessions attach over a loopback cdp_url, yet that Chrome is launched with the Bot Desktop DISPLAY, so it is the very browser a human who took over is typing into; keying the fence on "no cdp_url" let every command through. Fence whenever the session carries the `local` feature and exempt only remote/cloud/user-supplied CDP. Also fence while a human holds the lease even when the published DISPLAY is gone (dead Xvnc), matching computer_use instead of silently unfencing. (cherry picked from commit 7b8825bf32a67229666f1f848d3ac171ff3fbc93)
…ent to aux vision The epoch check ran only after _dispatch() returned, by which point the capture path had already written the PNG to the media cache, spilled the element tree to disk and routed the frame through auxiliary vision — the human's screen had left the process before being "discarded". A fence callable is threaded into _dispatch; capture and capture_after call it the moment backend.capture() returns, and the post-dispatch check stays for every other action. (cherry picked from commit 70c09e5fad89d2817f13aea772bd845ab7cc4c8a)
…e wrong shape
`[]`, `null` or `5` parsed fine and then raised AttributeError outside
the except tuple; `{}` or an unknown holder read as "agent holds". Any
of these is a torn or tampered file, and an untrusted lease must never
let the agent act on a screen a human may be using.
(cherry picked from commit dd77806ab619fcd55ce45698c5f3d739fdd63fa9)
… lease
The Desktop polls thumbnails on a timer, so every connected client kept
receiving frames of the screen a human had taken over — the same frames
computer_use refuses to capture. Answer {data_url: null, suppressed:
'human_has_control'} without touching the framebuffer.
(cherry picked from commit 9bb968026fe6563f793c4fb6aa67ce0577bc881e)
…ks another viewer's lease lease.release(None) skips the holder check, so a client that lost its viewer id (or any bare RPC) could take control away from whoever held it. Refuse with code viewer_mismatch unless params.force is set; display.stop and the CLI keep their unconditional release. (cherry picked from commit ae86da0a8d83ee60b538efb9f545d36213a3228c)
…wers the handoff
wait_for_release polled until the full timeout (10 min by default) even
when the holder never left AGENT, so an unseen request_handoff blocked the
turn instead of letting the model chase the user in chat. After `grace`
seconds (param, default 60, capped at the timeout) with the handoff still
pending and no human holding, return {ok: false, code: no_takeover}. Once
a human holds the screen the full timeout still applies to the hand-back.
(cherry picked from commit 1402af036044a7191ec885fc8fa5c24206880909)
/api/ws consumed any ticket and stamped its identity; display.observe mints provider "bot-desktop" tickets for viewers who may only be watching a screen, so one of those redeemed on /api/ws became a full authenticated session. Refuse bot-desktop tickets in _ws_auth_reason (reported as ticket_invalid, same as the display route refuses gateway tickets). (cherry picked from commit 753c3c35975fe1efab4a5bbc8bc6bbc9532b8521)
Reject oversized positive and extended clipboard lengths at the header. Process coalesced WebSocket data in bounded slices without rejecting valid multi-message frames. Includes fragmented-header and boundary regressions. Addresses the clipboard finding reported by carlotestor and corroborated by helix4u and other reviewers on NousResearch#108914. (cherry picked from commit b1fbe363266e6d6fa3d73d2605fe1ca383607859)
The header check alone removes the unbounded buffering: every other client message type is fixed-size or bounded by a 16-bit count / one byte, so once ClientCutText is capped nothing can grow _buf past ~256 KiB + a partial frame. The feed()/_feed() slicing wrapper was a second layer over the same fact and its coalesced-frame test exercised behaviour the base parser already had. Rename the test file into the rfb_filter family. (cherry picked from commit b9bfa7de4c7e968347923d109c3ae68a66ceb37d)
…akeover _bridge remembered "this viewer held control at some point" and never forgot it, so after a hand-back to the agent a takeover by another human closed the ex-holder's socket with 4000 control-taken although they were a plain watcher by then. The eviction rule is now _should_evict(held, lease, viewer_id): a lease held by the agent clears the memory; only a takeover while this viewer still held evicts. (cherry picked from commit 178abd0f87f980e603717b1eca39e4344816f7aa)
noVNC's code-less socket.close() and some proxies both surface as 1005 on the server, so the bridge cannot tell a deliberate pane close from a drop and must keep the lease. The Desktop closes with an explicit 1000 on unmount; this row pins the server side of that contract. (cherry picked from commit a62a7429f2b925d93d2122b7fa7f235144b8de44)
RfbClientFilter consulted lease.viewer_may_send_input per client message, which stats and reads lease.json on the event loop for every pointer move. The bridge now keeps one boolean, refreshed from the in-process on_change listener (same-process takeovers apply to the very next message) and by a file re-read at most every 250 ms (a takeover written by another process is seen within one interval). Test: a foreign takeover stops input in < 0.5 s (fails with the interval set to 5 s). (cherry picked from commit dcafcb2448c9668c80f2c449a4cda5ad95f82bab)
thumbnail_data_url swaps the process-wide os.environ["XAUTHORITY"] around ImageGrab.grab; a multiplexed gateway thumbnails several profiles from worker threads at once, so one grab could run with another profile's cookie (Xlib auth failure or the wrong screen) and the restore left the wrong value behind. A module lock serialises swap+grab+restore. (cherry picked from commit be789c85fdbae869a31d63f30a3434f03d350a50)
…sh, not the id display.observe took viewer_id from the client and display.status handed every client the holder's viewer_id, while the lease authorised input and release on string equality: any authenticated client could read the holder's id, mint an observe ticket with it and co-drive or release their lease. observe now mints viewer_id = secrets.token_urlsafe(16) and returns it; a requested id is honoured only when this same connection minted it earlier (a reconnecting pane keeps its lease), tracked per transport in a weak dict. Every lease payload leaving the gateway (display.status/start/stop/observe snapshots and the display.lease broadcast) goes through _lease_view, which replaces viewer_id with null plus viewer_hash = sha256(id)[:12] so the Desktop can still tell whether it is the holder. (cherry picked from commit 52ff67ab409ad547c05e8a4244eb6c24513adf65)
Match the existing gateway stop contract on supported hosts, including when the desktop has already exited. Keep the Linux lease import off unsupported hosts. Native Linux test exercises the CLI parser and real persisted lease. Addresses the CLI recovery finding from MrD1az and subsequent reviewers on NousResearch#108914. (cherry picked from commit 0b361bf11b45ea9667369214b93919fad4c1b0f5)
…ary->package map Fedora split xorg-x11-server-utils and xorg-x11-utils into per-binary packages (F35) and retired the umbrellas; dnf5 refuses the whole transaction on one unknown name, so `hermes computer-use screen install` was a no-op on Fedora. The dnf list now names xsetroot/xset/xdpyinfo/ xprop/setxkbmap directly. xprop (the launcher's WM-ready wait) joins REQUIRED_BINARIES and every distro list; apt gains x11-xkb-utils (setxkbmap) and pacman gains dbus (dbus-run-session), both previously reached only via transitive deps. BINARY_PACKAGES documents which package ships each binary per manager so a test can hold the lists to it. (cherry picked from commit dc5c4d70f7a0fd355fe21c96071513d8b5ae3744)
…t a pid pid_exists alone trusts a recycled pid: after a reboot or a long-lived gateway, an unrelated process can own the recorded number and status() reports the screen running while stop() SIGTERMs that stranger's whole process group. The pid file now stores "<pid> <psutil create_time>" and _launcher_pid() requires both to match; the pre-identity single-number format, a dead pid and an out-of-range digit string all read as not running (the safe direction: worst case is a spurious start()). (cherry picked from commit 2f9e0e40a764f5a6bb38c02194081dc34c4c8f0b)
Send close code 1000 before noVNC can send its statusless close. Keep reconnect teardown statusless so it does not release the human lease. Cover both lifecycle paths with component tests; verify the ordering against real noVNC and Xvnc separately. Addresses the close-code finding discussed by MrD1az, Xipong, and other reviewers on NousResearch#108914. (cherry picked from commit 6112341f0c79c710387de66db8c309a39e02eaf0)
Use the connection/profile event predicate in the portal, never match a legacy group to a remote namesake, and reset thumbnail state on owner changes. Component tests cover cross-host events, click routing, pending/rejected thumbnails, and stale responses. Addresses findings from Julientalbot, erosika, and BearHuddleston on NousResearch#108914. (cherry picked from commit 65155c92a808ce5bb94d783bb0f42d6a4fc7d94f)
…the lease hash `display.observe` now returns the viewer id this attach is known by, and lease payloads name the holder by `viewer_hash` (sha256(viewer_id)[:12]) instead of the raw id. The pane stores the minted id per attach ($screenState.viewer), passes it to display.lease.acquire/release, and derives iHold (pane) and the 'human' portal tone from `leaseHeldBy` — hash compare with a raw-id fallback for backends that still broadcast viewer_id. Why: the client-generated VIEWER_ID constant let a reloaded window mint a new identity while the lease still named the old one, and a raw id on the wire was a usable credential for anyone listening on the profile's event stream. (cherry picked from commit ad985457d185a99f23807c282a9a2eae7b496ecc)
…lds control
`display.thumbnail` answers `{data_url: null, suppressed: 'human_has_control'}`
while a human holds the lease. The hero captions that as "Hidden while someone
has control" (screen.heroSuppressed, all four locales) and treats it as a
successful refresh, so the miss counter never ages a withheld frame into
"Last seen — screen unreachable".
(cherry picked from commit 7da75f086fe30f0c7e329bba70f32b2dca283334)
… attach The SDK disposes an INACTIVE registry-routed bot's secondary socket as soon as its request count returns to zero, so `display.install.log/done` and the pane's `display.lease` events had no socket to arrive on — the install card sat at "Installing…" forever and the pane never learned about a take-over. `retainBotScreen` feature-detects `host.retainProfile` (like group-turns). The install card acquires it before `display.install` and releases on done/failed/unmount; the pane holds one for the attach lifetime, released by detach (reconnect swaps it, unmount drops it). (cherry picked from commit 1eed9c82e27d24a63abeda835f3349a9db66ebfe)
… close code
noVNC 1.7's `disconnect` detail carries only `{clean}`; the pane was searching
a `reason` that never arrives, so the "Another viewer took control" overlay was
unreachable and a take-over looked like a clean stop. The pane now listens on
the WebSocket it hands RFB (installed before RFB's own `onclose`) and treats
close code 4000 as control-taken.
(cherry picked from commit c0ed9a4cdb0d616cc727af26fd804f7e15a9f330)
…hecking forever A bot whose Hermes predates the display RPCs answered `display.status` with JSON-RPC -32601; the portal swallowed it and stayed on "Checking the screen…" for good, and the pane retried on every mount. `isDisplayUnavailable` (same shape the session-control store recognises) marks the bot `unavailable` in $screenState: the sidebar portal and hero render nothing, the routines-page portal and the pane show "Update the bot's Hermes to use Screen" (screen.portalUnavailable / screen.unavailableTitle, all locales), and no further status probes are issued. (cherry picked from commit 9ff358422b60e735ce2846f1e30427e15ae767fd)
…k a newer lease Lease events and status replies race on the wire; the cache took whichever landed last, so a status reply describing the pre-take-over lease could flip the pane back to "Bot is in control" while a human held it. DisplayLease keeps the backend's monotonic `epoch`; both setScreenStatus and setScreenLease drop a lease whose epoch is below the stored one. Payloads without an epoch (older backends) are applied as before. (cherry picked from commit 37f8b91067f8af8f11127249f6f689547238d8af)
…er owns
After a Desktop reload the pane attaches under a fresh server-minted viewer id
while the old one still holds the lease, so the docs' "click Hand back" had no
button to click — only Take over. When holder is human and this window is not
the holder, the pane now also offers "Hand back (force)", which calls
display.lease.release with {force: true} (the server refuses a plain release
from a non-holder).
i18n cleanup: drop the unused screen.recheck key, move the hardcoded
"Update Hermes Desktop…" notice and the tab title suffix in screen-open.tsx
into the bundle (screen.openNeedsUpdate / screen.title). All four locales.
(cherry picked from commit 5cf98770f72da261a5471467170993833d15118b)
…outer
The canvas host borrows `data-terminal` so type-to-focus / bare-key shortcuts
leave the remote screen's keystrokes alone — but `closeActiveTab` reads the
same marker as "a local terminal tab is focused" and closes the user's
terminal on Ctrl/⌘+W. The keybind listener sits on `window` in the capture
phase (and on macOS the chord arrives via Electron's before-input-event IPC),
so the pane cannot intercept it locally; it now carries a distinct
`data-remote-screen` marker for the close-tab router to exclude
(`isFocusWithin('[data-terminal]') && !isFocusWithin('[data-remote-screen]')`
in app/chat/close-tab.ts — a follow-up outside this plugin's files).
(cherry picked from commit e97b5923864b03c75a0502dd3b04fc5ce2dac34c)
…re-renders ProfileGroupScreenPortal rebuilt its synthesized RosterRow on every render, and the portal's effects (status probe, display.lease subscription) key on that row — so every sidebar paint re-subscribed the lease listener and re-ran the probe guard. The row is now memoised on (connectionId, profile, roster). The other half of the remount lives outside this plugin: gateway-groups.tsx hands ContribRender an inline `() => render(route)` closure, and ContribRender mounts its argument AS a component type, so the whole contribution remounts on every group-header render. That needs a stable component per (item, route) in ProfileGroupHeaderSlot (useMemo/useCallback), not a plugin-side change. (cherry picked from commit c8a606f76d8fb69c21773de7024c698cfbe22732)
… lease views everywhere, backend rebind on display change, footgun annotations, grace in schema
…-group header rows keep one render identity
…e fence, claimed install slot)
Signed-off-by: SmokeDev <degensmoke@gmail.com>
gaoanze888
reviewed
Sep 13, 2026
gaoanze888
left a comment
gaoanze888
left a comment
There was a problem hiding this comment.
Verified exact head d13f3425bb75acc9f99a1e8c2e40e8a42b98ee3a. The tests use the repository's native OS markers, dispatch through the real registered computer_use handler and lease/import path, and select the harmless noop backend rather than controlling the real desktop. Assertions cover both successful result shape and exactly one expected backend call.
On native macOS the file reports 4 passed / 4 Windows-skipped; Ruff and diff checks are clean. The OS test-file scanner recognizes both markers. This is complementary to the existing direct-handler coverage and I found no blocker.
alt-glitch
added
type/test
teknium1
force-pushed
the
hermes/hermes-b802e898
branch
from
September 13, 2026 02:00
d65af42 to
bc36ddb
Compare
Collaborator
teknium1
added a commit
that referenced
this pull request
Sep 13, 2026
…contract; native dispatch test stubs the approver Main gained a single typed GatewayEventMap sourced from the tui_gateway emitters (test_gateway_event_contract), so the six display.* notifications this PR emits must be listed there or the contract test and the desktop typecheck both fail. The native Windows/macOS dispatch test (#109505) ran on a runner with no approver, where the approval gate blocks click/type before dispatch; it now stubs `_request_approval` exactly like the capture-fence tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds native Windows/macOS registry-dispatch coverage for Bot Screen's computer-use integration. This is a test-only follow-up to #108914, targeting
hermes/hermes-b802e898atd65af42305c4; it does not reimplement thefcntlfix already integrated inf89e71e051.The existing missing-
fcntlsubprocess test verifies lease operations and imports. These additional cases exercise capture, click, typing and window discovery through the real registeredcomputer_usehandler on the actual OS, using the existing harmless noop backend.Related Issue
Complements the fix for the Windows regression reported by @helix4u and corroborated by other reviewers. #109446 was also checked for overlapping coverage.
Type of Change
Changes Made
One new file:
tests/tools/test_computer_use_native_platform.py.Cases carry
windows_onlyandmacos_onlymarkers, so the existing OS workflow's file scanner and marker selector discover them. No production code, configuration or workflow changes.How to Test
scripts/run_tests.sh tests/tools/test_computer_use_native_platform.py tests/tools/test_bot_desktop_lease.py -k 'native_computer_use or lease_works_without_fcntl' -j 2 --file-retries 0On native Windows at
d65af42305c4: 5 passed, 0 failed, 4 macOS cases skipped. This includes all four new registry cases plus upstream's missing-fcntltest. Both OS file-scanner selections include the new file. Changed-file Ruff and diff checks pass.Earlier native Windows registry probes reproduced the original
fcntlcrash atd947fc83a461. The final coverage now passes against the integrated fix. macOS execution belongs to its CI lane; no live screen/driver or full-suite result is claimed.SmokeDev