fix(browser/vnc): make end-to-end browser integration work on Jetson - #66
Conversation
β¦ to UI On Jetson Orin, the openclaw CLI takes ~10.5s just to start up (measured via 'time openclaw config get'). The browser/manage route used a 10-second timeout on both 'config set' calls, so the first set routinely timed out, the second never ran, persistBrowserEnabled(true) never wrote the flag, and the UI stuck forever at 'Enabling...' because subsequent status polls kept returning enabled: false. Changes: - All 'openclaw config set' calls in enable/disable use a 30s timeout (3x headroom over observed CLI startup). - When config set throws, return 500 with the error instead of swallowing it and continuing to restart the gateway. The UI now shows a real error instead of infinite spinner.
On a Jetson, :0 is owned by GDM. Even when GDM's Xauthority is shared with the clawbox user, windows drawn on :0 by apps launched through the clawbox-browser.service are hidden by the GDM greeter β the user sees only the NVIDIA wallpaper and no Chromium. start-vnc.sh took the mirror-:0 path whenever :0 was reachable, which on Jetson is always. Changes: - start-vnc.sh honors $CLAWBOX_VNC_MODE. When set to 'virtual', skips the :0 mirror attempt and goes straight to the Xvfb :99 path. The existing auto behavior is preserved when the env var is unset. - clawbox-vnc.service now sets Environment=CLAWBOX_VNC_MODE=virtual. This matches where the browser service launches Chromium (DISPLAY=:99) so both the browser and the VNC viewer share one Xvfb surface.
Adds copy/paste between the host browser and the VNC guest. Server-side (start-vnc.sh + install.sh): - Install and run autocutsel on the VNC :99 display to keep X CLIPBOARD and PRIMARY selections alive between windows. Without it, selections vanish when the owning window loses focus, breaking paste across apps. Client-side (VNCApp.tsx): - Listen to noVNC's 'clipboard' event and mirror the remote's cut-text into the host browser's clipboard via navigator.clipboard.writeText (works on insecure origins). Guest β host copy 'just works'. - The 'paste' DOM event never fires on a canvas, so Ctrl+V inside the VNC surface wouldn't inject the host clipboard. Added a keydown handler for Ctrl/Cmd+V that calls navigator.clipboard.readText() and forwards via rfb.clipboardPasteFrom. Secure-context only (silently no-ops on plain HTTP LAN URLs like http://clawbox.local). - For the HTTP-LAN case added a visible 'Paste to VNC' toolbar button (top-right of the surface). Opens a modal with a textarea β the user pastes into the textarea (Ctrl+V works natively in textareas), clicks Send, and the text lands in the guest X CLIPBOARD. Round-trip confirmed on the Jetson where Chrome blocks clipboard.readText over HTTP. - Freeze the VNC-input focus/key handlers while the modal is open so the textarea keeps focus for the native paste. Translations (en + bg + de + es + fr + it + ja + nl + sv + zh): - vnc.pasteToRemote, vnc.pasteHelp, vnc.pastePlaceholder, vnc.sendPaste
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 44 minutes and 0 seconds. β How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. π¦ How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. βΉοΈ Review infoβοΈ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: π Files selected for processing (2)
π WalkthroughWalkthroughThe changes introduce a VNC paste-to-remote feature with bidirectional clipboard support, refactor the VNC service to support virtual desktop mode with clipboard persistence via autocutsel, increase operation timeouts for browser service config updates to 30 seconds, and add comprehensive localization for the new paste UI across eight languages. Changes
Sequence Diagram(s)sequenceDiagram
participant User as User (Host)
participant VNC as VNC App
participant ClipboardAPI as Browser Clipboard API
participant Remote as Remote Session (VNC Server)
User->>VNC: Click "Paste to Remote" button
VNC->>VNC: Open paste modal
User->>VNC: Paste clipboard into textarea (or Ctrl+V)
VNC->>ClipboardAPI: navigator.clipboard.readText()
ClipboardAPI-->>VNC: Return clipboard text
VNC->>VNC: Display text in modal textarea
User->>VNC: Click "Send" or Ctrl+Cmd+Enter
VNC->>Remote: clipboardPasteFrom(text)
Remote-->>VNC: Clipboard updated on remote
Note over Remote: Remote application can paste (Ctrl+V) from clipboard
Remote->>VNC: clipboard event (guest copied text)
VNC->>ClipboardAPI: navigator.clipboard.writeText(guestText)
ClipboardAPI-->>User: Host clipboard updated
Estimated code review effortπ― 3 (Moderate) | β±οΈ ~25 minutes Possibly related PRs
Poem
π₯ Pre-merge checks | β 2 | β 1β Failed checks (1 warning)
β Passed checks (2 passed)
βοΈ Tip: You can configure your own custom pre-merge checks in the settings. β¨ Finishing Touchesπ§ͺ Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
β Actions performedReview triggered.
|
|
@coderabbitai full review |
β Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 4
π€ Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@install.sh`:
- Around line 970-992: The VNC systemd unit and related bits are only installed
by step_vnc_install, so existing installs won't get the updated
clawbox-vnc.service (with Environment=CLAWBOX_VNC_MODE=virtual) or the
autocutsel package; to fix, move the clawbox-vnc.service template out of
step_vnc_install into the shared config directory (e.g., config/) and add
deployment of that unit from step_systemd_services (or alternatively add an
idempotent refresh routine called during rebuild/update that reinstalls
autocutsel, updates start-vnc.sh/ensure-vnc-on-first-boot.sh ownership/perm, and
runs systemctl daemon-reload + systemctl restart/enable clawbox-vnc.service);
update references to start-vnc.sh, ensure-vnc-on-first-boot.sh, and the service
unit to ensure CLAWBOX_VNC_MODE=virtual is present and the unit is consistently
applied across installs.
In `@src/app/setup-api/browser/manage/route.ts`:
- Line 327: The handler that runs exec("/usr/bin/sudo", ["/usr/bin/systemctl",
"stop", "clawbox-browser.service"], { timeout: 30000 }) currently swallows
errors with an empty catch; update that catch in the close-browser route to at
least log the failure (e.g., console.warn or the module's logger like
processLogger.warn) including the error object and a short context message so
failures of systemctl stop are visible for debugging without changing the
fallback behavior (leave pkill + lock cleanup intact).
In `@src/components/VNCApp.tsx`:
- Around line 417-459: The modal markup for pasteOpen lacks dialog semantics and
the textarea lacks an accessible label; update the container to include
role="dialog" and aria-modal="true" and wire aria-labelledby to the modal title
(give the h3 a stable id like paste-dialog-title) and aria-describedby to the
help paragraph (give it an id like paste-dialog-desc), then add aria-labelledby
(or aria-label) on the textareaRef-controlled element (pasteTextareaRef)
pointing to that title/id so screen readers announce the dialog and textarea
properly; keep existing keyboard handlers (sendPaste, closePasteModal) and
ensure the title ids match the strings used in the JSX.
- Around line 217-235: The onKeyDown handler (used with pasteOpenRef,
vncFocusedRef and rfbRef.current.clipboardPasteFrom) must prevent the original
Ctrl/Cmd+V from propagating before starting the async
navigator.clipboard.readText: call event.preventDefault() and
event.stopPropagation() immediately once isPasteShortcut is detected, then
perform readText() and call rfbRef.current.clipboardPasteFrom(text) and only
after that (or on read error) call sendKey to emit the remote paste keystroke;
also ensure releaseRemoteModifiers is included in the useEffect dependency array
that registers onKeyDown so the closure sees it.
πͺ Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
βΉοΈ Review info
βοΈ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: a225002e-fa2b-43c9-acb8-5416c1f4a293
π Files selected for processing (8)
install.shscripts/start-vnc.shsrc/app/setup-api/browser/manage/route.tssrc/components/VNCApp.tsxsrc/lib/desktop-translations-part1.tssrc/lib/desktop-translations-part2.tssrc/lib/desktop-translations-part3.tssrc/lib/desktop-translations.ts
| step_vnc_install() { | ||
| # Install x11vnc, Xvfb (virtual framebuffer fallback), websockify, and a lightweight WM | ||
| apt-get install -y x11vnc xvfb websockify dbus-x11 openbox xterm x11-xserver-utils | ||
| apt-get install -y x11vnc xvfb websockify dbus-x11 openbox xterm x11-xserver-utils autocutsel | ||
|
|
||
| chmod +x "$PROJECT_DIR/scripts/start-vnc.sh" | ||
| chown "$CLAWBOX_USER:$CLAWBOX_USER" "$PROJECT_DIR/scripts/start-vnc.sh" | ||
| chmod +x "$PROJECT_DIR/scripts/ensure-vnc-on-first-boot.sh" | ||
| chown root:root "$PROJECT_DIR/scripts/ensure-vnc-on-first-boot.sh" | ||
|
|
||
| # Systemd service for VNC | ||
| # Systemd service for VNC β force virtual display mode. On headless | ||
| # Jetsons, :0 is GDM's greeter; apps launched into it are covered by | ||
| # the greeter and invisible to VNC viewers. Xvfb :99 gives a clean | ||
| # dedicated surface that matches what the browser service targets. | ||
| cat > /etc/systemd/system/clawbox-vnc.service <<VNCSVC | ||
| [Unit] | ||
| Description=ClawBox VNC (mirrors display or virtual desktop) | ||
| After=display-manager.service network.target | ||
| Description=ClawBox VNC (virtual desktop) | ||
| After=network.target | ||
|
|
||
| [Service] | ||
| Type=simple | ||
| User=$CLAWBOX_USER | ||
| Environment=CLAWBOX_VNC_MODE=virtual | ||
| ExecStart=$PROJECT_DIR/scripts/start-vnc.sh |
There was a problem hiding this comment.
Ensure the VNC fix is applied on existing installs.
Line 972 and Lines 983-992 only run from step_vnc_install; the update paths shown in this file donβt call that step, so already-installed devices can keep the old clawbox-vnc.service without CLAWBOX_VNC_MODE=virtual and without autocutsel. Move the service into config/ and deploy it from step_systemd_services, or invoke an idempotent VNC service/package refresh during rebuild/update.
π€ Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@install.sh` around lines 970 - 992, The VNC systemd unit and related bits are
only installed by step_vnc_install, so existing installs won't get the updated
clawbox-vnc.service (with Environment=CLAWBOX_VNC_MODE=virtual) or the
autocutsel package; to fix, move the clawbox-vnc.service template out of
step_vnc_install into the shared config directory (e.g., config/) and add
deployment of that unit from step_systemd_services (or alternatively add an
idempotent refresh routine called during rebuild/update that reinstalls
autocutsel, updates start-vnc.sh/ensure-vnc-on-first-boot.sh ownership/perm, and
runs systemctl daemon-reload + systemctl restart/enable clawbox-vnc.service);
update references to start-vnc.sh, ensure-vnc-on-first-boot.sh, and the service
unit to ensure CLAWBOX_VNC_MODE=virtual is present and the unit is consistently
applied across installs.
| const onKeyDown = (event: KeyboardEvent) => { | ||
| if (pasteOpenRef.current) return; | ||
| if (!vncFocusedRef.current) return; | ||
| const isPasteShortcut = | ||
| (event.ctrlKey || event.metaKey) && | ||
| !event.shiftKey && | ||
| !event.altKey && | ||
| event.key.toLowerCase() === "v"; | ||
| if (!isPasteShortcut) return; | ||
| if (!navigator.clipboard?.readText) return; | ||
| navigator.clipboard | ||
| .readText() | ||
| .then((text) => { | ||
| if (text) rfbRef.current?.clipboardPasteFrom(text); | ||
| }) | ||
| .catch(() => { | ||
| // Non-secure origin or clipboard permission denied β silently skip. | ||
| }); | ||
| }; |
There was a problem hiding this comment.
π§© Analysis chain
π Script executed:
# First, verify the file exists and check its size
wc -l src/components/VNCApp.tsxRepository: ID-Robots/clawbox
Length of output: 90
π Script executed:
# Read the VNCApp.tsx file with line numbers, focusing on the area around lines 217-235
cat -n src/components/VNCApp.tsx | sed -n '200,250p'Repository: ID-Robots/clawbox
Length of output: 2871
π Script executed:
# Find the useCallback definition that wraps this onKeyDown handler
cat -n src/components/VNCApp.tsx | sed -n '180,220p'Repository: ID-Robots/clawbox
Length of output: 2221
π Script executed:
# Look further up to find the useCallback or useEffect that contains these handlers
cat -n src/components/VNCApp.tsx | sed -n '140,185p'Repository: ID-Robots/clawbox
Length of output: 1906
π Script executed:
# Find the end of the useEffect (the dependency array)
cat -n src/components/VNCApp.tsx | sed -n '235,280p'Repository: ID-Robots/clawbox
Length of output: 2715
π Script executed:
# Search for releaseRemoteModifiers function
rg -n "releaseRemoteModifiers" src/components/VNCApp.tsxRepository: ID-Robots/clawbox
Length of output: 293
π Script executed:
# Read the releaseRemoteModifiers definition and surrounding context
cat -n src/components/VNCApp.tsx | sed -n '80,110p'Repository: ID-Robots/clawbox
Length of output: 1236
π Script executed:
# Check for sendKey calls and understand the API
rg -n "sendKey" src/components/VNCApp.tsx -A 1 -B 1Repository: ID-Robots/clawbox
Length of output: 885
π Script executed:
# Search for other keydown listeners or handlers that might process the event
rg -n "keydown|keypress|keyup" src/components/VNCApp.tsx -iRepository: ID-Robots/clawbox
Length of output: 1259
π Script executed:
# Let's examine the handleKeyDown function that processes keydown events
cat -n src/components/VNCApp.tsx | sed -n '276,310p'Repository: ID-Robots/clawbox
Length of output: 1643
π Script executed:
# Check the comment about noVNC forwarding keys and understand the event flow
cat -n src/components/VNCApp.tsx | sed -n '205,215p'Repository: ID-Robots/clawbox
Length of output: 904
π Script executed:
# Verify the sendKey method signature with keysym values like 0xffe3
rg -n "0xffe3|0x0076|ControlLeft|KeyV" src/components/Repository: ID-Robots/clawbox
Length of output: 132
π Script executed:
# Verify the KeyV keysym value
rg -n "KeyV|0x0076" src/components/Repository: ID-Robots/clawbox
Length of output: 43
π Script executed:
# Search for key mappings and getTrackedVncKey function
rg -n "getTrackedVncKey|keysym" src/components/VNCApp.tsx -A 2 -B 2 | head -50Repository: ID-Robots/clawbox
Length of output: 1828
π Script executed:
# Check the vnc-keys file to understand keysym values for V
find src -name "*vnc-keys*" -o -name "*keys*" | grep -v node_modulesRepository: ID-Robots/clawbox
Length of output: 80
π Script executed:
# Read the vnc-keys file to understand key mappings
cat -n src/lib/vnc-keys.ts | head -80Repository: ID-Robots/clawbox
Length of output: 2400
Prevent the original paste shortcut from propagating until the host clipboard is pushed to the remote.
The onKeyDown handler starts an async clipboard.readText() without preventing event propagation. Meanwhile, the document's handleKeyDown listener (lines 284β305) checks if (e.defaultPrevented) return and will forward the Ctrl/Cmd+V keystroke to the remote immediately, before clipboardPasteFrom() completes. This causes the remote to paste its stale clipboard contents on the first paste, not the freshly-synced host clipboard.
Call event.preventDefault() and event.stopPropagation() before the async read, then manually send the keystroke via sendKey() only after the clipboard update succeeds or fails. This serializes the clipboard sync before the remote paste shortcut fires.
Also add releaseRemoteModifiers to the dependency array at line 274 since the fix calls it.
π Proposed fix to serialize clipboard update before the remote paste shortcut
const onKeyDown = (event: KeyboardEvent) => {
if (pasteOpenRef.current) return;
if (!vncFocusedRef.current) return;
const isPasteShortcut =
(event.ctrlKey || event.metaKey) &&
!event.shiftKey &&
!event.altKey &&
event.key.toLowerCase() === "v";
if (!isPasteShortcut) return;
if (!navigator.clipboard?.readText) return;
+
+ const sendRemotePasteShortcut = () => {
+ releaseRemoteModifiers();
+ rfbRef.current?.sendKey(0xffe3, "ControlLeft", true);
+ rfbRef.current?.sendKey(0x0076, "KeyV");
+ rfbRef.current?.sendKey(0xffe3, "ControlLeft", false);
+ };
+
+ event.preventDefault();
+ event.stopPropagation();
navigator.clipboard
.readText()
.then((text) => {
- if (text) rfbRef.current?.clipboardPasteFrom(text);
+ if (!text) return;
+ rfbRef.current?.clipboardPasteFrom(text);
+ sendRemotePasteShortcut();
})
.catch(() => {
- // Non-secure origin or clipboard permission denied β silently skip.
+ // Clipboard permission denied β preserve normal remote paste behavior.
+ sendRemotePasteShortcut();
});
};- }, [activateVncInput, deactivateVncInput, focusVncSurface, status]);
+ }, [activateVncInput, deactivateVncInput, focusVncSurface, releaseRemoteModifiers, status]);π€ Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/components/VNCApp.tsx` around lines 217 - 235, The onKeyDown handler
(used with pasteOpenRef, vncFocusedRef and rfbRef.current.clipboardPasteFrom)
must prevent the original Ctrl/Cmd+V from propagating before starting the async
navigator.clipboard.readText: call event.preventDefault() and
event.stopPropagation() immediately once isPasteShortcut is detected, then
perform readText() and call rfbRef.current.clipboardPasteFrom(text) and only
after that (or on read error) call sendKey to emit the remote paste keystroke;
also ensure releaseRemoteModifiers is included in the useEffect dependency array
that registers onKeyDown so the closure sees it.
β¦s, add a11y to paste modal CodeRabbit PR ID-Robots#66 findings that were accurate and worth acting on: 1. browser/manage/route.ts close-browser handler: previously swallowed both systemctl-stop and pkill errors with empty catches, so wedged systemctl or real pkill failures were invisible. Now logs systemctl-stop failures with console.warn, and logs pkill failures only when the error isn't the benign 'no process matched' exit code (which is the happy path once systemctl has already stopped the service). Fallback behavior β pkill + cleanBrowserLocks β is unchanged. 2. VNCApp paste modal accessibility: the dialog was missing ARIA semantics, so screen readers announced neither the dialog itself nor the textarea purpose. Added: - role="dialog", aria-modal="true" on the modal container. - aria-labelledby pointing at the new id="vnc-paste-dialog-title" on the h3, so the dialog name is the 'Paste to VNC' heading. - aria-describedby pointing at id="vnc-paste-dialog-desc" on the help paragraph, so the dialog description is the usage hint. - Matching aria-labelledby / aria-describedby on the textarea so it inherits the same label + description without a separate <label>. CodeRabbit also suggested preventDefault+stopPropagation on the Ctrl/Cmd+V keydown before the async clipboard read, but that would block noVNC from forwarding the V keystroke to the guest β which is the intended 'both events ship, X CLIPBOARD arrives first' design. The suggestion to add releaseRemoteModifiers to the useEffect deps was also a false positive: onKeyDown doesn't reference it, so adding it would be a no-op dep. Skipped both.
β¦D-Robots#66 followup) The clawbox-vnc.service unit template and autocutsel package only land on a fresh install via step_vnc_install. Existing devices that update through the in-app updater never re-run that step, so they stay on the pre-PR ID-Robots#66 configuration: - clawbox-vnc.service unit without Environment=CLAWBOX_VNC_MODE=virtual, so start-vnc.sh takes the mirror-:0 path and VNC viewers see GDM's greeter instead of Chromium. - No autocutsel daemon, so copy/paste between the VNC client and the guest doesn't survive focus changes β the bidirectional clipboard wiring in PR ID-Robots#66 has nothing to keep the selection alive. This was the outside-diff 'Major' finding on the CodeRabbit review of PR ID-Robots#66 that we deliberately deferred at merge time. Adds step_vnc_refresh β a narrow idempotent subset of step_vnc_install: - apt-get install -y -qq autocutsel (no-op when already installed) - rewrite /etc/systemd/system/clawbox-vnc.service with the current unit contents (including CLAWBOX_VNC_MODE=virtual) - systemctl daemon-reload + restart clawbox-vnc.service Called from step_post_update, which runs after reboot on every in-app update. By then git_pull has already placed the updated start-vnc.sh on disk, so the restart picks up both the new env var and the script that honors it. Whitelisted in DISPATCH_STEPS so 'install.sh --step vnc_refresh' is runnable on its own for hotfix scenarios. Deliberately narrower than step_vnc_install β does not re-touch the firstboot-pending flag (which would cause clawbox-firstboot-vnc to re-run on every reboot), the websockify unit, or the clawbox-browser service unit. Those are already on-disk from the original install and aren't part of this specific pre-ID-Robots#66 regression. Verified on Jetson via 'install.sh --step vnc_refresh' β syntax clean, unit rewritten with the env var present, service restart completes.
Summary
Three coordinated fixes so Steps 1β3 of the browser integration page all succeed on a fresh Jetson Orin install, plus bidirectional VNC clipboard so the user can actually type into the remote browser.
Commits
openclaw config settimeout from 10s β 30s (CLI startup is ~10.5s on Jetson, so 10s times out every time), and surface config-set errors as HTTP 500 instead of swallowing and continuing. Fixes Step 2 stuck forever on "Enablingβ¦".CLAWBOX_VNC_MODE=virtualenv override and wire it intoclawbox-vnc.service. On Jetson,:0is GDM's greeter β apps launched into it are covered and invisible to VNC. Forcing the Xvfb:99path gives a clean surface the browser service and VNC viewer share.clipboardevent β host clipboard write (guest β host works out of the box). Host β guest needs secure context fornavigator.clipboard.readText, which fails on LAN HTTP β added a "π Paste to VNC" toolbar button that opens a textarea modal where paste works natively; clicking Send callsrfb.clipboardPasteFrom. Server-side addsautocutselto keep X CLIPBOARD / PRIMARY alive across windows.Test plan
install.shpicks upautocutselSummary by CodeRabbit
New Features
Improvements
Documentation