Skip to content

feat(lume): auto-generate VNC credentials and deliver via SSH - #1207

Merged
f-trycua merged 9 commits into
mainfrom
lume-cua-vnc
Mar 23, 2026
Merged

feat(lume): auto-generate VNC credentials and deliver via SSH#1207
f-trycua merged 9 commits into
mainfrom
lume-cua-vnc

Conversation

@f-trycua

@f-trycua f-trycua commented Mar 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Auto-generated VNC credentials: Lume generates a random port + 4-word passphrase for each VM's VNC server (no more hardcoded --vnc-password lume)
  • SSH-based VNC config delivery: After VM boot, lume automatically writes ~/.vnc.env inside the VM via SSH, bypassing macOS TCC restrictions that block VirtioFS reads from LaunchAgent processes
  • VNC backend fixes: Fixed coordinate corruption bug (fresh connection per operation), fixed scroll direction, fixed connection leak
  • setup-cua.sh cleanup: Removed all hardcoded VNC password/port defaults; websockify and start_server.sh now read from ~/.vnc.env
  • CI fixes: isort ordering, black formatting, MockVM signature, doc sync

Details

macOS TCC blocks LaunchAgent processes from reading VirtioFS mounts (Operation not permitted), so the previous VirtioFS-only approach for delivering VNC config to guest VMs didn't work reliably. The new approach:

  1. Lume writes ~/.vnc.env via SSH after VM boot (background task, 6 retries × 5s, silent on failure)
  2. VirtioFS remains as a fallback for interactive sessions
  3. start_server.sh and websockify both read from ~/.vnc.env

Each VM gets a unique random VNC port + password, ensuring VMs can't access each other's VNC servers.

Test plan

  • E2E verified on cloud host: screenshot, click, screen_size, key press, type text, move cursor
  • Verify VNC config auto-delivery after lume run
  • Verify websockify reads from ~/.vnc.env correctly
  • Verify no hardcoded "lume" VNC password remains

Summary by CodeRabbit

  • New Features

    • VNC password can now be configured via command-line option
    • Enhanced VNC interactions including drag operations and keyboard-based scrolling
    • Improved macOS CUA setup with automated noVNC web UI installation and configuration
  • Documentation

    • Updated CLI reference and API documentation to version 0.3.0

…scovery

Rewrite the VNC backend to use Twisted's global reactor directly instead
of vncdotool's api.connect(), which deadlocks inside uvicorn/asyncio on
Python 3.13. Each VNC operation now creates a fresh connection via
reactor.callFromThread() with threading.Event synchronization.

Add VirtioFS-based VNC port discovery: lume creates a shared "lume-config"
directory, writes vnc.env (port + password) after VNC starts, and the
guest reads it on boot — eliminating hardcoded VNC port/password defaults.

Key changes:
- vnc.py: global reactor pattern, @defer.inlineCallbacks closures,
  manual drag (replaces mouseDrag/doPoll), client.screen fix
- VM.swift: VirtioFS lume-config share, VNC URL parsing fix
  (URLComponents with vnc:// → http:// replacement)
- cli.py: only override CUA_VNC_PORT from CLI when explicitly provided
- setup-cua.sh: renamed from setup-cua-computer.sh, added "already
  mounted" check for lume-config VirtioFS share

Tested E2E: 19/19 operations passing (screenshot, click, type, scroll,
drag, hotkey, cursor position, screen size).
- Fix isort ordering in vnc.py (twisted.internet imports)
- Add vncPassword parameter to MockVM.run() to match VM superclass
- Regenerate lume CLI and HTTP API reference docs
- vnc.py: close client.transport in finally block of _with_client() to
  prevent TCP connection leaks accumulating over multiple operations
- VM.swift: remove old vnc.env before starting VM so the guest can't
  read stale port/password from a previous run
macOS TCC blocks LaunchAgent processes from reading VirtioFS mounts
("Operation not permitted"), so the start_server.sh VNC config discovery
now uses a priority-based approach:

1. Local vnc.env (~/.cua-server/vnc.env) — written by host via
   `lume ssh` or cached from a previous VirtioFS read. Not subject
   to TCC restrictions.
2. VirtioFS lume-config mount (fallback) — tries direct read, then
   sudo -n, and caches locally on success.

Also removes the sudoers entry approach (unreliable in LaunchAgent
context) and fixes the log output to show actual VNC target values.
After VNC starts, lume now spawns a background task that waits for
SSH to become available and writes vnc.env directly to ~/.cua-server/
inside the VM. This works around macOS TCC blocking LaunchAgent
processes from reading VirtioFS mounts.

The SSH write is best-effort and silent — if SSH is disabled on the
VM, it gives up quietly after ~30s. The guest can still discover
VNC config via VirtioFS or use hardcoded defaults as fallback.
Use a generic location (~/.vnc.env) that doesn't depend on
cua-server being installed. Keeps lume's VNC config delivery
decoupled from cua-specific paths.
Lume now auto-generates a random VNC port and password for each VM
and delivers them to ~/.vnc.env via SSH. Remove all hardcoded "lume"
VNC password and 5901 port defaults from setup-cua.sh:

- Remove hardcoded HOST_VNC_PORT=5901 and VNC_PASSWORD=lume defaults
- Remove VNC port/password from LaunchAgent plist env vars
- Make websockify read VNC port from ~/.vnc.env via wrapper script
- Mark --host-vnc-port and --vnc-password flags as deprecated
- Update comments and final output to reflect auto-config approach
@vercel

vercel Bot commented Mar 23, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Mar 23, 2026 11:54pm

Request Review

@coderabbitai

coderabbitai Bot commented Mar 23, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 29a675ad-41c7-47c0-8ccb-04f038a9995c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR updates Lume documentation to version 0.3.0, replaces the legacy macOS CUA setup script with a comprehensive new installer adding VNC/noVNC support with websockify integration, and threads VNC password support through the entire stack from CLI commands down to the Python backend while refactoring the VNC handler to use per-operation connections instead of persistent ones.

Changes

Cohort / File(s) Summary
Documentation Updates
docs/content/docs/lume/reference/cli-reference.mdx, docs/content/docs/lume/reference/http-api.mdx
Updated VersionHeader component props and version switcher metadata to reflect Lume release version 0.3.0 (from 0.2.86), with current version set to 0.3 and backward reference to v0.2 documentation.
Bash Setup Scripts
libs/lume/scripts/setup-cua-computer.sh (deleted), libs/lume/scripts/setup-cua.sh (added)
Replaced legacy setup script with comprehensive new installer that bootstraps Homebrew, Python 3.13, system deps, CUA server, and adds VNC/noVNC support via websockify with VirtioFS lume-config mount for VNC credentials sharing, LaunchAgent auto-start, and system configuration (sleep/lock/login settings).
Swift CLI & VM Integration
libs/lume/src/Commands/Run.swift, libs/lume/src/LumeController.swift, libs/lume/src/VM/VM.swift
Added --vnc-password CLI option and threaded vncPassword: String? parameter through the command → controller → VM stack, with new static helper VM.writeVNCConfigViaSSH(...) for guest-side VNC credential distribution via SSH with retry logic.
VNC Service Layer
libs/lume/src/VNC/VNCService.swift
Updated protocol and implementation signatures to accept optional password parameter; if not provided, a passphrase is generated internally as before.
Swift Test Mocks
libs/lume/tests/Mocks/MockVM.swift, libs/lume/tests/Mocks/MockVNCService.swift
Updated mock signatures to accept new vncPassword/password parameters and forward them to parent implementations.
Python Backend
libs/python/computer-server/computer_server/cli.py
Conditionally set CUA_VNC_PORT environment variable only when CLI port differs from default (5900) or env var is unset, avoiding unconditional overwrites.
VNC Handler Refactoring
libs/python/computer-server/computer_server/handlers/vnc.py
Replaced persistent vncdotool client connection model with per-operation connections via _with_client(fn) factory; added drag_to(...) and drag_path(...) methods using stepwise mouse movements; changed scroll semantics from mouse wheel buttons to arrow key presses with inverted sign logic in handler.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant CLI as CLI (Run.swift)
    participant Ctrl as LumeController
    participant VM as VM.swift
    participant VNC as VNCService
    participant Host as Host System
    participant Guest as Guest VM
    participant SSH as SSH Client

    User->>CLI: lume run --vnc-port 5900 --vnc-password secret
    CLI->>Ctrl: runVM(..., vncPort: 5900, vncPassword: "secret")
    Ctrl->>VM: run(..., vncPort: 5900, vncPassword: "secret")
    
    VM->>Host: Create lume-config VirtioFS dir
    VM->>VNC: setupSession(port: 5900, password: "secret")
    VNC->>Host: start(port: 5900, password: "secret")
    Host->>Host: Use provided password or generate
    
    Host->>Host: Extract port/password from vnc:// URL
    Host->>Host: Write vnc.env locally
    Host->>Guest: writeVNCConfigViaSSH() with retry
    
    SSH->>Guest: Poll for IP, SSH availability
    SSH->>Guest: echo VNC_PORT/VNC_PASSWORD >> ~/.vnc.env
    Guest-->>Host: VNC config written (best-effort)
    
    Host-->>User: VNC session ready
Loading
sequenceDiagram
    participant Handler as VNCHandler
    participant Conn as _VNCConnection
    participant ThreadPool as asyncio.to_thread
    participant Factory as VNCDoToolFactory
    participant Reactor as Twisted reactor
    participant VNC as VNC Server

    Handler->>ThreadPool: drag_to(start, end)
    ThreadPool->>Conn: drag_to(start_x, start_y, end_x, end_y)
    
    Conn->>Factory: _with_client(drag operation)
    Factory->>Reactor: Create fresh connection per operation
    Reactor->>VNC: Connect
    VNC-->>Reactor: Client ready
    
    Conn->>VNC: mouseDown(button)
    Conn->>VNC: mouseMove(intermediate_x, intermediate_y) [stepwise]
    Conn->>VNC: mouseMove(end_x, end_y)
    Conn->>VNC: mouseUp(button)
    
    Factory->>Reactor: Close transport
    Reactor-->>Conn: Operation complete
    Conn-->>Handler: Return
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

release:pypi/computer-server, release:lume

Suggested reviewers

  • ddupont808

Poem

🐰 A new CUA setup hops into place,
VNC passwords flow through the stack with grace,
Per-operation connections spring to life so spry,
While websockify bridges the VNC to the sky,
Version 0.3 brings infrastructure delight! 🌱

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 29.31% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'feat(lume): auto-generate VNC credentials and deliver via SSH' directly and accurately reflects the main change described across all modified files: adding VNC password generation and delivery mechanisms, updating VNC services to accept passwords, threading this through the Swift command layer, implementing SSH-based credential delivery, and updating shell scripts and VNC handlers to support this new workflow.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch lume-cua-vnc

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Publishable packages changed

  • lume

Add release:<service> labels to auto-release on merge (+ optional bump:minor or bump:major, default is patch).
Or add no-release to skip.

@f-trycua f-trycua added the release:lume Release lume on merge label Mar 23, 2026
@f-trycua
f-trycua merged commit 8ec453a into main Mar 23, 2026
7 of 9 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Publishable packages changed

  • lume — will auto-release on merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:lume Release lume on merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant