Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 37 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,11 @@ jobs:
strategy:
fail-fast: false
matrix:
# the harness must stay portable Node — the windows leg is the
# guardrail for the in-progress Windows port (POSIX-only tests
# self-skip there)
# The harness and pure desktop capabilities stay portable across all
# supported host platforms. POSIX-only process tests self-skip.
os: [macos-latest, ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 15
env:
# tests never launch Electron; skip the binary download
ELECTRON_SKIP_BINARY_DOWNLOAD: "1"
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
Expand All @@ -34,6 +30,41 @@ jobs:
- run: pnpm install --frozen-lockfile
- run: pnpm typecheck
- run: pnpm test
- run: pnpm check:electron
- name: production UI build
if: matrix.os == 'ubuntu-latest'
run: pnpm exec vite build

package-linux:
name: package + smoke (Ubuntu 24.04 x64)
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
Comment thread
coderabbitai[bot] marked this conversation as resolved.
with:
persist-credentials: false
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 24
cache: pnpm
- name: Install package validation tools
run: sudo apt-get update && sudo apt-get install -y desktop-file-utils xvfb dbus-x11
- run: pnpm install --frozen-lockfile
- run: pnpm package:linux
- run: node scripts/verify-linux-package.mjs
- name: Configure Chromium sandbox for the unpacked app
run: |
sudo chown root:root release/linux-unpacked/chrome-sandbox
sudo chmod 4755 release/linux-unpacked/chrome-sandbox
test "$(stat -c '%U:%G %a' release/linux-unpacked/chrome-sandbox)" = "root:root 4755"
- name: Launch packaged app and verify lifecycle
run: dbus-run-session -- xvfb-run -a node scripts/smoke-linux-package.mjs
- uses: actions/upload-artifact@v4
if: always()
with:
name: openmausbot-ubuntu-x64
path: |
release/*.deb
release/*.AppImage
if-no-files-found: error
19 changes: 16 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,28 @@ makes a PR easy to merge. Read it once before opening anything; it's short on pu

Requirements: **Node 24+**, **pnpm**, and for actually chatting with a bot, at least one agent CLI
([`claude`](https://claude.com/claude-code) or [`codex`](https://github.com/openai/codex)) installed
and logged in. macOS is the primary platform; the harness server itself is portable Node and the
test suite runs on macOS, Linux, and Windows.
and logged in. macOS is the primary release platform and Ubuntu 24.04 x64 is the Linux desktop beta;
the harness server itself is portable Node and the test suite runs on macOS, Linux, and Windows.

```sh
git clone https://github.com/milind-soni/OpenMausBot && cd OpenMausBot
pnpm install

pnpm dev:server # harness server → 127.0.0.1:8799
pnpm dev # app → http://127.0.0.1:5199
pnpm dev:desktop # Electron shell (macOS)
pnpm dev:desktop # Electron shell (macOS/Ubuntu; keep server + Vite running)

pnpm typecheck # app + server
pnpm test # vitest suite (server unit + driver contract + API smoke)
pnpm test:watch # same, in watch mode
pnpm check:electron # syntax-check the plain JS Electron entrypoints

pnpm package:mac # DMG + ZIP; requires Swift/Xcode tools
pnpm package:linux # Ubuntu x64 .deb + AppImage; no Swift required
```

For Ubuntu installation and real desktop checks, see [`docs/linux-desktop.md`](docs/linux-desktop.md).

## Repo map

| Path | What lives there |
Expand Down Expand Up @@ -92,6 +98,11 @@ The SPI in [`server/contracts.ts`](server/contracts.ts) is deliberately small. A

- The harness (`server/`) must stay portable Node. Anything macOS-only (TCC, Swift helpers,
`~/Library` paths) belongs in `electron/` behind a `process.platform === "darwin"` gate.
- Renderer code must consume the desktop capability contract rather than infer support from Electron,
the user agent, or the presence of a preload bridge. Screen preview, dictation, and local control are
independent capabilities.
- Test Ubuntu platform claims on a real GNOME session. Xvfb proves packaging and lifecycle, not Wayland
portal behavior or local computer control.
- **Never build command strings for a shell.** No `shell: true`, no spawning through `cmd.exe` with
quoted strings — model names, personas, and MCP config JSON travel through argv, and cmd.exe
metacharacter expansion is a real injection class. On Windows, resolve `.cmd` shims to their JS
Expand All @@ -108,6 +119,8 @@ responses or events, no baking them into argv where another local process could
## Before you open the PR

- [ ] `pnpm typecheck` and `pnpm test` pass
- [ ] `pnpm check:electron` passes for desktop-shell changes
- [ ] Ubuntu packaging changes pass `pnpm package:linux` and `node scripts/verify-linux-package.mjs`
- [ ] New server behavior has a test; driver changes keep the contract tests green
- [ ] No `dist-server/` churn, no lockfile churn beyond your actual dependency change
- [ ] macOS-only code is platform-gated; nothing breaks the packaged app
Expand Down
50 changes: 38 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Talk to them like contacts. Watch them work. Approve what matters.

![TypeScript](https://img.shields.io/badge/TypeScript-strict-3178C6?logo=typescript&logoColor=white)
![React](https://img.shields.io/badge/React-19-61DAFB?logo=react&logoColor=black)
![Electron](https://img.shields.io/badge/Electron-macOS%20%C2%B7%20Windows-2B2E3A?logo=electron&logoColor=9FEAF9)
![Electron](https://img.shields.io/badge/Electron-macOS%20%C2%B7%20Windows%20%C2%B7%20Ubuntu-2B2E3A?logo=electron&logoColor=9FEAF9)
![Agents](https://img.shields.io/badge/agents-Claude%20·%20Codex-d97757)
![PRs](https://img.shields.io/badge/PRs-welcome-38d591)

Expand Down Expand Up @@ -50,8 +50,8 @@ already have:
— your existing logins and subscriptions, no new accounts, no proxy in the middle.
- **Local first.** One small harness server on `127.0.0.1` owns every agent process. Transcripts, keys, and
events live in `~/.openmausbot`, not a cloud.
- **Agents with hands.** Each bot can get a real computer — a cloud Linux desktop it drives while you watch
live, or your own Mac — plus 500+ apps through Composio Connect.
- **Agents with hands.** Each bot can get a cloud Linux desktop it drives while you watch live, plus 500+ apps
through Composio Connect. Local computer control is currently available on macOS and tracked for Linux.

## Features

Expand All @@ -72,7 +72,7 @@ providers dimmed with the reason. Switch a bot's model mid-conversation.
### 🖥️ Every bot gets a computer

Open the Computer panel and the bot's cloud desktop spins up on its own — live screen preview while it
works, "Open desktop" to take over in your browser, or point the bot at *this Mac* instead.
works, "Open desktop" to take over in your browser, or — on macOS — point the bot at *this computer* instead.

<img src="docs/screenshots/computer-panel.png" alt="Computer panel with live screen preview" width="100%">

Expand Down Expand Up @@ -143,7 +143,7 @@ flowchart LR
REG[Driver registry] --> BUS[Event bus → SSE]
BROKER[Permission broker]
end
subgraph agents ["Agents on your Mac"]
subgraph agents ["Agents on your computer"]
CL[claude CLI]
CX[codex CLI]
end
Expand All @@ -161,17 +161,21 @@ flowchart LR
| Harness | `server/harness/` | Registry (configs → live instances) and the fan-in event bus every client folds. |
| API | `server/index.ts` | Bots, turns, approvals, model catalog, computer lifecycle, connectors, config — HTTP + SSE. |
| App | `src/` | The chat shell. Server-backed store, one reducer, zero client-side transports. |
| Desktop | `electron/` | macOS + Windows shells: dictation helper (SFSpeechRecognizer, macOS only), local screen capture, CUA bridge (macOS only). |
| Desktop | `electron/` | macOS, Windows, and Ubuntu shells with an embedded harness and platform capabilities; Apple speech and current local CUA stay macOS-only. |

## Quick start

**Easiest:** grab the build for your machine — the harness server is embedded, so there's no setup either way.
**Released builds:** the harness server is embedded, so macOS and Windows need no separate server setup.

| | Download | Install |
|---|---|---|
| **macOS** (Apple silicon) | [OpenMausBot.dmg](https://github.com/milind-soni/openmausbot-releases/releases/latest/download/OpenMausBot.dmg) | Drag it to Applications, open it. Signed & notarized. |
| **Windows** (x64) | [OpenMausBot-setup.exe](https://github.com/milind-soni/openmausbot-releases/releases/latest/download/OpenMausBot-setup.exe) | Run it — one-click, per-user, no admin rights. The installer isn't code-signed yet, so SmartScreen shows "unknown publisher": **More info → Run anyway**. |

**Ubuntu Desktop beta:** build the `.deb` or AppImage from source using the commands below. Release downloads
will be linked here once Linux publishing is enabled. See [the Ubuntu Desktop guide](docs/linux-desktop.md) for
installation, capabilities, and troubleshooting.

**From source:**

```sh
Expand All @@ -180,13 +184,34 @@ pnpm install

pnpm dev:server # harness server → 127.0.0.1:8799
pnpm dev # app → http://127.0.0.1:5199
pnpm dev:desktop # or the Electron shell
pnpm dev:desktop # Electron shell; keep the two commands above running
```

Requirements: **macOS or Windows**, **Node 24+**, **pnpm**, and at least one agent CLI — [`claude`](https://claude.com/claude-code),
Requirements: **macOS, Windows, or Ubuntu 24.04 x64**, **Node 24+**, **pnpm**, and at least one agent CLI — [`claude`](https://claude.com/claude-code),
[`codex`](https://github.com/openai/codex), or [`grok`](https://x.ai/cli) — installed and logged in. They appear
in the model picker automatically.

Package the desktop application:

```sh
pnpm package:mac # macOS: DMG + ZIP; requires Swift/Xcode tools
pnpm package:win # Windows: installer + ZIP
pnpm package:linux # Ubuntu x64: .deb + AppImage; no Swift required
```

### Desktop capability status

| Capability | macOS | Ubuntu 24.04 Xorg | Ubuntu 24.04 Wayland |
|---|---|---|---|
| Packaged app, embedded harness, local agent CLIs | Supported | Beta | Beta |
| Composio and Box/cloud computers | Supported | Beta | Beta |
| Local screen preview and computer control | Supported | Planned | Planned after compositor validation |
| Native on-device dictation | Supported | Planned | Planned |

Unavailable native features fail closed on Ubuntu without blocking chat or cloud features. Linux local computer
control, Wayland capture/automation, dictation, and ARM64 are tracked in
[#29](https://github.com/milind-soni/OpenMausBot/issues/29) and are not claimed by the baseline package.

Optional, pasted once in **App Settings** (gear in the sidebar footer):

| Key | Unlocks |
Expand All @@ -197,16 +222,17 @@ Optional, pasted once in **App Settings** (gear in the sidebar footer):

```sh
pnpm typecheck # app + server
pnpm test # unit, driver, API, and desktop capability tests
pnpm build # typecheck + production build
pnpm package:win # Windows installer + zip → release/
pnpm check:electron # syntax-check Electron main/preload files
```

## Status

Early but real — the loop works end to end: message → agent → streamed reply → tools → approvals →
computer use. Rough edges to expect: routines (scheduled tasks) are a placeholder, sidebar sections aren't
built yet, and the Linux shell hasn't been attempted (macOS and Windows both run end to end; the harness
itself is portable Node).
computer use. macOS and Windows have released builds; Ubuntu 24.04 x64 packages are in beta with the
capability limits above. Rough edges to expect: routines are a placeholder and sidebar sections aren't built yet.

Contributions welcome — the driver SPI in [`server/contracts.ts`](server/contracts.ts) is deliberately
small; adding a provider is one file in [`server/drivers/`](server/drivers/) plus a one-line registration.
Expand Down
155 changes: 155 additions & 0 deletions docs/linux-desktop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Ubuntu Desktop

OpenMausBot has an Ubuntu 24.04 LTS x86_64 desktop beta. The Electron package embeds the harness server, so
installed builds do not require Node, pnpm, Swift, or a terminal at runtime.

## What works

- The native Electron window and embedded OpenMausBot server on GNOME Xorg and GNOME Wayland.
- Local Claude, Codex, Grok, Gemini, and other configured agent CLIs.
- Chat, streaming turns, approvals, bot-to-bot communication, and local data storage.
- Composio connected apps and Box cloud computers.
- External documentation and OAuth links in the default browser.

The first beta intentionally does **not** claim Linux dictation, local screen preview, or control of this
computer. Those controls are unavailable in the UI and fail closed in the Electron and server layers. Use a
Cloud box when a bot needs a computer. Xorg computer control, Wayland validation, bundled CUA, dictation, and
ARM64 are follow-ups in [issue #29](https://github.com/milind-soni/OpenMausBot/issues/29).

## Build packages

Requirements for building from source:

- Ubuntu 24.04 LTS x86_64
- Node.js 24 or newer
- pnpm 10.33.0 (Corepack can install the version declared by the project)

```sh
git clone https://github.com/milind-soni/OpenMausBot.git
cd OpenMausBot
corepack enable
pnpm install --frozen-lockfile
pnpm package:linux
```

The build creates:

- `release/OpenMausBot-<version>-amd64.deb`
- `release/OpenMausBot-<version>-x86_64.AppImage`

The AppImage uses a static runtime and does not require the legacy `libfuse2` package.

## Install and run

Install the Debian package with APT so its desktop dependencies are resolved:

```sh
sudo apt install ./release/OpenMausBot-*-amd64.deb
```

Then open **OpenMausBot** from the GNOME application launcher. To remove it:

```sh
sudo apt remove openmausbot
```

The portable AppImage does not install system files:

```sh
chmod +x release/OpenMausBot-*-x86_64.AppImage
./release/OpenMausBot-*-x86_64.AppImage
```

Application data remains local in `~/.openmausbot`. Electron browser data and window state use the normal XDG
configuration directory (`~/.config/openmausbot` unless the environment overrides it).

## Develop the desktop shell

Development mode uses three processes. Keep each command running in its own terminal:

```sh
pnpm dev:server
pnpm dev
pnpm dev:desktop
```

For a package-shaped build without creating `.deb` or AppImage artifacts:

```sh
pnpm package:linux:dir
./release/linux-unpacked/openmausbot
```

## Agent CLI discovery

Applications launched from GNOME do not inherit the same interactive shell `PATH` as a terminal. OpenMausBot
keeps the inherited path and adds existing common locations such as:

- `~/.local/bin`
- `~/.claude/local`
- `~/.volta/bin`
- `~/.bun/bin`
- `~/.asdf/shims`
- `~/.deno/bin`
- `~/.nvm/versions/node/*/bin`
- `/usr/local/bin`

It also probes the login shell in the background. If a CLI still is not detected, set an explicit additional
path before launching the app from a terminal and verify it there:

```sh
OMB_EXTRA_PATH=/your/custom/bin ./release/OpenMausBot-*-x86_64.AppImage
```

Restart OpenMausBot after installing or signing in to a CLI.

## Xorg and Wayland

The baseline shell, chat, cloud computers, and connected apps work in both GNOME session types. OpenMausBot
detects Wayland before XWayland when both `WAYLAND_DISPLAY` and `DISPLAY` exist, so future capture features do
not accidentally bypass portal-mediated behavior.

Local computer control remains disabled on both session types in this beta. Future Xorg support will require a
validated `cua-driver`; Wayland support will remain disabled until the exact GNOME/Mutter action surface has
real capture, input, scaling, permission, and lifecycle evidence.

## Validate a package change

```sh
pnpm typecheck
pnpm test
pnpm check:electron
pnpm package:linux
node scripts/verify-linux-package.mjs
dbus-run-session -- xvfb-run -a node scripts/smoke-linux-package.mjs
```

The verifier checks `.deb` metadata, desktop identity, resources, artifact permissions, and the absence of
unsupported native binaries. The smoke test launches the unpacked production app without `--no-sandbox`,
validates the renderer/preload capabilities and embedded health endpoint, then proves clean shutdown. It is not
a substitute for manual testing on a real GNOME Xorg and Wayland desktop.

## Troubleshooting

### An agent CLI is missing

Run the CLI directly in a terminal, finish its sign-in flow, then restart OpenMausBot. If it lives outside the
common directories above, use `OMB_EXTRA_PATH` while testing and report the install location so it can be
considered for automatic discovery.

### A bot needs computer tools

Choose **Cloud box** in the Computer panel and add a Box token in App Settings. **This computer** is disabled on
Linux until local CUA control is implemented and validated.

### The AppImage does not start

Confirm the executable bit and architecture:

```sh
chmod +x OpenMausBot-*-x86_64.AppImage
file OpenMausBot-*-x86_64.AppImage
```

Run it from a terminal once to collect the startup output. Do not install `libfuse2` just for this AppImage; the
package is built with the static runtime.
Binary file added docs/screenshots/ubuntu-computer-panel.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading