Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ providers dimmed with the reason. Switch a bot's model mid-conversation.
### 🖥️ Every bot gets a computer

Open the Computer panel and the bot's cloud desktop spins up on its own — live screen preview while it
works, "Open desktop" to take over in your browser, or point the bot at *this Mac* instead.
works, "Open desktop" to take over in your browser, or point the bot at *this computer* instead.

<img src="docs/screenshots/computer-panel.png" alt="Computer panel with live screen preview" width="100%">

Expand Down Expand Up @@ -134,7 +134,7 @@ flowchart LR
REG[Driver registry] --> BUS[Event bus → SSE]
BROKER[Permission broker]
end
subgraph agents ["Agents on your Mac"]
subgraph agents ["Agents on your machine"]
CL[claude CLI]
CX[codex CLI]
end
Expand All @@ -152,12 +152,13 @@ flowchart LR
| Harness | `server/harness/` | Registry (configs → live instances) and the fan-in event bus every client folds. |
| API | `server/index.ts` | Bots, turns, approvals, model catalog, computer lifecycle, connectors, config — HTTP + SSE. |
| App | `src/` | The chat shell. Server-backed store, one reducer, zero client-side transports. |
| Desktop | `electron/` | macOS shell: dictation helper (SFSpeechRecognizer), local screen capture, CUA bridge. |
| Desktop | `electron/` | Desktop shell: macOS dictation helper (SFSpeechRecognizer), local screen capture, CUA bridge. |

## Quick start

**Easiest:** [download the latest .dmg](https://github.com/milind-soni/openmausbot-releases/releases/latest),
drag it to Applications, open it. The harness server is embedded — no setup.
**Easiest:** [download the latest release](https://github.com/milind-soni/openmausbot-releases/releases/latest)
— `.dmg` for macOS (drag to Applications) or `.exe` for Windows (NSIS installer). The harness server is
embedded — no setup.

**From source:**

Expand All @@ -170,10 +171,15 @@ pnpm dev # app → http://127.0.0.1:5199
pnpm dev:desktop # or the Electron shell
```

Requirements: **macOS**, **Node 24+**, **pnpm**, and at least one agent CLI — [`claude`](https://claude.com/claude-code)
Requirements: **macOS or Windows**, **Node 24+**, **pnpm**, and at least one agent CLI — [`claude`](https://claude.com/claude-code)
or [`codex`](https://github.com/openai/codex) — installed and logged in. They appear in the model picker
automatically.

**Platform notes:** computer use resolves the `cua-driver` binary automatically — the CuaDriver app
(macOS) or the trycua Cua app / `cua-driver.exe` on PATH (Windows) — and reuses a running daemon when one
is up. Native dictation is macOS-only (Swift/SFSpeechRecognizer); the composer mic button is hidden
elsewhere. Windows packaging: `pnpm package:win`.

Optional, pasted once in **App Settings** (gear in the sidebar footer):

| Key | Unlocks |
Expand All @@ -190,8 +196,9 @@ pnpm build # typecheck + production build
## Status

Early but real — the loop works end to end: message → agent → streamed reply → tools → approvals →
computer use. Rough edges to expect: routines (scheduled tasks) are a placeholder, sidebar sections aren't
built yet, and Windows/Linux shells haven't been attempted (the harness itself is portable Node).
computer use, on macOS and Windows. Rough edges to expect: routines (scheduled tasks) are a placeholder,
sidebar sections aren't built yet, native dictation is macOS-only, and Linux packaging hasn't been
attempted (the harness itself is portable Node).

Contributions welcome — the driver SPI in [`server/contracts.ts`](server/contracts.ts) is deliberately
small; adding a provider is one file in [`server/drivers/`](server/drivers/) plus a one-line registration.
Binary file added build/icon.ico
Binary file not shown.
35 changes: 15 additions & 20 deletions dist-server/drivers/claude.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,14 @@
// - Composio Connect (connected apps → tools) over streamable HTTP
// - the bot's cloud computer (box.ascii.dev) via server/computer-proxy.ts
// — screenshot/exec/open_url, the CUA-on-the-box bridge
import { spawn } from "node:child_process";
import { execFile } from "node:child_process";
import { existsSync, unlinkSync } from "node:fs";
import { createServer as createNetServer } from "node:net";
import { homedir } from "node:os";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";
import { DATA_DIR } from "../config.js";
import { newEventId, newId } from "../contracts.js";
import { cliExec, cliVersion, killProcessTree, spawnCliHidden } from "./cli.js";
import { appendNative } from "./native.js";
const DRIVER_KIND = "claudeAgent";
// model catalog ported from upstream packages/contracts/src/model.ts
Expand Down Expand Up @@ -56,6 +55,11 @@ function askSummary(ask) {
}
function permissionSocketPath(threadId) {
const tag = threadId.replace(/[^\w-]/g, "").slice(0, 8);
// Windows has no unix sockets; Node maps a listen() path to a named pipe,
// and drive-letter paths (with ':') are invalid pipe names (EACCES).
// Use an explicit \\.\pipe\ name — both sides get the same string via argv.
if (process.platform === "win32")
return `\\\\.\\pipe\\ogb-perm-${tag}`;
return join(DATA_DIR, `perm-${tag}.sock`);
}
function createPermissionBroker(opts) {
Expand Down Expand Up @@ -272,7 +276,7 @@ export const ClaudeDriver = {
delete env.ANTHROPIC_API_KEY;
delete env.CLAUDECODE;
delete env.CLAUDE_CODE_ENTRYPOINT;
const child = spawn(config.cli, args, {
const child = spawnCliHidden(config.cli, args, {
cwd: turn.cwd ?? homedir(),
env,
stdio: ["pipe", "pipe", "pipe"],
Expand Down Expand Up @@ -370,17 +374,7 @@ export const ClaudeDriver = {
settle(false, "exit_before_result");
}
});
const stop = () => {
try {
process.kill(-child.pid, "SIGTERM");
}
catch {
try {
child.kill("SIGTERM");
}
catch { }
}
};
const stop = () => killProcessTree(child.pid);
active.set(threadId, { stop, turnId, broker });
emit({ ...base(threadId, turnId), type: "turn.started" });
// prompt over stdin as a stream-json message — never argv (ARG_MAX)
Expand All @@ -391,9 +385,7 @@ export const ClaudeDriver = {
return { turnId };
};
const snapshot = async () => {
const version = await new Promise((resolve) => {
execFile(config.cli, ["--version"], { timeout: 8000 }, (err, stdout) => resolve(err ? null : stdout.trim()));
});
const version = await cliVersion(config.cli);
if (!version)
return { state: "unavailable", reason: `\`${config.cli}\` CLI not found` };
const authenticated = existsSync(join(homedir(), ".claude", ".credentials.json"));
Expand Down Expand Up @@ -430,9 +422,12 @@ export const ClaudeDriver = {
return () => listeners.delete(listener);
},
},
generateText: (prompt) => new Promise((resolve, reject) => {
execFile(config.cli, ["-p", prompt, "--model", "claude-haiku-4-5", "--output-format", "text"], { timeout: 60_000, env: { ...process.env } }, (err, stdout) => (err ? reject(err) : resolve(stdout.trim())));
}),
generateText: async (prompt) => {
const res = await cliExec(config.cli, ["-p", prompt, "--model", "claude-haiku-4-5", "--output-format", "text"], { timeout: 60_000, env: { ...process.env } });
if (!res.ok)
throw new Error(res.stderr || `\`${config.cli}\` failed`);
return res.stdout.trim();
},
dispose: async () => {
for (const { stop } of active.values())
stop();
Expand Down
231 changes: 231 additions & 0 deletions dist-server/drivers/cli.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
// Windows CLI helpers.
//
// CLIs installed via npm/yarn/pnpm on Windows ship as .cmd batch shims
// (e.g. `codex.cmd` in %APPDATA%\npm). child_process cannot execute .cmd
// files directly — spawn/execFile fail with ENOENT unless the command runs
// through cmd.exe, and going through cmd.exe re-opens argv to its quoting
// and %VAR% expansion rules. Instead we resolve the shim to the real JS
// entry and run it with process.execPath — no shell at all. Native
// installers (the claude installer → claude.exe) resolve to their .exe.
//
// All helpers are async (or spawn async children) — nothing here blocks
// the harness event loop.
import { execFile, spawn, spawnSync } from "node:child_process";
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, isAbsolute, join, resolve } from "node:path";
const IS_WIN = process.platform === "win32";
const SHIM_RE = /\.(cmd|bat)$/i;
// `where` results don't change often; cache per CLI so per-turn spawns
// don't pay a fresh `where` process each time.
const whereCache = new Map();
const WHERE_TTL = 60_000;
/** Resolve a CLI name to a path that can actually be spawned. */
export function resolveCli(cli) {
if (!IS_WIN)
return cli;
const hit = whereCache.get(cli);
if (hit && Date.now() - hit.at < WHERE_TTL)
return hit.path;
let resolved = cli;
try {
const out = spawnSync("where", [cli], { encoding: "utf8", timeout: 5000, windowsHide: true });
if (out.status === 0 && out.stdout) {
const candidates = out.stdout
.split(/\r?\n/)
.map((c) => c.trim())
.filter(Boolean);
const exe = candidates.find((c) => /\.exe$/i.test(c));
const shim = candidates.find((c) => SHIM_RE.test(c));
resolved = exe ?? shim ?? cli;
}
}
catch {
/* keep the raw name */
}
whereCache.set(cli, { path: resolved, at: Date.now() });
return resolved;
}
// npm/pnpm/yarn .cmd shims are thin wrappers that exec node on a JS entry
// (e.g. `"%dp0%\node_modules\@openai\codex\bin\codex.js" %*`). Extract that
// entry so we can spawn process.execPath directly and skip cmd.exe.
function shimScriptTarget(shim) {
try {
const text = readFileSync(shim, "utf8");
const m = text.match(/"([^"]+\.(?:[cm]?js))"/);
if (!m)
return null;
const raw = m[1].replace(/%(?:~)?dp0%/gi, dirname(shim) + "\\");
if (raw.includes("%"))
return null; // unknown var token — give up
return isAbsolute(raw) ? raw : resolve(dirname(shim), raw);
}
catch {
return null;
}
}
/**
* How to spawn a CLI on this platform:
* - POSIX: the raw name (resolved via PATH by the shell-less spawn).
* - Windows: the resolved .exe, or — for .cmd shims — node running the
* shim's real JS entry, which needs no cmd.exe at all.
* Falls back to the raw name when nothing better can be resolved.
*/
export function resolveCliCommand(cli) {
if (!IS_WIN)
return { command: cli, args: [] };
const resolved = resolveCli(cli);
if (SHIM_RE.test(resolved)) {
const script = shimScriptTarget(resolved);
if (script) {
return {
command: process.execPath,
args: [script],
// packaged: process.execPath is the Electron binary; run as plain node
env: { ELECTRON_RUN_AS_NODE: "1" },
};
}
}
return { command: resolved, args: [] };
}
/** execFile equivalent that also works for .cmd shims on Windows. */
export function cliExec(cli, args, opts = {}) {
const { command, args: prefix, env: runEnv } = resolveCliCommand(cli);
return new Promise((resolvePromise) => {
const execOpts = {
timeout: opts.timeout,
env: { ...opts.env, ...runEnv },
windowsHide: true,
};
const cb = (err, stdout, stderr) => resolvePromise({ ok: !err, stdout, stderr: stderr ?? "" });
// last-resort: a .cmd shim we couldn't unwrap runs through cmd.exe
if (IS_WIN && SHIM_RE.test(command)) {
execFile(command, [...prefix, ...args], { ...execOpts, shell: true }, cb);
}
else {
execFile(command, [...prefix, ...args], execOpts, cb);
}
});
}
/** `cli --version` probe; null when the CLI is missing or errors. */
export function cliVersion(cli, timeoutMs = 8000) {
return cliExec(cli, ["--version"], { timeout: timeoutMs }).then((r) => r.ok && r.stdout.trim() ? r.stdout.trim() : null);
}
/**
* Kill a spawned CLI and its whole process tree (child MCP servers, the
* codex app-server worker, etc.). POSIX uses the process group
* (children are spawned detached); Windows uses taskkill /T /F because
* process.kill(-pid) throws ESRCH and plain kill() leaves orphans.
*/
export function killProcessTree(pid) {
if (!pid)
return;
if (IS_WIN) {
try {
spawnSync("taskkill", ["/pid", String(pid), "/T", "/F"], {
windowsHide: true,
timeout: 5000,
});
}
catch {
/* already gone */
}
return;
}
try {
process.kill(-pid, "SIGTERM");
}
catch {
try {
process.kill(pid, "SIGTERM");
}
catch {
/* already gone */
}
}
}
// PowerShell wrapper that runs a CLI with a hidden console. windowsHide on
// the direct spawn would give the CLI NO console — then every console-app
// it spawns (cmd.exe for device-id probes, MCP servers like cua-driver.exe)
// would create its own VISIBLE console window. A hidden console instead is
// inherited by the whole subtree, so nothing ever flashes. Args travel in a
// JSON file, so there is no cmd/PowerShell quoting hazard.
const PS_HIDDEN_WRAPPER = `param([string]$Cli, [string]$ArgsFile)
$ErrorActionPreference = 'Stop'
$ArgList = @(Get-Content -Raw -LiteralPath $ArgsFile | ConvertFrom-Json)
& $Cli @ArgList
exit $LASTEXITCODE
`;
// PowerShell 5.1 (built into Windows) mangles native args that contain
// embedded quotes — fatal for --mcp-config JSON. PowerShell 7 (pwsh)
// passes argv correctly, so prefer it and fall back to a plain
// windowsHide spawn (direct child hidden; grandchildren may flash).
function resolvePwsh() {
const candidates = [
join(process.env.ProgramFiles ?? "C:\\Program Files", "PowerShell", "7", "pwsh.exe"),
...(process.env.ProgramW6432 ? [join(process.env.ProgramW6432, "PowerShell", "7", "pwsh.exe")] : []),
];
for (const c of candidates) {
if (existsSync(c))
return c;
}
return null;
}
/**
* Spawn a CLI so that no console window ever appears on Windows — including
* for anything the CLI itself spawns. POSIX: plain spawn (no-op).
* Callers pass stdio pipes (["pipe","pipe","pipe"]) and get a child with
* live stdout/stderr streams.
*/
export function spawnCliHidden(cli, args, opts) {
const { command, args: prefix, env: runEnv } = resolveCliCommand(cli);
if (!IS_WIN) {
return spawn(command, [...prefix, ...args], opts);
}
// `detached` is a POSIX-only need here (killProcessTree uses the process
// group). On Windows it maps to DETACHED_PROCESS — the child gets NO
// console, and pwsh then exits 0 immediately without running the script
// or writing a byte, which surfaces as "cli exited 0 before result".
// Windows reaps the tree with taskkill /T /F, so drop the flag.
const { detached: _detached, ...winOpts } = opts;
const pwsh = resolvePwsh();
if (!pwsh) {
// no pwsh: plain spawn with the direct child hidden (grandchildren may
// flash their own consoles — acceptable degradation)
return spawn(command, [...prefix, ...args], {
...winOpts,
env: { ...opts.env, ...runEnv },
windowsHide: true,
});
}
// NOTE: no windowsHide here — PowerShell must keep a (hidden) console so
// the CLI and its console descendants attach to it instead of flashing.
const dir = mkdtempSync(join(tmpdir(), "omb-spawn-"));
const argsFile = join(dir, "args.json");
const wrapper = join(dir, "wrap.ps1");
writeFileSync(argsFile, JSON.stringify([...prefix, ...args]));
writeFileSync(wrapper, PS_HIDDEN_WRAPPER);
const child = spawn(pwsh, [
"-NoProfile",
"-NonInteractive",
"-ExecutionPolicy",
"Bypass",
"-WindowStyle",
"Hidden",
"-File",
wrapper,
"-Cli",
command,
"-ArgsFile",
argsFile,
], { ...winOpts, env: { ...opts.env, ...runEnv } });
child.once("close", () => {
try {
rmSync(dir, { recursive: true, force: true });
}
catch {
/* best-effort cleanup */
}
});
return child;
}
Loading