diff --git a/.agents/skills/commit-all/SKILL.md b/.agents/skills/commit-all/SKILL.md new file mode 100644 index 000000000..9f59dab0c --- /dev/null +++ b/.agents/skills/commit-all/SKILL.md @@ -0,0 +1,36 @@ +--- +name: commit-all +description: Use this skill when the user asks to "commit all", "commit everything", or wants all outstanding changes committed. Groups unrelated changes into separate, well-described commits instead of one catch-all commit. +--- + +# Commit All + +## Goal + +Commit every outstanding change in the working tree — but group unrelated changes into separate, informative commits so the git history stays useful. + +## Workflow + +1. **Survey all changes.** Run `git status` and `git diff` (staged + unstaged) to see the full picture. Include untracked files. +2. **Identify logical groups.** Cluster files by the change they belong to. A "group" is a set of files that were modified for the same reason (e.g. a bug fix, a new feature, a config tweak, a dependency update). Use file paths, diff content, and your understanding of the codebase to decide. +3. **Order commits.** Infra/config/dependency changes first, then library/core changes, then feature/UI changes, then docs/polish. +4. **For each group, create one commit:** + - Stage only the files belonging to that group (`git add ...`). Never use `git add -A` or `git add .`. + - Write a concise, informative commit message that describes *what* changed and *why*. Follow the repo's existing commit style (check `git log --oneline -10`). + - Do not lump unrelated changes together just because they're small. +5. **Verify.** After all commits, run `git status` to confirm the tree is clean. Run `git log --oneline -n ` (where N = number of commits created) to show the user what was committed. + +## Commit Message Rules + +- Keep the subject line under 72 characters. +- Use imperative mood ("add", "fix", "update", not "added", "fixes"). +- If a change is trivial (whitespace, typo, formatting), it's fine to batch those into one commit labeled accordingly. +- End every commit message with the Co-Authored-By trailer. + +## Hard Rules + +- Never combine unrelated changes in one commit. +- Never skip or discard changes — everything gets committed. +- Never use `git add -A` or `git add .`. +- Do not push. Only commit locally. +- Do not commit files that look like they contain secrets (`.env`, credentials, tokens). Warn the user about those instead. diff --git a/.agents/skills/writing-guide/SKILL.md b/.agents/skills/writing-guide/SKILL.md new file mode 100644 index 000000000..422ca50aa --- /dev/null +++ b/.agents/skills/writing-guide/SKILL.md @@ -0,0 +1,109 @@ +--- +name: writing-guide +description: Use when writing or editing any Spacebot copy — README sections, docs, release notes, marketing text, design doc summaries. Covers voice, tone, patterns to avoid, and what good Spacebot writing sounds like. +--- + +# Writing Guide + +Spacebot copy should sound like a confident engineer wrote it, not a language model. The test: would a developer reading the README think "someone knows what they're talking about" or "this was AI-generated"? These rules exist because the second outcome is common and costs credibility. + +## Voice + +Direct. Technical. No hedging. Short sentences with real content. Lead with the fact, not the framing. State what something is — not what it isn't. + +The tagline sets the tone: "The agent harness that runs teams, communities, and companies." That's a claim. It doesn't explain itself. Good Spacebot copy makes claims and lets the detail below earn them. + +## Patterns to Avoid + +These are the specific patterns that make copy sound AI-generated. Avoid all of them. + +**Em dashes in prose sentences.** Em dashes are fine in bullet point labels ("**Shell** — run arbitrary commands") but not inside sentences. Replace with a comma, a period, or restructure. + +Bad: "The cortex sees across all channels — the only process with full system scope." +Good: "The cortex is the only process that sees across all channels." + +**"Not X. Not Y." openers.** Starting a description by saying what something isn't. + +Bad: "Not markdown files. Not unstructured vectors. Spacebot's memory is..." +Good: "Spacebot's memory is a typed, graph-connected knowledge system." + +**"This isn't X, it's Y."** Classic AI construction. Just say the thing. + +Bad: "This isn't a generic claim — it's four specific mechanisms." +Good: "Spacebot builds on itself through four specific mechanisms." + +**"The result is..." and "The through-line:"** Setup language that delays the actual point. + +Bad: "The result is an agent that works out of the box." +Good: "It works out of the box." + +**"No X. No Y." closers.** Ending a paragraph with a string of negatives. + +Bad: "No heartbeat.json. No drift." +Good: Just cut it, or fold it into the sentence that precedes it. + +**"This is the most important X."** Let the reader decide what's important. + +Bad: "This is the most important structural difference between Spacebot and every other agent harness." +Good: Just state the difference. + +**Semicolons in prose.** Use a period. + +Bad: "The agent proposes; you decide." +Good: "The agent proposes. You decide." + +**Parallel triplets.** Three consecutive "same X, same Y, same Z" constructions sound mechanical. + +Bad: "Same context, same memories, same understanding." +Good: "They have the full conversation history." + +**"Not only X, but also Y."** Pick one. + +**Generic improvement claims.** Never say "self-improving" or "gets smarter." Name the actual mechanism. + +Bad: "Spacebot learns from experience." +Good: "After a conversation goes idle, a background branch reviews the history and saves skills and memories worth keeping." + +## What Spacebot Is Opinionated About + +When describing Spacebot's opinions, name them specifically: the process model, memory schema, and task lifecycle. Don't say its opinions are about "one thing" — they cover multiple things. The unifying idea is that state belongs in structured storage, not markdown files the LLM manages. + +## What Is and Isn't a Differentiator + +**Is a differentiator:** +- The task system as the structural foundation of autonomy +- True process-level concurrency across users +- Typed memory graph in SQLite with graph edges and hybrid search +- Autonomy channel with full context on wake, state tracked through tasks not files +- Spacedrive integration for cross-device execution and safe data access + +**Is not a differentiator:** +- Skills (every harness has skills, the format is not special) +- "Self-improving" as a generic claim +- Internal implementation details the user doesn't see + +Don't advertise internal improvements as external features. If Spacebot previously sent cold context to workers and now sends full context, that's an internal fix. The user-visible claim is "the autonomy channel wakes with full context" — not "unlike the old approach." + +## The Spacedrive Story + +Two layers: + +**What exists today:** multi-device access via P2P, remote execution via Spacedrive's permission system, file system intelligence via context nodes, safe data access via Prompt Guard 2 screening. + +**Where it's going:** team + personal library switching, org graph delegation, full company deployment model. + +Be honest about which is which. Don't present the vision as current capability. + +## Prose Structure + +Opening paragraphs should be 3-4 sentences. Lead with the strongest claim. No setup sentences ("In this section, we'll cover..."). No closing sentences that summarize what was just said. + +Bullet points are for lists of discrete items. Prose is for explanation, argument, and narrative. Don't bullet-point things that belong in prose. + +Section headers are short nouns or short sentences. No gerunds ("Building the Memory System"). No questions ("What Is the Task System?"). + +## Words to Avoid + +comprehensive, utilize, harness (as a verb), unlock, revolutionary, groundbreaking, remarkable, pivotal, powerful, exciting, cutting-edge, seamless, robust, leverage, paradigm, ecosystem (when not literally true), journey, dive deep, explore, embark. + +Also avoid: "at its core," "under the hood," "out of the box" (unless used once, intentionally), "world-class," "best-in-class." diff --git a/.gitignore b/.gitignore index 78f85fdef..c50cebe07 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,9 @@ .DS_Store .fastembed_cache .build/ +*.db +*.db-shm +*.db-wal # Interface interface/node_modules/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..2fe95bf87 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,187 @@ +# Contributing + +Contributions welcome. Read [RUST_STYLE_GUIDE.md](RUST_STYLE_GUIDE.md) before writing any code, and [AGENTS.md](AGENTS.md) for the full implementation guide. + +--- + +## Prerequisites + +- **Rust** 1.85+ with `rustfmt` and `clippy` +- **protoc** (protobuf compiler) +- **bun** (for frontend/interface work) +- **just** (`brew install just` or `cargo install just --locked`) + +Optional: [Nix flakes](https://nixos.org/) for isolated dev environments (`nix develop` gives you everything). + +--- + +## Getting Started + +1. Fork the repo and create a feature branch +2. Run `./scripts/install-git-hooks.sh` (installs a pre-commit hook that runs `cargo fmt`) +3. `cargo build` to verify the backend compiles +4. For frontend work: `cd interface && bun install` +5. Make your changes +6. Run `just preflight && just gate-pr` +7. Submit a PR + +--- + +## PR Gate + +Every PR must pass `just gate-pr` before merge. This mirrors CI and checks: + +1. **Migration safety** — new migrations only, never edit existing ones +2. **Formatting** — `cargo fmt --all -- --check` +3. **Compile** — `cargo check --all-targets` +4. **Lints** — `cargo clippy --all-targets -Dwarnings` +5. **Tests** — `cargo test --lib` +6. **Integration compile** — `cargo test --tests --no-run` + +Use `just gate-pr --fast` to skip clippy and integration compile during iteration. + +The frontend CI (`interface-ci.yml`) runs `bun ci` and `bunx tsc --noEmit` on interface changes. + +--- + +## Project Structure + +Single binary crate (no workspace). Key directories: + +``` +src/ +├── main.rs — CLI entry, config, startup +├── lib.rs — re-exports +├── config.rs — config loading/validation +├── error.rs — top-level Error enum +├── llm/ — LlmManager, model routing, providers +├── agent/ — Channel, Branch, Worker, Compactor, Cortex +├── hooks/ — SpacebotHook, CortexHook +├── tools/ — reply, branch, spawn_worker, memory_*, etc. +├── memory/ — MemoryStore, hybrid search, graph ops +├── messaging/ — Discord, Telegram, Slack, webhook adapters +├── conversation/ — history persistence, context assembly +├── cron/ — scheduler, CRUD +├── identity/ — SOUL.md, IDENTITY.md, USER.md loading +├── secrets/ — encrypted credentials (AES-256-GCM) +├── settings/ — key-value settings +└── db/ — SQLite migrations, connection setup + +interface/ — Dashboard UI (Vite + React + TypeScript) +prompts/ — LLM prompts as markdown (not Rust strings) +docs/ — Documentation site (MDX) +desktop/ — Tauri desktop app +scripts/ — Dev tooling (hooks, gates, builds) +``` + +Module roots use `src/module.rs`, **not** `src/module/mod.rs`. + +--- + +## Rust Conventions + +The full guide is in [RUST_STYLE_GUIDE.md](RUST_STYLE_GUIDE.md). Key points: + +**Imports** — three tiers separated by blank lines: (1) crate-local, (2) external crates, (3) std. + +**Error handling** — domain errors per module, wrapped by top-level `Error` enum via `#[from]`. Use `?` and `.context()`. Never silently discard with `let _ =`. + +**Async** — native RPITIT for async traits (not `#[async_trait]`). `tokio::spawn` for concurrent work. Clone before moving into async blocks. + +**Logging** — `tracing` crate, never `println!`. Structured key-value fields. `#[tracing::instrument]` for spans. + +**Lints** (enforced in Cargo.toml): `dbg_macro = "forbid"`, `todo = "forbid"`, `unimplemented = "forbid"`. + +**Testing** — `#[cfg(test)]` at end of file. `#[tokio::test]` for async. `.unwrap()` is fine in tests only. + +--- + +## Frontend (Interface) + +Use **bun** exclusively — never npm, pnpm, or yarn. + +```bash +cd interface +bun install # install deps +bun run dev # dev server +bun run build # production build +``` + +### SpaceUI Packages + +The dashboard uses `@spacedrive/*` packages published to npm from the [spaceui](https://github.com/spacedriveapp/spaceui) monorepo: + +- `@spacedrive/primitives` — base UI components +- `@spacedrive/ai` — AI chat components +- `@spacedrive/forms` — form components +- `@spacedrive/explorer` — file explorer components +- `@spacedrive/tokens` — design tokens + +`package.json` points to npm versions (e.g. `"^0.2.0"`). CI pulls from the registry. For local development, `bun link` overrides them with your local copies. + +**Local SpaceUI development:** + +Clone the spaceui repo adjacent to this one, then run the link command: + +```bash +git clone https://github.com/spacedriveapp/spaceui ../spaceui +just spaceui-link +``` + +This builds SpaceUI, registers all packages as global links, and connects them to `interface/`. Use `bun run watch` in the SpaceUI repo for automatic rebuilds. + +To unlink and restore npm versions: `just spaceui-unlink`. + +--- + +## Useful Commands + +```bash +just preflight # validate git/remote state +just gate-pr # full PR gate (mirrors CI) +just gate-pr --fast # skip clippy + integration compile +just typegen # generate TypeScript API types +just check-typegen # verify types match +just build-opencode-embed # build OpenCode embed bundle +just bundle-sidecar # build Tauri sidecar +just desktop-dev # run desktop app in dev mode +just update-frontend-hash # update Nix hash after frontend dep changes +``` + +--- + +## Migrations + +SQLite migrations are **immutable**. Never edit an existing migration file. Always create a new timestamped migration for schema changes. + +--- + +## Architecture + +See [ARCHITECTURE.md](ARCHITECTURE.md) for the full design. The short version: five process types, each with one job. + +- **Channels** — user-facing LLM, stays responsive, never blocks on work +- **Branches** — fork channel context to think, return conclusion, get deleted +- **Workers** — independent task execution with focused tools, no conversation context +- **Compactor** — programmatic context monitor, triggers compaction before channels fill up +- **Cortex** — system observer, generates memory bulletins, supervises processes + +Key rule: **never block the channel**. Branch to think, spawn workers to act. + +--- + +## Release Process + +Releases are triggered by git tags (`v*`). The CI workflow: + +1. Verifies `Cargo.toml` version matches the tag +2. Builds multi-platform binaries (x86_64/aarch64, Linux/macOS) +3. Builds Docker images (amd64/arm64) +4. Creates a GitHub release with binaries +5. Updates the Homebrew tap + +--- + +## License + +FSL-1.1-ALv2 ([Functional Source License](https://fsl.software/)), converting to Apache 2.0 after two years. diff --git a/README.md b/README.md index bc79c3346..e921e9f54 100644 --- a/README.md +++ b/README.md @@ -5,9 +5,7 @@

Spacebot

- An AI agent for teams, communities, and multi-user environments.
- Thinks, executes, and responds — concurrently, not sequentially.
- Never blocks. Never forgets. + The agent harness that runs teams, communities, and companies.

@@ -21,15 +19,17 @@ - [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/spacedriveapp/spacebot) + + +

spacebot.shHow It Works • - Architecture • + Goals & TasksQuick Start • - Tech Stack • + SpacedriveDocs

@@ -41,194 +41,105 @@ --- -## The Problem +Spacebot is opinionated agent infrastructure, built for teams and usable by anyone. **State belongs in structured storage, not markdown files the LLM manages.** Memory lives in a typed graph in SQLite. Autonomy runs on a task state machine linked to goals, not a heartbeat.json. The LLM reasons. The system holds state. -Most AI agent frameworks run everything in a single session. One LLM thread handles conversation, thinking, tool execution, memory retrieval, and context compaction — all in one loop. When it's doing work, it can't talk to you. When it's compacting, it goes dark. When it retrieves memories, raw results pollute the context with noise. +**It gets smarter the more you use it.** After complex tasks, the agent captures what it learned as reusable skills. After conversations go idle, a background process silently saves skills and memories worth keeping. Every session builds on the last — without any user action. -[OpenClaw](https://github.com/anomalyco/openclaw) _does_ have subagents, but handles them poorly and there's no enforcement to their use. The session is the bottleneck for everything. - -Spacebot splits the monolith into specialized processes that only do one thing, and delegate everything else. +It works out of the box and scales from one person to a whole community. --- -## Built for Teams and Communities - -Most AI agents are built for one person in one conversation. Spacebot is built for many people working together — a Discord community with hundreds of active members, a Slack workspace with teams running parallel workstreams, a Telegram group coordinating across time zones. - -This is why the architecture exists. A single-threaded agent breaks the moment two people talk at once. Spacebot's delegation model means it can think about User A's question, execute a task for User B, and respond to User C's small talk — all at the same time, without any of them waiting on each other. - -**For communities** — drop Spacebot into a Discord server. It handles concurrent conversations across channels and threads, remembers context about every member, and does real work (code, research, file operations) without going dark. Fifty people can interact with it simultaneously. - -**For fast-moving channels** — when messages are flying in, Spacebot doesn't try to respond to every single one. A message coalescing system detects rapid-fire bursts, batches them into a single turn, and lets the LLM read the room — it picks the most interesting thing to engage with, or stays quiet if there's nothing to add. Configurable debounce timing, automatic DM bypass, and the LLM always knows which messages arrived together. - -**For teams** — connect it to Slack. Each channel gets a dedicated conversation with shared memory. Spacebot can run long coding sessions for one engineer while answering quick questions from another. Workers handle the heavy lifting in the background while the channel stays responsive. - -**For multi-agent setups** — run multiple agents on one instance. A community bot with a friendly personality on Discord, a no-nonsense dev assistant on Slack, and a research agent handling background tasks. Each with its own identity, memory, and security permissions. One binary, one deploy. +## The Problem -### Deploy Your Way +Most AI agent frameworks run everything in a single session. One LLM thread handles conversation, thinking, tool execution, memory retrieval, and context compaction, all in one loop. When it's doing work, it can't talk to you. When it's compacting, it goes dark. When it retrieves memories, raw results pollute the context with noise. -| Method | What You Get | -| -------------------------------------- | ------------------------------------------------------------------------------------------- | -| **[spacebot.sh](https://spacebot.sh)** | One-click hosted deploy. Connect your platforms, configure your agent, done. | -| **Self-hosted** | Single Rust binary. No Docker, no server dependencies, no microservices. Clone, build, run. | -| **Docker** | Container image with everything included. Mount a volume for persistent data. | +Spacebot splits the monolith into specialized processes that only do one thing, and delegate everything else. --- -## Capabilities +## Built for Teams -### Task Execution +No other agent harness handles concurrent multi-user conversations, shared memory across channels, and true process-level concurrency. A Discord community with hundreds of active members, a Slack workspace running parallel workstreams, a Telegram group coordinating across time zones. Spacebot handles all of it without any user waiting on another. -Workers come loaded with tools for real work: +Solo users get the same infrastructure. Better memory, better concurrency, better structure. Everything teams rely on, for one person. -- **Shell** — run arbitrary commands with configurable timeouts -- **File** — read, write, and list files with auto-created directories -- **Exec** — run specific programs with arguments and environment variables -- **[OpenCode](https://opencode.ai)** — spawn a full coding agent as a persistent worker with codebase exploration, LSP awareness, and deep context management -- **Browser** — headless Chrome automation with an accessibility-tree ref system. Navigate, click, type, screenshot, manage tabs — the LLM addresses elements by short refs (`e0`, `e1`) instead of fragile CSS selectors -- **[Brave](https://brave.com/search/api/) web search** — search the web with freshness filters, localization, and configurable result count - -### Messaging +**For communities:** drop Spacebot into a Discord server. It handles concurrent conversations across channels and threads, remembers context about every member, and does real work without going dark. Fifty people can interact simultaneously. A message coalescing system detects rapid-fire bursts, batches them into a single turn, and lets the agent read the room. -Native adapters for Discord, Slack, Telegram, Twitch, and Webchat with full platform feature support: - -- **Message coalescing** — rapid-fire messages are batched into a single LLM turn with timing context, so the agent reads the room instead of spamming replies -- **File attachments** — send and receive files, images, and documents -- **Rich messages** — embeds/cards, interactive buttons, select menus, and polls (Discord). Block Kit messages and slash commands (Slack) -- **Threading** — automatic thread creation for long conversations -- **Reactions** — emoji reactions on messages -- **Typing indicators** — visual feedback while the agent is thinking -- **Message history backfill** — reads recent conversation context on first message -- **Per-channel permissions** — guild, channel, and DM-level access control, hot-reloadable -- **Webchat** — embeddable portal chat with SSE streaming, per-agent session isolation - -### Memory +**For teams:** connect it to Slack. Each channel gets a dedicated conversation with shared memory. One engineer gets a deep coding session while another gets a quick answer. Workers handle heavy lifting in the background while the channel stays responsive. -Not markdown files. Not _unstructured_ blocks in a vector database. Spacebot's memory is a typed, graph-connected knowledge system — and this opinionated structure is why agents are productive out of the box. +**For multi-agent setups:** run multiple agents on one instance. A community bot on Discord, a dev assistant on Slack, a research agent handling background tasks. Each with its own identity, memory, and security permissions. One binary, one deploy. -Every memory has a type, an importance score, and graph edges connecting it to related memories. The agent doesn't just "remember things" — it knows the difference between a fact it learned, a decision that was made, a goal it's working toward, and a preference the user expressed. This structure is what lets the cortex synthesize a useful briefing instead of dumping raw search results into context. - -- **Eight memory types** — Fact, Preference, Decision, Identity, Event, Observation, Goal, Todo -- **Graph edges** — RelatedTo, Updates, Contradicts, CausedBy, PartOf -- **Hybrid recall** — vector similarity + full-text search merged via Reciprocal Rank Fusion -- **Memory import** — dump files into the `ingest/` folder and Spacebot extracts structured memories automatically. Supports text, markdown, and PDF files. Migrating from OpenClaw? Drop your markdown memory files in and walk away. -- **Cross-channel recall** — branches can read transcripts from other conversations -- **Memory bulletin** — the cortex generates a periodic briefing of the agent's knowledge, injected into every conversation -- **Warmup readiness contract** — branch/worker/cron dispatch checks `ready_for_work` (warm state + embedding ready + fresh bulletin), records cold-dispatch metrics, and triggers background forced warmup without blocking channels - -### Scheduling - -Cron jobs created and managed from conversation or config: +--- -- **Natural scheduling** — "check my inbox every 30 minutes" becomes a cron job with a delivery target -- **Strict wall-clock schedules** — use cron expressions for exact local-time execution (for example, `0 9 * * *` for 9:00 every day) -- **Legacy interval compatibility** — existing `interval_secs` jobs still run and remain configurable -- **Configurable timeouts** — per-job `timeout_secs` to cap execution time (defaults to 120s) -- **Active hours** — restrict jobs to specific time windows (supports midnight wrapping) -- **Circuit breaker** — auto-disables after 3 consecutive failures -- **Full agent capabilities** — each job gets a fresh channel with branching and workers +## How It Works -### Model Routing +Five process types. Each does one job. -Four-level routing system that picks the right model for every LLM call. Structural routing handles the common case — process types and task types are known at spawn time. Prompt-level routing handles the rest, scoring user messages to downgrade simple requests to cheaper models automatically. +**Channels** are the user-facing LLM process. One per conversation, with soul, identity, and personality. A channel never executes tasks or searches memories directly. It branches to think, spawns workers to act, and stays responsive. -- **Process-type defaults** — channels get the best conversational model, workers get something fast and cheap, compactors get the cheapest tier -- **Task-type overrides** — a coding worker upgrades to a stronger model, a summarization worker stays cheap -- **Prompt complexity scoring** — lightweight keyword scorer classifies user messages into three tiers (light/standard/heavy) and routes to the cheapest model that can handle it. Scores the user message only — system prompts and context are excluded. <1ms, no external calls -- **Fallback chains** — when a model returns 429 or 502, the next model in the chain takes over automatically -- **Rate limit tracking** — 429'd models are deprioritized across all agents for a configurable cooldown -- **Per-agent routing profiles** — eco, balanced, or premium presets that shift what models each tier maps to. A budget agent routes simple messages to free models while a premium agent stays on opus +**Branches** fork from the channel's context to think. They have the full conversation history and run concurrently. The channel sees the conclusion, not the working. -```toml -[defaults.routing] -channel = "anthropic/claude-sonnet-4" -worker = "anthropic/claude-haiku-4.5" +**Workers** are independent processes. Each gets a specific task, a focused prompt, and task-appropriate tools, with no conversation context. Fire-and-forget for one-shot tasks, or interactive for longer sessions where follow-up routes to the active worker. -[defaults.routing.task_overrides] -coding = "anthropic/claude-sonnet-4" +**The Compactor** is a programmatic monitor (not an LLM) that watches context size per channel and triggers compaction before the channel fills up. Compaction workers run alongside without blocking. -[defaults.routing.prompt_routing] -enabled = true -process_types = ["channel", "branch"] +**The Cortex** sees across all channels, workers, and branches simultaneously. It generates the memory bulletin, a periodically refreshed briefing of the agent's knowledge injected into every conversation. It supervises processes, maintains the memory graph, detects patterns, and provides an admin chat with full tool access. -[defaults.routing.fallbacks] -"anthropic/claude-sonnet-4" = ["anthropic/claude-haiku-4.5"] ``` +User sends message + → Channel receives it + → Branches to think (has channel's context) + → Branch recalls memories, decides what to do + → Branch might spawn a worker for heavy tasks + → Branch returns conclusion + → Branch deleted + → Channel responds to user -**Z.ai (GLM) example** — use GLM models directly with a [GLM Coding Plan](https://z.ai) subscription: - -```toml -[llm] -zhipu_key = "env:ZHIPU_API_KEY" - -[defaults.routing] -channel = "zhipu/glm-4.7" -worker = "zhipu/glm-4.7" - -[defaults.routing.task_overrides] -coding = "zhipu/glm-4.7" +Channel context hits 80% + → Compactor notices + → Spins off a compaction worker + → Worker summarizes old context + → Compacted summary swaps in + → Channel never interrupted ``` -**Ollama example** — run against a local Ollama instance: +For process capabilities, tool access by type, memory internals, cron, and multi-agent isolation, see [ARCHITECTURE.md](ARCHITECTURE.md). -```toml -[llm] -ollama_base_url = "http://localhost:11434" +--- -[defaults.routing] -channel = "ollama/gemma3" -worker = "ollama/gemma3" +## Goals and Tasks -[defaults.routing.task_overrides] -coding = "ollama/qwen3" -``` +Spacebot is built around a task system. Goals set direction. Tasks carry work. The agent executes, remembers, and improves whether or not you're present. -**Custom provider example** — add any OpenAI-compatible or Anthropic-compatible endpoint: +On a configured interval, the **autonomy channel** wakes with full context: identity, memory, working memory, the complete task state, active goals, and a summary of its last few runs. It picks the most important ready task, executes it with full tool access, and exits. -```toml -[llm.provider.my-provider] -api_type = "openai_completions" # or "openai_chat_completions", "openai_responses", "anthropic" -base_url = "https://my-llm-host.example.com" -api_key = "env:MY_PROVIDER_KEY" +State lives in tasks. Progress notes go on the task itself. After a crash, the next wake reads task metadata and picks up where things left off. At the end of each run the autonomy channel writes a summary of what happened. On next wake, that summary is the first thing it reads. -[defaults.routing] -channel = "my-provider/my-model" -``` +**The agent proposes. You decide.** Tasks the autonomy channel creates land in `pending_approval`. Nothing runs autonomously until you approve it. -**Azure OpenAI Service** — configure Azure OpenAI deployments: +--- -```toml -[llm.provider.azure] -api_type = "azure" -base_url = "https://{resource-name}.openai.azure.com" -api_key = "env:AZURE_API_KEY" -api_version = "2024-06-01" # required -deployment = "gpt-4o" # required — your deployment name +## What It Does -[defaults.routing] -channel = "azure/gpt-4o" -worker = "azure/gpt-4o-mini" -``` +### Memory -Important notes: -- `base_url` must end with `.openai.azure.com` -- `api_version` and `deployment` are required fields -- API key authentication is handled automatically via the `api-key` header -- For Azure AI Foundry (accessing Anthropic, Llama, or other models through Azure's model catalog), use `api_type = "openai_chat_completions"` instead and configure the deployment endpoint accordingly +Spacebot's memory is a typed, graph-connected knowledge system in SQLite and LanceDB. Every memory has a type, an importance score, and graph edges to related memories. The agent distinguishes facts from decisions, preferences from goals. That structure lets the cortex synthesize a useful briefing rather than dumping raw search results into context. -Additional built-in providers include **Kilo Gateway**, **OpenCode Go**, **NVIDIA**, **MiniMax**, **Moonshot AI (Kimi)**, and **Z.AI Coding Plan** — configure with `kilo_key`, `opencode_go_key`, `nvidia_key`, `minimax_key`, `moonshot_key`, or `zai_coding_plan_key` in `[llm]`. +- **Eight memory types** — Fact, Preference, Decision, Identity, Event, Observation, Goal, Todo +- **Graph edges** — RelatedTo, Updates, Contradicts, CausedBy, PartOf +- **Hybrid recall** — vector similarity + full-text search merged via Reciprocal Rank Fusion +- **Memory import** — drop files into `ingest/` and Spacebot extracts structured memories automatically. Supports text, markdown, and PDF. +- **Memory bulletin** — the cortex generates a periodic knowledge briefing injected into every conversation ### Skills -Extensible skill system integrated with [skills.sh](https://skills.sh): +Skills are reusable procedures injected into worker system prompts. The agent writes them from experience — and they accumulate automatically over time. -- **skills.sh registry** — install any skill from the public ecosystem with one command -- **CLI management** — `spacebot skill add owner/repo` to install, list, remove, and inspect skills +- **Autonomous skill capture** — when a channel identifies a workflow that required multiple steps or problem-solving, it delegates to a branch to write it as a skill. The skill loads into the next session and every session after +- **Post-conversation reflection** — after a conversation goes idle, a background branch silently reviews the history and saves skills and memories worth keeping. No user action required +- **AI-assisted authoring** — describe a skill in plain language, the agent generates it and shows a preview before saving - **Worker injection** — skills are injected into worker system prompts for specialized tasks -- **Bundled resources** — scripts, references, and assets packaged with skills -- **OpenClaw compatible** — drop in existing OpenClaw skills, or any skill from skills.sh - -**Install skills from the registry:** +- **skills.sh registry** — install any skill from the public ecosystem with one command. Compatible with any skill from the public registry ```bash spacebot skill add vercel-labs/agent-skills @@ -236,204 +147,103 @@ spacebot skill add anthropics/skills/pdf spacebot skill list ``` -### MCP Integration - -Connect workers to external [MCP](https://modelcontextprotocol.io/) (Model Context Protocol) servers for arbitrary tool access -- databases, APIs, SaaS products, custom integrations -- without native Rust implementations: - -- **Per-agent config** — each agent declares its own MCP servers in `config.toml` -- **Both transports** — stdio (subprocess) for local tools, streamable HTTP for remote servers -- **Automatic tool discovery** — tools are discovered via the MCP protocol and registered on worker ToolServers with namespaced names (`{server}_{tool}`) -- **Automatic retry** — failed connections retry in the background with exponential backoff (5s initial, 60s cap, 12 attempts). A broken server never blocks agent startup -- **Hot-reloadable** — add, remove, or change servers in config and they reconcile live -- **API management** — full CRUD API under `/api/mcp/` for managing server definitions and monitoring connection status programmatically - -```toml -[[mcp_servers]] -name = "filesystem" -transport = "stdio" -command = "npx" -args = ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"] - -[[mcp_servers]] -name = "sentry" -transport = "http" -url = "https://mcp.sentry.io" -headers = { Authorization = "Bearer ${SENTRY_TOKEN}" } -``` - -### Security - -Spacebot runs autonomous LLM processes that execute arbitrary shell commands and spawn subprocesses. Security isn't an add-on — it's a layered system designed so that no single failure exposes credentials or breaks containment. - -#### Credential Isolation - -Secrets are split into two categories: **system** (LLM API keys, messaging tokens — never exposed to subprocesses) and **tool** (CLI credentials like `GH_TOKEN` — injected as env vars into workers). The category is auto-assigned based on the secret name, or set explicitly. - -- **Environment sanitization** — every subprocess starts with a clean environment (`--clearenv` on Linux, `env_clear()` everywhere else). Only safe baseline vars (`PATH`, `HOME`, `LANG`), tool-category secrets, and explicit `passthrough_env` entries are present. In sandbox mode, `HOME` is set to the workspace; in passthrough mode, `HOME` uses the parent environment. System secrets never enter any subprocess -- **Secret store** — credentials live in a dedicated redb database, not in `config.toml`. Config references secrets by alias (`anthropic_key = "secret:ANTHROPIC_API_KEY"`), so the config file is safe to display, screenshot, or `cat` -- **Encryption at rest** — optional AES-256-GCM encryption with a master key derived via Argon2id. The master key lives in the OS credential store (macOS Keychain, Linux kernel keyring) — never on disk, never in an env var, never accessible to worker subprocesses -- **Keyring isolation** — on Linux, workers are spawned with a fresh empty session keyring via `pre_exec`. Even without the sandbox, workers cannot access the parent's kernel keyring where the master key lives -- **Output scrubbing** — all tool secret values are redacted from worker output before it reaches channels or LLM context. A rolling buffer handles secrets split across stream chunks. Channels see `[REDACTED]`, never raw values -- **Worker secret management** — workers can store credentials they obtain (API keys from account creation, OAuth tokens) via the `secret_set` tool. Stored secrets are immediately available to future workers - -#### Process Containment - -- **Process sandbox** — shell and exec tools run inside OS-level filesystem containment. On Linux, [bubblewrap](https://github.com/containers/bubblewrap) creates a mount namespace where the entire filesystem is read-only except the agent's workspace and configured writable paths. On macOS, `sandbox-exec` enforces equivalent restrictions via SBPL profiles. Kernel-enforced, not string-filtered -- **Dynamic sandbox mode** — sandbox settings are hot-reloadable. Toggle via the dashboard or API without restarting the agent -- **Workspace isolation** — file tools canonicalize all paths and reject anything outside the agent's workspace. Symlinks that escape are blocked -- **Leak detection** — secret-pattern checks are enforced at channel egress (`reply` and plaintext fallback output) across plaintext, URL-encoded, base64, and hex encodings. Outbound text matching a secret pattern is blocked; worker tool outputs no longer hard-fail the worker -- **Library injection blocking** — the exec tool blocks dangerous environment variables (`LD_PRELOAD`, `DYLD_INSERT_LIBRARIES`, `NODE_OPTIONS`, etc.) that could hijack child process loading -- **SSRF protection** — the browser tool blocks requests to cloud metadata endpoints, private IPs, loopback, and link-local addresses -- **Identity file protection** — writes to `SOUL.md`, `IDENTITY.md`, and `USER.md` are blocked at the application level -- **Durable binary storage** — `tools/bin` directory on PATH survives hosted rollouts. Workers are instructed to install binaries there instead of ephemeral package manager locations - -```toml -[agents.sandbox] -mode = "enabled" # "enabled" (default) or "disabled" -writable_paths = ["/home/user/projects/myapp"] # additional writable dirs beyond workspace -passthrough_env = ["CUSTOM_VAR"] # forward specific env vars to workers -``` - ---- +### Scheduling -## How It Works +Cron jobs created and managed from conversation: -Five process types. Each does one job. +- **Natural scheduling** — "check my inbox every 30 minutes" becomes a cron job with a delivery target +- **Strict wall-clock schedules** — cron expressions for exact local-time execution +- **Single delivery** — all reply calls are buffered during the run and flushed as one message when the job completes. No mid-run fragments. +- **Circuit breaker** — auto-disables after 3 consecutive failures +- **Full agent capabilities** — each job gets a fresh channel with branching and workers -### Channels +### Task Execution -The user-facing LLM process — the ambassador to the human. One per conversation (Discord thread, Slack channel, Telegram DM, etc). Has soul, identity, and personality. Talks to the user. Delegates everything else. +Workers come loaded with tools for real work: -A channel does **not**: execute tasks directly, search memories itself, or do any heavy tool work. It is always responsive — never blocked by work, never frozen by compaction. +- **Shell** — run arbitrary commands with configurable timeouts +- **File** — read, write, and list files with auto-created directories +- **Browser** — headless Chrome automation with accessibility-tree refs. Navigate, click, type, screenshot, manage tabs +- **[OpenCode](https://opencode.ai)** — spawn a full coding agent as a persistent worker with codebase exploration, LSP awareness, and deep context management +- **[Brave](https://brave.com/search/api/) web search** — search the web with freshness filters, localization, and configurable result count -When it needs to think, it branches. When it needs work done, it spawns a worker. +### Messaging -### Branches +Native adapters for Discord, Slack, Telegram, Twitch, Signal, Mattermost, Email, and Webchat, plus a generic Webhook receiver: -A fork of the channel's context that goes off to think. Has the channel's full conversation history — same context, same memories, same understanding. Operates independently. The channel never sees the working, only the conclusion. +- **Message coalescing** — rapid-fire messages are batched into a single LLM turn with timing context +- **File attachments** — send and receive files, images, and documents. Attachments are saved to the workspace and recalled by ID +- **Rich messages** — embeds/cards, interactive buttons, select menus, and polls (Discord). Block Kit and slash commands (Slack) +- **Email** — IMAP polling + SMTP delivery with TLS, UID-based dedup, allowed sender filtering, and attachment limits. Works with local bridges like Proton Bridge +- **Webchat** — embeddable portal chat with SSE streaming, per-agent session isolation +- **Per-channel permissions** — guild, channel, and DM-level access control, hot-reloadable -``` -User A: "what do you know about X?" - → Channel branches (branch-1) +### Model Routing -User B: "hey, how's it going?" - → Channel responds directly: "Going well! Working on something for A." +Four-level routing picks the right model for every call. Channels get the best conversational model. Workers get something fast and cheap. Coding workers upgrade automatically. Simple user messages are downgraded to cheaper models by a sub-millisecond prompt scorer with no external calls. Voice messages route to a dedicated voice model. -Branch-1 resolves: "Here's what I found about X: [curated memories]" - → Channel sees the branch result on its next turn - → Channel responds to User A with the findings -``` +Any OpenAI-compatible or Anthropic-compatible endpoint works, including Ollama for local models, Z.ai GLM models, Azure OpenAI, and custom providers. Built-in support for Kilo Gateway, NVIDIA, MiniMax, Moonshot AI, Gemini, GitHub Copilot, OpenCode Go, and more. -Multiple branches run concurrently. First done, first incorporated. Each branch forks from the channel's context at creation time, like a git branch. +### MCP Integration -### Workers +Connect workers to external [MCP](https://modelcontextprotocol.io/) servers for arbitrary tool access — databases, APIs, SaaS products, custom integrations. Both stdio and streamable HTTP transports. Automatic tool discovery, hot-reloadable, exponential-backoff retry so a broken server never blocks startup. -Independent processes that do jobs. Get a specific task, a focused system prompt, and task-appropriate tools. No channel context, no soul, no personality. +### Security -**Fire-and-forget** — do a job and return a result. Summarization, file operations, one-shot tasks. +Spacebot runs autonomous LLM processes that execute arbitrary shell commands. Security is layered so no single failure exposes credentials or breaks containment. -**Interactive** — long-running, accept follow-up input from the channel. Coding sessions, multi-step tasks. +**Credential isolation:** secrets split into system credentials (LLM API keys, messaging tokens, never exposed to subprocesses) and tool credentials (CLI tokens injected as env vars into workers). Every subprocess starts with a sanitized environment. System secrets never enter any subprocess. -``` -User: "refactor the auth module" - → Branch spawns interactive coding worker - → Branch returns: "Started a coding session for the auth refactor" +- **Secret store** — credentials live in a dedicated encrypted database, referenced by alias. Plain config files never contain secrets +- **Encryption at rest** — optional AES-256-GCM with a master key derived via Argon2id, stored in the OS credential store (macOS Keychain, Linux kernel keyring), never on disk or in an env var +- **Output scrubbing** — all tool secret values are redacted from worker output before it reaches channels or LLM context. A rolling buffer handles secrets split across stream chunks -User: "actually, update the tests too" - → Channel routes message to active worker - → Worker receives follow-up, continues with its existing context -``` +**Process containment:** shell and exec tools run inside OS-level filesystem containment. On Linux, [bubblewrap](https://github.com/containers/bubblewrap) creates a mount namespace where the filesystem is read-only except the agent's workspace. On macOS, `sandbox-exec` enforces equivalent restrictions via SBPL profiles. Enforced at the kernel level. -Workers are pluggable. Any process that accepts a task and reports status can be a worker. +- **Dynamic sandbox** — toggle sandbox mode via dashboard or API without restarting +- **Workspace isolation** — file tools reject paths outside the agent's workspace. Symlinks that escape are blocked. +- **Leak detection** — secret-pattern checks at channel egress across plaintext, URL-encoded, base64, and hex encodings +- **SSRF protection** — browser tool blocks requests to cloud metadata endpoints, private IPs, loopback, and link-local addresses -**Built-in workers** come with shell, file, exec, and browser tools out of the box. They can write code, run commands, manage files, browse the web — enough to build a whole project from scratch. +--- -**[OpenCode](https://opencode.ai) workers** are a built-in integration that spawns a full OpenCode coding agent as a persistent subprocess. OpenCode brings its own codebase exploration, LSP awareness, and context management — purpose-built for deep coding sessions. When a user asks for a complex refactor or a new feature, the channel can spawn an OpenCode worker that maintains a rich understanding of the codebase across the entire session. Both built-in and OpenCode workers support interactive follow-ups. +## Gets Better with Use -### The Compactor +Spacebot builds on itself over time through four specific mechanisms. -Not an LLM process. A programmatic monitor per channel that watches context size and triggers compaction before the channel fills up. +**Branches write skills from experience.** When a channel identifies a workflow that required multiple steps, problem-solving, or domain knowledge, it delegates to a branch to capture it as a structured skill. The skill goes to disk and loads into the next session. Future workers get it injected into their system prompt. -| Threshold | Action | -| --------- | -------------------------------------------- | -| **>80%** | Background compaction (summarize oldest 30%) | -| **>85%** | Aggressive compaction (summarize oldest 50%) | -| **>95%** | Emergency truncation (hard drop, no LLM) | +**Post-conversation reflection saves what's worth keeping.** After a conversation goes idle, a background branch reviews the history and silently saves skills and memories worth keeping. It runs with a capped turn budget, produces no user-visible output, and fires only when there's enough conversation to learn from. -Compaction workers run alongside the channel without blocking it. Summaries stack chronologically at the top of the context window. +**Memory deepens with every interaction.** Each conversation adds facts, preferences, decisions, and observations to a typed graph with importance scoring and graph edges. The cortex synthesizes this into a briefing every future conversation benefits from. -### The Cortex +**Goals drive autonomous work between conversations.** The autonomy channel wakes on its interval, picks up ready tasks, and works through them. Working memory records what happened, so the next conversation picks up where things left off. -The agent's inner monologue. The only process that sees across all channels, workers, and branches simultaneously. Generates a **memory bulletin** — a periodically refreshed, LLM-curated briefing of the agent's knowledge injected into every conversation. Supervises running processes (kills hanging workers, cleans up stale branches). Maintains the memory graph (decay, pruning, merging near-duplicates, cross-channel consolidation). Detects patterns across conversations and creates observations. Also provides a direct interactive admin chat with full tool access for system inspection and manual intervention. +Everything goes through typed tools into structured storage. Nothing drifts. --- -## Architecture - -``` -User sends message - → Channel receives it - → Branches to think (has channel's context) - → Branch recalls memories, decides what to do - → Branch might spawn a worker for heavy tasks - → Branch returns conclusion - → Branch deleted - → Channel responds to user - -Channel context hits 80% - → Compactor notices - → Spins off a compaction worker - → Worker summarizes old context + extracts memories - → Compacted summary swaps in - → Channel never interrupted -``` - -### What Each Process Gets - -| Process | Type | Tools | Context | -| --------- | ------------------ | ----------------------------------------- | ----------------------------------- | -| Channel | LLM | Reply, branch, spawn workers, route | Conversation + compaction summaries | -| Branch | LLM | Memory recall, memory save, spawn workers | Fork of channel's context | -| Worker | Pluggable | Shell, file, exec, browser (configurable) | Fresh prompt + task description | -| Compactor | Programmatic | Monitor context, trigger workers | N/A | -| Cortex | LLM + Programmatic | Memory, consolidation, system monitor | Entire agent scope | +## Spacebot + Spacedrive -### Memory System +Spacebot pairs with [Spacedrive](https://github.com/spacedriveapp/spacedrive), an open-source cross-platform file manager built on a virtual distributed filesystem. Neither requires the other. When paired, Spacebot is the only agent harness with direct integration into a cross-device filesystem. -Memories are structured objects, not files. Every memory is a row in SQLite with typed metadata and graph connections, paired with a vector embedding in LanceDB. +### What Pairing Enables Today -- **Eight types** — Fact, Preference, Decision, Identity, Event, Observation, Goal, Todo -- **Graph edges** — RelatedTo, Updates, Contradicts, CausedBy, PartOf -- **Hybrid search** — Vector similarity + full-text search, merged via Reciprocal Rank Fusion -- **Three creation paths** — Branch-initiated, compactor-initiated, cortex-initiated -- **Importance scoring** — Access frequency, recency, graph centrality. Identity memories exempt from decay. - -### Cron Jobs +**Multi-device access:** one Spacebot instance, all your devices. Talk to your agent from your phone while a worker executes on your server. Spacedrive's P2P layer (Iroh/QUIC) routes from every device through the paired node to Spacebot. No separate SDK, no separate auth. -Scheduled recurring tasks. Each cron job gets a fresh short-lived channel with full branching and worker capabilities. +**Remote execution:** workers can target any device in your library. A task that needs your home server's GPU, your work laptop's local repos, or your phone's camera routes through Spacedrive's permission system to the target device. From the agent's perspective, the tool call is identical. -- Multiple cron jobs run independently on wall-clock schedules (or legacy intervals) -- Stored in the database, created via config, conversation, or programmatically -- Cron expressions execute against the resolved cron timezone for predictable local-time firing -- Persisted `next_run_at` cursor for deterministic restart behavior and missed-run fast-forwarding -- Claim-before-run scheduling so multi-process or restarted schedulers do not double-fire recurring jobs -- Run-once jobs use at-most-once claiming semantics and disable before execution starts -- Per-job `timeout_secs` to cap execution time -- Circuit breaker auto-disables after 3 consecutive failures -- Active hours support with midnight wrapping -- Execution and delivery outcomes are logged separately, with bounded retry/backoff for proactive sends +**File system intelligence:** every directory can carry context nodes describing what it contains and what policies apply. When the agent navigates your filesystem it gets that context, not a blind listing. -### Multi-Agent +**Safe data access:** Spacedrive indexes external sources (Gmail, Slack, Obsidian, GitHub, Apple Notes, contacts, calendar, browser history) as searchable data the agent can query. Every record passes through a local prompt injection classifier (Prompt Guard 2) before reaching the agent. The agent can search your emails without a malicious email hijacking it. -Each agent is an independent entity with its own workspace, databases, identity files, cortex, and messaging bindings. All agents share one binary, one tokio runtime, and one set of API keys. +### Where This Is Going ---- +A company deploys Spacebot + Spacedrive on their infrastructure. Employees install Spacedrive on their devices and join the company library. The company agent has access to employee devices through Spacedrive's permission system, with individual-level controls. The org graph in Spacebot defines hierarchy and delegation: which agents report to which, who can approve what, how tasks flow. -### Spacedrive Integration (Future) +An employee talks to the company agent from their MacBook. The agent knows their projects, their device, their role, and can spawn workers on any authorized machine. They switch to their personal Spacedrive library and connect to their home Spacebot, with personal data and personal context. The app is the same. The agent is different. -Spacebot is the AI counterpart to [Spacedrive](https://github.com/spacedriveapp/spacedrive) — an open source cross-platform file manager built on a virtual distributed filesystem. Both projects are independent and fully functional on their own, but complementary by design. Spacedrive indexes files across all your devices, clouds, and platforms with content-addressed identity, semantic search, and local AI analysis. Spacebot brings autonomous reasoning, memory, and task execution. Together, an agent that can think, remember, and act — backed by terabytes of queryable data across every device you own. - -Read the full vision in the [roadmap](docs/content/docs/(deployment)/roadmap.mdx). +No other agent harness is building this. It's a category. --- @@ -442,7 +252,7 @@ Read the full vision in the [roadmap](docs/content/docs/(deployment)/roadmap.mdx ### Prerequisites - **Rust** 1.85+ ([rustup](https://rustup.rs/)) -- An LLM API key from any supported provider (Anthropic, OpenAI, OpenRouter, Kilo Gateway, Z.ai, Groq, Together, Fireworks, DeepSeek, xAI, Mistral, NVIDIA, MiniMax, Moonshot AI, OpenCode Zen, OpenCode Go) — or use `spacebot auth login` for Anthropic OAuth +- An LLM API key from any supported provider (Anthropic, OpenAI, OpenRouter, Kilo Gateway, Z.ai, Groq, Together, Fireworks, DeepSeek, xAI, Mistral, NVIDIA, MiniMax, Moonshot AI, Gemini, GitHub Copilot, OpenCode Zen, OpenCode Go), or use `spacebot auth login` for Anthropic OAuth ### Build and Run @@ -457,35 +267,7 @@ cd spacebot cargo build --release ``` -### Minimal Config - -Create `config.toml`: - -```toml -[llm] -openrouter_key = "env:OPENROUTER_API_KEY" - -[defaults.routing] -channel = "anthropic/claude-sonnet-4" -worker = "anthropic/claude-sonnet-4" - -[[agents]] -id = "my-agent" - -[messaging.discord] -token = "env:DISCORD_BOT_TOKEN" - -[[bindings]] -agent_id = "my-agent" -channel = "discord" -guild_id = "your-discord-guild-id" - -# Optional: route a named adapter instance -[[bindings]] -agent_id = "my-agent" -channel = "discord" -adapter = "ops" -``` +### Run ```bash spacebot # start as background daemon @@ -500,7 +282,7 @@ The binary creates all databases and directories automatically on first run. See ### Authentication -Spacebot supports Anthropic OAuth as an alternative to static API keys. Use your Claude Pro, Max, or API Console subscription directly: +Spacebot supports Anthropic OAuth as an alternative to static API keys: ```bash spacebot auth login # OAuth via Claude Pro/Max (opens browser) @@ -510,7 +292,17 @@ spacebot auth refresh # manually refresh the access token spacebot auth logout # remove stored credentials ``` -OAuth tokens are stored in `anthropic_oauth.json` and auto-refresh transparently before each API call. When OAuth credentials are present, they take priority over a static `ANTHROPIC_API_KEY`. +OAuth tokens are stored in `anthropic_oauth.json` and auto-refresh before each API call. When OAuth credentials are present, they take priority over a static `ANTHROPIC_API_KEY`. + +--- + +## Deploy Your Way + +| Method | What You Get | +| -------------------------------------- | ------------------------------------------------------------------------------------------- | +| **[spacebot.sh](https://spacebot.sh)** | One-click hosted deploy. Connect your platforms, configure your agent, done. | +| **Self-hosted** | Single Rust binary. No Docker, no server dependencies, no microservices. Clone, build, run. | +| **Docker** | Container image with everything included. Mount a volume for persistent data. | --- @@ -518,22 +310,22 @@ OAuth tokens are stored in `anthropic_oauth.json` and auto-refresh transparently | Layer | Technology | | --------------- | --------------------------------------------------------------------------------------------------------------- | -| Language | **Rust** (edition 2024) | +| Language | **Rust** (edition 2024) — single binary, no runtime dependencies, no GC pauses | | Async runtime | **Tokio** | | LLM framework | **[Rig](https://github.com/0xPlaygrounds/rig)** v0.31 — agentic loop, tool execution, hooks | -| Relational data | **SQLite** (sqlx) — conversations, memory graph, cron jobs | +| Relational data | **SQLite** (sqlx) — conversations, memory graph, tasks, goals, cron jobs | | Vector + FTS | **[LanceDB](https://lancedb.github.io/lancedb/)** — embeddings (HNSW), full-text (Tantivy), hybrid search (RRF) | | Key-value | **[redb](https://github.com/cberner/redb)** — settings, encrypted secrets | | Embeddings | **FastEmbed** — local embedding generation | | Crypto | **AES-256-GCM** — secret encryption at rest | -| Discord | **Serenity** — gateway, cache, events, rich messages, interactions | -| Slack | **slack-morphism** — Socket Mode, events, Block Kit, slash commands, streaming via message edits | -| Telegram | **teloxide** — long-poll, media attachments, group/DM support | -| Twitch | **twitch-irc** — chat integration with trigger prefix | -| Browser | **Chromiumoxide** — headless Chrome via CDP | +| Discord | **Serenity** — gateway, cache, events, rich messages, interactions | +| Slack | **slack-morphism** — Socket Mode, events, Block Kit, slash commands | +| Telegram | **teloxide** — long-poll, media attachments, group/DM support | +| Twitch | **twitch-irc** — chat integration with trigger prefix | +| Browser | **Chromiumoxide** — headless Chrome via CDP | | CLI | **Clap** — command line interface | -No server dependencies. Single binary. All data lives in embedded databases in a local directory. +Single binary, no server dependencies. All data lives in embedded databases in a local directory. --- @@ -541,33 +333,18 @@ No server dependencies. Single binary. All data lives in embedded databases in a | Doc | Description | | -------------------------------------- | -------------------------------------------------------- | -| [Quick Start](docs/content/docs/(getting-started)/quickstart.mdx) | Setup, config, first run | -| [Config Reference](docs/content/docs/(configuration)/config.mdx) | Full `config.toml` reference | -| [Agents](docs/content/docs/(core)/agents.mdx) | Multi-agent setup and isolation | -| [Memory](docs/content/docs/(core)/memory.mdx) | Memory system design | -| [Tools](docs/content/docs/(features)/tools.mdx) | All available LLM tools | -| [Compaction](docs/content/docs/(core)/compaction.mdx) | Context window management | -| [Cortex](docs/content/docs/(core)/cortex.mdx) | Memory bulletin and system observation | -| [Cron Jobs](docs/content/docs/(features)/cron.mdx) | Scheduled recurring tasks | -| [Routing](docs/content/docs/(core)/routing.mdx) | Model routing and fallback chains | -| [Secrets](docs/content/docs/(configuration)/secrets.mdx) | Credential storage, encryption, and output scrubbing | -| [Sandbox](docs/content/docs/(configuration)/sandbox.mdx) | Process containment and environment sanitization | -| [Messaging](docs/content/docs/(messaging)/messaging.mdx) | Adapter architecture (Discord, Slack, Telegram, Twitch, Webchat, webhook) | -| [Discord Setup](docs/content/docs/(messaging)/discord-setup.mdx) | Discord bot setup guide | -| [Browser](docs/content/docs/(features)/browser.mdx) | Headless Chrome for workers | -| [MCP](docs/content/docs/(features)/mcp.mdx) | External tool servers via Model Context Protocol | -| [OpenCode](docs/content/docs/(features)/opencode.mdx) | OpenCode as a worker backend | -| [Philosophy](docs/content/docs/(core)/philosophy.mdx) | Why Rust | - ---- - -## Why Rust - -Spacebot isn't a chatbot — it's an orchestration layer for autonomous AI processes running concurrently, sharing memory, and delegating to each other. That's infrastructure, and infrastructure should be machine code. - -Rust's strict type system and compiler mean there's one correct way to express something. When multiple AI processes share mutable state and spawn tasks without human oversight, "the compiler won't let you do that" is a feature. The result is a single binary with no runtime dependencies, no garbage collector pauses, and predictable resource usage. - -Read the full argument in [docs/philosophy](docs/content/docs/(core)/philosophy.mdx). +| [Quick Start](docs/content/docs/(getting-started)/quickstart.mdx) | Setup, config, first run | +| [Config Reference](docs/content/docs/(configuration)/config.mdx) | Full `config.toml` reference | +| [Architecture](ARCHITECTURE.md) | Process types, tool access, memory internals, multi-agent | +| [Memory](docs/content/docs/(core)/memory.mdx) | Memory system design | +| [Tools](docs/content/docs/(features)/tools.mdx) | All available LLM tools | +| [Routing](docs/content/docs/(core)/routing.mdx) | Model routing and fallback chains | +| [Secrets](docs/content/docs/(configuration)/secrets.mdx) | Credential storage, encryption, output scrubbing | +| [Sandbox](docs/content/docs/(configuration)/sandbox.mdx) | Process containment and environment sanitization | +| [Cron Jobs](docs/content/docs/(features)/cron.mdx) | Scheduled recurring tasks | +| [MCP](docs/content/docs/(features)/mcp.mdx) | External tool servers via Model Context Protocol | +| [OpenCode](docs/content/docs/(features)/opencode.mdx) | OpenCode as a worker backend | +| [Messaging](docs/content/docs/(messaging)/messaging.mdx) | Adapter architecture and platform setup | --- @@ -583,10 +360,14 @@ Contributions welcome. Read [RUST_STYLE_GUIDE.md](RUST_STYLE_GUIDE.md) before wr 6. Run `just preflight` and `just gate-pr` 7. Submit a PR +### SpaceUI (Frontend Components) + +The dashboard uses [`@spacedrive/*`](https://github.com/spacedriveapp/spaceui) packages from npm. For local development with linked packages, see [CONTRIBUTING.md](CONTRIBUTING.md). + Formatting is still enforced in CI, but the hook catches it earlier by running `cargo fmt --all` before each commit. `just gate-pr` mirrors the CI gate and includes migration safety, compile checks, and test verification. --- ## License -FSL-1.1-ALv2 — [Functional Source License](https://fsl.software/), converting to Apache 2.0 after two years. See [LICENSE](LICENSE) for details. +FSL-1.1-ALv2, [Functional Source License](https://fsl.software/), converting to Apache 2.0 after two years. See [LICENSE](LICENSE) for details. diff --git a/SPACEUI_MIGRATION.md b/SPACEUI_MIGRATION.md new file mode 100644 index 000000000..1a1839bc1 --- /dev/null +++ b/SPACEUI_MIGRATION.md @@ -0,0 +1,342 @@ +# SpaceUI Migration + +**345 files changed | +39,873 / -17,225** + +Migrates the entire frontend to [SpaceUI](https://github.com/spacedriveapp/spaceui), Spacedrive's component library. The local UI primitives (~25 components, ~3000 lines) are gone — replaced by `@spacedrive/primitives`, `@spacedrive/ai`, `@spacedrive/forms`, and `@spacedrive/explorer`. Tailwind v3 is out, Tailwind v4 via `@tailwindcss/vite` is in, with SpaceUI's design token system and theme imports. + +But this isn't just a component swap. The interface has been restructured from the ground up. + +## Layout + +The old flat nav is replaced with a persistent sidebar (220px, Spacedrive-style) with labeled sections, accordion agent sub-nav, and a global workers popover in the footer. The org chart has its own dedicated tab instead of being crammed into the main view. + +A new **Dashboard** serves as the landing page with real data wired to action items (notifications), token usage, and recent activity cards. + +## UI rewrites + +Every major view was decomposed from monolithic files into modular components: + +- **Settings** — 2900-line monolith → 12 section components +- **AgentConfig** — 1452-line monolith → ConfigSidebar, section editors, shared types +- **TopologyGraph** — 2074-line monolith → OrgGraph, ProfileNode, GroupNode, edge/config panels, graph builder +- **Workbench** — rewritten with modular WorkbenchSidebar, WorkerColumn, and OpenCode theme inheriting CSS custom properties + +**Tasks** got the biggest conceptual change — the kanban board is gone, replaced with a Linear-style task list with detail views, GitHub metadata badges, and SSE-driven updates. + +## New systems + +**Wiki** — full implementation from scratch. SQLite-backed (`wiki_pages` + `wiki_page_versions`), tolerant multi-pass edit matching, 6 tools (create/edit/read/list/search/history), REST API, frontend route with page browser and wiki-link navigation. + +**Notifications** — SQLite store, API endpoints, SSE real-time broadcasting, `useNotifications` hook with optimistic dismiss, wired into the dashboard's action items card. Task approval and worker failure notifications emitted automatically. + +**Portal** — webchat renamed to portal throughout. Modular PortalPanel/Timeline/Composer/Header replacing the monolithic WebChatPanel. File attachments with multipart upload, drag-and-drop, and timeline rendering. Conversation persistence with full CRUD. + +**Streaming** — `prompt_once_streaming` on SpacebotHook with token-by-token `WorkerText` deltas. OpenAI Responses API SSE streaming (~636 lines) supporting function call deltas, text deltas, reasoning summaries. + +**Built-in skills** — compiled into the binary via `include_str!`, starting with a wiki-writing skill. + +## Backend + +**Projects** elevated from per-agent to instance level — migration with full dedup logic, dropped `agent_id` from the projects table, updated all store/API/tool paths. Auto logo detection scanning `.github/`, `public/`, `src-tauri/`. + +**Conversation settings** — `ConversationSettings` struct with memory mode, delegation, worker context, model selection. Per-channel persistence via `ChannelSettingsStore` with resolution chain (per-channel DB > binding defaults > agent defaults). `ModelOverrides` threaded through Channel, Branch, Worker, Compactor. Settings hot-reload via `ProcessEvent::SettingsUpdated`. + +**Channel settings unification** — `ResponseMode` enum (Active/Observe/MentionOnly) replacing the old `listen_only_mode` system. Per-channel persistence, `[bindings.settings]` TOML support, slash commands that persist to DB. MentionOnly fixed to retain context and capture memories even when not responding. + +**Direct mode** — channels can now get full worker-level tools (shell, file, browser, wiki, web search, memory) with tool calls rendered in the portal timeline. + +**Tool-use enforcement** — configurable per-model prompt injection ("you MUST use tools") across all process types. + +**Token usage tracking** — `token_usage` table with `UsageAccumulator` flushed per-process, wired to the dashboard chart. + +## Design docs + +Added: conversation-settings, wiki, token-usage-tracking, cron-outcome-delivery, skill-authoring, worker-briefing, attachment-portal-and-defaults, slash-commands, autonomy, goals. Some implemented, others queued. + +--- + +## Commit Log (newest first) + +> 14 merge commits omitted — no code changes. + +### 1. `248d740` — "tweaks" `+153 / -92` +Reformatted Sidebar imports to multi-line, renamed task approval notifications from "Approval"/"Approve" to "Review" with Clock icon, unified cron job button variants to "gray", fixed workbench EmptyState centering. + +### 2. `de9e680` — "tweaks" `+6 / -480` +Deleted the 474-line `autonomy-loop.md` design doc. Minor Tauri-to-desktop platform abstraction renames (`IS_TAURI` → `IS_DESKTOP`). + +### 3. `945ea8e` — "probably bad, likely revert" `+62 / -16` +Rewrote `apply_history_after_turn` to extract `chat_history` from `PromptCancelled` and `MaxTurnsError` variants instead of the external history param (which Rig doesn't update on error). Flushes accumulated tool calls into `chat_history` before returning `PromptCancelled` in SpacebotHook. + +### 4. `28e7951` — "ui" `+164 / -9` +New `ApprovalModal` component — full dialog for task approval notifications with task detail view, approve/dismiss actions, and query invalidation. Replaced the inline action button in ActionItemsCard. + +### 5. `7bb1874` — "Add CONTRIBUTING.md, SpaceUI link workflow, and npm version refs" `+219 / -5` +Added CONTRIBUTING.md, switched `@spacedrive/*` deps from `link:` to `^0.2.0` (npm published with local bun link override), added `just spaceui-link` / `just spaceui-unlink` commands. + +### 6. `b841d1c` — "progress" `+458 / -406` +Abstracted Tauri into a generic desktop host layer (`IS_DESKTOP`, `hasBundledServer`, `spawnBundledProcess`). Added active-project highlighting in sidebar, pinned "main" agent first, added macOS traffic-light padding, added stubs for cortex_chat and new tool registration. + +### 7. `1da3fbf` — "a lot of stuff" `+1,343 / -500` +Built-in skills system (compiled via `include_str!`, wiki-writing skill). Added `company_name` to global settings with InstanceSection page. Implemented "direct mode" for channels giving them full worker-level tools (shell, file, browser, wiki, web search, memory). Added tool_call_run timeline items to portal view. Company name switcher popover in sidebar. Ingest section in agent config. Browser tool structs made `pub(crate)`. + +### 8. `731efa2` — "workers" `+43 / -47` +Cosmetic Rust formatting only — line-break adjustments in usage.rs, wiki.rs, model.rs, pricing.rs, set_outcome.rs. No functional changes. + +### 9. `2790fac` — "workers" `+1,033 / -800` +Workers panel refactor: added cancel button with `channel_id` tracking, extracted `channel_dispatch.rs` from `channel.rs`, added retrigger prompt fragment, improved wiki store error handling, switched hardcoded hex colors to semantic tokens (`text-status-success`, etc.). + +### 10. `bc097ea` — "wiki pt.2" `+91 / -2` +Wired wiki tools into branches, workers, and channel dispatch. Added `wiki_enabled` flag to branch/worker prompt templates, `wiki_write: bool` to `WorkerContextMode`, propagated wiki store through all process spawn paths. + +### 11. `855db49` — "wiki" `+2,590 / -13` +Full wiki system: SQLite migration (`wiki_pages`, `wiki_page_versions`), `WikiStore` with tolerant multi-pass text matching for edits, 6 wiki tools (create/edit/read/list/search/history), REST API endpoints, frontend Wiki route with page browser/viewer/create form and wiki-link navigation, TypeScript types. + +### 12. `7b704b4` — "wiki" `+419 / -11` +Wiki design document (398 lines — page types, storage schema, tolerant edit matching, tool definitions, versioning, link syntax, agent access patterns, implementation phases). Added sidebar nav for Wiki, wired RecentActivityCard to real task data, added `.db*` to gitignore. + +### 13. `5cd0398` — "cron outcome delivery rewrite, token usage tracking, dashboard UI, task notifications" `+1,968 / -309` +Replaced CronReplyBuffer with `set_outcome()` tool for clean cron delivery. Token usage tracking with new migration/table and `UsageAccumulator`. Connected TokenUsageCard to real API data. Rewired dashboard cards to SpaceUI primitives. WorkersPanel modal → inline slide-over. Wired `ApiState` through `AgentDeps` for task notification emission. Added token-usage-tracking and slash-commands design docs. + +### 14. `055bdba` — "cron, openai streaming improvements, ui" `+918 / -417` +Wall-clock cron expression support alongside legacy interval mode. OpenAI Responses API SSE streaming in the LLM layer. Created `set_outcome` tool and prompt. Token-usage-tracking design doc. README rewrite. Dashboard cards switched to dark variant. + +### 15. `24b7abd` — "ui" `+146 / -226` +Redesigned ChannelCard from list-style to square aspect-ratio with fade-masked message stream. Replaced inline WorkerBadge/BranchBadge with compact pills. Added PlatformIcon, CircleButton, typing indicator, auto-scroll on send to PortalTimeline. + +### 16. `6118909` — "design docs, new readme and attachments" `+2,401 / -443` +Complete README rewrite. 6 design docs (autonomy, goals, cron-outcome-delivery, skill-authoring, worker-briefing, attachment-portal-and-defaults). File attachment support for portal chat (multipart upload API, drag-and-drop composer, attachment rendering in timeline, metadata in conversation history). + +### 17. `5059a55` — "ui" `+451 / -19` +Global WorkersPanel as a sidebar footer popover with search, history/interactive tabs, per-agent worker queries, live SSE integration, modal detail view. Added PortalHeader component. + +### 18. `9ce5d23` — "notifications" `+1,167 / -293` +Full notification system: SQLite migration, `NotificationStore` with CRUD/filtering, API endpoints, SSE real-time broadcasting, `useNotifications` hook with optimistic dismiss, wired ActionItemsCard to real data, cortex observation notifications on worker failure, task approval notification emission. + +### 19. `aa0f389` — "ui" `+1,692 / -895` +Dashboard route with four cards (ActionItemsCard, TokenUsageCard, RecentActivityCard, GetSpacedriveCard). Extracted AgentSkills into modular components (SkillsSidebar, SkillsDirectory, SkillInspector, BundledSkills, InstalledSkillRow, RegistrySkillRow). + +### 20. `4331139` — "ui" `+1,497 / -1,457` +Decomposed monolithic 1452-line AgentConfig.tsx into modular components: ConfigSidebar, ConfigSectionEditor (routing, tuning, compaction, cortex, coalesce, memory, browser, sandbox, projects), GeneralEditor, IdentityEditor, SaveBar, shared types/constants/utils. Added Config sub-item to sidebar agent nav. + +### 21. `7157fe3` — "settings" `+3,055 / -2,915` +Decomposed monolithic 2900-line Settings page into 12 section components (ApiKeysSection, AppearanceSection, ChangelogSection, ChannelsSection, ChatGptOAuthDialog, ConfigFileSection, OpenCodeSection, ProviderCard, SecretsSection, ServerSection, UpdatesSection, WorkerLogsSection). Replaced ReactFlow Controls in OrgGraph with custom zoom buttons. + +### 22. `5b66aea` — "better workbench" `+715 / -407` +Rewrote Workbench into modular components (WorkbenchSidebar, WorkerColumn, EmptyState, utils). Replaced hardcoded hex colors in OpenCode embed theme with CSS custom property references (`var(--color-*)`) for theme inheritance. + +### 23. `e92674b` — "fixes" `+60 / -17` +Renamed Orchestrate → Workbench (route + sidebar). Fixed OpenCode proxy path prefix stripping. Fixed worker list to resolve project names from global ProjectStore. Added `@layer opencode-portals` CSS to prevent style conflicts. + +### 24. `c6b3489` — "ui" `+2,108 / -2,080` +Decomposed 2074-line TopologyGraph.tsx into modular Org Chart components: OrgGraph, OrgGraphInner, ProfileNode, GroupNode, LinkEdge, EdgeConfigPanel, GroupConfigPanel, AgentEditDialog, HumanEditDialog, buildGraph, constants, handles, storage utils. + +### 25. `b1a6be5` — "color fix" `+5,080 / -2,298` +Foundational SpaceUI component integration across 32 files: converted all UI from inline Tailwind/raw HTML to `@spacedrive/primitives` (Button, Input, Dialog, Badge, Switch, Select, etc.). Replaced hardcoded color classes with semantic design tokens. Added new TopBar component. + +### 26. `8a12e3d` — "progress" `+2,094 / -1,051` +Replaced monolithic WebChatPanel and TopBar with modular Portal system (PortalPanel, PortalTimeline, PortalComposer, PortalHeader, PortalHistoryPopover, PortalActiveWorkers). Added AgentProjects route. Migrated projects from per-agent SQLite DBs to shared instance DB with full dedup migration logic. + +### 27. `2432984` — "ui" `+1,201 / -1,730` +Rewrote task management: deleted 868-line TaskBoard (kanban) and replaced with Linear-style TaskList/TaskDetail/TaskCreateForm. Added GitHub metadata badges (issues/PRs) via TaskUtils.tsx. Rebuilt GlobalTasks with full task CRUD and SSE-driven updates. + +### 28. `b8ceff1` — "Rename @spaceui/* packages to @spacedrive/* (npm org)" `+56 / -56` +Pure package rename across all imports and config: `@spaceui/*` → `@spacedrive/*` in package.json, vite.config.ts, styles.css, bun.lock. + +### 29. `c53a1e3` — "SpaceUI migration: sidebar redesign, global projects, logo detection" `+983 / -641` +Three changes: (1) Sidebar redesign to Spacedrive style — 220px, labeled nav items, section headers, accordion agent sub-nav. (2) Projects made global by dropping `agent_id` from projects table + all Rust store/API/tool updates. (3) Auto project logo detection scanning `.github/`, `public/`, `src-tauri/` with new `GET /projects/{id}/logo` endpoint. + +### 30. `fb5c0f3` — "remove Channel Behaviour config tab" `+3 / -18` +Removed the Channel Behaviour tab from agent config — per-channel response modes in channel settings replaced it. + +### 31. `7e223f4` — "rename Quiet to Observe, remove Listen Only toggle" `+89 / -88` +Renamed `Quiet` response mode to `Observe` across full stack (Rust enums with serde alias, TS types, UI labels, docs). Observe = agent learns from context and captures memories but never responds. + +### 32. `ce2e9d7` — "docs: add Configuring Channels guide" `+141 / -1` +New getting-started doc covering per-channel settings: response modes, model overrides, memory modes, delegation, worker context, slash commands. Added Channel Settings section to core channels doc. + +### 33. `f07cecc` — "fix helper text to include commands as a trigger" `+2 / -2` +Updated two helper text strings to mention commands (not just mentions/replies) trigger responses in MentionOnly mode. + +### 34. `d421b03` — "address review: compaction guard, doc fixes, checkbox accessibility" `+31 / -17` +Compaction check after injecting suppressed MentionOnly messages. Drops history write lock before async calls. Wraps checkbox inputs in `