Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 23 additions & 10 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,24 @@
.idea
.claude

# Markdown that isn't a runtime asset
# Docs
docs
README.md
CONTRIBUTING.md
SECURITY.md
CODE_OF_CONDUCT.md
THIRD_PARTY_NOTICES.md
AGENTS.md
*.md
# Re-include plugin manifests — bundled into the image via Dockerfile and
# read at boot to build the plugin catalog.
!docs/harness-platform
!docs/harness-platform/examples
!docs/harness-platform/examples/*.yaml
# Re-include agent-integration boilerplate — Plugin-Builder (B.1+)
# kopiert die Files zur Codegen-Zeit; Dockerfile mountet das Verzeichnis
# unter /app/boilerplate/agent-integration. Nur das eine Subverzeichnis
# allowen, der Rest von docs/ bleibt excluded.
!docs/harness-platform/boilerplate
!docs/harness-platform/boilerplate/agent-integration
!docs/harness-platform/boilerplate/agent-integration/**

# Dev frontend (separate deploy target with its own Dockerfile under web-dev/)
# Dev frontend (separate deploy target)
web-dev

# Middleware local state / build artefacts
Expand All @@ -26,26 +35,30 @@ middleware/dist
middleware/.memory
middleware/.env
middleware/.env.*
middleware/fly-memory-pull
# Workspace-Pakete: Source bleibt drin (Builder-Stage compiliert frisch),
# aber lokale dist/ + tsbuildinfo werden im Container neu gebaut. Defense
# gegen stale-build-artefacts shipping.
middleware/packages/*/dist
middleware/packages/*/tsconfig.tsbuildinfo
middleware/packages/*/node_modules
middleware/scripts
middleware/test
middleware/*.log
# Re-include the build-asset copier — it runs as part of `npm run build`
# inside the Docker builder stage.
!middleware/scripts/copy-build-assets.mjs
# Re-include the Node-version guard — preinstall hook from
# middleware/package.json that runs before `npm ci` in both stages.
!middleware/scripts/check-node-version.mjs
middleware/test
middleware/*.log

# Optional skills directory zips (when present)
# Legacy Managed-Agent zip bundles in skills/ — we only need the SKILL.md trees
skills/*.zip
skills/.DS_Store
skills/**/.DS_Store

# Standalone configs that aren't used by the runtime
agent-config*.yaml

# OS junk
**/.DS_Store
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
## Test plan
<!-- Checklist of what you actually ran. Be specific:
- [ ] `npm run lint && npm run typecheck && npm run test` in middleware
- [ ] `docker compose up -d` + middleware boot ok
- [ ] `docker compose -f infra/docker-compose.yml up postgres` + 9 migrations
- [ ] manual: ___ -->

## Risk / blast radius
Expand Down
24 changes: 18 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,19 @@ jobs:

- uses: actions/setup-node@v4
with:
node-version: '20'
node-version: '22'
cache: 'npm'
cache-dependency-path: middleware/package-lock.json

- name: npm ci (middleware + workspaces)
run: npm ci --no-audit --no-fund
run: npm ci --include=optional --no-audit --no-fund

# sharp's native binary is platform-specific. package-lock.json was
# generated on macOS (darwin-arm64) and has no `node_modules/@img/
# sharp-linux-x64` entry, so `npm ci` even with --include=optional
# skips it. Force-install the linux-x64 variant for the CI runner.
- name: Install sharp linux-x64 native binary
run: npm install --no-save --no-audit --no-fund --os=linux --cpu=x64 sharp

- name: Build workspace packages (compiled dist/ for cross-package imports)
run: npm run build
Expand All @@ -60,6 +67,11 @@ jobs:
- name: Test (node --test via tsx)
run: npm run test

# Smoke scripts (middleware/scripts/smoke-*.ts) live outside this
# public repo by design — they hit byte5-internal endpoints. The
# privacy-shield v2 smoke that previously ran here has the same
# coverage as the in-suite privacyOutputValidator tests.

# ------------------------------------------------------------------
# Web-dev: Next.js admin UI — vitest + lint + typecheck
# ------------------------------------------------------------------
Expand All @@ -75,12 +87,12 @@ jobs:

- uses: actions/setup-node@v4
with:
node-version: '20'
node-version: '22'
cache: 'npm'
cache-dependency-path: web-ui/package-lock.json

- name: npm ci
run: npm ci --no-audit --no-fund
run: npm ci --include=optional --no-audit --no-fund

- name: Lint
run: npm run lint
Expand Down Expand Up @@ -182,12 +194,12 @@ jobs:

- uses: actions/setup-node@v4
with:
node-version: '20'
node-version: '22'
cache: 'npm'
cache-dependency-path: ${{ matrix.workspace }}/package-lock.json

- name: npm ci
run: npm ci --no-audit --no-fund
run: npm ci --include=optional --no-audit --no-fund

- name: npm audit (--audit-level=high)
run: npm audit --audit-level=high
46 changes: 46 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -79,3 +79,49 @@ tmp/
# OSS-Core export scope (OB-30) and must never be checked in alongside the
# harness platform sources. See marketing-site/docs/00-isolation.md.
marketing-site/

# ─── byte5-specific agent + deployment configs (private) ──────────────────
# These describe a specific byte5 deployment (Fly app, Neon project, kroki
# sidecars, customer-specific agent configs) and must never land in public
# Omadia. Generic equivalents ship with the OSS docs (see docs/).
agent-config.yaml
agent-config-*.yaml
fly.toml
compose.yml
infra/
kroki/
ollama/
middleware/DEPLOY.md
middleware/data/
middleware/.uploaded-packages/

# ─── Internal-only docs (session handoffs, plans, customer evaluations) ───
# Session-context that served its purpose during development and would
# pollute the public docs tree. The harness-platform/ architecture docs
# themselves ARE public — only HANDOFF-* session snapshots are excluded.
docs/plans/
docs/day-one-learnings-*.md
docs/dev-frontend-handoff.md
docs/middleware-agent-handoff.md
docs/northdata-agent-plan.md
docs/softgarden-agent-evaluation.md
docs/harness-platform/HANDOFF-*.md

# ─── Smoke scripts (internal validation harness, not part of public API) ──
# Boot-test scripts that hit live byte5 Fly endpoints / Neon project. The
# OSS audience doesn't have access to those targets; equivalent guidance
# lives in the public test/ directories per package.
middleware/scripts/smoke-*.ts

# ─── Private byte5 plugins (channels + integrations) ──────────────────────
# byte5-specific channel adapters (Teams, Telegram) and SaaS integrations
# (Confluence, MS365, Odoo). Canonical source lives in the separate private
# repo ~/sources/omadia-byte5-plugins (workspace: packages/channel-*,
# packages/integration-*). The copies here are working-dev mirrors and
# must never reach the public Omadia repo. Reconciliation between the two
# locations is a separate cleanup (see HANDOFF when scheduled).
middleware/packages/harness-channel-teams/
middleware/packages/harness-channel-telegram/
middleware/packages/harness-integration-confluence/
middleware/packages/harness-integration-microsoft365/
middleware/packages/harness-integration-odoo/
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ discussions, code reviews, public events held under the project name).

Instances of unacceptable behaviour can be reported privately to:

> `conduct@byte5.de`
> `info@omadia.ai`

Reports are reviewed by the project maintainers and handled with
discretion. We follow the **enforcement guidelines** chapter of the
Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,19 @@ Bootstrap:
```bash
git clone https://github.com/byte5ai/omadia.git
cd omadia
cp middleware/.env.example middleware/.env # set ANTHROPIC_API_KEY
docker compose up -d minio kroki ollama # sidecars (skip middleware/web-ui — those run via npm)
cp infra/.env.example infra/.env # set ANTHROPIC_API_KEY
docker compose -f infra/docker-compose.yml --env-file infra/.env up -d postgres

# Middleware (Express + plugin runtime + builder)
cd middleware
nvm use
npm install
npm run dev # starts on :8080
npm run dev # starts on :3979

# Admin UI (Next.js 15)
cd ../web-ui
npm install
npm run dev # starts on :3000
npm run dev # starts on :3300
```

The middleware re-builds and re-types every workspace package on `npm run
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2026 byte5.ai
Copyright (c) 2026 byte5 GmbH

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
105 changes: 38 additions & 67 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,80 +16,43 @@ model. You bring your own LLM API key, run the stack on a single machine
> is supported but the upgrade path is hand-rolled today; an automated
> migration runner is on the v1.0 roadmap.

> **Heads-up — `main` was force-pushed on 2026-05-12** to purge a documentation
> file that contained internal identifiers. The `v0.1.0` tag is unchanged, but
> if you cloned this repository before that date your next `git pull` will fail
> with `non-fast-forward` / `Updates were rejected`. To recover, discard the
> stale local history and reset to the rewritten remote:
>
> ```bash
> git fetch origin
> git reset --hard origin/main
> ```
>
> If you have local commits on top of the old `main`, cherry-pick them onto the
> new base instead (`git log` on the old SHA is still reachable locally for ~90
> days via the reflog).

## Quickstart
## Quickstart (~60 seconds after the first image pull)

```bash
git clone https://github.com/byte5ai/omadia.git
cd omadia

# 1. Provide an Anthropic API key. The middleware will not boot without one.
# Every other env var has a working default for the docker-compose stack.
cp middleware/.env.example middleware/.env
$EDITOR middleware/.env # set ANTHROPIC_API_KEY=sk-ant-...
# 1. Provide an Anthropic API key. Other env vars have sane local defaults.
cp infra/.env.example infra/.env
$EDITOR infra/.env # set ANTHROPIC_API_KEY=...

# 2. Bring up the full stack. First build pulls ~3 GB of images
# (postgres+pgvector, kroki, minio, ollama, presidio sidecar build).
docker compose up -d --build
# 2. Bring up the stack (postgres + middleware + admin UI).
docker compose -f infra/docker-compose.yml --env-file infra/.env up -d

# 3. Watch it come up — middleware needs ~60-90s on first boot for KG
# migrations + plugin activations + ollama model pulls.
docker compose logs -f middleware

# 4. Open the management UI and complete the first-admin wizard.
open http://localhost:3333 # /setup walks you through
# 3. Open the management UI and complete the first-admin wizard.
open http://localhost:3300 # /setup walks you through
```

The first user-creation flow lands on `/setup`. Once an administrator exists,
`/setup` self-locks (returns `410 Gone`) and the regular `/login` page takes
over.

### Re-running

`docker compose up -d` brings the stack back up; volumes (postgres data,
vault, memory, uploaded plugins) survive. To start completely fresh:
### Optional Compose profiles

```bash
docker compose down -v && docker compose up -d --build
```
# Mermaid / PlantUML / Vega rendering for the diagrams plugin
docker compose -f infra/docker-compose.yml --profile diagrams up -d

> **Heads up — browser localStorage**: chats are cached in the browser
> (offline-friendly). If you've ever used another Omadia instance on the
> same `http://localhost:3333` (e.g. a previous deployment), those cached
> chats will surface in this fresh install too. Browser DevTools →
> Application → Local Storage → `http://localhost:3333` → "Clear All" gives
> you a clean slate. (A first-install detection that does this
> automatically is on the v0.2 roadmap.)

### Service map

| Service | Host port | Purpose |
|---|---|---|
| `web-ui` | `3000` | Admin UI (Next.js) |
| `middleware` | `8080` | Kernel API + plugin runtime |
| `postgres` | `5432` | Postgres + pgvector — knowledge graph / routines / verifier persistence (default user/password/db: `omadia`) |
| `kroki` | `8765` | Diagram rendering (Mermaid, PlantUML, Vega, …) |
| `minio` | `9000` / `9001` | S3-compatible object storage (console: `minioadmin` / `minioadmin`) |
| `ollama` | `11434` | In-tenant embeddings + small NER model (`nomic-embed-text` + `llama3.2:3b`) |
| `presidio` | `5001` | Python NER sidecar for the privacy detector plugin (FastAPI, `~1.5 GB` first build) |

Stop the stack with `docker compose down`; add `-v` to also wipe the
persistent volumes (`middleware-data`, `postgres-data`, `minio-data`,
`ollama-data`).
# In-tenant embeddings via Ollama (no external API required)
docker compose -f infra/docker-compose.yml --profile embeddings up -d

# Presidio NER sidecar for the privacy-proxy detector plugin
docker compose -f infra/docker-compose.yml --profile privacy-presidio up -d

# All optional profiles in one command
docker compose -f infra/docker-compose.yml \
--profile diagrams --profile embeddings --profile privacy-presidio up -d
```

## What's in the box

Expand Down Expand Up @@ -133,9 +96,9 @@ persistent volumes (`middleware-data`, `postgres-data`, `minio-data`,
(Postgres + pgvector) (Ollama / API) (AES-256-GCM file)
```

More detailed walk-throughs of the plugin loading sequence, capability
registry, and the multi-provider authentication layer will be published
alongside the v0.2 release.
A more detailed walk-through of the plugin loading sequence, capability
registry, and the multi-provider authentication layer lives under
[`docs/`](docs/).

## Plugin development

Expand All @@ -156,11 +119,19 @@ the differentiating logic, and verifying with the smoke runner before install.
## Deployment

- **Local / single-tenant** — `docker compose up`, see Quickstart above
- **Bring-your-own** — the runtime is a stock Node service plus the
sidecars in `docker-compose.yaml` (Kroki, MinIO, Ollama). Any host
capable of running Docker works (Kubernetes, ECS, Fly.io, plain VM).
Postgres is optional — without `DATABASE_URL` the kernel uses the
in-memory knowledge graph.
- **Fly.io** — single-app deployment, multi-region supported. The compose
stack and the Fly image are baked from the same `Dockerfile`.
- **Bring-your-own** — the runtime is a stock Node + Postgres app; any host
capable of running both works (Kubernetes, ECS, plain VM).

> **Required production secret.** The shipped image runs with
> `NODE_ENV=production`, which makes `VAULT_KEY` mandatory at boot — without
> it the middleware refuses to start (this is intentional; the dev fallback
> writes the master key into the data volume, which is not safe at rest).
> Generate one with `openssl rand -base64 32` and wire it as a platform
> secret (Fly: `fly secrets set VAULT_KEY=…`) before the first deploy. The
> local Compose stack pins `NODE_ENV=development` so the dev fallback stays
> available for `docker compose up` without configuration.

> **Required production secret.** The shipped image runs with
> `NODE_ENV=production`, which makes `VAULT_KEY` mandatory at boot — without
Expand Down Expand Up @@ -188,7 +159,7 @@ Active development tracks:

## License

[MIT](LICENSE) — Copyright (c) 2026 byte5.ai
[MIT](LICENSE) — Copyright (c) 2026 byte5 GmbH

Third-party dependency licenses and notices are documented in
[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md). The dependency tree is
Expand Down
Loading
Loading