From b7e9121328211f4884b6e380230c57d653325cfe Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 21 Aug 2026 08:48:23 +0200 Subject: [PATCH] docs: make the plugin hub-ready and self-explanatory for operators MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The hub does not store this manifest, it PROJECTS it. `parsePublish` (omadia-hub/lib/manifest.ts:113-231) keeps identity, compat.core, setup.fields, setup.guide, provides/requires/depends_on and permissions, and throws away everything else — including `optional_requires` and every comment in the file. So the four capabilities this plugin is designed to survive the absence of, and the degradation it takes for each, were invisible to anyone deciding whether to install it. manifest.yaml - setup.guide (en + de) now carries what the registry cannot: the two operator grants and how to perform them, the runner backends, the ledger handoff dry-run, and a degradation table for each optional capability. The German guide was three sections behind; it is at parity. - Every setup.fields entry gets label + help as `{ en, de }` maps, the shape core normalises through manifestLocalized.normalizeLocalized (byte5ai/omadia#602). 21 fields, both locales. - compat.core `>=1.0 <2.0` -> `>=1.5 <2.0`. Each of C4/C6/C7/C9/C11 is traced to the core PR that shipped it; C11 is newest, and it landed in @omadia/plugin-api 1.5.0. Core never semver-compares compat_core (manifestLoader.ts:503 and registryClient.ts:415 only carry the string through to the store view), which is precisely why it has to be honest rather than permissive: on a 1.4.0 core `ctx.sql.seedLedger` is undefined and the handoff silently does not happen. - A header comment recording the kept/stripped split with file:line, so the next operator-facing block does not get added somewhere that is discarded at publish time. The SQL grant has no UI. `pluginSqlGrants.ts` says so twice in its own source — nothing in core's `src/` calls `grant()`, and the bundled ramp covers four bundled plugins this one is not among. Documented as the manual INSERT it currently is, rather than left as "grant the SQL permission" with no way to do it. The public-path grant does have a shipped surface and is documented as the endpoint it is. packages/plugin/README.md ships INSIDE the ZIP and still described 0.1.0: "Nothing yet", "no routes", "Permissions: none declared". It is the first thing an operator reads after unzipping. Rewritten for what 0.3.0 does. Also: docs/OPERATOR-GUIDE.md (the runbook the READMEs link to), a root README rewritten from "Status: P0 - scaffold" to the current release with an architecture diagram, CHANGELOG entries for 0.1.0 and 0.2.0 reconstructed from the history plus an Unreleased section, and a CONTRIBUTING section on keeping the four operator-facing documents in step. Verified: typecheck, build, 1269 tests and `npm run package` all pass; version unchanged at 0.3.0 so the drift guard still agrees. The cut ZIP was run through the hub's own parsePublish to confirm every gate passes and to check what the index actually ends up holding. --- CHANGELOG.md | 164 ++++++++++- CONTRIBUTING.md | 29 ++ README.md | 284 ++++++++++++------- docs/OPERATOR-GUIDE.md | 496 ++++++++++++++++++++++++++++++++++ packages/plugin/README.md | 106 ++++++-- packages/plugin/manifest.yaml | 489 +++++++++++++++++++++++++++++---- 6 files changed, 1396 insertions(+), 172 deletions(-) create mode 100644 docs/OPERATOR-GUIDE.md diff --git a/CHANGELOG.md b/CHANGELOG.md index fb404f8..94dde2d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,11 @@ All notable changes to `@omadia/dev-platform`. The version that matters is `packages/plugin/manifest.yaml` — the hub reads the manifest, and `npm run package` aborts if it disagrees with `packages/plugin/package.json`. +Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This +project versions the ARTIFACT, not the repository: a release is a ZIP an +operator can install, so anything that does not change the ZIP does not get a +version. + ## 0.3.1 — 2026-08-21 Declares the ledger handoff in the manifest so the KERNEL performs it, closing @@ -49,6 +54,51 @@ key and keep the previous behaviour. filename present in `migrations/`, and a `ledger` that agrees with the manifest's so core does not warn about a split-brain dry run. +### Documentation + +The hub does not store `manifest.yaml`, it PROJECTS it: `parsePublish` +(`omadia-hub/lib/manifest.ts:113-231`) keeps a fixed set of fields and discards +the rest — including `optional_requires` and every comment in the file. So the +four capabilities this plugin is written to survive the absence of, and the +degradation it takes for each, were invisible to anyone deciding whether to +install it. + +- **`setup.guide` (en + de)** now carries what the registry cannot: both + operator grants and how to perform each, the Fly-versus-Docker runner choice, + the ledger-handoff dry run, and a degradation table for every + `optional_requires` capability. The German guide had fallen three sections + behind English and is back at parity. +- **Every `setup.fields` entry carries `label` and `help` as `{ en, de }` maps** + — the shape core normalises through `manifestLocalized.normalizeLocalized` + (byte5ai/omadia#602). 21 fields, both locales. +- **`identity.description`** rewritten for the storefront card, and + `categories` extended with `automation` and `github`; the hub builds its + category selector from the union of published plugins' categories, so these + are browse paths rather than tags. +- **A header comment recording the kept/stripped split** with `omadia-hub` + file:line references, so the next operator-facing block does not get added + somewhere that is discarded at publish time. +- **`docs/OPERATOR-GUIDE.md`** (new) — install, the two operator grants, + credentials, runner backends, the migration handoff, uninstall/purge, + troubleshooting and known issues, in one place. +- **`packages/plugin/README.md`**, which ships INSIDE the ZIP, still described + 0.1.0 ("Nothing yet", "no routes", "Permissions: none declared"). It is the + first thing an operator reads after unzipping; rewritten for what this + release actually does. +- **`README.md`** rewritten from "Status: P0 — scaffold" to the current + release, with an architecture diagram; **`CONTRIBUTING.md`** gains a section + on keeping the four operator-facing documents in step. + +### Changed (documentation pass) + +- **`compat.core` `>=1.0 <2.0` → `>=1.5 <2.0`.** `@omadia/plugin-api` 1.5.0 is + the first core carrying C4, C6, C7, C9 and C11 — the surface this plugin + cannot activate without. Core never semver-compares `compat_core` + (`manifestLoader.ts:503`, `registryClient.ts:415` only carry the string + through to the store view), which is precisely why it has to be honest rather + than permissive. Note this release's own `permissions.sql.handoff` wants + **1.6.0** and degrades cleanly below it, so it does not raise the floor. + ## 0.3.0 — 2026-08-21 Verification pass against omadia core `origin/main` (`9feb3ad3`), the first core @@ -105,6 +155,114 @@ that carries the whole C9–C11 contract surface: `optional_requires`, ## 0.2.0 — 2026-08-20 -Extraction of the Dev Platform out of omadia core into this repository (epic -byte5ai/omadia#470, P1–P5 plus C11). See -`docs/ACCEPTANCE-RUN-2026-08-20.md` for the acceptance run that closed it. +The extraction itself. The Dev Platform moved out of omadia core into this +repository — plugin tree, operator SPA, runner sidecars and the migration +handoff (epic byte5ai/omadia#470, phases P2–P5 plus C11). Core stayed live and +unchanged throughout; this release is the copy that proves the destination +works, not the deletion of the origin. See `docs/ACCEPTANCE-RUN-2026-08-20.md` +for the run that closed it: **71 PASS / 0 FAIL / 2 BLOCKED** against +`main`+C6+C7. + +### Added + +- **The middleware tree (P3).** 60 of 62 dev-platform source files, 52 of 58 + suites and all nine migrations, ported against the C6 and C7 extension + points: dev jobs, the analyze/plan/implement/review pipeline, gate and diff + policy, the LLM budget proxy, HTTP routers, chat tools and background + workers. +- **The operator SPA (P2).** The 28 files under core's + `web-ui/app/admin/dev-platform/**` rebuilt as a standalone Vite/React 19 + bundle in `packages/ui`, shipped inside the ZIP as `ui/` and served by core at + `/p//ui/`. Four screens: hub, job detail, repo detail, add-repo + wizard. `next-intl` became a 300-key-per-locale `src/lib/i18n.tsx`; + `next/navigation` became a hash router, because a path route would 404 on + reload against a static mount and a fragment never reaches the server. + `scripts/check-ui-vocabulary.mjs` gates the build against the 690 classes core + actually serves — all 334 Tailwind arbitrary values are gone, since the ZIP + allowlist rejects that shape and a class core never saw renders unstyled + rather than erroring. +- **Runner sidecars and the supply chain (P4).** `sidecars/dev-runner`, + `sidecars/dev-runner-daemon` (control plane + egress proxy), + `sidecars/dev-dind`, the protocol shim in `packages/runner-shim`, the compose + overlay and the operator transcript CLI. Core's other four sidecars + (pii-detector, privacy-detector-presidio, skillspector, updater) are core's + and stayed there. +- **Migration handoff (C11).** `ctx.sql.seedLedger()` records core's already-run + slots 0022–0030 as applied in this plugin's own ledger instead of re-applying + all nine — but never on core's word alone. Each of the nine files carries a + WITNESS in `src/ledgerHandoff.ts` proving the schema object it creates is + actually present. `handoff-plan.json` ships in the ZIP so an operator can + dry-run the handoff against production **before** installing. +- `scripts/acceptance-local.mjs` — an idempotent acceptance harness, 39 handler + probes. + +### Fixed + +- **`activate()` double-registered every chat tool**, via `register` and + `registerHandler` — two doors into one name-keyed kernel map that both throw + on a duplicate. The root cause was a hand-narrowed context type missing the + handler parameter, which made the correct call unwritable. +- **The ZIP shipped 145 sourcemaps** (1,736,840 bytes). Archive 965,433 → + 517,009 bytes. +- **The first ZIP was cut without `migrations/`** — it installed clean and then + failed at activation with the plugin's nine tables absent. `migrations/` is + now a required directory in `build-zip.mjs`, alongside `dist` and `ui`. +- **Parallel suites sharing one database claimed each other's jobs**, reporting + defects that existed in neither. With a database configured the runner now + switches to `--test-concurrency=1`; the pure run stays parallel. +- Two real defects surfaced by turning on the daemon's never-wired `typecheck`: + `proxyClient.mjs` passed its abort callback as `withDeadline`'s third + argument (`label`), so the abort never fired and a hung egress proxy leaked + its fetch until process exit; and `buildEgressProxyClient` passed `tokens[0]` + through unchecked, so an empty `DEV_RUNNER_DAEMON_TOKEN` authenticated the + daemon to its own proxy as `Bearer undefined` — every job then saw 407 on + every request, which inside a runner looks like a total network outage. Now a + boot refusal. + +### Reported upstream + +Six core gaps found by the acceptance run and filed against omadia: +byte5ai/omadia#794, #795, #796, #797, #798, plus C12's static `publicPaths` +literals. None was plugin-side and none blocked staging. + +## 0.1.0 — 2026-08-20 + +Scaffold. A repository that builds, tests and cuts a real, installable — and +deliberately empty — plugin artifact. **No dev-platform code had moved yet.** + +That order was the point. A repository that cannot cut its own release artifact +ends up publishing from whatever tree someone last built in, a failure this +plugin set had already lived through when a package kept being released from a +frozen monorepo branch. The pipeline got proven while the payload was empty and +a mistake cost nothing. + +### Added + +- npm workspaces layout: `packages/plugin-api` (types-only), + `packages/plugin`, `packages/ui`. Conventions follow + `omadia-integration-odoo`: ESM, `tsc` build, the `@omadia/plugin-api` sibling + linked by `file:` with a `"*"` peer, and a `build-zip.mjs` that emits a flat + `out/-.zip`. +- `identity.kind: "extension"`, decided rather than defaulted. + `toolPluginRuntime` activates only `tool`/`extension`/`integration`, and the + agent-builder treats `kind === "integration"` as "an external system an agent + may read from" — which the Dev Platform is not. There is no `platform` kind. +- Packaging guards, all mutation-checked rather than merely green: version + drift, identity drift, a non-activatable kind, an uncommented permissions + block, a missing locale and a missing build artifact each fail the build or + the suite. +- CI that checks out this repo and `byte5ai/omadia` side by side, builds only + `middleware/packages/plugin-api` from the sibling, then runs `npm ci`, + typecheck, build, test and package, and uploads the ZIP. + +### Fixed + +- The first CI run went red on two traps that are invisible locally. + `middleware/package.json` is an npm workspace root, so `npm install` run from + inside `packages/plugin-api` is hijacked to the root and leaves the package's + own `node_modules` empty — the local clean-room check had missed it by + copying the package into a bare temp dir where no parent workspace exists. + And `npx tsc` does not fail when TypeScript is absent: it downloads an + unrelated abandoned `tsc` package and exits 1 with a misleading message. The + compiler is now invoked by explicit path, and both notes are carried in + README and CONTRIBUTING where a contributor hits the same traps. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 80a2369..751c57e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -104,6 +104,35 @@ inside is byte-identical, and `test/migrationSql.test.ts` proves it against `client.query`, not the file, so an escaping bug cannot hide behind a symmetrical un-escape in the test. +## Documentation + +Four documents are operator-facing, and two of them ship to people who never see +this repository. Keep them in step: + +| File | Audience | Ships in the ZIP | +|---|---|---| +| `packages/plugin/manifest.yaml` → `setup.guide` | Whoever browses the hub or runs the install wizard | **yes**, and it is the only operator text the hub renders | +| `packages/plugin/README.md` | Whoever unzips the artifact | **yes** | +| `docs/OPERATOR-GUIDE.md` | Whoever runs it in production | no — linked from both | +| `README.md` | Whoever builds or contributes | no | + +Two rules that are easy to get wrong: + +- **The hub strips every manifest block it does not name**, including + `optional_requires` and *all comments*. Anything an operator must know before + installing has to live in `identity.description`, `setup.fields` or + `setup.guide` — see the header comment in `manifest.yaml` for the kept/stripped + list with `omadia-hub` file:line references. +- **`setup.fields` `label` and `help` are `{ en, de }` maps.** Core normalises + them through `manifestLocalized.normalizeLocalized` (byte5ai/omadia#602). A + bare string is tolerated and read as English; adding a field with an + English-only label is how the German install wizard ends up half-translated. + +Changing anything an operator does — a new permission, a new required grant, a +changed default, a capability moving between `requires` and `optional_requires` +— means updating `setup.guide` in **both locales** and the operator guide in the +same PR. + ## Conventions - **Issues and PRs in English.** Code comments too. diff --git a/README.md b/README.md index 2be5b54..7a0905e 100644 --- a/README.md +++ b/README.md @@ -2,47 +2,98 @@ The **Omadia Dev Platform**, packaged as an installable plugin. -The Dev Platform runs autonomous coding jobs against a repository: it provisions -a runner, drives an LLM through an analyze/plan/implement/review pipeline, and -opens a pull request. It ships its own HTTP routes, chat tools, background -loops, database migrations and UI. +An agent clones one of your GitHub repositories into an isolated runner, works a +job through an analyze → plan → implement → review pipeline, and opens a pull +request. Human approval gates, a diff policy, a per-job cost budget and a +default-deny egress proxy bound what it can do at every step. -Today all of that lives inside omadia core. This repository is where it moves -to, under [epic byte5ai/omadia#470][epic]. +Extracted from omadia core under [epic byte5ai/omadia#470][epic]. Current +release **0.3.1**. [epic]: https://github.com/byte5ai/omadia/issues/470 -## Status: P0 — scaffold - -**Nothing has moved yet.** What is here is a repository that builds, tests and -cuts a real, installable, empty plugin artifact. +> **Operators start here:** [`docs/OPERATOR-GUIDE.md`](./docs/OPERATOR-GUIDE.md) +> — install, the two required grants, credentials, runner backends, migration +> handoff, uninstall and troubleshooting. The rest of this file is about +> building the thing. + +## Why a plugin + +The Dev Platform was ~49,000 lines inside core: its own routes, tables, UI, +sidecars and background workers, all coupled to a host that did not need any of +it. As a plugin it installs on demand, owns its own schema, declares the +permissions it needs, and can be removed without leaving core carrying its +weight. Core keeps the extension points; the payload lives here. + +## Architecture + +```mermaid +flowchart TB + subgraph host["omadia core (host process)"] + kernel["Plugin kernel
capability resolver · route registry · vault"] + pool[("graphPool
Postgres")] + ui_host["web-ui
/plugin-ui/<id>"] + end + + subgraph plugin["@omadia/dev-platform (this repo)"] + activate["activate()
seed ledger → migrate → register"] + routes["HTTP routes
admin · runner · webhooks"] + tools["chat tools · background workers"] + spa["operator SPA (ui/)"] + migs["migrations/ → 9 tables"] + end + + subgraph runners["Runner backend (one per job, ephemeral)"] + daemon["dev-runner-daemon
control plane"] + proxy["egress proxy
default-deny"] + job["runner + shim
agent-written code"] + end + + kernel -->|"ctx.services / ctx.sql / ctx.routes"| activate + activate --> routes & tools & migs + migs --> pool + kernel --> pool + ui_host -->|"serves ui/ at /p/<id>/ui/"| spa + + routes -->|"provision job"| daemon + daemon --> job + job -->|"all egress"| proxy + proxy -->|"LLM · git · registries"| net(["internet"]) + job -.->|"phone home: bearer token
/api/v1/dev-runner"| routes + gh(["GitHub"]) -.->|"webhook: HMAC
/api/webhooks/github"| routes + + classDef c fill:#0f172a,stroke:#38bdf8,color:#e2e8f0 + classDef p fill:#0f172a,stroke:#a78bfa,color:#e2e8f0 + classDef r fill:#0f172a,stroke:#fbbf24,color:#e2e8f0 + class kernel,pool,ui_host c + class activate,routes,tools,spa,migs p + class daemon,proxy,job r +``` -That order is deliberate. A repository that cannot cut its own release artifact -ends up publishing from whatever tree someone last built in — a failure this -plugin set has already lived through, when a package kept being released from a -frozen monorepo branch. So the pipeline gets proven while the payload is empty -and a mistake costs nothing. +Three properties are load-bearing: -| Phase | What arrives | Where | -|---|---|---| -| **P0** | This scaffold — repo, build, tests, ZIP, CI | ✅ here | -| **P1** | `@omadia/dev-platform-plugin-api` published | `packages/plugin-api` | -| **P2** | The SPA — 26 files ported to Vite/React | `packages/ui` | -| **P3** | The middleware tree: `src/devplatform/**`, routers, migrations, config | `packages/plugin` | -| **P4** | Runner sidecars, the protocol shim, supply chain | new | -| **P5** | Hub publish + the staging proof gate | — | - -Phases **C10–C13** then delete ~49k LOC from core and hand the migrations over. -The full sequence, the capability gaps and the blocking decisions are in -`specs/470-dev-platform-plugin/` in the omadia repo. +- **The middleware never holds docker credentials.** Only the runner daemon + does, and the compose topology test asserts the middleware has no socket, no + `DOCKER_HOST` and no route to the engine. +- **The runner reaches the internet only through the egress proxy**, which is + default-deny. +- **No long-lived credential enters a runner.** With a GitHub App the runner + gets a freshly minted, single-repo, read-only token that is revoked when the + job ends. ## Layout ``` packages/ - plugin-api/ @omadia/dev-platform-plugin-api 1.0.0 types-only DevJob* contract - plugin/ @omadia/dev-platform 0.1.0 the plugin itself - ui/ placeholder — P2 + plugin-api/ @omadia/dev-platform-plugin-api types-only DevJob* contract + plugin/ @omadia/dev-platform 0.3.1 the plugin itself + ui/ Vite/React 19 operator SPA + runner-shim/ @omadia/dev-runner-shim the in-runner protocol shim +sidecars/ + dev-runner/ image that runs exactly one job + dev-runner-daemon/ control plane + egress proxy + dev-dind/ the one privileged service +docs/ operator guide, secrets, supply chain, acceptance runs ``` `packages/plugin` is what installs into an Omadia host. Its `manifest.yaml` is @@ -51,14 +102,14 @@ in that file for why not `integration` or `tool`. ## Building it -Node 22 (`nvm use 22.22.3`). This repo is an npm workspaces monorepo. +Node 22 (`nvm use 22.22.3`). npm workspaces monorepo. ```bash npm install npm run typecheck npm run build npm test -npm run package -w packages/plugin # → packages/plugin/out/omadia-dev-platform-0.1.0.zip +npm run package -w packages/plugin # → packages/plugin/out/omadia-dev-platform-0.3.1.zip ``` `npm run package` refuses to build if `identity.version` in `manifest.yaml` and @@ -68,20 +119,17 @@ drift, the published artifact carries a version that maps to no commit, or an upgrade installs a second plugin beside the first instead of replacing it. Bump both, together. -## The sibling-checkout dependency +The ZIP is flat — `manifest.yaml`, `package.json`, `dist/`, `migrations/`, +`ui/`, `handoff-plan.json`, `README.md` and `LICENSE` at the archive root, no +wrapping directory. `dist`, `migrations` and `ui` are **required** directories +and `handoff-plan.json` a required file: each was once omitted, and each +produced an artifact that installed cleanly and then failed at activation. -The plugin types itself against `@omadia/plugin-api`, which is a **private -workspace package inside omadia core**. It is on no registry. This repo links it -from a sibling checkout, exactly as the other byte5 plugin repos do: - -```jsonc -// package.json (root) -"devDependencies": { - "@omadia/plugin-api": "file:../odoo-bot/middleware/packages/plugin-api" -} -``` +## The sibling-checkout dependency -So the layout on disk must be: +The plugin types itself against `@omadia/plugin-api`, a **private workspace +package inside omadia core**. It is on no registry, so this repo links it from a +sibling checkout, exactly as the other byte5 plugin repos do: ``` ~/sources/ @@ -93,7 +141,7 @@ Two consequences worth knowing before the first confusing error: 1. **The sibling must be built.** `middleware/packages/plugin-api/dist/` is gitignored in core, and `tsconfig.json` here resolves the package through its - emitted `.d.ts`. Build it once: + emitted `.d.ts`: ```bash cd ../odoo-bot/middleware @@ -105,69 +153,119 @@ Two consequences worth knowing before the first confusing error: inside an npm **workspace root**, so an install started there is hijacked to the root and leaves the package's own `node_modules` empty. And invoke the compiler by path — a bare `npx tsc` does not fail when TypeScript is missing, - it downloads an unrelated abandoned `tsc` package instead. Both of these cost - a red CI run already; the workflow carries the same notes. -2. **The dependency is types-only.** Every import of it is `import type` and - vanishes from the emitted JavaScript, so the shipped ZIP has no runtime - dependency on core's package at all. `npm run package` strips - `devDependencies` for exactly this reason: those `file:` paths describe one - machine's directory layout and must never reach a published artifact. + it downloads an unrelated abandoned `tsc` package instead. Both cost a red CI + run already. +2. **The dependency is types-only.** Every import is `import type` and vanishes + from the emitted JavaScript, so the shipped ZIP has no runtime dependency on + core's package. `npm run package` strips `devDependencies` for exactly this + reason: those `file:` paths describe one machine's directory layout and must + never reach a published artifact. -CI reproduces this by checking out both repositories side by side — see -`.github/workflows/ci.yml`. +To build against an unmerged core branch, use `OMADIA_CORE_DIR` with +`npm run link:core` rather than editing the committed `file:` path — see +[CONTRIBUTING.md](./CONTRIBUTING.md). CI reproduces the layout by checking out +both repositories side by side. -Alternatives to the `file:` sibling (a vendored `.d.ts`, or a git dependency on -a tag) were considered and are still open; the sibling checkout is the pattern -already proven in production across six byte5 plugins. +## Testing -## Installing it +```bash +npm test +``` -The artifact is a flat ZIP — `manifest.yaml`, `package.json`, `dist/` and -`LICENSE` at the archive root, no wrapping directory. +About a third of the suite needs a real Postgres and **skips loudly** without one +— a skipped suite must never read as a passing one: -- **Via the hub:** `hub.omadia.ai` serves the registry that Omadia hosts install - from. Publishing is a `POST` of the ZIP to `/api/publish`. Before publishing - anything, read the current `registry/index.json` — the hub has been ahead of a - repo's `main` before, and a higher version number is not proof of newer - content. -- **Via direct upload:** an operator can upload the ZIP in the Omadia admin UI. +```bash +docker run -d --name omadia-devplatform-pgtest -p 55438:5432 \ + -e POSTGRES_USER=test -e POSTGRES_PASSWORD=test -e POSTGRES_DB=test \ + pgvector/pgvector:pg16 -Installing this P0 release is safe and reversible: it registers no routes, no -tools and no migrations, so uninstalling leaves nothing behind. +export GRAPH_PG_TEST_URL=postgres://test:test@127.0.0.1:55438/test +export OMADIA_CORE_DIR=../odoo-bot # core's base migrations 0001-0021 +npm test +``` -## Open questions +The plugin's schema builds on core's, so the bootstrap applies core's base +migrations first and then this package's nine — from the **shipped `.js` +artifacts**, so a pg suite exercises what the ZIP contains rather than the +`.sql` they were generated from. -**Should `@omadia/dev-platform-plugin-api` exist at all?** Two documents in the -epic disagree, and this is worth settling before anything publishes: +There is also an end-to-end acceptance harness that drives a live instance: -- `middleware/src/devplatform/devJobTypes.ts` (in core, current `main`) states in - its header that these types should stay **core-local** and travel with the - dev-platform tree — "deliberately NOT a new published package, which would be - the same speculative generality the accessor was." -- `specs/470-dev-platform-plugin/implementation.md`, phase row **P0–P1**, says to - publish `@omadia/dev-platform-plugin-api`. +```bash +BASE_URL=http://localhost:3000 DATABASE_URL=… node scripts/acceptance-local.mjs +``` -The package is scaffolded here because P0 asked for it. If the core-local -reading wins, delete `packages/plugin-api` and fold the types into -`packages/plugin/src/` — cheap now, expensive after a publish. +It is idempotent and its exit code equals the FAIL count. Recorded runs: -Related: `ctx.devJobs` and two of its six types (`DevJobCreateRequest`, -`DevJobsAccessor`) were **deleted** from core, not moved — zero providers, zero -consumers, threw on every call. They are deliberately not resurrected here. See -the header of `packages/plugin-api/src/index.ts`. +| Run | Against | Outcome | +|---|---|---| +| [2026-08-20](./docs/ACCEPTANCE-RUN-2026-08-20.md) | 0.2.0, patched core | 71 PASS / 0 FAIL / 2 BLOCKED | +| [2026-08-21](./docs/ACCEPTANCE-RUN-2026-08-21.md) | 0.3.0, plain `origin/main` | 38/33/2 on `main`; **71/0/2** with C12 — all 33 traced to one core residue, since merged as [#807][c12] | -Still undecided upstream, and blocking later phases: +[c12]: https://github.com/byte5ai/omadia/pull/807 -- **H3, the chat card.** Core's chat page hardcodes `tool.name === - 'dev_job_start'` and renders a compiled React card. An iframe per tool call is - not acceptable; the choice is a declarative card schema or an accepted - degradation to a plain tool row. Needed before P4. -- **G7 fallback.** If the plugin asset pipeline (C8) proves too costly, the - fallback is an npm-published UI package that `web-ui` optionally installs. +## Installing it + +- **Via the hub:** `hub.omadia.ai` serves the registry Omadia hosts install + from. Publishing is a `POST` of the ZIP to `/api/publish`. Before publishing, + read the live `registry/index.json` — the hub has been ahead of a repo's + `main` before, and a higher version number is not proof of newer content. +- **Via direct upload:** upload the ZIP in the Omadia admin UI. + +**Two grants are required after install** and neither is implied by installing +— the SQL permission (which core has no UI for yet) and the public-path +consents. Both are in +[Operator Guide §4](./docs/OPERATOR-GUIDE.md#4-the-two-operator-grants). + +**Never re-upload a changed ZIP under the same version.** The already-`import()`ed +module is reused from Node's ESM cache and the old code keeps serving until the +middleware restarts. Bump the version instead. + +## Supply chain -## Contributing +The runner image is `ghcr.io/byte5ai/omadia-dev-runner`, published only on a tag +push, signed keyless over the immutable digest, and verified at daemon boot. +During the transition from core's publisher the identity regexp accepts either +signer — anchored at both ends, because cosign matches unanchored. Details and +the narrowing step: [`docs/SUPPLY_CHAIN.md`](./docs/SUPPLY_CHAIN.md). -See [CONTRIBUTING.md](./CONTRIBUTING.md). +## Documentation + +| Document | For | +|---|---| +| [`docs/OPERATOR-GUIDE.md`](./docs/OPERATOR-GUIDE.md) | Installing and running it | +| [`docs/SECRETS.md`](./docs/SECRETS.md) | Every credential and how to rotate it | +| [`docs/SUPPLY_CHAIN.md`](./docs/SUPPLY_CHAIN.md) | Image signing, SBOM, verification | +| [`packages/plugin/SEAMS.md`](./packages/plugin/SEAMS.md) | Every core seam and its degradation | +| [`CHANGELOG.md`](./CHANGELOG.md) | What changed, per release | +| [`CONTRIBUTING.md`](./CONTRIBUTING.md) | Setup, migrations, conventions | + +## Open questions + +**Should `@omadia/dev-platform-plugin-api` exist at all?** Two documents in the +epic disagree, and it is worth settling before anything publishes. +`middleware/src/devplatform/devJobTypes.ts` in core states these types should +stay **core-local**; `specs/470-dev-platform-plugin/implementation.md` phase row +P0–P1 says to publish the package. It is scaffolded here because P0 asked for +it. If the core-local reading wins, fold the types into `packages/plugin/src/` — +cheap now, expensive after a publish. + +Still undecided upstream: + +- **H3, the chat card.** Core's chat page hardcodes `tool.name === + 'dev_job_start'` and renders a compiled React card. The choice is a + declarative card schema or an accepted degradation to a plain tool row. +- **G7 on older cores.** Fixed in 0.3.1 by declaring + `permissions.sql.handoff`, which makes the kernel run the ledger handoff + ahead of its own migration runner — but that key needs `@omadia/plugin-api` + 1.6.0. Below it the key is ignored, the pre-activate migration run still + pre-empts the handoff, and the `skippedNoWitness` alarm never fires. On such a + core, run the dry-run CLI by hand + ([Operator Guide §7](./docs/OPERATOR-GUIDE.md#7-migration-handoff)). +- **Grant lifecycle.** `plugin_sql_grants` and `plugin_public_path_grants` rows + are not revoked on uninstall. Orphaned rather than a live hole — the runtime + stops honouring them — but the intended lifecycle is still open. ## License diff --git a/docs/OPERATOR-GUIDE.md b/docs/OPERATOR-GUIDE.md new file mode 100644 index 0000000..623555e --- /dev/null +++ b/docs/OPERATOR-GUIDE.md @@ -0,0 +1,496 @@ +# Operator Guide — `@omadia/dev-platform` + +Everything needed to install, consent to, credential, run, upgrade and remove +the Dev Platform. The [README](../README.md) is the project overview; this is +the runbook. + +- [1. What you are installing](#1-what-you-are-installing) +- [2. Prerequisites](#2-prerequisites) +- [3. Install](#3-install) +- [4. The two operator grants](#4-the-two-operator-grants) +- [5. Credentials — you enter these yourself](#5-credentials--you-enter-these-yourself) +- [6. Choosing a runner backend](#6-choosing-a-runner-backend) +- [7. Migration handoff](#7-migration-handoff) +- [8. Optional capabilities and what you lose](#8-optional-capabilities-and-what-you-lose) +- [9. Supply chain](#9-supply-chain) +- [10. Uninstall and purge](#10-uninstall-and-purge) +- [11. Troubleshooting](#11-troubleshooting) +- [12. Known open issues](#12-known-open-issues) + +--- + +## 1. What you are installing + +An agent clones one of your repositories into an isolated runner, works a job +through an analyze → plan → implement → review pipeline, and opens a pull +request. Human approval gates, a diff policy, a per-job cost budget and a +default-deny egress proxy bound what it can do. + +The plugin contributes nine database tables it migrates itself, three chat +tools, three background workers, an operator SPA, and HTTP routes — three of +which are served without a kernel session and therefore need your explicit +consent (§4). + +## 2. Prerequisites + +| Requirement | Why | +|---|---| +| **omadia core ≥ 1.5** (`@omadia/plugin-api` 1.5.0) | The floor: `ctx.sql.seedLedger` (the migration handoff) arrived there. Below it the handoff silently does not happen. **1.6.0 is recommended** — it honours `permissions.sql.handoff`, which is what makes the handoff run *before* core's own migration runner (G7, §12). | +| **Postgres-backed knowledge graph** | The job, repo, gate and artifact tables live in `graphPool`. With an in-memory graph the plugin **refuses to activate** rather than pretending to work — set `DATABASE_URL` and install the Neon knowledge-graph plugin first. | +| **A runner backend** | Fly Machines or local Docker. See §6. | +| **An LLM provider key** | The proxy forwards to it. Also required for core's own orchestrator to publish `chatAgent@1`. | + +`graphPool@1` is the plugin's only hard capability requirement. If core cannot +provide it, install is refused with HTTP 409 `install.missing_capability`. + +## 3. Install + +**From the hub.** In your omadia instance, open **Admin → Registries**, add +`https://hub.omadia.ai`, then install **Dev Platform** from +**Admin → Plugins → Store**. + +**From a ZIP.** Build it yourself and upload in the admin UI: + +```bash +npm ci && npm run build +npm run package -w packages/plugin +# → packages/plugin/out/omadia-dev-platform-.zip +``` + +The artifact is flat — `manifest.yaml`, `package.json`, `dist/`, `migrations/`, +`ui/`, `handoff-plan.json` and `LICENSE` at the archive root, no wrapping +directory. + +> **Never re-upload a changed ZIP under the same version.** The previously +> `import()`ed module is reused from Node's ESM cache, so the old code keeps +> serving until you restart the middleware. Bump the version instead. This cost +> real time during the acceptance run; it will cost an operator more. + +## 4. The two operator grants + +The manifest *asks*; a human has to *agree*. Neither grant is implied by +installing. + +### 4.1 The SQL permission (`permissions.sql`) — no UI yet + +Core provisions `ctx.sql` only when a row exists in `plugin_sql_grants` whose +`ledger` matches the manifest exactly. + +> **Core ships no surface for this.** Not a UI, not an API route — +> `middleware/src/platform/pluginSqlGrants.ts` says so twice in its own source: +> *"`plugin_sql_grants` still has no shipped grant surface — nothing in `src/` +> calls `grant()`."* There is a hardcoded ramp for four **bundled** core +> plugins; `@omadia/dev-platform` is installed, not bundled, so it is not on it +> and could not be. + +Until core ships one, insert the row by hand: + +```sql +INSERT INTO plugin_sql_grants (plugin_id, ledger, granted_by) +VALUES ('@omadia/dev-platform', + 'plg_omadia_dev_platform_migrations', + 'you@example.com'); +``` + +Then restart the middleware. The ledger name must match +`permissions.sql.ledger` character for character — core compares the two and +treats any difference as no grant at all. + +The name is not arbitrary: core derives `plg__` from the id +*it* knows and rejects anything outside it. `plg_` is kernel-reserved so no core +table can live there, and the folded id means one plugin cannot nominate +another's ledger and forge its migration history. + +### 4.2 The public-path grants (`permissions.public_paths`) + +Three prefixes are served without a kernel session, because the caller has none +to present. Each is still authenticated: + +| Prefix | Authenticated by | +|---|---| +| `/api/v1/dev-runner` | A per-job bearer token, verified inside the router. | +| `/api/webhooks/github` | HMAC-SHA256 over the raw request body. | +| `/api/v1/dev-platform` | The GitHub App manifest-conversion callback, bound to a kernel-signed, plugin-audience state token. | + +This grant **does** have a shipped surface, behind operator auth: + +```bash +# What the plugin asks for, and what you have already granted +curl -X GET "$BASE/api/v1/admin/runtime/installed/@omadia%2Fdev-platform/public-paths" + +# Grant. This is the COMPLETE set — omitting a prefix revokes it. +curl -X PUT "$BASE/api/v1/admin/runtime/installed/@omadia%2Fdev-platform/public-paths" \ + -H 'Content-Type: application/json' \ + -d '{"paths":["/api/v1/dev-runner","/api/webhooks/github","/api/v1/dev-platform"]}' +``` + +Consent cannot exceed the declaration — core rejects with +`runtime.public_path_not_declared` any prefix this manifest does not request, so +the consent endpoint can never itself be used to make an arbitrary URL public. + +Without these grants: the runner cannot phone home, webhooks 401, and GitHub App +onboarding cannot complete. + +## 5. Credentials — you enter these yourself + +**Nothing is migrated for you, and that is deliberate.** The plugin stores +credentials in its own vault namespace, so after installing — and after any +reinstall — you re-connect each repository by hand. + +A one-time core migration hook was rejected because it would move credentials +silently. The right moment to notice that a GitHub App private key has entered a +plugin's namespace is *while it is happening*, not during an incident. + +What the plugin holds: + +| Secret | Written when | Used for | +|---|---|---| +| GitHub App private key (PEM) | You register or bind a GitHub App | Minting short-lived installation tokens | +| GitHub App client secret | App registration | The OAuth leg of App setup | +| GitHub App webhook secret | App registration | HMAC over the raw webhook body | +| Per-repo clone token (PAT) | You connect a repo with a PAT | Read-only clone inside the runner | +| Per-repo device-flow token | You complete the device flow | Same | + +**Steps.** Open **Dev Platform → Repositories** and connect a credential per +repo: + +- **GitHub App — recommended.** The runner receives a freshly minted, + single-repository, read-only installation token that is revoked when the job + ends. No long-lived credential ever reaches the runner. +- **PAT** — scope it to the one repository. +- **Device flow** — currently **dormant**: `activate()` passes no device-flow + provider, so `POST /repos/:id/connect/device*` answers `503 + devplatform.device_flow_unconfigured`. PAT and App onboarding are unaffected. + +Until a repo has a credential, its jobs fail at clone — visibly, at job start. + +**Rotation** is the same action as entering one: re-connect the repository, or +re-bind the App. The old value is replaced in place; jobs already running keep +the installation token they were minted, which expires on its own. + +Two secrets belong to the deployment rather than the plugin, and live in +`docker-compose.dev-platform.yaml`: + +- `DEV_RUNNER_DAEMON_TOKEN` — a **comma-separated list**, for zero-downtime + rotation. Both ends accept every token in the list and send the first, so the + procedure is: prepend the new token → restart → drop the old one. +- The LLM provider key the proxy forwards with. + +Uninstall and purge never reach into the vault — deleting credentials is the +host's operation on the host's namespace. Remove them through your host's secret +management once the plugin is gone. + +## 6. Choosing a runner backend + +Backends register only when their prerequisites are configured. Everything is +**off by default** — a missing token means "not registered", never "registered +and insecure". + +### Docker / compose (`kind=docker`) — the default + +Selected by `DEV_PLATFORM_BACKEND` (default `docker`; only the literal `local` +selects otherwise). Registers when `DEV_RUNNER_DAEMON_URL` **and** +`DEV_RUNNER_DAEMON_TOKEN` are both set. + +```bash +docker compose \ + -f /path/to/omadia/docker-compose.yaml \ + -f /path/to/omadia-dev-platform/docker-compose.dev-platform.yaml up -d +``` + +Four services, and the separation between them is the security model: + +- **`dev-runner-daemon`** — the only holder of docker credentials. +- **`dev-dind`** — the nested engine; the only `privileged: true` service in the + whole stack. Internal networks only, no host port, TLS-only (the daemon + refuses a plaintext 2375 engine). +- **`dev-egress-proxy`** — default-deny, the only path from a job container to + the internet. Pinned at `172.28.5.3` because job containers are created by + dind and never see compose DNS. +- **`middleware`** — gains **no** docker socket, **no** `DOCKER_HOST` and no + route to `dev-engine`. The compose-topology test asserts that absence; it is + the single most important property of that file. + +Set `DEV_EGRESS_BASE_ALLOWLIST` to include your package registry. Without a +registry route an auto-detected `npm ci` **hangs** against the proxy's +default-deny rather than failing cleanly. + +### Fly Machines (`kind=fly`) — the hosted path + +One ephemeral Machine per job. Registered only when `DEV_FLY_RUNNER_APP` is set +*and* a runner image is configured; it is additive and orthogonal to +`DEV_PLATFORM_BACKEND`. + +```bash +flyctl apps create --org +# then store a Fly deploy token SCOPED TO THAT APP in Vault +``` + +Two boot-time refusals, both of which log and simply skip registration: + +- `DEV_FLY_RUNNER_APP` equal to this app's `FLY_APP_NAME` — refusing to + provision runners into the middleware's own app. +- `DEV_FLY_RUNNER_APP` set but no runner image. + +Missing token → `devplatform.fly_deploy_token_missing`. The token is read from +Vault per API operation, never held on the instance. Endpoint selection is +automatic: on Fly, the internal Machines API; off Fly, `api.machines.dev`. + +Sizing: `DEV_FLY_REGION`, `DEV_FLY_GUEST_CPUS` (1), `DEV_FLY_GUEST_MEMORY_MB` +(1024), `DEV_FLY_MAX_CPUS` (4), `DEV_FLY_MAX_MEMORY_MB` (8192). + +### Local process (`kind=local`) — development only + +`DEV_PLATFORM_BACKEND=local`, plus setup fields `unsafe_local: true` **and** +`unsafe_local_uid`. Agent-written code then runs on the middleware host itself. +A missing uid is a hard activation refusal, not a warning. + +## 7. Migration handoff + +If you ran the Dev Platform *inside* core, migration slots 0022–0030 are already +applied and recorded in **core's** ledger. This plugin's ledger starts empty, so +without a handoff `runMigrations()` re-applies all nine. + +`ctx.sql.seedLedger()` records them as applied instead — but never on core's +word. Each of the nine files carries a **witness** proving the schema object it +creates is actually present. + +The case that makes this necessary is *rows present, tables absent*: a restore +from a snapshot older than the migrations, a version-skewed rollback, an +operator who dropped a table during an incident. A handoff that trusted core's +rows would activate green and 500 on every request. With witnesses the seed +declines, the migration runner applies the files, and that is the repair. + +### Dry-run it against production first + +`handoff-plan.json` ships in the ZIP precisely so you can see the plan against +your real database **before** the plugin is installed and before a single row is +written. Core ships the CLI (`middleware/scripts/plugin-ledger-handoff.mjs`, +epic #470 C11): + +```bash +cd /path/to/omadia/middleware +npm run build # the CLI imports from dist/ + +DATABASE_URL=postgres://…/omadia \ +node scripts/plugin-ledger-handoff.mjs \ + --plan /path/to/omadia-dev-platform/packages/plugin/handoff-plan.json +``` + +| Flag | Effect | +|---|---| +| `--plan ` | **Required.** `migrationsDir` inside it is resolved relative to the plan file, so a plan copied out of the ZIP works from wherever you put it. | +| *(none)* | **Dry run — the default.** `--apply` is the only way to write. The inverse default would be wrong for a tool whose whole value is being run against production by someone who has not read it. | +| `--apply` | Actually writes the ledger rows. | +| `--database-url ` | Overrides `$DATABASE_URL`. | +| `--json` | Machine-readable report. | + +Exit codes: **0** plan computed (or applied) · **1** the handoff refused · **2** +usage or plan-file error. + +The dry run costs one read-only transaction, and that is literally true rather +than a claim: witnesses execute inside a read-only subtransaction over +PostgreSQL's extended protocol, so a multi-statement witness is refused by the +server before it can escape the dry run, and a writing witness is refused before +it can touch the donor ledger or any bystander table. + +The CLI names no plugin and no table — the plan file supplies the id, the +ledger, the migrations directory and the entries. That is core's decoupling +ratchet at work, not tidiness: no core file may name the extracted plugin. + +**Reading the report.** `seeded` were adopted on proof · `applied` were left for +the migration runner · `alreadySeeded` were already in this plugin's ledger · +**`skippedNoWitness` is the alarm** — the donor ledger records them, but their +witness says the schema object is absent. On a healthy installation it is empty +and the CLI prints `✓ no disagreement between the donor ledger and the live +catalog`. When it is not empty the CLI is explicit that this is the handoff +working, not failing: the migration runner will apply those files and that is +the repair. Confirm the database is the one you think it is before continuing. + +Running the CLI is optional — installing the plugin performs the same handoff +itself. It exists so the most irreversible-looking step of the epic can be read +before it is taken. + +Since 0.3.1 the manifest also declares `permissions.sql.handoff`, so a core with +`@omadia/plugin-api` 1.6.0 or newer runs this plan **before** its own migration +runner and the handoff reports real numbers. **On an older core the key is +ignored** — the pre-activate migration run gets there first, `seedLedger` can +only answer `alreadySeeded`, and `skippedNoWitness` never fires. That is exactly +when running the dry run by hand is worth it: it is the only way to see the +disagreement. See G7 in §12. + +## 8. Optional capabilities and what you lose + +Four capabilities are declared `optional_requires`. The plugin installs and runs +without them; each absence is logged, none is silent. + +**The registry does not carry this.** The hub reads `requires`, `provides` and +`depends_on` and never `optional_requires`, so the storefront cannot show it. + +| Capability | Absent means | +|---|---| +| `turnContext@1` | The three chat tools (`dev_job_start`, `dev_job_status`, `dev_job_list`) are **not registered at all**. They authorize per call against the human driving the turn; with no envelope there is nothing to authorize against. Registering tools that refuse every call is worse — the model keeps retrying and the refusals read as a bug. Use the operator UI. | +| `githubAppJwt@1` | Falls back to a local RS256 signer. Functionally equivalent; the cost is a duplicated security primitive. Core publishes no provider today. | +| `usageTelemetry@1` | No rows in the operator cost dashboard. **Per-job budgets keep enforcing** — they meter the plugin's own tables. Drops are counted and reported at deactivate. | +| `conductorRoles@1` | Repositories whose approver is a **role** open gates nobody can approve; the job waits until its deadline expires. Fail-closed, the safe direction. **Workaround: configure a named user approver.** This is the largest functional gap. | + +## 9. Supply chain + +The runner image is `ghcr.io/byte5ai/omadia-dev-runner`, published only by +`.github/workflows/release-runner-image.yml` on a tag push or manual dispatch — +never on a branch push. Signing is keyless (Fulcio + Rekor) and always over the +immutable **digest**, never a tag. `sidecars/dev-runner-daemon/src/imageVerify.mjs` +verifies at daemon boot. + +Because the image was previously published by core, verification accepts either +signer during the transition: + +``` +^(?:https://github\.com/byte5ai/omadia/\.github/workflows/publish-images\.yml|https://github\.com/byte5ai/omadia-dev-platform/\.github/workflows/release-runner-image\.yml)@refs/(?:heads|tags)/[A-Za-z0-9._/-]+$ +``` + +Two deliberate properties. It is **anchored at both ends** — cosign compiles +with Go RE2 and matches *unanchored*, so without `^…$` a URL like +`https://evil.example/?x=` would satisfy it. And it is +**narrow**: two exact repo+workflow pairs, never "anything under `byte5ai`". + +| Configuration | cosign flag | +|---|---| +| `DEV_IMAGE_COSIGN_IDENTITY_REGEXP` set | `--certificate-identity-regexp ` — validated at boot; unanchored or uncompilable is a refusal naming the variable | +| `DEV_IMAGE_COSIGN_IDENTITY` set to one of the two signers | `--certificate-identity-regexp ` — widened, logged loudly once per boot | +| `DEV_IMAGE_COSIGN_IDENTITY` set to anything else | `--certificate-identity ` | +| Neither set | verification **skips**, with a warning | + +`DEV_IMAGE_COSIGN_ISSUER` must be set in any enforcing configuration — a regexp +alone is not a pin. `DEV_IMAGE_VERIFY=off` is the only full escape hatch. + +Verify by hand: + +```bash +cosign verify \ + --certificate-identity-regexp '^https://github\.com/byte5ai/omadia-dev-platform/\.github/workflows/release-runner-image\.yml@refs/(?:heads|tags)/[A-Za-z0-9._/-]+$' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + ghcr.io/byte5ai/omadia-dev-runner@sha256: + +cosign verify-attestation --type spdxjson \ + --certificate-identity-regexp '' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + ghcr.io/byte5ai/omadia-dev-runner@sha256: +``` + +**On Fly there is no pull hook for cosign** — the platform pulls the image +itself at deploy time. The guarantee there is digest pinning in the daemon +config plus the CI-verified signature at release. Boot-time verification still +runs everywhere the daemon can reach the registry. + +The plugin ZIP deliberately ships **no runner shim**: a second copy would create +a parallel provenance path verified by nothing, since the hub checks no +signature and the ZIP carries no attestation. + +**One release after the first publish from this repository**, narrow the regexp +to this repo alone — the code is written so that is a deletion, not a rewrite. + +## 10. Uninstall and purge + +**Uninstall never drops a table.** `close()` disposes routes, tools, nav entries +and background loops; it touches no schema, so a reinstall is lossless +(measured: ledger 9 with 0 re-applied, rows intact, routes back to 200). + +What stops: routers unmount, the nav entry disappears, background workers stop, +and granted public paths stop being honoured (`/api/v1/dev-runner/...` goes +200 → 404). + +What survives: all nine `dev_*` tables and every row, plus the migration ledger. + +> Grant rows are **not** revoked on uninstall — `plugin_sql_grants` and +> `plugin_public_path_grants` keep their rows. Orphaned, not a live hole: the +> runtime stops honouring them. The lifecycle decision is still open upstream. + +To actually destroy the data, call the purge route — explicit, destructive and +type-to-confirm: + +```bash +curl -X POST "$BASE/api/v1/admin/dev-platform/admin/purge" \ + -H 'Content-Type: application/json' \ + -d '{"confirm":"@omadia/dev-platform"}' +``` + +Without the confirm phrase you get `400 devplatform.purge_not_confirmed` and the +message spelling out what would be destroyed. It drops all nine tables in one +transaction, in dependency order, without `CASCADE` — and the ledger with them, +because tables dropped while the ledger stays populated is the single worst end +state available here. + +Purge does not touch the vault (§5). + +## 11. Troubleshooting + +**A changed ZIP had no effect.** You re-uploaded the same `id@version`. The +module was reused from Node's ESM cache. Restart the middleware, and bump the +version next time. + +**`install.missing_capability` (HTTP 409).** + +``` +plugin requires capabilities not yet provided: +``` + +A hard `requires:` entry has no *declared* provider. Note the sharp edge: +`details.available_providers: []` means "nobody declares providing this" — the +resolver reads manifest `provides:`, while `services.get` reads the live +registry, so a capability can exist at runtime and still fail this gate. For +this plugin only `graphPool@1` can trigger it; the other four moved to +`optional_requires` in 0.3.0. + +**Activation fails naming the ledger.** The ledger must sit inside +`plg_omadia_dev_platform_` and match `permissions.sql.ledger` exactly. A grant +row with a different ledger name is not a partial grant — it is no grant. + +**Every route 404s but the plugin reports `status: "active"`.** Known core bug +(G4, §12). The install path writes `status: 'active'` before running the +`onInstalled` hook and never revises it when the hook throws. Check the +middleware log rather than the status field; the boot path is correct. + +**Runner starts, then every request inside it fails with 407.** An empty +`DEV_RUNNER_DAEMON_TOKEN` — the daemon authenticates to its own egress proxy as +`Bearer undefined`. Inside a runner this presents as a total network outage. Now +a boot refusal, so check the daemon's startup log. + +**`npm ci` inside a job hangs.** The egress proxy is default-deny and your +package registry is not in `DEV_EGRESS_BASE_ALLOWLIST` (default +`registry.npmjs.org`). + +**The proxy answers 500 with no policy.** `llm_allowed_models` is empty. From +inside the runner this is indistinguishable from a wiring bug — set it. + +**Role-approver gates can never be approved.** `conductorRoles@1` has no +provider. Configure a named user approver (§8). + +**Device-flow connect returns 503.** Expected — `devplatform.device_flow_unconfigured`. +Use a GitHub App or a PAT. + +## 12. Known open issues + +| # | Issue | Status | +|---|---|---| +| **G4** | A failed activation still reports `status: "active"`. The install path writes the status before the `onInstalled` hook and never revises it. Makes every other verdict less trustworthy. | **Open** in core | +| **G6** | Core's `publicPaths.ts` carried two static dev-platform exemptions that collided with this plugin's declarations. This single residue caused all 33 failures of the 2026-08-21 run against `main`. | **Fixed on core `main`** — C12, core PR #807 (`e1e31f62`, 2026-08-21). Use a core at or after that commit. | +| **G7** | Core's pre-activate migration run happened **before** `activate()`, pre-empting the C11 handoff: `seedLedger` found all nine already applied and `skippedNoWitness` — the one alarm the feature exists to raise — never fired. The 2026-08-21 run measured `0 seeded, 9 already seeded` on the exact upgrade the feature was built for, and nothing went red. | **Fixed in 0.3.1** (C15, core issue byte5ai/omadia#814). `permissions.sql.handoff` declares the plan so the **kernel** runs it ahead of its own migration runner. Needs core with `@omadia/plugin-api` **1.6.0**; on anything older the key is ignored and the `activate()` fallback still runs — so on a core below 1.6.0 the gap remains, and the §7 dry run is the way to see it. | + +Acceptance runs: + +| Run | Against | Outcome | +|---|---|---| +| [`ACCEPTANCE-RUN-2026-08-20.md`](./ACCEPTANCE-RUN-2026-08-20.md) | 0.2.0 vs. a **patched** core | 71 PASS / 0 FAIL / 2 BLOCKED. Found G1–G6. | +| [`ACCEPTANCE-RUN-2026-08-21.md`](./ACCEPTANCE-RUN-2026-08-21.md) | 0.3.0 vs. **plain** `origin/main`, no patches | 38 / 33 / 2 on `main`; **71 / 0 / 2** with C12. All 33 failures traced to G6 alone. 1,316 tests green. | + +Both reproduce with `node scripts/acceptance-local.mjs` (exit code equals the +FAIL count), driven by `BASE_URL` and `DATABASE_URL`. + +## See also + +- [`SECRETS.md`](./SECRETS.md) — the full credential list and rotation +- [`SUPPLY_CHAIN.md`](./SUPPLY_CHAIN.md) — signing, SBOM, the narrowing step +- [`../packages/plugin/SEAMS.md`](../packages/plugin/SEAMS.md) — every core seam + and its degradation +- [`iframe-credentials.md`](./iframe-credentials.md) diff --git a/packages/plugin/README.md b/packages/plugin/README.md index f7226d9..94b61bd 100644 --- a/packages/plugin/README.md +++ b/packages/plugin/README.md @@ -4,42 +4,98 @@ The Omadia Dev Platform as an installable plugin. `kind: extension`. This README ships **inside the plugin ZIP**, so it is written for whoever installs the artifact rather than for whoever builds it. Build and contribution -instructions live in the [repository README](../../README.md). +instructions live in the [repository README](../../README.md); the full runbook +is [`docs/OPERATOR-GUIDE.md`](../../docs/OPERATOR-GUIDE.md). ## What this release does -**Nothing yet.** Version 0.1.0 activates, logs once, and deactivates cleanly. It -contributes no routes, no chat tools, no background jobs and no database -migrations, and there is nothing to configure. +Runs autonomous coding jobs against your GitHub repositories. An agent clones a +repo into an isolated runner, works the job through an analyze → plan → +implement → review pipeline, and opens a pull request. Human approval gates, a +diff policy, a per-job cost budget and a default-deny egress proxy bound what it +can do. -It exists so that the packaging and install path are proven before the Dev -Platform itself moves in — roughly 49,000 lines currently living in omadia core. -Installing it is safe and fully reversible: uninstalling leaves no tables, no -routes and no scheduled work behind. +It contributes an operator SPA, HTTP routes, three chat tools, background +workers, and **nine database tables it migrates itself**. -## What it will do +## Before you install -Run autonomous coding jobs against a repository — provision a runner, drive an -LLM through an analyze → plan → implement → review pipeline, and open a pull -request — plus the admin UI, the chat tools and the webhook surfaces that go -with it. +- **omadia core ≥ 1.5** (`@omadia/plugin-api` 1.5.0) — the newest extension + point used here is `ctx.sql.seedLedger`, the migration handoff. +- **A Postgres-backed knowledge graph.** The job, repo, gate and artifact tables + live in `graphPool`. With an in-memory graph the plugin **refuses to + activate** rather than pretending to work. +- **A runner backend** — Fly Machines or local Docker. -That payload arrives in phases P2 through P4 of -[epic byte5ai/omadia#470](https://github.com/byte5ai/omadia/issues/470). +`graphPool@1` is the only hard capability requirement. Without a declared +provider, install is refused with `install.missing_capability` (HTTP 409). -## Permissions +## Two operator grants are required -None declared. Two are documented as comments in `manifest.yaml` for the -releases that will need them: +The manifest asks; a human has to agree. Neither is implied by installing, and +without them the plugin does not work. -- **`sql`** — gates the existing `graphPool@1` capability so the plugin can own - its own tables and run its own migrations. -- **`public_paths`** — an operator-consented grant for the two unauthenticated - prefixes the dev runner phones home to. +1. **`permissions.sql`** — core provisions `ctx.sql` only when a row exists in + `plugin_sql_grants` for ledger `plg_omadia_dev_platform_migrations`. + **Core ships no UI or API for this yet**, so the row is inserted by hand. +2. **`permissions.public_paths`** — three prefixes served without a kernel + session (each still authenticated: a per-job bearer token, a webhook HMAC, + and a signed state token). Granted through + `PUT /api/v1/admin/runtime/installed/@omadia%2Fdev-platform/public-paths`. -Both stay commented until the core capability that enforces them has shipped. An -unknown manifest key is silently ignored rather than rejected, so declaring them -early would produce a plugin that activates with no grant and no error. +Exact steps for both: +[Operator Guide §4](../../docs/OPERATOR-GUIDE.md#4-the-two-operator-grants). + +## Optional capabilities + +Four capabilities are declared `optional_requires`: the plugin installs and runs +without them, and each absence is logged rather than silent. **The registry does +not carry optional capabilities**, so this is the only place it is written down +before install. + +| Absent | You lose | +|---|---| +| `turnContext@1` | The three chat tools are not registered at all. Use the operator UI. | +| `githubAppJwt@1` | Falls back to a local RS256 signer. Functionally equivalent. | +| `usageTelemetry@1` | No cost-dashboard rows. **Per-job budgets keep enforcing.** | +| `conductorRoles@1` | Role-approver gates become unapprovable (fail-closed). Use named approvers. | + +## Credentials — you enter these yourself + +Nothing is migrated for you. The plugin stores GitHub App signing material, +webhook secrets and per-repo clone tokens in its **own** vault namespace, so +after installing — and after any reinstall — you connect each repository by hand +under **Dev Platform → Repositories**. + +That is a deliberate choice over a one-time migration hook: the right moment to +see credentials move into a plugin's namespace is while it is happening. A +GitHub App is the recommended credential — the runner then receives a freshly +minted, single-repo, read-only token that is revoked when the job ends. See +[`docs/SECRETS.md`](../../docs/SECRETS.md). + +## Upgrading from the Dev Platform inside core + +If you ran this subsystem inside core, migration slots 0022–0030 are already +applied and recorded in **core's** ledger. `handoff-plan.json` ships in this ZIP +so you can dry-run the adoption against your real database before installing: + +```bash +cd /path/to/omadia/middleware && npm run build +DATABASE_URL=… node scripts/plugin-ledger-handoff.mjs --plan /handoff-plan.json +``` + +Dry run is the default; `--apply` is the only way to write. Each of the nine +files is adopted only where a **witness** proves the schema object it creates is +actually present — the case that matters is rows present, tables absent. +[Operator Guide §7](../../docs/OPERATOR-GUIDE.md#7-migration-handoff). + +## Uninstall + +**Uninstall never drops a table.** Routes, tools, nav entries and background +loops are disposed; the nine `dev_*` tables and all rows survive, so a reinstall +is lossless. To actually destroy the data there is an explicit, type-to-confirm +purge route — see +[Operator Guide §10](../../docs/OPERATOR-GUIDE.md#10-uninstall-and-purge). ## License diff --git a/packages/plugin/manifest.yaml b/packages/plugin/manifest.yaml index 5a9428b..fae6dfa 100644 --- a/packages/plugin/manifest.yaml +++ b/packages/plugin/manifest.yaml @@ -3,6 +3,49 @@ schema_version: "1" # --------------------------------------------------------------------------- # The hub reads THIS file, not package.json. `identity.version` and the # package.json `version` must agree — `npm run package` aborts on drift. +# +# WHAT THE HUB KEEPS, AND WHAT IT THROWS AWAY +# ------------------------------------------- +# `omadia-hub/lib/manifest.ts` does not store this file. It PROJECTS it into a +# fixed index shape (`parsePublish`, :113-231) and everything it does not name +# is gone the moment the ZIP is published. What survives: +# +# identity.{id,name,version,kind,domain,description,categories, +# authors,license,icon_url} → lib/manifest.ts:205-216 +# compat.core → :219 (compat.node is DROPPED) +# setup.fields[] → :224 verbatim objects +# setup.guide → :225 {locale: markdown} map +# provides / requires / depends_on → :221-223 strings only +# permissions → :226 verbatim object +# +# Everything else is STRIPPED. Three of those matter here: +# +# 1. `optional_requires:` is NOT in that list. The hub never reads it +# (:221-223 name `provides`, `requires` and `depends_on` and nothing +# else), so the four capabilities below and — more importantly — the +# degradation each one takes when absent are INVISIBLE on the storefront. +# That is why `setup.guide` carries the degradation table in prose. Do not +# delete it as duplication: it is the only copy an operator can read +# before installing. +# 2. EVERY COMMENT IN THIS FILE. `parseYaml` keeps values, not trivia. The +# rationale comments are for whoever edits this file — never for the +# operator. Anything the operator must know belongs in `setup.guide`. +# 3. `lifecycle.entry`. Core reads it at install; the hub does not. +# +# And two rendering traps on the hub's detail page: +# - `summarizePermission` (app/p/[...id]/page.tsx:309-317) collapses a NESTED +# array to its length: `network.outbound` renders as "outbound: [4]", so the +# four hosts are invisible. A top-level array value survives, but keyed by +# index — `public_paths` renders as "0: /api/v1/dev-runner · 1: …", which is +# legible but not self-explanatory. Either way the operator-facing reason +# for each entry has to be spelled out in `setup.guide`; the permissions +# grid shows values, never rationale. +# - `SetupRow` (:243-259) prints `label` through `asText`, which +# JSON.stringifies a non-string. The `{en,de}` maps below are the shape core +# normalises (`manifestLocalized.ts`, OM-17/#602) and the install wizard +# renders correctly; the hub's storefront table still needs a one-line fix +# to resolve them. Tracked — do not flatten these back to bare strings to +# work around a display bug in a different repository. # --------------------------------------------------------------------------- identity: @@ -23,18 +66,52 @@ identity: kind: "extension" version: "0.3.1" domain: "dev-platform" - description: "The Omadia Dev Platform — dev jobs, runner orchestration, the job pipeline, its own HTTP routes, chat tools and database migrations. Extracted from omadia core into its own installable plugin (epic byte5ai/omadia#470)." + description: "Runs autonomous coding jobs against your GitHub repositories. An agent clones a repo into an isolated runner, drives an LLM through an analyze/plan/implement/review pipeline, and opens a pull request — bounded at every step by human approval gates, a diff policy, a per-job cost budget and an egress allowlist. Ships its own operator UI, HTTP routes, chat tools, background workers and nine database tables it migrates itself. Runners execute on Fly Machines or on local Docker; neither ever sees a long-lived credential. Extracted from omadia core into its own installable plugin (epic byte5ai/omadia#470)." authors: - name: "byte5 GmbH" email: "info@omadia.ai" url: "https://omadia.ai" license: "MIT" + # The hub's storefront facets by `kind`, and its category selector is built + # from the union of every published plugin's `categories` — so a term here is + # a browse path, not a tag cloud. Keep it to terms an operator would actually + # filter on. categories: - "extension" - "developer-tools" + - "automation" + - "github" compat: - core: ">=1.0 <2.0" + # `>=1.5` names @omadia/plugin-api 1.5.0 — the first core release carrying the + # WHOLE contract surface this plugin activates against. Not a guess; each of + # the five extension points landed in a numbered core PR: + # + # C4 operator-granted `permissions.public_paths` byte5ai/omadia#791 era + # C6 plugin route auth modes + route-local raw body #791 + # C7 `permissions.sql` gate + advisory-locked #787 + # plugin migrations + # C9 `optional_requires` + `ctx.services.getOptional` #802 → plugin-api 1.3.0 + # + `pluginUi` nav entries + # C11 `ctx.sql.seedLedger` migration handoff #806 → plugin-api 1.5.0 + # + # C11 is the newest of the five, so 1.5.0 is the floor. Pinning lower would be + # a claim this plugin cannot honour: on a 1.4.0 core `ctx.sql.seedLedger` is + # `undefined`, the ledger handoff silently does not happen, and the migrations + # re-run against a schema core already created. + # + # C15 (`permissions.sql.handoff`, below) wants plugin-api 1.6.0 and does NOT + # raise this floor, because it degrades cleanly: a core older than 1.6.0 + # ignores the unknown key and the `activate()` `seedLedger` call remains the + # thing that performs the handoff. The floor tracks what this plugin cannot + # run without, not what it would prefer. + # + # NOTE, so nobody mistakes this for a gate: core does NOT enforce it. Nothing + # in `middleware/src` semver-compares `compat_core` — `manifestLoader.ts:503` + # and `registryClient.ts:415` only carry the string through to the store view + # (`web-ui/app/store/[id]/page.tsx:434`). It is a declaration to the operator, + # so it must be honest rather than permissive. + core: ">=1.5 <2.0" node: ">=20" lifecycle: @@ -185,13 +262,98 @@ setup: ### Before you install + **omadia core 1.5 or newer.** The plugin activates against extension + points that landed across five core releases; the newest is + `ctx.sql.seedLedger` (the migration handoff). On an older core the handoff + silently does not happen and the migrations re-run against a schema core + already created. + **A Postgres-backed knowledge graph is required.** The job spine, repo, gate and artifact tables live there. With an in-memory graph the plugin refuses to activate rather than pretending to work — set `DATABASE_URL` and install the Neon knowledge-graph plugin first. - **Grant the SQL permission.** This plugin owns nine tables and migrates - them itself, into its own ledger (`omadia_dev_platform_migrations`). + ### Two operator grants — do these, or the plugin does not work + + This plugin declares two permissions that a manifest alone cannot give it. + Core requires a human to agree, separately, after install. + + **1. The SQL permission (`permissions.sql`).** The plugin owns nine tables + and migrates them itself into its own ledger, + `plg_omadia_dev_platform_migrations`. Core provisions `ctx.sql` only when + a row exists in `plugin_sql_grants`. + + > **There is no button for this yet.** Core ships no UI and no API route + > that writes that row — `middleware/src/platform/pluginSqlGrants.ts` + > records it twice in its own source ("no shipped grant surface — nothing + > in `src/` calls `grant()`"). Until core ships one, an operator with + > database access inserts it by hand: + > + > ```sql + > INSERT INTO plugin_sql_grants (plugin_id, ledger, granted_by) + > VALUES ('@omadia/dev-platform', + > 'plg_omadia_dev_platform_migrations', + > 'you@example.com'); + > ``` + > + > The ledger name must match the manifest exactly — core compares them and + > treats a mismatch as no grant at all. Restart the middleware afterwards. + + **2. The public-path grants (`permissions.public_paths`).** Three URL + prefixes are served without a kernel session, because the caller has no + session to present. Each is still authenticated, just not by one: + + - `/api/v1/dev-runner` — a per-job bearer token, verified in the router. + - `/api/webhooks/github` — HMAC-SHA256 over the raw request body. + - `/api/v1/dev-platform` — the GitHub App manifest-conversion callback, + bound to a kernel-signed, plugin-audience state token. + + This one **does** have a shipped surface, behind operator auth: + + ``` + GET /api/v1/admin/runtime/installed/@omadia%2Fdev-platform/public-paths + PUT /api/v1/admin/runtime/installed/@omadia%2Fdev-platform/public-paths + { "paths": ["/api/v1/dev-runner", + "/api/webhooks/github", + "/api/v1/dev-platform"] } + ``` + + `PUT` takes the COMPLETE set you consent to — omitting a prefix revokes + it. Consent cannot exceed the declaration: core rejects any prefix this + manifest does not ask for. Without these grants the runner cannot phone + home, webhooks 401, and GitHub App onboarding cannot complete. + + ### What this plugin does without, and what you lose + + Four capabilities are declared **optional**. The plugin installs and runs + when core does not provide them, and each absence is logged rather than + silent. This table is the only place you can read it before installing — + the registry does not carry optional capabilities. + + | Capability | Absent means | + |---|---| + | `turnContext@1` | The chat dev-job tools are **not registered at all**, rather than registered and always refusing. They authorize per call against the human driving the turn; with no envelope there is nothing to authorize against. Use the operator UI instead. | + | `githubAppJwt@1` | The plugin falls back to a local RS256 signer. Functionally equivalent; the App private key is held by the plugin rather than by core. | + | `usageTelemetry@1` | No rows in the operator cost dashboard's ledger. **Per-job budgets are unaffected** — they meter this plugin's own tables and keep enforcing. | + | `conductorRoles@1` | Repositories whose approver is a ROLE rather than a person cannot have their gates approved. Fail-CLOSED — the safe direction, but a regression you should know about. Assign named approvers instead. | + + Only `graphPool@1` is mandatory. If core cannot provide it the install is + refused with `install.missing_capability` (HTTP 409). + + ### Where your runners execute + + Two backends, chosen by what you configure in the deployment — not here: + + - **Fly Machines** — one ephemeral Machine per job, in a dedicated Fly + app named by `DEV_FLY_RUNNER_APP`. This is the production path. The app + must exist before the first job. + - **Local Docker** — the compose overlay shipped with this repository + (`docker-compose.dev-platform.yaml`), which brings up the runner daemon, + the egress proxy and one privileged docker-in-docker service. Best for + evaluation and self-hosting. + + Either way the runner reaches this middleware at the phone-home URL below, + so a loopback address only works when the runner shares the host. ### What lives here, and what lives in your deployment @@ -214,6 +376,28 @@ setup: deliberate: both modes put a credential or your host filesystem within reach of code an agent wrote. + ### Upgrading from the Dev Platform that ran inside core + + Migration slots 0022–0030 are already applied on your database and + recorded in CORE's ledger. This plugin adopts them instead of re-applying + them — but only where a WITNESS proves the schema object each file creates + is actually present. The case that makes this necessary is rows present, + tables absent: a restore from an older snapshot, a version-skewed + rollback, a table dropped during an incident. + + `handoff-plan.json` ships inside the ZIP so you can see the plan against + your real database BEFORE installing. Dry run is the default: + + ``` + cd /middleware && npm run build + DATABASE_URL=… node scripts/plugin-ledger-handoff.mjs \ + --plan /handoff-plan.json + ``` + + Read `skippedNoWitness` in the output. Empty is healthy. Non-empty means + core's ledger and your live schema disagree — not a failure of the + handoff, but a reason to confirm the database is the one you think it is. + ### After installing — you re-enter the credentials Connect a repository under **Dev Platform → Repositories**. A GitHub App @@ -221,8 +405,9 @@ setup: single-repo, read-only token that is revoked when the job ends. **GitHub App keys and repository tokens are not migrated for you.** This - plugin stores credentials in its own vault namespace, so App private keys, - webhook secrets and per-repo clone tokens are entered here once, by you. + plugin stores credentials in its own vault namespace, so App signing + material, webhook secrets and per-repo clone tokens are entered here once, + by you. That is a deliberate choice over a one-time migration hook: the right moment to see that a GitHub App private key has moved into a plugin's namespace is while it is happening. The same applies after a reinstall. @@ -237,14 +422,102 @@ setup: ### Vor der Installation + **omadia Core 1.5 oder neuer.** Das Plugin aktiviert gegen + Erweiterungspunkte aus fünf Core-Releases; der jüngste ist + `ctx.sql.seedLedger` (die Migrations-Übergabe). Auf älterem Core findet + die Übergabe still nicht statt und die Migrationen laufen erneut gegen ein + Schema, das Core bereits angelegt hat. + **Ein Postgres-gestützter Knowledge Graph ist Voraussetzung.** Job-, Repo-, Gate- und Artefakt-Tabellen liegen dort. Mit einem In-Memory-Graph verweigert das Plugin die Aktivierung, statt Betrieb vorzutäuschen — setze zuerst `DATABASE_URL` und installiere das Neon-Knowledge-Graph-Plugin. - **Erteile die SQL-Berechtigung.** Das Plugin besitzt neun Tabellen und - migriert sie selbst, in sein eigenes Ledger - (`omadia_dev_platform_migrations`). + ### Zwei Freigaben durch dich — ohne sie läuft das Plugin nicht + + Das Manifest *fragt*; zustimmen muss ein Mensch. Keine der beiden Freigaben + ergibt sich aus der Installation. + + **1. Die SQL-Berechtigung (`permissions.sql`).** Das Plugin besitzt neun + Tabellen und migriert sie selbst in sein eigenes Ledger + `plg_omadia_dev_platform_migrations`. Core stellt `ctx.sql` nur bereit, + wenn in `plugin_sql_grants` eine passende Zeile existiert. + + > **Dafür gibt es noch keinen Button.** Core liefert weder UI noch + > API-Route, die diese Zeile schreibt — + > `middleware/src/platform/pluginSqlGrants.ts` hält das zweimal im eigenen + > Quelltext fest („no shipped grant surface — nothing in `src/` calls + > `grant()`"). Bis Core eine Oberfläche liefert, trägt sie jemand mit + > Datenbankzugriff von Hand ein: + > + > ```sql + > INSERT INTO plugin_sql_grants (plugin_id, ledger, granted_by) + > VALUES ('@omadia/dev-platform', + > 'plg_omadia_dev_platform_migrations', + > 'du@example.com'); + > ``` + > + > Der Ledger-Name muss exakt dem Manifest entsprechen — Core vergleicht + > beide und wertet jede Abweichung als gar keine Freigabe. Danach die + > Middleware neu starten. + + **2. Die Public-Path-Freigaben (`permissions.public_paths`).** Drei + Präfixe werden ohne Kernel-Session ausgeliefert, weil der Aufrufer keine + hat. Authentifiziert sind sie trotzdem, nur anders: + + - `/api/v1/dev-runner` — Bearer-Token pro Job, im Router geprüft. + - `/api/webhooks/github` — HMAC-SHA256 über den rohen Request-Body. + - `/api/v1/dev-platform` — der GitHub-App-Callback, gebunden an ein + kernel-signiertes State-Token mit Plugin-Audience. + + Hierfür gibt es eine fertige Schnittstelle hinter Operator-Auth: + + ``` + GET /api/v1/admin/runtime/installed/@omadia%2Fdev-platform/public-paths + PUT /api/v1/admin/runtime/installed/@omadia%2Fdev-platform/public-paths + { "paths": ["/api/v1/dev-runner", + "/api/webhooks/github", + "/api/v1/dev-platform"] } + ``` + + `PUT` erwartet die VOLLSTÄNDIGE Menge — ein weggelassenes Präfix wird + entzogen. Die Zustimmung kann die Deklaration nicht überschreiten: Core + lehnt jedes Präfix ab, das dieses Manifest nicht anfragt. Ohne diese + Freigaben erreicht der Runner die Middleware nicht, Webhooks laufen auf + 401, und das GitHub-App-Onboarding kann nicht abschließen. + + ### Worauf dieses Plugin verzichten kann — und was dich das kostet + + Vier Capabilities sind **optional** deklariert. Das Plugin installiert und + läuft auch ohne sie, und jede Abwesenheit wird geloggt, keine ist still. + Diese Tabelle ist die einzige Stelle, an der du das vor der Installation + nachlesen kannst — die Registry führt optionale Capabilities nicht. + + | Capability | Abwesenheit bedeutet | + |---|---| + | `turnContext@1` | Die Chat-Tools werden **gar nicht erst registriert**, statt registriert zu sein und jeden Aufruf abzulehnen. Sie autorisieren pro Aufruf gegen den Menschen im Turn; ohne Envelope gibt es nichts, wogegen autorisiert werden könnte. Nutze stattdessen die Operator-UI. | + | `githubAppJwt@1` | Rückfall auf einen lokalen RS256-Signer. Funktional gleichwertig; die App-Signaturschlüssel liegen dann beim Plugin statt bei Core. | + | `usageTelemetry@1` | Keine Zeilen im Kosten-Dashboard. **Budgets pro Job sind nicht betroffen** — sie messen die eigenen Tabellen des Plugins und greifen weiter. | + | `conductorRoles@1` | Repositories, deren Freigeber eine ROLLE ist, lassen sich nicht mehr freigeben. Fail-CLOSED — die sichere Richtung, aber eine Einschränkung, die du kennen solltest. Trage stattdessen namentliche Freigeber ein. | + + Nur `graphPool@1` ist zwingend. Kann Core es nicht bereitstellen, wird die + Installation mit `install.missing_capability` (HTTP 409) abgelehnt. + + ### Wo deine Runner laufen + + Zwei Backends, ausgewählt über dein Deployment — nicht hier: + + - **Fly Machines** — eine kurzlebige Machine pro Job, in einer eigenen + Fly-App (`DEV_FLY_RUNNER_APP`). Das ist der Produktionspfad; die App + muss vor dem ersten Job existieren. + - **Lokales Docker** — das Compose-Overlay aus diesem Repository + (`docker-compose.dev-platform.yaml`) mit Runner-Daemon, Egress-Proxy und + genau einem privilegierten Docker-in-Docker-Dienst. Gut für Evaluierung + und Self-Hosting. + + In beiden Fällen erreicht der Runner diese Middleware über die + Callback-URL unten — eine Loopback-Adresse funktioniert also nur, wenn der + Runner denselben Host teilt. ### Was hier steht — und was in dein Deployment gehört @@ -268,6 +541,30 @@ setup: Credential oder dein Host-Dateisystem in Reichweite von Code, den ein Agent geschrieben hat. + ### Umstieg von der Dev Platform, die in Core lief + + Die Migrations-Slots 0022–0030 sind auf deiner Datenbank bereits angewandt + und in CORES Ledger vermerkt. Dieses Plugin übernimmt sie, statt sie + erneut anzuwenden — aber nur dort, wo ein ZEUGE belegt, dass das + Schema-Objekt der jeweiligen Datei tatsächlich existiert. Der Fall, der das + nötig macht, ist: Zeilen vorhanden, Tabellen NICHT — ein Restore aus einem + älteren Snapshot, ein versionsschiefer Rollback, eine im Störfall + gelöschte Tabelle. + + `handoff-plan.json` liegt im ZIP, damit du den Plan VOR der Installation + gegen deine echte Datenbank sehen kannst. Dry-Run ist die Voreinstellung: + + ``` + cd /middleware && npm run build + DATABASE_URL=… node scripts/plugin-ledger-handoff.mjs \ + --plan /handoff-plan.json + ``` + + Lies `skippedNoWitness` in der Ausgabe. Leer ist gesund. Nicht leer heißt: + Cores Ledger und dein tatsächliches Schema widersprechen sich — kein + Fehler der Übergabe, aber ein Grund, sich zu vergewissern, dass es die + Datenbank ist, für die du sie hältst. + ### Nach der Installation — die Credentials trägst du selbst ein Verbinde ein Repository unter **Dev Platform → Repositories**. Eine GitHub @@ -284,119 +581,189 @@ setup: dann, wenn es passiert. Nach einer Neuinstallation gilt dasselbe. Die vollständige Liste und die Rotation stehen in `docs/SECRETS.md`. + # `label` and `help` are `{ : text }` maps. Core normalises both + # through `manifestLocalized.normalizeLocalized` (OM-17 / byte5ai/omadia#602) + # and the catalog projection and the install wizard share that one function, + # so the store view and the form can never disagree about what a field says. + # A bare string is still tolerated and read as English — the map is the + # canonical form, and it is what puts German in front of a German operator. fields: # --- reachability ------------------------------------------------------- - key: "runner_base_url" type: "url" - label: "Runner phone-home base URL" - help: "The base URL a runner reaches this middleware at, e.g. https://bot.example.com. Runners are launched OUTSIDE this process and call back to it; a loopback address only works when the runner shares the host." + label: + en: "Runner phone-home base URL" + de: "Callback-URL für Runner" + help: + en: "The base URL a runner reaches this middleware at, e.g. https://bot.example.com. Runners are launched OUTSIDE this process and call back to it, so a loopback address only works when the runner shares the host. Leave empty and the plugin derives it from the incoming request, which is wrong behind a proxy that rewrites Host." + de: "Basis-URL, unter der ein Runner diese Middleware erreicht, z. B. https://bot.example.com. Runner laufen AUSSERHALB dieses Prozesses und rufen zurück — eine Loopback-Adresse funktioniert nur, wenn der Runner denselben Host teilt. Leer lassen heißt: aus dem eingehenden Request abgeleitet, was hinter einem Proxy mit umgeschriebenem Host falsch ist." required: false - key: "commit_author" type: "string" - label: "Commit author" - help: "Identity on commits the platform pushes, as `Name `." + label: + en: "Commit author" + de: "Commit-Autor" + help: + en: "Identity on commits the platform pushes, as `Name `. This is what shows up in your repository history and in blame — pick something that is obviously a bot." + de: "Identität für Commits, die die Plattform pusht, im Format `Name `. Das steht später in der Repo-History und in `git blame` — nimm etwas, das erkennbar ein Bot ist." default: "omadia dev platform " required: false # --- concurrency + time bounds ----------------------------------------- - key: "max_concurrent_jobs" type: "integer" - label: "Max concurrent jobs" - help: "How many jobs may run at once. Each one holds a runner." + label: + en: "Max concurrent jobs" + de: "Maximal gleichzeitige Jobs" + help: + en: "How many jobs may run at once. Each one holds a runner, so this is also the ceiling on how many Fly Machines or containers exist at peak — size it against your runner budget, not against your queue." + de: "Wie viele Jobs gleichzeitig laufen dürfen. Jeder belegt einen Runner, also ist das zugleich die Obergrenze gleichzeitiger Fly-Machines bzw. Container — orientiere dich am Runner-Budget, nicht an der Warteschlange." default: "2" required: false - key: "job_wall_clock_ms" type: "integer" - label: "Job wall-clock limit (ms)" - help: "Hard ceiling on one job's total runtime. Exceeding it terminates the runner and fails the job." + label: + en: "Job wall-clock limit (ms)" + de: "Laufzeitlimit pro Job (ms)" + help: + en: "Hard ceiling on one job's total runtime. Exceeding it terminates the runner and fails the job. Default 2700000 = 45 minutes." + de: "Harte Obergrenze für die Gesamtlaufzeit eines Jobs. Wird sie überschritten, wird der Runner beendet und der Job schlägt fehl. Default 2700000 = 45 Minuten." default: "2700000" required: false - key: "heartbeat_timeout_ms" type: "integer" - label: "Heartbeat timeout (ms)" - help: "A runner silent for longer is treated as stalled and reaped." + label: + en: "Heartbeat timeout (ms)" + de: "Heartbeat-Timeout (ms)" + help: + en: "A runner silent for longer is treated as stalled and reaped. Set it below the wall-clock limit or a hung runner burns the full budget before anything notices. Default 300000 = 5 minutes." + de: "Ein Runner, der länger schweigt, gilt als hängend und wird eingesammelt. Muss unter dem Laufzeitlimit liegen — sonst verbrennt ein hängender Runner das volle Budget, bevor es jemand merkt. Default 300000 = 5 Minuten." default: "300000" required: false # --- LLM policy + budget ------------------------------------------------ - key: "llm_allowed_models" type: "host_list" - label: "Allowed models" - help: "Model ids a job's agent may call through the proxy. EMPTY MEANS NO JOB CAN RUN — the proxy answers 500 with no policy. A model with no entry in the price table cannot have its cost budget enforced; the plugin logs a warning naming it at activation." + label: + en: "Allowed models" + de: "Erlaubte Modelle" + help: + en: "Model ids a job's agent may call through the proxy. EMPTY MEANS NO JOB CAN RUN — with no policy the proxy answers 500, which surfaces inside the runner as a total LLM outage. A model with no entry in the price table cannot have its cost budget enforced; the plugin logs a warning naming it at activation rather than silently metering it at zero." + de: "Modell-IDs, die der Agent eines Jobs über den Proxy aufrufen darf. LEER HEISST: KEIN JOB LÄUFT — ohne Policy antwortet der Proxy mit 500, was im Runner wie ein kompletter LLM-Ausfall aussieht. Für ein Modell ohne Eintrag in der Preistabelle lässt sich das Kostenbudget nicht durchsetzen; das Plugin warnt beim Aktivieren mit Modellnamen, statt es still mit 0 zu verrechnen." required: false - key: "llm_default_budget_usd" type: "integer" - label: "Default budget per job (USD)" - help: "Cost ceiling for one job's LLM spend, unless the repository overrides it. Crossing it marks the job budget_exceeded and terminates the runner mid-call." + label: + en: "Default budget per job (USD)" + de: "Standardbudget pro Job (USD)" + help: + en: "Cost ceiling for one job's LLM spend, unless the repository overrides it. Crossing it marks the job budget_exceeded and terminates the runner mid-call — enforcement is not deferred to the end of the turn." + de: "Kostenobergrenze für die LLM-Ausgaben eines Jobs, sofern das Repository sie nicht überschreibt. Wird sie überschritten, markiert das den Job als budget_exceeded und beendet den Runner mitten im Call — die Durchsetzung wartet nicht auf das Ende des Turns." default: "5" required: false - key: "llm_max_output_tokens" type: "integer" - label: "Max output tokens per call" - help: "Clamp on max_tokens. Bounds how far a single response can overshoot the budget before enforcement sees it." + label: + en: "Max output tokens per call" + de: "Maximale Output-Tokens pro Call" + help: + en: "Clamp on max_tokens. Bounds how far a single response can overshoot the budget before enforcement sees it — the budget is checked between calls, so one very large response is the blind spot this closes." + de: "Begrenzt max_tokens. Bestimmt, wie weit eine einzelne Antwort das Budget überschreiten kann, bevor die Durchsetzung greift — geprüft wird zwischen den Calls, eine einzelne sehr große Antwort ist genau der blinde Fleck, den dieser Wert schließt." default: "8192" required: false # --- data lifecycle ----------------------------------------------------- - key: "event_retention_days" type: "integer" - label: "Event retention (days)" - help: "Age at which low-value job telemetry is pruned by the nightly sweep." + label: + en: "Event retention (days)" + de: "Aufbewahrung von Events (Tage)" + help: + en: "Age at which low-value job telemetry is pruned by the nightly sweep. Audit-grade events are NOT covered by this — they follow the audit retention below." + de: "Alter, ab dem geringwertige Job-Telemetrie vom nächtlichen Sweep gelöscht wird. Audit-relevante Events fallen NICHT darunter — für sie gilt die Audit-Aufbewahrung unten." default: "30" required: false - key: "audit_retention_days" type: "integer" - label: "Audit retention (days)" - help: "Outer bound for audit-grade events (gate decisions, policy verdicts, token mint and revoke)." + label: + en: "Audit retention (days)" + de: "Aufbewahrung von Audit-Events (Tage)" + help: + en: "Outer bound for audit-grade events: gate decisions, policy verdicts, token mint and revoke. These are the records that answer 'who approved this change, and what was it allowed to touch' — set it to your compliance retention, not to your disk budget." + de: "Obergrenze für audit-relevante Events: Gate-Entscheidungen, Policy-Verdicts, Ausstellung und Widerruf von Tokens. Das sind die Datensätze, die beantworten, wer eine Änderung freigegeben hat und worauf sie zugreifen durfte — richte den Wert an deiner Compliance-Aufbewahrung aus, nicht am Speicherplatz." default: "365" required: false - key: "max_events_per_job" type: "integer" - label: "Max events per job" - help: "Per-job event cap, enforced at write time. A chatty runner is truncated rather than allowed to fill the table." + label: + en: "Max events per job" + de: "Maximale Events pro Job" + help: + en: "Per-job event cap, enforced at write time. A chatty runner is truncated rather than allowed to fill the table." + de: "Obergrenze für Events pro Job, durchgesetzt beim Schreiben. Ein geschwätziger Runner wird abgeschnitten, statt die Tabelle volllaufen zu lassen." default: "5000" required: false - key: "artifact_max_bytes" type: "integer" - label: "Max inline artifact size (bytes)" - help: "Ceiling on a single stored artifact (diff, brief, verdict)." + label: + en: "Max inline artifact size (bytes)" + de: "Maximale Artefaktgröße inline (Bytes)" + help: + en: "Ceiling on a single stored artifact — a diff, a brief, a review verdict. Anything larger is truncated with a marker, so raising this trades database size for the ability to read very large diffs in the UI. Default 1048576 = 1 MiB." + de: "Obergrenze für ein einzelnes gespeichertes Artefakt — Diff, Briefing, Review-Verdict. Größeres wird mit Marker abgeschnitten; ein höherer Wert tauscht also Datenbankgröße gegen die Lesbarkeit sehr großer Diffs in der UI. Default 1048576 = 1 MiB." default: "1048576" required: false # --- webhook triggers --------------------------------------------------- - key: "webhooks_enabled" type: "boolean" - label: "Accept GitHub webhooks" - help: "Let labelled issues create jobs automatically. Off means no webhook route is registered at all." + label: + en: "Accept GitHub webhooks" + de: "GitHub-Webhooks annehmen" + help: + en: "Let labelled issues create jobs automatically. Off means the webhook route is not registered at all — not registered-and-refusing, so `/api/webhooks/github` simply 404s and GitHub shows the delivery as failed." + de: "Lässt gelabelte Issues automatisch Jobs erzeugen. Aus heißt: Die Webhook-Route wird gar nicht erst registriert — nicht registriert-und-ablehnend. `/api/webhooks/github` antwortet dann mit 404 und GitHub zeigt das Delivery als fehlgeschlagen." default: "true" required: false - key: "webhook_max_jobs_per_repo_hour" type: "integer" - label: "Webhook jobs per repo per hour" - help: "Rate ceiling per repository. A webhook is a public trigger — this is what bounds a label-spam incident." + label: + en: "Webhook jobs per repo per hour" + de: "Webhook-Jobs pro Repo und Stunde" + help: + en: "Rate ceiling per repository. A webhook is a PUBLIC trigger — anyone who can apply a label can start a job that spends money. This is what bounds a label-spam incident." + de: "Ratenobergrenze pro Repository. Ein Webhook ist ein ÖFFENTLICHER Trigger — wer ein Label setzen kann, kann einen Job starten, der Geld kostet. Dieser Wert begrenzt einen Label-Spam-Vorfall." default: "10" required: false - key: "webhook_max_jobs_per_sender_hour" type: "integer" - label: "Webhook jobs per sender per hour" - help: "Rate ceiling per GitHub user." + label: + en: "Webhook jobs per sender per hour" + de: "Webhook-Jobs pro Absender und Stunde" + help: + en: "Rate ceiling per GitHub user, applied on top of the per-repo ceiling. The per-repo limit alone does not stop one account from consuming a repository's whole hourly allowance." + de: "Ratenobergrenze pro GitHub-Account, zusätzlich zur Repo-Grenze. Die Repo-Grenze allein verhindert nicht, dass ein einzelner Account das komplette Stundenkontingent eines Repositories aufbraucht." default: "5" required: false - key: "egress_base_allowlist" type: "host_list" - label: "Extra egress hosts for runners" - help: "Hosts a runner may reach beyond the defaults (package registries and the like). Private and link-local addresses are rejected." + label: + en: "Extra egress hosts for runners" + de: "Zusätzliche Egress-Hosts für Runner" + help: + en: "Hosts a runner may reach beyond the defaults (package registries and the like). Everything not on the list is refused by the egress proxy. Private and link-local addresses are rejected outright — an allowlist entry cannot be used to reach your internal network." + de: "Hosts, die ein Runner zusätzlich zu den Defaults erreichen darf (Paket-Registries und Ähnliches). Alles, was nicht auf der Liste steht, lehnt der Egress-Proxy ab. Private und Link-local-Adressen werden grundsätzlich abgewiesen — über einen Allowlist-Eintrag kommt niemand in dein internes Netz." required: false # --- the two interlocked safety switches -------------------------------- @@ -406,34 +773,54 @@ setup: # boot-time safety refusal (plan.md §6). - key: "subscription_mode" type: "boolean" - label: "Subscription mode" - help: "Run jobs against a CLI subscription credential inside the runner. REQUIRES the acknowledgment below — activation is refused without it." + label: + en: "Subscription mode" + de: "Subscription-Modus" + help: + en: "Run jobs against a CLI subscription credential inside the runner instead of a metered API key. REQUIRES the acknowledgment below — activation is REFUSED without it, the plugin does not start degraded." + de: "Führt Jobs mit einer CLI-Abo-Credential im Runner aus statt mit einem abgerechneten API-Key. ERFORDERT die Bestätigung unten — ohne sie wird die Aktivierung VERWEIGERT; das Plugin startet nicht im eingeschränkten Modus." default: "false" required: false - key: "subscription_ack" type: "string" - label: "Subscription acknowledgment" - help: "Type your name to acknowledge that subscription jobs run your CLI credential inside a runner executing agent-written code. Required whenever subscription mode is on." + label: + en: "Subscription acknowledgment" + de: "Bestätigung Subscription-Modus" + help: + en: "Type your name to acknowledge that subscription jobs run your CLI credential inside a runner that is executing code an agent wrote. Required whenever subscription mode is on. A free-text name rather than a checkbox on purpose: the point is that a person, nameable afterwards, agreed to it." + de: "Trage deinen Namen ein, um zu bestätigen, dass Subscription-Jobs deine CLI-Credential in einem Runner verwenden, der von einem Agenten geschriebenen Code ausführt. Erforderlich, sobald der Subscription-Modus an ist. Bewusst ein Freitextname statt einer Checkbox: Es geht darum, dass eine Person zugestimmt hat, die sich hinterher benennen lässt." required: false - key: "unsafe_local" type: "boolean" - label: "Unsafe local backend" - help: "Run the shim as a local process instead of in a container. Development only. REQUIRES the uid below — activation is refused without it." + label: + en: "Unsafe local backend" + de: "Unsicheres lokales Backend" + help: + en: "Run the shim as a local process instead of in a container. DEVELOPMENT ONLY — agent-written code then executes on the middleware host itself, with only the uid below between it and your filesystem. REQUIRES that uid; activation is refused without it." + de: "Startet den Shim als lokalen Prozess statt im Container. NUR FÜR ENTWICKLUNG — von einem Agenten geschriebener Code läuft dann auf dem Middleware-Host selbst, getrennt von deinem Dateisystem nur durch die UID unten. Diese UID ist ERFORDERLICH; ohne sie wird die Aktivierung verweigert." default: "false" required: false - key: "unsafe_local_uid" type: "integer" - label: "Local runner uid" - help: "Dedicated unprivileged uid the jailed shim runs as. NEVER 0. Required whenever the unsafe local backend is on." + label: + en: "Local runner uid" + de: "UID für lokalen Runner" + help: + en: "Dedicated unprivileged uid the jailed shim runs as. NEVER 0, and never the uid the middleware itself runs as — that would give the job write access to the middleware's own files. Required whenever the unsafe local backend is on." + de: "Dedizierte unprivilegierte UID, unter der der eingesperrte Shim läuft. NIEMALS 0 und niemals die UID der Middleware selbst — sonst bekäme der Job Schreibzugriff auf deren eigene Dateien. Erforderlich, sobald das unsichere lokale Backend an ist." required: false # --- dormant ------------------------------------------------------------ - key: "tracker_polling_enabled" type: "boolean" - label: "Tracker polling (not yet available)" - help: "Poll a Jira or Linear tracker for labelled tickets. NOT WIRED in this release — six hardening fixes gate switch-on (cold-start budget, sender allowlist, label-application trigger, cross-source dedupe, ticket-id namespacing, frozen provider contract). Turning it on records the intent and logs a warning; nothing polls." + label: + en: "Tracker polling (not yet available)" + de: "Tracker-Polling (noch nicht verfügbar)" + help: + en: "Poll a Jira or Linear tracker for labelled tickets. NOT WIRED in this release — six hardening fixes gate switch-on (cold-start budget, sender allowlist, label-application trigger, cross-source dedupe, ticket-id namespacing, frozen provider contract). Turning it on records the intent and logs a warning; nothing polls. It is shown rather than hidden so the roadmap is visible and the key is stable when it does land." + de: "Fragt einen Jira- oder Linear-Tracker nach gelabelten Tickets ab. In diesem Release NICHT VERDRAHTET — sechs Härtungen stehen davor (Cold-Start-Budget, Absender-Allowlist, Trigger beim Label-Setzen, quellenübergreifende Deduplizierung, Namespacing der Ticket-IDs, eingefrorener Provider-Contract). Einschalten hält die Absicht fest und schreibt eine Warnung ins Log; gepollt wird nichts. Das Feld ist sichtbar statt versteckt, damit die Roadmap erkennbar bleibt und der Key stabil ist, wenn es soweit ist." default: "false" required: false