Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
50b9e42
refactor(api): extract a context-free submitTask() service (M2 Track T1)
hutusi Aug 3, 2026
bd04d82
feat(api): project API keys — Bearer agr_ auth, scopes, project bindi…
hutusi Aug 3, 2026
6147abd
fix(api): archived projects accept no new work
hutusi Aug 3, 2026
9d9e257
refactor: move submission, quota, and worker-liveness into @agrippa/o…
hutusi Aug 3, 2026
1e4f189
feat: scheduled task submission (M2 Track T3)
hutusi Aug 3, 2026
bd84369
feat: make a scheduled run actually useful — validated params and dat…
hutusi Aug 3, 2026
7cc5695
feat: inbound webhook triggers (M2 Track T4)
hutusi Aug 3, 2026
cb3d50e
fix(api): bound the trigger body before reading it, and warn where th…
hutusi Aug 3, 2026
4ba495b
fix: address codex round-1 review — enqueue durability, trigger dedup…
hutusi Aug 3, 2026
b81119e
fix: address CodeRabbit round-1 — per-row reconciliation, attempt-age…
hutusi Aug 3, 2026
111ee44
fix: address codex round-2 — atomic submission, live calendar recover…
hutusi Aug 3, 2026
22c1052
docs: say plainly that duplicate suppression depends on the sender
hutusi Aug 3, 2026
41f8c7c
fix: address codex round-3 — idempotent firings, calendar read order,…
hutusi Aug 3, 2026
f021d3a
fix: self-review before merge — the redelivery guard was in the wrong…
hutusi Aug 3, 2026
9393dd2
fix: address round-5 review — notifications were shipping empty
hutusi Aug 4, 2026
733fe19
fix: self-review of 9393dd2 — the last retry was a silent one
hutusi Aug 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ Browser (SPA) ──REST──▶ api ──singleton-keyed send──▶ pg-bos
|---|---|
| `packages/core` | Domain vocabulary (roles, tiers, statuses), the pure run state machine, zod request schemas shared by API and SPA, `pickLocale`, queue names. Depends on nothing internal. |
| `packages/db` | Drizzle schema (one file per aggregate), committed SQL migrations, idempotent seed, AES-256-GCM secrets crypto, the advisory-locked migrator. |
| `packages/orchestration` | The platform brain: `compile.ts` (YAML → validated compiled JSON), `expression.ts` (the deliberately non-Turing-complete `${...}` language), `engine/engine.ts` (the run loop), `queue.ts` (pg-boss factory), `engine/bus.ts` + `redis-bus.ts` (live-event fan-out), `resolve.ts` (submit-time param validation + model-role resolution), `seed-builtins.ts`. |
| `packages/orchestration` | The platform brain: `compile.ts` (YAML → validated compiled JSON), `expression.ts` (the deliberately non-Turing-complete `${...}` language), `engine/engine.ts` (the run loop), `queue.ts` (pg-boss factory + the best-effort post-commit send), `engine/bus.ts` + `redis-bus.ts` (live-event fan-out), `resolve.ts` (submit-time param validation + model-role resolution), `submit.ts` + `run-plan.ts` (the one submission path, whatever triggered it), `usage.ts` (quota gate + reporting), `worker-executors.ts` (which executors are live), `authority.ts` (may this owner still submit — checked at every firing), `schedule-fire.ts` + `trigger-fire.ts` (unattended entry points), `notifications.ts`, `audit.ts`, `artifact-store.ts`, `remote/` (daemon transport + routing), `seed-builtins.ts`. |
| `packages/executor-core` | The `Executor` interface and normalized event stream (ADR-0005), `UsageMeter`, and `FakeExecutor` — the compliance fixture every executor must satisfy. |
| `packages/executor-claude` | Maps a step request onto Claude Agent SDK `query()` options and SDK messages back onto normalized events; scans `.agrippa/artifacts/` on success. |
| `packages/executor-codex` | Wraps the OpenAI Codex CLI (`codex exec --json`): JSONL events → normalized events, native OS sandbox for containment, fenced-json artifact synthesis for read-only steps (ADR-0011). |
| `packages/i18n` | en/zh-CN resources by namespace + the server-side error-message lookup; a parity test keeps locales in lockstep. |
| `apps/api` | Hono: better-auth, RBAC middleware, registries, template publishing, task submission (grant + quota gating), run lifecycle, approvals, artifacts, SSE, audit log. Serves the built SPA in production. Never imports executors. |
| `apps/api` | Hono: better-auth + `agr_` API keys, RBAC middleware, registries, template publishing, run lifecycle, approvals, artifacts, SSE, audit log, and the management surfaces for schedules, webhook triggers and keys. Submission itself lives in `@agrippa/orchestration` (the worker fires schedules and triggers, so it cannot live here). Serves the built SPA in production. Never imports executors. |
| `apps/worker` | pg-boss consumers around the engine; real deps: `GitWorkspaceManager` (clone with scrubbed credentials), `GitScmService` (branch/push/PR via provider REST — ADR-0011), `DbResourceMaterializer` (skills → disk, MCP + secrets), `DiskArtifactStore`, `DemoExecutor` (token-free, loop-round-aware), checkpoint-expiry handler, reconciliation sweeper. Registers executors and asserts them against core's `EXECUTOR_CATALOG` at boot. |
| `apps/web` | React SPA: TanStack Router/Query, shadcn/ui (GitLab-style sidebar shell, indigo/violet tokens, light+dark), react-i18next. The load-bearing piece is `TaskParamsForm` — rendered from compiled template inputs, so new task types need zero frontend work. |
| `templates/` | Builtin templates (6 × `agrippa/v3` + the `swdev.bug-localize-fix.yaml` v1 fixture), shared subagent prompts, builtin skills. Compiled + published at boot, checksum-guarded. Author new templates in v3 (ADR-0016); v1/v2 still accepted. |
Expand All @@ -36,7 +36,7 @@ Dependency direction is enforced by `scripts/check-deps.ts` (runtime deps only).

## Key flows

**Submit → run.** `POST /projects/:id/tasks` validates params against the compiled input schema (the same schema the SPA rendered the form from), verifies each `repoRef` is owned by the project, resolves model roles → concrete granted models (frozen into `runs.model_resolution`), pins the authorized skills/MCP into `runs.resource_manifest` (required grants enforced, optional included only when granted), checks hard-stop quota headroom, then inserts task+run in one transaction and sends a pg-boss job keyed by run id. A worker sweeper re-enqueues stragglers (and runs left paused by a lost approval-resume), so a run can never be stranded nor double-queued.
**Submit → run.** Four entry points converge on one `submitTask()` in `@agrippa/orchestration`: a browser session, an `agr_` API key on the same route, a cron `schedule.fire` job, and a signed `POST /api/triggers/:token` recorded as a delivery and submitted by the worker. Everything below is therefore enforced once, for all four — and the three unattended ones re-check at fire time that their owner may still submit, since a standing order must never outlive its author's access. `POST /projects/:id/tasks` validates params against the compiled input schema (the same schema the SPA rendered the form from), verifies each `repoRef` is owned by the project, resolves model roles → concrete granted models (frozen into `runs.model_resolution`), pins the authorized skills/MCP into `runs.resource_manifest` (required grants enforced, optional included only when granted), checks hard-stop quota headroom, then inserts task+run in one transaction and sends a pg-boss job keyed by run id. A worker sweeper re-enqueues stragglers (and runs left paused by a lost approval-resume), so a run can never be stranded nor double-queued — which is why every post-commit send is best-effort: the row is durable and the sweeper delivers it, so reporting a send failure as a submission failure would only invite a retry that creates a second run.

**Engine loop.** The engine (worker-side) walks flow nodes (phases and bounded loops): `when:` conditions, optional-resource skips, per-step retries, token-limit checks at every boundary. Loop rounds key every row by `(stepId, iteration)` and derive the resume iteration from persisted rows. Agent steps resolve their slot's executor + faber from `runs.agent_bindings` and stream normalized executor events which the engine persists to `run_events` (per-run monotonic `seq`), mirrors to the bus, and projects into `run_steps`/`token_usage`/`artifacts`. Before every agent attempt the worker replaces executor project configuration and rematerializes only authorized skills. System steps cover checkout plus the platform git write-path (`git.branch`/`git.push`/`pr.open` — the PR body includes accepted-findings waivers). At the end it enforces the artifact contract — *succeeded* always means the contracted outputs exist (latest loop iteration wins).

Expand All @@ -60,7 +60,9 @@ Dependency direction is enforced by `scripts/check-deps.ts` (runtime deps only).
8. **The worker trusts only the pinned manifest**: repos are project-scoped and skills/MCP resolve solely from `runs.resource_manifest`, never the mutable global registry.
9. **Lifecycle mutations are atomic**: run status transitions are compare-and-swap on the expected status and event `seq` is allocated by the database, so concurrent writers can't clobber a status or collide on a seq (`run-lifecycle.ts`).
10. **Approved evidence is the published tree**: platform Git reads only its private gitdir/index, excludes runtime configuration from both evidence and publication, and refuses errors, empty snapshots, or any patch mismatch before push (ADR-0012).
11. **A project provider credential outranks worker env, and slots resolve single-provider**: the executor overlay replaces the wire protocol's whole auth-var family (never merges), and submit-time resolution satisfies all of a slot's scoped roles from one provider — a step's base URL is process-wide, so a mixed slot could never execute (ADR-0013).
11. **Unattended work still names a human, and its authority is re-checked at the point of use**: `tasks.created_by`/`runs.created_by` are NOT NULL, so a schedule or webhook trigger submits as its author, and `checkWorkAuthority` re-tests project membership, project status and task-type availability at every firing rather than freezing them at creation — a standing order can do exactly what its owner could do right now, and never more.
12. **One firing, one run**: every unattended submission writes an idempotency key with the run — `runs.origin_key`, unique where not null, set inside the submission transaction — so a queue that re-delivers a job whose transaction already committed cannot charge the project a second time. Only the human paths omit it, where a second request really is a second intent.
13. **A project provider credential outranks worker env, and slots resolve single-provider**: the executor overlay replaces the wire protocol's whole auth-var family (never merges), and submit-time resolution satisfies all of a slot's scoped roles from one provider — a step's base URL is process-wide, so a mixed slot could never execute (ADR-0013).

## Where to look

Expand Down
Loading
Loading