feat: add sanitized Kanban REST API - #61982
Conversation
… index The REST adapter's task creation did a SELECT-before-INSERT on idempotency_key with no uniqueness constraint, so two concurrent POSTs with the same key both passed the check and created duplicate tasks. Add a partial UNIQUE index on tasks(idempotency_key) scoped to live (non-archived) rows — matching exactly how create_task looks the key up (WHERE idempotency_key = ? AND status != 'archived'). Boards are separate SQLite files, so this gives board-scoped idempotency. Archiving a task still frees its key for reuse. The migration runs in the existing connect()/_migrate_add_optional_columns path so legacy DBs upgrade on open: it first nulls out any duplicate live keys (keeping the row the lookup would return) so the index can build, then drops the old non-unique idx_tasks_idempotency and creates the partial UNIQUE index. It uses plain statements (the CREATE UNIQUE INDEX DDL commits the dedupe UPDATE) to avoid nesting inside the migration's implicit txn. The adapter now catches the resulting IntegrityError on insert and returns the winning task with created=false / HTTP 200, preserving the documented idempotency semantics. Also removes the always-true `created = existing is None` dead code (the existing case returns earlier). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The adapter reached into private kanban_db internals (_normalize_board_slug, _append_event) and ran a raw UPDATE against the tasks table from the API layer. Promote a public normalize_board_slug() wrapper and add a public edit_task_fields() domain function that owns the title/body/priority UPDATE and the "edited" event, so the adapter imports only public API. edit_task_fields() also enforces state-machine honesty: editing the title or body of a completed (done) or archived task is refused (RuntimeError -> HTTP 409), consistent with the other transition guards. priority stays editable. The adapter checks the terminal state up front so a mixed assignee+title PATCH doesn't partially apply. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ections list_boards opened two SQLite connections per board (init_db then connect) just to count tasks; connect() already runs the schema/migration pass on first open, so _board_counts now uses a single connect_closing() per board. The events and runs endpoints loaded a task's full history and sliced [-limit:] in memory. Add a `limit` parameter to list_events()/list_runs() that applies the LIMIT in SQL (newest N by time, re-sorted oldest-first) so long histories don't materialise every row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…own link Three response-hardening fixes to the REST adapter: - kanban_db ValueError messages went straight into HTTP `detail`, which could leak internal specifics. Introduce _client_error(): it logs the raw exception server-side and returns a stable message — a known-safe validation string when recognised, otherwise a generic fallback. Applied to board-slug, list-filter, create, comment, and block errors. - _AUTH_HEADER_RE only matched `Authorization:` at the start of a line (`^` multiline), so a token inside a dumped curl command survived redaction. Match the header anywhere in a line. - link_tasks now _require_task's both parent and child, so linking a nonexistent task returns 404 (consistent with unlink) instead of a 400 from kanban_db's ValueError. Genuine invalid links (self/cycle) stay 400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… changes Extend the adapter test suite: idempotency uniqueness under a duplicate live insert (and archived-key reuse), the API returning the winner on an IntegrityError race, link 404s for missing parent/child, PATCH 409 on completed and archived tasks, events/runs limit correctness (newest N, oldest-first), and sanitized vs. safe error detail. Document the two deliberate contract improvements (link 404 instead of 400; PATCH 409 on completed/archived) and the now-real idempotency guarantee (partial UNIQUE index; concurrent duplicates resolve to one task). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
teknium1
left a comment
There was a problem hiding this comment.
Thanks for separating a narrow DTO-based contract from the rich operator dashboard surface. The premise is valid: current plugins/kanban/dashboard/plugin_api.py:158-176 serializes the complete task dataclass, and current main has no sanitized adapter.
Problems
website/docs/user-guide/features/kanban-rest-api.md:20-25documents a bearer credential for external controllers, but current main accepts Bearer only when it is the ephemeral dashboard_SESSION_TOKEN(hermes_cli/web_server.py:319-336). On gated deployments, routes require a verified cookie session (hermes_cli/web_server.py:361-369). Generic service bearer auth is limited to exact registered paths (hermes_cli/dashboard_auth/token_auth.py:54-74,144-164), and this change registers none.- The advertised idempotency is not concurrency-safe.
hermes_cli/kanban_db.py:2539-2552checks beforeBEGIN IMMEDIATEat :2580, andidx_tasks_idempotencyis non-unique (:1996-1999). The existing stress contract requires concurrent creates with one key to return one id and persist one row (tests/stress/test_atypical_scenarios.py:712-751).
Suggested changes
- Wire a real service-auth path and test it through the mounted dashboard app, or scope the documentation to dashboard-session clients.
- Make keyed creation atomic at the storage layer and add a concurrent-create regression.
Automated hermes-sweeper review.
| returned. The log endpoint returns only a bounded excerpt after Hermes secret | ||
| redaction and absolute-path removal. | ||
|
|
||
| Use the same authentication required by the Hermes dashboard deployment. In |
There was a problem hiding this comment.
This bearer flow is not available to an external controller on a gated dashboard: current main only accepts Bearer <_SESSION_TOKEN> in loopback mode (hermes_cli/web_server.py:319-336), while gated mode requires a cookie session. The generic service-token seam also only applies to exact registered paths (hermes_cli/dashboard_auth/token_auth.py:54-74), and this router registers none. Please add a real service-auth integration and E2E coverage, or change the contract/docs to dashboard-session authentication.
The partial UNIQUE index made a lost create race surface as sqlite3.IntegrityError: the retry loop only generated a fresh task id, so the same idempotency_key failed again and the caller crashed — violating the stress contract (idempotency_key_race) that concurrent creators with one key must all get the winner's id. Move the recovery into the storage layer: create_task_idempotent now owns the full contract (fast-path lookup, insert, and IntegrityError resolution to the live winner) and returns (task_id, created); create_task stays a same-signature wrapper returning the id. The REST adapter drops its duplicated lookup + IntegrityError handling and maps created=False to HTTP 200. Covered by a deterministic lost-race regression (winner injected via the _new_task_id seam), a 4-thread concurrent-create regression, and the existing stress scenario.
…n-auth seam Exact-path registration cannot cover routers with parameterised paths, so the seam gains register_token_route_prefix(prefix, exclude=...) with excluded subtrees (an external plugin surface can be token-authable while its interactive dashboard subtree stays on the cookie gate). Registrations may now demand a required scope: a verified principal lacking it is refused with 403 + an audit event. With a second service credential joining the stack this is what keeps one credential (e.g. the drain secret) from opening another surface's routes. The drain plugin passes its configured scope through; unscoped registrations keep the old accept-any-verified-principal behaviour. Also extracts the shared secret entropy gate out of the drain plugin into dashboard_auth.secret_strength (drain re-exports for compat).
New dashboard_auth plugin (mirroring the drain plugin) that gives external control planes a real authentication path to the sanitized kanban REST adapter — previously the documented bearer flow did not exist: loopback only accepted the ephemeral SPA session token and gated binds required a cookie session. The operator provisions HERMES_KANBAN_API_SECRET (>=256-bit, fail-closed entropy gate); the plugin registers a kanban-scoped token provider plus the /api/plugins/kanban/ prefix in the token-auth seam, excluding the interactive /dashboard subtree which stays on cookie/session auth. E2E-covered through the mounted dashboard app in gated mode: bearer drives the external surface, missing/wrong token is 401, the drain credential is refused with 403, and the dashboard subtree never accepts the bearer.
Replace the fictional HERMES_DASHBOARD_TOKEN bearer flow with the real contract: provisioning the service secret, its kanban-only scope, the excluded interactive dashboard subtree, and the fallback/token-only semantics with and without the secret.
…the REST API Gives an external control plane what it needs to visualise and drive orchestrator fan-out without touching the internal dashboard API: - GET /profiles — sanitized assignee roster: name, description (the operator-facing profile.yaml text the built-in decomposer routes on), and has_description. Models, providers, filesystem paths, env/config state, and skill inventories stay private; 503 with a generic detail when listing fails. capabilities now advertises profiles_api: true (profile_execution stays false — routing happens via task assignment only). - Task DTO gains created_by, so a consumer can attribute which profile/surface created each card, not just who executes it. - Run entries gain profile — the executor of that specific attempt, which diverges from task.assignee when a task is reassigned between retries. The sanitization contract test now pins the widened run shape while still refusing summary/metadata/error leakage. The new endpoint sits under the token-guarded /api/plugins/kanban/ prefix, so the service-credential scope rules apply unchanged (E2E-covered through the mounted app).
Two scenarios in the atypical stress suite were stale against deliberate kernel changes and failed on every run: - workspace_nonexistent_path read task.spawn_failures, a field renamed to consecutive_failures when NousResearch#20410 unified the circuit-breaker counter across spawn/timeout/crash outcomes. It also never actually reached the workspace-failure path anymore: since the dispatcher crash-loop fix (2026-05-05) tasks whose assignee is not a real profile on disk are skipped before claiming, so the scenario ran zero of its assertions. The scenario now creates the profile dir first and asserts on consecutive_failures — the counter provably ticks to 1 on a nonexistent dir: workspace. - parent_in_different_status_states expected a child of an archived parent to stay in todo, contradicting the documented recompute_ready contract ('done' OR 'archived' parents unblock children) introduced by the "treat archived parent tasks as terminal for dependency resolution" fix. The expectation now matches the kernel: archived, like done, is terminal. Full suite: 28 scenarios, 0 failures, 0 skips.
|
Thanks — both findings were valid. Addressed in four commits:
Wired a real service-auth path instead of scoping the docs down:
E2E coverage goes through the real mounted dashboard app in gated mode (
Correct — the partial UNIQUE index made a lost race surface as sqlite3.IntegrityError: the retry loop only
Verified live as well: 6 concurrent POST /tasks with one Idempotency-Key against a running server → exactly one |
|
Note on the stress-suite commit ( While verifying this PR against Both are cases of the stress suite going stale against deliberate kernel changes from May 2026:
Happy to split this commit out into its own PR if you'd rather land the test repair independently — the failures |
Resolves four collisions with a month of upstream drift: - kanban_db.create_task_idempotent: keep the (task_id, created) return while picking up the new project_source_task_id parameter and the _inherit_notify_subs call upstream added. - kanban_db.create_task: forward model_override, provider_override, reasoning_effort and project_source_task_id. The thin wrapper was introduced here after those parameters landed upstream, so the dashboard plugin_api call site broke on the merged tree. - kanban_db: keep edit_task_fields alongside the set_model_override / set_reasoning_effort pair upstream inserted at the same spot. - test_dashboard_token_auth: drop the two seam tests upstream pruned intentionally, keep the new prefix/scope coverage. - kanban.md: keep the REST API pointer on top of the widened toolset list.
_connection called init_db() before connect(), which evicts the path from _INITIALIZED_PATHS and so forces connect() down its slow path on every single request: header validation, the full integrity probe, the schema executescript and the whole optional-column migration, all behind the cross-process init lock — plus a second connection for the actual work. connect() already creates the parent directory and auto-runs schema and migrations on a path's first open, which is why _board_counts dropped the same redundant init_db call. Route the task endpoints through connect_closing for the same reason: this is a polling surface for external control planes, and it should not pay a migration per GET.
website/sidebars.ts enumerates docs explicitly, so the new page was reachable only through the inline link from the Kanban page.
contributor-check resolves every non-merge commit author against this map and fails the run on an unmapped, non-noreply address.
plugin_api.router is now the composed router: the sanitized external surface at the root plus the operator dashboard under /dashboard. Tests that mount it on a bare app and call /api/plugins/kanban/<route> get a 404 for every dashboard route. The same three-line change was already applied to the dashboard tests that existed when this branch was cut; these four files landed upstream in the meantime and need it too.
Sanitized Kanban REST API for external control planes
Summary
Adds a narrow, authenticated, safe-by-default REST API over the existing Hermes Kanban store, mounted at
/api/plugins/kanban. It gives external control planes a stable integration boundary for creating, routing,inspecting, and managing Kanban tasks — without reading raw SQLite state, touching internal dashboard routes, or
executing profiles directly.
Design constraints, all preserved end to end:
hermes_cli.kanban_db— same database, WAL/transaction behavior, task state machine, dependency promotionrules, and dispatcher lifecycle. No second schema, no second queue.
launches it.
API surface
GET /health,GET /capabilitiesGET /boards,GET /boards/{id-or-name}GET /profiles— sanitized assignee roster (read-only)GET /tasks,POST /tasks,GET /tasks/{id},PATCH /tasks/{id}POST /tasks/{id}/comment,/complete,/block,/unblock,/archivePOST /tasks/{parent}/links/{child},DELETE /tasks/{parent}/links/{child}GET /tasks/{id}/events,/runs,/logAll request models are strict Pydantic (
extra="forbid"); unknown fields are rejected.Authentication
External controllers authenticate with a dedicated service credential, wired through the dashboard-auth framework
rather than bolted onto the routes:
and required scopes: a verified principal lacking the route's scope is refused with 403 + an audit event. With
multiple service credentials stacked, one credential can no longer open another surface's routes — the drain secret
cannot drive the kanban API and vice versa.
dashboard_authplugin (mirroring the drain plugin) verifiesAuthorization: Bearer $HERMES_KANBAN_API_SECRETwith a constant-time compare and vouches for akanban-scoped principal. Aweak/short/low-entropy secret (< 256 bits) fails closed at load; the shared entropy gate now lives in
dashboard_auth.secret_strengthand is reused by both plugins.Without the secret set, the plugin is a no-op and prior behavior is unchanged.
Idempotency (concurrency-safe)
POST /tasksaccepts anIdempotency-Keyheader oridempotency_keybody field. A repeat returns the existing livetask with HTTP 200 and
created: false; a fresh insert returns HTTP 201 andcreated: true.The guarantee holds under concurrency and is enforced at the storage layer:
idempotency_key(live, non-archived tasks; legacy duplicates deduped duringmigration) closes the check-then-insert race. Archiving a task frees its key.
create_task_idempotentresolves a lost race to the winner's row instead of surfacingIntegrityError— everycaller (CLI, tools, dispatcher, REST) gets one task id, satisfying the multi-process stress contract
(
idempotency_key_race).Sanitization
Task and run DTOs expose workflow state plus routing attribution —
assignee,created_by(which profile/surfacecreated the card), and the executing
profileper run — and deliberately omit: task bodies and results, commenttext, workspace paths, branch names, claim locks, worker PIDs, session IDs, idempotency keys, run
summaries/metadata/errors, and raw event payloads.
GET /profilesreturns onlyname,description, andhas_description— no models, providers, paths, env, orskill inventories.
GET /tasks/{id}/logreturns a bounded excerpt (8 KiB default, 32 KiB max) after Hermes secretredaction,
Authorization-header scrubbing, and absolute-path replacement with[PATH].Error details follow an allowlist: known-safe validation messages pass through verbatim; anything else collapses to a
stable generic detail with the raw error logged server-side only.
Dashboard compatibility
The first-party dashboard needs a richer API than should be exposed externally, so its existing routes are namespaced
under
/api/plugins/kanban/dashboard(bundled dashboard JS updated). Both routers share the samekanban_dbimplementation. Third-party consumers of the previously undocumented dashboard routes should migrate to the sanitized
API or adopt the new namespace.
Documentation
website/docs/user-guide/features/kanban-rest-api.md— credential provisioning, the full endpoint contract,idempotency semantics, and a worked example (parent operation → dependent children → dependency links → completion →
polling sanitized events/runs/logs).
Testing
Affected suites (REST adapter, kanban store, token-auth seam, auth middleware, both auth plugins):
Auth is covered end-to-end through the real mounted dashboard app in gated mode: valid bearer drives the external
surface, missing/wrong token → 401, a foreign-scoped service credential → 403, and the dashboard subtree never
accepts the bearer.
Idempotency race coverage: a deterministic lost-race regression at the storage layer, a threaded concurrent-create
regression, and the multi-process stress scenario. Exercised live as well: 6 concurrent POST /tasks with one key →
exactly one 201 + five 200s, one row persisted.
Full atypical stress suite on this branch:
python tests/stress/test_atypical_scenarios.py
28 scenarios, 0 failures, 0 skips
▎ Note: one commit (test(stress): catch atypical scenarios up to current kanban contracts) repairs two stress
▎ scenarios that fail on current main independently of this PR — they had gone stale against the spawn_failures →
▎ consecutive_failures rename (#20410) and the archived-parents-are-terminal dependency fix. Included so the suite is
▎ green on this branch; happy to split it into its own PR.
Out of scope
Profile execution or configuration endpoints, raw worker/session state, product-specific workflow types, and any
second Kanban schema or storage layer.
Follow-ups