feat(kanban): surface-agnostic notification substrate + orchestrator supervision loop

[aeberts]

What does this PR do?

Implements NousResearch#49190. Today, Kanban can only send task notifications to chat apps (Telegram/Discord/Slack). This PR lets those notifications reach any surface — CLI, TUI, web, or an AI orchestrator — and uses that to let an orchestrator wait for its tasks to finish and react, instead of constantly polling.

For example, an orchestrator can sign up for updates on its tasks, hand control back, and be woken again later — either when all of its sub-tasks have finished (so it can review the combined result), or when one gets stuck (so it can step in). It no longer has to repeatedly poll the board with kanban_list to find out what happened. The change is purely additive: it doesn't alter how tasks are scheduled or recorded (the task_events log is untouched) — it only watches the existing system and delivers the notifications.

For reviewers — this PR can be split. It's structured as clean, per-feature commits: the reusable substrate (F01–F06) is self-contained, and the supervision loop (F07–F15, M01) builds on top of it. They're submitted together because the loop is what exercises the substrate end to end, but I'm happy to split this into a substrate-first PR plus a supervision follow-up if that's easier to review — just say the word.

The gap. Kanban already has a durable task-event log and a "claim-once" notification loop, but both are tied to the gateway messaging Platform. A subscription is the tuple (task_id, platform, chat_id, thread_id), and the only runtime consumer is the gateway watcher, which can only deliver to a connected messaging adapter. So every non-gateway surface (CLI, TUI, WebUI, or an orchestrator run) has to reinvent its own partial notifier, and subscriber identity is inferred from whoever calls kanban_create (a worker that creates a child subscribes itself, not the user).

The recommended way to run a dependency graph today, per the kanban tutorial, Story 1, is for a human to script the whole DAG up front (--parent-at-create). The injected KANBAN_GUIDANCE protocol is "decompose, then complete your own task", with no "subscribe, wait for fan-in, then judge or triage" step. A supervising agent's only option is to busy-poll, which burns turns, is unreliable, and ends with its turn so it never sees the result.

This is a recurring gap in practice. The common workarounds are a messaging-platform subscription (kanban notify-subscribe --platform telegram) or a separate CLI watcher (kanban watch --kinds completed), both of which are the gateway-bound paths this substrate generalizes.

The primary use case for this PR is to enable LLM-driven kanban use, reusing as much of the existing kanban tools and code as possible while staying close to the Nous Research Kanban tutorial examples.

Related Issue

Closes NousResearch#49190.

Also references NousResearch#53234, a separate chat -q / openai-codex provider bug that gates the fully-automated live loop (see Limitations). It is not introduced or fixed here.

Type of Change

• New feature (non-breaking change that adds functionality)

Changes Made

Note: F01-F15 and M01 are internal tracking IDs that group the commits. Issue details (rationale and trade-offs considered) are available on request.

Substrate (reusable; F01-F06).

• F01 generalize kanban_notify_subs: surrogate id plus subscriber_kind plus target (additive _add_column_if_missing migration; subscriber_kind="gateway" keeps the old 4-tuple as its target, so existing rows are unchanged). hermes_cli/kanban_db.py
• F02 re-key the claim/cursor (exactly-once) API on the surrogate id; a gateway-tuple shim keeps callers unchanged. hermes_cli/kanban_db.py
• F03 delivery-adapter registry keyed by subscriber_kind; the gateway watcher becomes the first registered adapter rather than the only path. gateway/kanban_delivery.py, gateway/kanban_watchers.py
• F04 explicit kanban_subscribe primitive (declare subscriber_kind and target), which fixes the inferred-identity problem at the contract level. hermes_cli/kanban_db.py, hermes_cli/kanban.py
• F05 / F06 CLI and TUI delivery adapters on a shared, surface-agnostic notice store. gateway/kanban_delivery.py

Supervision loop (built on the substrate; F07, F09-F15, M01).

• F07 orchestrator supervision subscription: subscribe to a whole subtree, emit a curated fan-in snapshot on child completion. gateway/kanban_delivery.py, hermes_cli/kanban_db.py
• F09 / F10 the two minimal re-engagement triggers: fan-in ("done, then judge", writes a [kanban:reengage] root handoff) and blocked ("stuck, then triage", a [kanban:triage] handoff). They are mutually exclusive per snapshot. hermes_cli/kanban_db.py, hermes_cli/kanban.py
• F11 / F12 live engine: the dispatcher auto-wakes an orchestrator-mode supervisor on a blocked child; re-engagement runs in the dispatch tick; subtree subs are delivered live via a non-advancing peek. hermes_cli/kanban_db.py, gateway/kanban_watchers.py
• F13 agent-facing kanban_subscribe tool plus subscribe-and-yield guidance (parents-at-create). tools/kanban_tools.py, toolsets.py, agent/prompt_builder.py
• F14 the notifier tick delivers surface-agnostic subs with no messaging platform connected. gateway/kanban_watchers.py
• M01 / F15 live-wake delivery: supervision notices surface into an idle interactive session (CLI and TUI) at turn boundaries, never mid-turn (see Critical: Background notifications repeatedly overwrite user input, TUI becomes unresponsive NousResearch/hermes-agent#42173), with debounce and a completion-wake. cli.py, tui_gateway/server.py, hermes_cli/kanban_db.py

Reuses upstream's existing past-tense event vocabulary (completed, blocked, and so on); there is no parallel task.done namespace. 25 files changed (9 source, 16 test), about +6.7k/-222 lines. No dependency changes.

Architecture & data flow

These diagrams make two things visual: (1) we extend the existing event/claim/deliver mechanism rather than replace it, and (2) we don't break the one delivery path that exists today (gateway → messaging) — every non-gateway surface is purely additive, gated behind a new registry seam. In the architecture diagrams, nodes shown in green are new in this PR; everything else is unchanged from upstream.

1. Overall architecture

Everything upstream of the delivery step is untouched: the worker/dispatcher, the task_events log, recompute_ready scheduling, the kanban_notify_subs table (additively extended, old rows unchanged), and the watcher's claim/cursor + failure-accounting + multi-board fan-out. The only new seam is the delivery registry keyed by subscriber_kind. The gateway branch through it is the old inlined send, moved verbatim; CLI/TUI/orchestrator are new branches that register without touching the watcher.

Today (upstream)

flowchart LR
  Wk["Worker / dispatcher"] --> TE[("task_events log")]
  TE --> RR["recompute_ready scheduler"]
  NS[("kanban_notify_subs<br/>task_id, platform, chat_id, thread_id")] --> CL
  TE --> CL
  subgraph GW["Gateway notifier watcher"]
    CL["claim_unseen_events_for_sub"] --> SND["inlined per-event send"]
  end
  SND --> MSG["Messaging surface<br/>Telegram / Discord / Slack"]
  Other["CLI / TUI / orchestrator"] -. "no delivery path:<br/>busy-poll kanban_list" .-> TE

Loading

With this PR

flowchart LR
  Wk["Worker / dispatcher"] --> TE[("task_events log<br/>UNCHANGED")]
  TE --> RR["recompute_ready scheduler<br/>UNCHANGED"]
  NS[("kanban_notify_subs<br/>+ surrogate id, subscriber_kind, target<br/>old rows unchanged")] --> CL
  TE --> CL
  subgraph GW["Gateway notifier watcher — same claim/cursor, failure accounting, fan-out"]
    CL["claim"] --> REG{"delivery registry<br/>by subscriber_kind"}
  end
  REG -->|gateway| GA["GatewayDeliveryAdapter<br/>(old send path, verbatim)"]
  GA --> MSG["Messaging surface<br/>Telegram / Discord / Slack"]
  REG -->|cli| CA["CLIDeliveryAdapter"]
  REG -->|tui| TA["TUIDeliveryAdapter"]
  REG -->|orchestrator| OA["OrchestratorDeliveryAdapter<br/>subtree claim + fan-in snapshot"]
  CA --> NOT[("kanban_notices store")]
  TA --> NOT
  OA --> NOT
  NOT --> LW["live-wake / hermes kanban notices<br/>idle CLI + TUI re-engage"]

  classDef new fill:#e6ffed,stroke:#22863a,color:#22863a;
  class REG,CA,TA,OA,NOT,LW new;

Loading

2. A kanban event reaching the CLI / orchestrator

There is no CLI delivery path today — a CLI orchestrator can only busy-poll kanban_list, and because it polls inside a turn it ends the turn before the result lands, so it never sees fan-in. After this PR the same terminal event is claimed by the registered adapter, persisted as one notice, and surfaced into the idle REPL at a turn boundary (never mid-turn), which re-engages the orchestrator.

Today (upstream)

sequenceDiagram
  participant U as User (CLI)
  participant O as Orchestrator turn
  participant B as task_events
  participant Wk as Worker
  Note over U,Wk: no CLI delivery path
  O->>B: kanban_list (poll)
  O-->>U: turn ends (no result yet)
  Wk->>B: child completed / blocked
  Note over O: turn already ended — never sees fan-in<br/>must busy-poll to notice

Loading

With this PR

sequenceDiagram
  participant U as User (idle CLI REPL)
  participant O as Orchestrator
  participant Sub as kanban_notify_subs
  participant B as task_events
  participant Wk as Worker
  participant NW as Gateway notifier tick
  participant OA as OrchestratorDeliveryAdapter
  participant N as kanban_notices
  O->>Sub: kanban_subscribe(root, kind=orchestrator, scope=subtree)
  O-->>U: yield turn (idle)
  Wk->>B: child completed / blocked
  NW->>B: tick: subtree has unseen events? (non-advancing peek)
  NW->>OA: route by subscriber_kind
  OA->>B: claim_unseen_subtree_events_for_sub (claim-once)
  OA->>N: persist ONE supervision notice (fan-in snapshot)
  U->>N: turn-boundary drain (_drain_kanban_live_notices)
  N-->>O: re-engage queued on _pending_input (next turn)
  O->>O: judge fan-in / triage block / goal complete

Loading

3. A kanban event reaching a gateway-mediated surface (Discord)

This is the path that already works, and the point of the diagram is that it does not change. The watcher still polls subs, still claims with the same cursor, still runs the same failure accounting and artifact upload, and still calls self.adapters[Discord].send(...) with a byte-for-byte identical message. The only difference is one indirection hop — the send now resolves through get_delivery_adapter('gateway') instead of being inlined.

Discord is just the example here: the gateway adapter resolves the messaging platform generically (self.adapters[platform]), so every chat platform Hermes supports today (Telegram, Slack, and the rest) keeps working through this same path with no new per-platform adapters — the only "adapters" this PR adds are the four subscriber_kind delivery adapters (gateway/cli/tui/orchestrator), all included here.

Today (upstream)

sequenceDiagram
  participant Wk as Worker
  participant B as task_events
  participant NW as Gateway notifier watcher
  participant Sub as kanban_notify_subs
  participant D as Discord adapter
  Wk->>B: completed
  NW->>Sub: poll subs
  NW->>B: claim_unseen_events_for_sub
  NW->>D: self.adapters[Discord].send(chat_id, msg)
  D-->>Wk: ✔ Kanban {id} done — {title}

Loading

With this PR

sequenceDiagram
  participant Wk as Worker
  participant B as task_events
  participant NW as Gateway notifier watcher
  participant Sub as kanban_notify_subs
  participant REG as delivery registry
  participant GA as GatewayDeliveryAdapter
  participant D as Discord adapter
  Wk->>B: completed
  NW->>Sub: poll subs (subscriber_kind='gateway')
  NW->>B: claim_unseen_events_for_sub (same claim/cursor)
  NW->>REG: get_delivery_adapter('gateway')
  REG->>GA: deliver(batch)
  GA->>D: self.adapters[Discord].send(chat_id, msg)
  D-->>Wk: ✔ Kanban {id} done — {title} (identical message)

Loading

How to Test

Automated gate. Run the substrate test files explicitly (-k kanban pulls in about 19 pre-existing upstream cross-file isolation failures that pass per-file):

pytest \
  tests/gateway/test_kanban_orchestrator_delivery.py \
  tests/gateway/test_kanban_cli_delivery.py \
  tests/gateway/test_kanban_tui_delivery.py \
  tests/gateway/test_kanban_delivery_registry.py \
  tests/gateway/test_kanban_watchers_orchestrator_gate.py \
  tests/hermes_cli/test_kanban_live_notices.py \
  tests/hermes_cli/test_kanban_reengage.py \
  tests/hermes_cli/test_kanban_notify.py \
  tests/hermes_cli/test_kanban_notify_id_rekey.py \
  tests/hermes_cli/test_kanban_subtree_closure.py \
  tests/hermes_cli/test_kanban_db_init.py \
  tests/hermes_cli/test_kanban_dispatch_supervisor.py \
  tests/tools/test_kanban_subscribe_tool.py \
  tests/tools/test_kanban_tools.py \
  tests/tui_gateway/test_kanban_live_wake.py -q
# 224 passed

Manual live testing. Set up a small board the dependency-safe way and supervise it from an idle session:

1. In hermes chat (orchestrator mode): create 2-3 prerequisite cards, then the root in one kanban_create call with the subtasks as parents (so it is born todo); kanban_subscribe(root); then yield.
2. Let the dispatcher work the board. A worker blocks on a decision, the supervisor is woken into your idle session to surface it, you answer in chat, it unblocks, fan-in happens, then a goal-complete wake arrives.
3. The full loop (subscribe, yield, block-wake, decide, fan-in, goal-complete) runs hands-off beyond the one human decision, on both CLI and TUI surfaces.

The two live transcripts and the gate output can be attached to this PR on request.

Demonstrated behaviour change (A/B)

The patch changes orchestrator behaviour, not just plumbing:

• Fan-in / quorum (F09). Re-engaged on fan-in: without the curated handoff a manager redundantly rebuilds the deliverable from scratch; with it, the manager verifies the child's existing artifact and finishes. Redo, then verify-and-finish.
• Human-in-the-loop / blocked (F10/F11). The deadlock gap (a blocked child means the root stays todo and the supervisor never wakes) versus a woken supervisor triaging directly off the handoff. Shown end-to-end live on the TUI: block surfaced, human decides, supervisor records and unblocks, then goal-complete.

Relationship to existing features

• kanban swarm. The swarm command builds a fixed topology up front (a blackboard root, N parallel workers, a verifier gated on all workers, then a synthesizer). This PR is the general subscribe / deliver / re-engage event mechanism that such a shape sits on, not a competing topology. The re-engagement handoff deliberately mirrors the kanban_swarm blackboard comment shape, so the two share a convention rather than fork one.
• auto_subscribe_on_create. The existing config gate that auto-subscribes the origin session on create still works unchanged. The explicit kanban_subscribe primitive (F04) and tool (F13) are complementary: they let a caller declare the target deliberately, which is what fixes the worker-subscribes-itself case, without removing the auto-subscribe default.
• Typed block reasons and the unblock breaker (feat(kanban): typed block reasons + unblock-loop breaker NousResearch/hermes-agent#52848). That change types each block (dependency, needs_input, capability, transient), auto-resumes dependency blocks, and after a recurrence limit routes a repeatedly re-blocked task to triage. The re-engagement here sits on top of it. A supervisor wakes only on the blocked event feat(kanban): typed block reasons + unblock-loop breaker NousResearch/hermes-agent#52848 reserves for the human-needing kinds; a dependency block emits dependency_wait and auto-resumes, so it never wakes a supervisor. F11's own brake counts a different signal (consecutive supervisor turns that did not resolve a block) and mirrors the existing dispatcher consecutive-failure machinery, so it does not duplicate feat(kanban): typed block reasons + unblock-loop breaker NousResearch/hermes-agent#52848's per-task recurrence counter.

Scope / non-goals

• No new primitives; additive schema migration; the task_events log is untouched.
• The re-engagement code only observes scheduling. It supplies the missing summary or handoff; it never promotes, reschedules, or writes task status (it relies on the existing recompute_ready).
• Re-engage with a fresh turn, never mid-turn. Waking a running session by injecting a message mid-thought is a known hazard (see Critical: Background notifications repeatedly overwrite user input, TUI becomes unresponsive NousResearch/hermes-agent#42173) and is out of scope; live-wake delivers only at idle or turn boundaries.
• Explicit subscribe, not auto-enroll-everything. This avoids notification spam and keeps the manager in control of what it watches.
• One coherent change. This is deliberately part substrate, part MVP / proof-of-concept: the re-engagement loop is what exercises the substrate end to end, so the two are submitted together. The branch is clean per-feature commits, so it can be split into substrate-first (F01-F06) plus a supervision follow-up (F07-F15) if you prefer smaller PRs.

Limitations

The substrate mechanisms (subscribe, wake, fan-in, triage, live-wake delivery) are all demonstrated live. The fully hands-off auto-loop on the stock worker-spawn path is not claimed here: it is gated on a separate provider bug, NousResearch#53234 (chat -q quiet mode does not execute tool calls with openai-codex; tools execute under -z but not under chat -q). The live runs used a local -z worker workaround that is not part of this PR. Once NousResearch#53234 is fixed, the same loop runs hands-off on the stock path.

No bundled agent skill, by design. This PR ships the plumbing — subscribe, deliver, re-engage — not an opinionated workflow layered on top of it. The injected KANBAN_GUIDANCE gains only the minimal subscribe-and-yield step; we deliberately stop short of a packaged "supervisor" skill or preset so adopters can compose the substrate however they like (orchestrator, CLI watcher, TUI, or their own). An opinionated skill can follow once the primitives have settled in use.

Future work

• F08 substrate generalization: a scope field (task/subtree/board), new event kinds (artifact_available, dependency_satisfied, fan_in_ready), and webhook/node adapters.
• WebUI/API wake adapter: resume the origin session when a subscribed task finishes after the browser stream closed. The substrate already supports it; it just needs the adapter.
• [Bug]: chat -q quiet mode does not execute tool calls with the openai-codex provider (tool calls emitted as text) NousResearch/hermes-agent#53234: the chat -q / Codex worker bug that gates the fully-automated live loop.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (feat(kanban):, fix(kanban):, test(kanban):)
• I searched for existing PRs to make sure this isn't a duplicate (no competing substrate PR; [Feature]: Generalize Kanban notifications into an event substrate — any-surface subscribers + delivery-adapter registry NousResearch/hermes-agent#49190 is open and unassigned)
• My PR contains only changes related to this feature (no unrelated commits)
• I've run the test suite (scripts/run_tests.sh, CI-parity). The focused substrate gate (the 15 files listed under How to Test) passes 224/224. The full suite shows some failures on my local macOS dev box, but they are pre-existing and unrelated: all in other subsystems (acp, anthropic-adapter, wecom, gateway-service, state-db, file-tools, browser), and the one kanban-named failure (test_signal_handler_kanban_worker, a SIGTERM-timing test) reproduces identically on a clean upstream/main checkout with none of these commits. None of this PR's 25 files touch those paths.
• I've added tests for my changes (16 test files: 14 new, 2 modified; about 5k lines)
• I've tested on my platform: macOS Apple-Silicon (ARM, Python 3.12.8) — substrate gate 224/224, and the full CI-parity suite (scripts/run_tests.sh) triaged clean (every failure pre-existing/environmental, reproduced on a clean upstream/main checkout). Previously also verified on macOS Intel (Python 3.11.15). Not run directly on Windows or Linux; scripts/check-windows-footguns.py is clean across all 25 changed files.

Documentation & Housekeeping

• I've updated relevant documentation (the in-prompt KANBAN_GUIDANCE subscribe-and-yield guidance) or N/A
• N/A: no config keys added or changed (cli-config.yaml.example untouched)
• N/A: no architecture or workflow change to CONTRIBUTING.md / AGENTS.md
• I've considered cross-platform impact (footgun scan clean; touches subprocess spawn and file paths, with no POSIX-only assumptions added)
• I've updated tool descriptions/schemas (new kanban_subscribe tool schema and guidance)