Skip to content

Add user orchestrator for agent messaging and observation - #207

Merged
sethkarten merged 86 commits into
mainfrom
feature/user-orchestrator
Jul 2, 2026
Merged

Add user orchestrator for agent messaging and observation#207
sethkarten merged 86 commits into
mainfrom
feature/user-orchestrator

Conversation

@sethkarten

@sethkarten sethkarten commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary

This combines #164 and #173 into one user orchestrator surface for supervising multi-agent Prime Agent workflows:

  • adds the agent-message skill and daemon-backed direct session messaging
  • adds the agent-observe skill for read-only inspection of active agent sessions
  • adds the orchestration-heartbeat skill to initialize or refresh an internal orchestrator RLM heartbeat
  • exposes daemon CLI controls for listing queued messages, sending direct follow-ups, pausing/resuming delivery, and clearing a session queue
  • wires the host-backed capabilities through native Prime Agent session services

The design is based on the same constraint structure as cognitive load theory and the multi-agent communication framing in Interpretable Learned Emergent Communication for Human-Agent Teams: keep coordination channels sparse, bounded, interpretable, and directly relevant to the user/operator task. The orchestrator should reduce the need to manually track session state across tmux panes, files, and memory while avoiding an unbounded chat fabric between agents.

Orchestration heartbeat

orchestration_heartbeat.initialize() creates or updates a labeled internal RLM heartbeat for the current orchestrator session. The heartbeat prompt tells the orchestrator to:

  • inspect active sessions with agent_observe
  • summarize each session as active, waiting, blocked, error, or completed
  • include progress and explicit blockers
  • recommend next actions and draft target messages when intervention is useful
  • ask for user approval before cross-session messages unless a specific messaging policy was already approved
  • refresh against newly visible sessions during recurring checks

This is intentionally layered on RLM heartbeats, not the user-visible /heartbeat, so the model cannot mutate the user-level heartbeat while managing orchestration checks.

Safety model

  • No broadcast. Messages must target one known active session.
  • No offline inbox or remote/SSH delivery in this PR.
  • Message bodies are bounded by MAX_AGENT_MESSAGE_CHARS.
  • Pending queues are capped by MAX_PENDING_AGENT_MESSAGES.
  • Sends are rate-limited per source/target pair.
  • Queues can be paused, resumed, listed, and cleared from the daemon CLI.
  • Observation is read-only, count/character bounded, and exposes tool call names without tool arguments.
  • Skills fail closed when no daemon controller is available.

Combined PRs

Validation

  • npm run build
  • npx vitest --run packages/coding-agent/test/agent-observe.test.ts packages/coding-agent/test/agent-session-bus.test.ts packages/coding-agent/test/agent-session-services.test.ts packages/coding-agent/test/kernel-agent-message-skill.test.ts packages/coding-agent/test/kernel-agent-observe-skill.test.ts packages/coding-agent/test/kernel-orchestration-heartbeat-skill.test.ts packages/coding-agent/test/kernel-rlm-heartbeat-skill.test.ts packages/coding-agent/test/suite/agent-session-observe.test.ts packages/coding-agent/test/builtin-skills.test.ts packages/coding-agent/test/daemon-command.test.ts packages/coding-agent/test/daemon-mode.test.ts
  • npm run check
  • pre-commit npm run check

Note

High Risk
Large changes to AgentSession prompt/queue lifecycle and concurrent delivery in the daemon; bugs could mis-route messages, deadlock busy sessions, or leak bounded observe previews across sessions.

Overview
Adds daemon-backed multi-session orchestration: active Prime Agent sessions can message each other, inspect peers read-only, and run a recurring internal orchestrator heartbeat.

Agent messaging routes direct text to one live session via new daemon RPC (send_message) and CLI (daemon send, daemon agent-messages). Delivery uses daemon-derived sender identity, no broadcast, size/queue/rate limits, per-target locking, and pause/resume/clear controls. AgentSession gains acceptAgentMessagePrompt / queueAgentMessagePrompt and a refactored _prompt path (skip extensions, queueIfBusy, delivery receipts) so cross-session prompts do not race normal user prompts.

Agent observe exposes bounded list/get/recent-message previews through host handlers and the agent-observe Python skill; observation cannot mutate other sessions.

Orchestration heartbeat (orchestration-heartbeat skill) composes agent_observe and rlm_heartbeat to create or refresh a labeled internal RLM heartbeat with compact progress/blocker/action instructions (separate from user /heartbeat).

Kernel bootstrap now discovers sibling Python skill dependencies from pyproject.toml and installs them in topological order. Orchestration-related skills are hidden unless the matching daemon controllers are wired on the session.

Reviewed by Cursor Bugbot for commit 9454cba. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add agent-to-agent messaging, observation, and orchestration heartbeat skill to daemon

  • Introduces cross-session agent messaging via AgentSession.acceptAgentMessagePrompt/queueAgentMessagePrompt and a daemon sendAgentSessionMessage method with per-target locking, rate limiting, queue capacity guards, and pause/resume controls.
  • Adds two new Python skills: agent-message (list agents, send messages) and agent-observe (list/get agents, fetch recent messages), each bridged to host handlers in AgentSession.
  • Introduces the orchestration-heartbeat Python skill that generates recurring heartbeat instructions by observing active sessions and managing labeled RLM heartbeats.
  • Extends daemon-mode.ts with new RPC commands (send_message, agent_messages_status/pause/resume/clear) and AgentObserveController wiring; adds daemon send and daemon agent-messages CLI subcommands.
  • Refactors kernel bootstrap in bootstrap.ts to auto-discover sibling Python skill dependencies from pyproject.toml and install them in topological order.
  • Risk: _prompt in AgentSession is significantly refactored with new accepted-prompt lifecycle tracking; existing callers relying on preflight callback shape receive a new queued flag parameter.

Changes since #207 opened

  • Modified AgentSession.acceptAgentMessagePrompt to defer and queue accepted agent messages when the session becomes busy between preflight validation and prompt handoff [7ca681b]
  • Added test coverage for queueing accepted agent messages when the session becomes busy before handoff [7ca681b]
  • Modified AgentSession.acceptAgentMessagePrompt to fold pending next-turn messages into queued agent message prompts when the session is busy or becomes busy at handoff [7bed563]
  • Added AgentSession._queuePromptWithPendingNextTurnMessages helper method and AgentSession._buildPromptContent utility to construct combined prompt content arrays [7bed563]
  • Extended AgentSession._queueSteer and AgentSession._queueFollowUp to accept optional prebuilt content arrays [7bed563]
  • Added test case verifying that pending next-turn context is folded into accepted agent messages queued while the session is busy [7bed563]
  • Fixed context restoration in AgentSession.acceptAgentMessagePrompt when handoff busy rejection occurs without streamingBehavior [c6c4ad9]
  • Moved queue update event emissions to occur after agent method invocations in AgentSession._queueSteer and AgentSession._queueFollowUp [c6c4ad9]
  • Added test case verifying next-turn context restoration during handoff busy rejection in agent-session-prompt.test.ts [c6c4ad9]
  • Added test case verifying queue clearing behavior with queue_update listeners in agent-session-queue.test.ts [c6c4ad9]
  • Added synchronization to AgentDaemon prompt handler to await in-flight agent-message accept operations before starting daemon-originating prompts [9454cba]

Macroscope summarized 009ef1c.

Comment thread packages/coding-agent/skills/agent-observe/pyproject.toml
@sethkarten
sethkarten marked this pull request as ready for review June 18, 2026 21:37
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
This was referenced Jun 23, 2026
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/cli/daemon-command.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/cli/daemon-command.ts
@sethkarten

Copy link
Copy Markdown
Contributor Author

@kevinjosethomas addressed the agent-message queue and startup-race comments.

Changes pushed to feature/user-orchestrator:

  • agent_messages_clear now removes only queued agent-message prompts instead of clearing unrelated user/heartbeat queue entries.
  • Agent-message delivery treats existing pending queued work as busy, so new messages enqueue behind existing work instead of jumping ahead.
  • Daemon session state is registered before extension binding, so session_start handlers can safely call agent-message / heartbeat host APIs.
  • Added regression coverage for selective clear, pending-work ordering, and session-start host-controller availability.
  • npm run check passed locally and via pre-commit.

GitHub CI is running on the pushed head now.

Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 14 total unresolved issues (including 13 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c6c4ad9. Configure here.

Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
@sethkarten
sethkarten merged commit 2c30b48 into main Jul 2, 2026
3 checks passed
@kevinjosethomas
kevinjosethomas deleted the feature/user-orchestrator branch July 8, 2026 00:29
zhengr pushed a commit to zhengr/prime-agent that referenced this pull request Aug 8, 2026
…lect-ai#207)

* Add user orchestrator agent messaging and observe

* Add orchestration heartbeat skill

* Address user orchestrator PR review comments

* Tighten agent message rate limiting

* Harden agent message queue safety

* Preserve agent message reservations on queue clear

* Serialize agent messages per target

* Handle queued agent message edge cases

* Preserve orchestration heartbeat schedule on refresh

* Fix agent message queue handling

* Fix agent message review issues

* Defer agent message acknowledgment past stream start

* Fix agent message acceptance race

* Avoid blocking agent message send on pre-prompt work

* Avoid blocking agent messages on pre-prompt hooks

* Hide daemon prompt fast path from public options

* fix(coding-agent): harden agent message delivery

* fix(coding-agent): keep failed agent messages from consuming limits

* fix(coding-agent): address agent orchestrator review issues

* fix(coding-agent): address agent orchestrator review issues

* fix(coding-agent): queue sends during accepted retries

* fix(coding-agent): address remaining orchestrator bot findings

* test(coding-agent): update streaming prompt guard expectation

* test(coding-agent): remove duplicate daemon queue case

* fix(coding-agent): harden prompt acceptance races

* fix(coding-agent): serialize agent message clearing

* fix(coding-agent): lock agent message safety controls

* fix(coding-agent): harden agent message pause

* fix(coding-agent): queue agent messages internally

* fix(coding-agent): release prompt lock when streaming starts

* fix(coding-agent): reject messages paused while locked

* fix(coding-agent): reject messages after session switch

* fix(coding-agent): serialize session close with messages

* fix(coding-agent): reject messages for closing sessions

* fix(coding-agent): queue messages during compaction

* fix(coding-agent): avoid blocking messages during prompt prep

* fix(coding-agent): track queued agent messages

* fix(coding-agent): preserve agent message queue identity

Fixes PrimeIntellect-ai#207

* fix(coding-agent): parse canonical agent message ids

Fixes PrimeIntellect-ai#207

* fix(coding-agent): clear accepted agent messages

Fixes PrimeIntellect-ai#207

* fix(coding-agent): sanitize agent message metadata

Fixes PrimeIntellect-ai#207

* fix(coding-agent): suppress cleared accepted messages

Fixes PrimeIntellect-ai#207

* fix(coding-agent): clear accepted prompt state on failure

Fixes PrimeIntellect-ai#207

* fix(coding-agent): account for accepted message delivery

Fixes PrimeIntellect-ai#207

* fix(coding-agent): close accepted prompt busy gap

Fixes PrimeIntellect-ai#207

* fix(coding-agent): harden accepted message cleanup

Fixes PrimeIntellect-ai#207

* fix(coding-agent): report accepted messages as pending

Fixes PrimeIntellect-ai#207

* fix(coding-agent): wait for accepted message delivery

Fixes PrimeIntellect-ai#207

* fix(coding-agent): report duplicate follow-up preflight

Fixes PrimeIntellect-ai#207

* fix(coding-agent): reject cleared accepted messages

Fixes PrimeIntellect-ai#207

* fix(coding-agent): reject failed agent message delivery

Fixes PrimeIntellect-ai#207

* fix(coding-agent): emit cleared accepted agent end

Fixes PrimeIntellect-ai#207

* fix(coding-agent): unblock idle prompts after agent delivery

Fixes PrimeIntellect-ai#207

* fix(coding-agent): wait for agent message delivery

Fixes PrimeIntellect-ai#207

* fix(coding-agent): avoid direct delivery waiter hang

Fixes PrimeIntellect-ai#207

* fix(coding-agent): settle late agent message waits

Fixes PrimeIntellect-ai#207

* fix(coding-agent): reject delivery waits on dispose

Fixes PrimeIntellect-ai#207

* fix(coding-agent): acknowledge queued agent messages

* fix(coding-agent): review fixes for agent messaging admission, capacity, lifecycle, and CLI parsing

* fix(coding-agent): drain agent messages after bash

* fix(coding-agent): preserve queued message order after bash

* fix(coding-agent): resolve queued agent-message sends without blocking on delivery; settle waiters on abort

* fix(coding-agent): report busy observed sessions

* fix(coding-agent): settle agent-message lifecycle edge cases

* fix(coding-agent): keep deferred agent messages ordered

* fix(coding-agent): harden queued message cleanup

* fix(coding-agent): report queued direct accepts

* fix(coding-agent): harden agent message delivery state

* fix(coding-agent): restore next-turn context after failed accept

* fix(coding-agent): defer cron while agent accepts messages

* fix(coding-agent): queue cron during agent-message accept

* fix(coding-agent): normalize orchestration heartbeat intervals

* fix(coding-agent): recheck busy state before direct accept

* fix(coding-agent): preserve next-turn context in queued accepts

* fix(coding-agent): settle queued accept edge cases

* fix(coding-agent): serialize daemon prompts with agent messages

---------

Co-authored-by: Kevin Thomas <kevin.jt2007@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants