Skip to content

feat: voice first-class channel + cross-channel guardian notifications#7539

Merged
noanflaherty merged 14 commits into
mainfrom
feature/voice-cross-guardian
Feb 24, 2026
Merged

feat: voice first-class channel + cross-channel guardian notifications#7539
noanflaherty merged 14 commits into
mainfrom
feature/voice-cross-guardian

Conversation

@noanflaherty
Copy link
Copy Markdown
Contributor

@noanflaherty noanflaherty commented Feb 24, 2026

Summary

This feature makes voice a first-class channel and adds cross-channel guardian notifications for AI phone calls.

Key changes:

  • Voice as first-class channel: 'voice' added to ChannelId type with per-call conversations (key pattern: asst:${assistantId}:voice:call:${callSessionId})
  • ASK_USER → ASK_GUARDIAN rename: Renamed the marker the AI uses when it needs human input during a call
  • Cross-channel guardian dispatch: When the AI needs input, questions are fanned out to all configured channels (mac desktop, Telegram, SMS) — first valid response wins
  • DTMF callee verification: Optional verification step where the callee enters a numeric code via keypad before the call proceeds
  • Guardian action data model: Two new SQLite tables (guardian_action_requests, guardian_action_deliveries) tracking cross-channel dispatch and resolution
  • Answer resolution: First-writer-wins semantics via atomic DB check, with stale response handling and request-code disambiguation
  • Expiry sweep: 60-second periodic sweep for expired guardian requests, with notices sent to all delivery channels
  • Voice thread visibility: Voice channel threads appear in the desktop conversation list
  • Voice settings card: New card in Settings > Connect showing Twilio voice readiness
  • Bridge removal: Removed the call-bridge that routed desktop chat messages to calls (replaced by cross-channel dispatch)
  • Pointer messages: Concise status messages in the initiating chat thread
  • Documentation: Updated SKILL.md and ARCHITECTURE.md to reflect all changes

Milestones:

# Milestone Issue PR
M1 ASK_USER → ASK_GUARDIAN rename #7489 #7507
M2 Voice channel identity + per-call conversations #7490 #7512, #7524
M3 Voice event projection + bridge removal #7491 #7529
M4 DTMF callee verification #7493 #7533
M5 Guardian data model + dispatch #7494 #7534
M6 Guardian answer resolution #7495 #7535
M7 Settings UI + thread visibility + expiry #7497 #7536
M8 Docs and architecture alignment #7498 #7538

Test plan

  • Verify voice calls work end-to-end with per-call conversations
  • Verify ASK_GUARDIAN dispatches to all configured channels
  • Verify first-response-wins answer resolution across channels
  • Verify DTMF verification flow
  • Verify voice threads appear in desktop thread list
  • Verify voice card shows correct status in Settings
  • Verify expiry sweep expires stale requests
  • Verify pointer messages appear in initiating chat

🤖 Generated with Claude Code


Open with Devin

noanflaherty and others added 9 commits February 23, 2026 22:21
Co-authored-by: Claude <noreply@anthropic.com>
)

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
When a guardian action request is dispatched to telegram/sms/mac channels
during a voice call, replies on any of those channels are now intercepted,
validated, and used to resume the call:

- Channel inbound (telegram/sms): intercept guardian answers early in
  handleChannelInbound(), with identity verification, single/multi-delivery
  disambiguation via request codes, and first-writer-wins resolution
- Mac thread: intercept in session-process processMessage() before the
  agent loop, routing the user message as a guardian answer
- Guardian dispatch: create mac conversations server-side with
  getOrCreateConversation() and seed them with the question text
- Store: add getPendingDeliveryByConversation() for mac channel routing

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…e settings card (#7536)

Add periodic sweep (60s interval) for expired cross-channel guardian action
requests. When a request expires: marks request+deliveries as expired, expires
pending questions, and sends expiry notices to external channels and mac threads.

Allow voice-channel threads to appear in the desktop thread list by updating
the session filter in both ThreadSessionRestorer and ThreadManager to pass
through sessions with sourceChannel == "voice".

Add a Voice (Phone Calls) card to the Settings Connect tab showing Twilio
credential and phone number readiness for voice calls.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…-M7 changes (#7538)

Reflect the cross-channel guardian architecture in documentation:
- SKILL.md: add DTMF callee verification section, update answering
  questions to describe ASK_GUARDIAN cross-channel dispatch with
  first-response-wins semantics, note mid-call steering via desktop
  chat is no longer supported, add accepted regressions section
- ARCHITECTURE.md: update outgoing calls intro to describe voice as
  first-class channel with per-call conversations, replace bridge-based
  Mermaid diagram flow with guardian dispatch flow, replace call-bridge
  key component with guardian-dispatch/guardian-action-store/guardian-
  action-sweep, replace Call Bridge section with Cross-Channel Guardian
  Consultation, add guardian_action_requests and guardian_action_deliveries
  SQLite tables, add guardian modules to Channel Guardian Security table

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…K test)

Merge main into feature/voice-cross-guardian to resolve two conflicts:
- relay-server.ts: integrate CALL_WELCOME_GREETING static greeting skip
  with DTMF verification (verify first, then check static greeting)
- call-orchestrator.test.ts: keep new CALL_OPENING_ACK test from main
  alongside ASK_GUARDIAN comment rename from feature branch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 17f50c60e7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread assistant/src/calls/relay-server.ts Outdated
Comment thread assistant/src/calls/guardian-dispatch.ts Outdated
Comment thread assistant/src/runtime/routes/channel-routes.ts
Comment thread assistant/src/calls/guardian-dispatch.ts Outdated
devin-ai-integration[bot]

This comment was marked as resolved.

@noanflaherty
Copy link
Copy Markdown
Contributor Author

Thanks for the large refactor — I walked through the new voice + guardian flow end-to-end and found a few blocking issues that should be fixed before we rely on this in prod.

1) [P1] Voice-thread transcript/completion data can be dropped when no Session exists for the new per-call voice conversation

Where

  • assistant/src/calls/call-domain.ts:232-251 (call session is re-pointed to a new per-call voice conversation)
  • assistant/src/daemon/session.ts:168 (call notifiers are only registered when a Session is constructed)
  • assistant/src/calls/call-state.ts:45-69 (notifier fire functions are no-ops if no callback exists)
  • assistant/src/calls/relay-server.ts:437-440 and assistant/src/calls/call-orchestrator.ts:418-421,476-479 (transcript/completion are emitted only via notifier callbacks)

Why this is a problem
With first-class voice channels, the call now runs in a dedicated voice conversation that usually has no active desktop Session unless the user opens that thread during the call. In that case, fireCallTranscriptNotifier / fireCallCompletionNotifier do nothing, so the voice thread can miss transcript/completion messages entirely.

Impact
The core “per-call voice thread” UX can be empty or incomplete even though call events occurred.

Suggested fix
Persist transcript/question/completion messages directly to conversation_store in the call path (relay/orchestrator) and treat notifier callbacks as optional live fanout, or proactively bootstrap a Session for each newly created voice conversation.


2) [P1] Guardian answers are marked resolved before verifying answerCall succeeded (can permanently lose answers)

Where

  • assistant/src/runtime/routes/channel-routes.ts:623-633
  • assistant/src/daemon/session-process.ts:239-250
  • assistant/src/memory/guardian-action-store.ts:175-213 (resolution is destructive: request/deliveries become answered)
  • assistant/src/calls/call-domain.ts:402-427 (answerCall can fail with 400/404/409)

Why this is a problem
Both inbound paths call resolveGuardianActionRequest(...) first, then call answerCall(...) fire-and-forget and ignore the result. If answerCall fails (no pending question, orchestrator gone, not waiting, empty answer), the request is already irreversibly marked answered and other channels are blocked by first-response-wins.

There is also an empty-answer edge case in the disambiguation flow: stripping the request code can produce "", which still resolves the request before answerCall rejects it.

Impact
False “resolved/relayed” acknowledgements and unrecoverable lost guardian responses.

Suggested fix

  • Validate non-empty answerText after code stripping.
  • Await answerCall and only mark guardian request resolved on success.
  • On call transition away from waiting_on_user (timeout/end/cancel), proactively cancel/expire associated guardian action requests instead of waiting for periodic sweep.

3) [P1] Orchestrator isn’t wired with assistant scope or broadcast callback, so cross-channel dispatch is mis-scoped and mac thread push can be skipped

Where

  • assistant/src/calls/call-orchestrator.ts:74-79 (defaults: assistantId = 'self', broadcast = undefined)
  • assistant/src/calls/relay-server.ts:339 (constructs CallOrchestrator without opts)
  • assistant/src/calls/guardian-dispatch.ts:87-103,130-137 (binding lookup uses assistantId; mac thread creation event only emitted when broadcast exists)

Why this is a problem
Because opts are never passed from relay setup:

  • assistant-scoped calls can look up guardian bindings under 'self' instead of the call’s actual assistant id.
  • guardian_request_thread_created may never be broadcast to mac clients, so they won’t get immediate thread-open notifications.

Impact
Cross-assistant routing errors and degraded mac cross-channel notification UX.

Suggested fix
Thread assistant context + broadcast callback from daemon/runtime server -> relay connection -> orchestrator constructor options, and use session.assistantId when building orchestrator.


If helpful, I can put up a follow-up patch plan that addresses these in the smallest safe sequence (data integrity first, then notification fanout wiring).

1. Pass broadcast/assistantId to CallOrchestrator from RelayConnection via
   module-level setRelayBroadcast wired in lifecycle.ts, so mac desktop
   receives guardian_request_thread_created IPC events and multi-assistant
   deployments use the correct assistant ID.

2. Thread bearer token through guardian dispatch deliverToExternalChannel
   so gateway /deliver/{channel} calls include Authorization header.

3. Swap resolve/answerCall ordering in channel-routes guardian answer
   interception: call answerCall first, resolve only on success, so
   failed answers leave the request pending for retry.

4. Use content block array format for addMessage calls in
   guardian-dispatch.ts and guardian-action-sweep.ts to match codebase
   convention (JSON.stringify([{type:'text',text:'...'}])).

5. Expire deliveries in 'sent' status (not just 'pending') in
   expireGuardianActionRequest using inArray.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

All review feedback addressed in #7550:

  1. P1 — Missing broadcast/assistantId in orchestrator (Codex + Devin): Added setRelayBroadcast() plumbing from lifecycle → relay-server, and passing { broadcast, assistantId } to CallOrchestrator constructor.
  2. P1 — Missing bearer auth on guardian deliveries (Codex): Now reading readHttpToken() and passing it through to deliverChannelReply.
  3. P1 — Resolve-before-answerCall ordering (Codex): Swapped order — answerCall() now runs first; request only resolved on success.
  4. P2 — Bare JSON string for addMessage (Codex + Devin): Fixed to use content block array format [{type:'text', text}] in guardian-dispatch.ts and guardian-action-sweep.ts.
  5. P2 — Delivery expiry misses 'sent' status (Devin): Changed filter to inArray(['pending', 'sent']) in expireGuardianActionRequest.

@noanflaherty
Copy link
Copy Markdown
Contributor Author

Thanks for the fast follow-up — this commit fixes a lot of what I raised (assistant/broadcast wiring, bearer token threading, external-channel answer ordering, delivery expiry status handling). I still see two blocking gaps:

1) [P1] Voice transcript/completion persistence still depends on an active Session callback

Where

  • assistant/src/calls/relay-server.ts:455-458
  • assistant/src/calls/call-orchestrator.ts:452-456
  • assistant/src/calls/call-state.ts:45-52,68-69

Issue
Transcript/completion events are still emitted only via notifier maps (fireCallTranscriptNotifier / fireCallCompletionNotifier). If the per-call voice conversation has no live daemon Session (common unless the user opens that thread during the call), there is no registered callback and these events are dropped.

Why this matters
With first-class voice threads, this can leave the voice thread empty/incomplete despite an active call.

Suggested fix
Persist transcript/question/completion messages directly to conversation_store in relay/orchestrator paths, and keep notifier callbacks as optional live fanout.


2) [P1] Mac guardian-answer path still resolves request before call delivery succeeds

Where

  • assistant/src/daemon/session-process.ts:239-250

Issue
The mac path still does:

  1. resolveGuardianActionRequest(...)
  2. void answerCall(...)

So if answerCall fails (orchestrator gone, timed out, empty answer), the request is already marked answered and cannot be retried from another channel.

Suggested fix
Mirror the updated external-channel flow in channel-routes.ts:

  1. call answerCall first and check result,
  2. only resolveGuardianActionRequest on success,
  3. return a retry/failure message otherwise while keeping request pending.

Once those two are handled, this should close out the original blocking feedback from my review.

devin-ai-integration[bot]

This comment was marked as resolved.

1. Fix mac channel guardian-answer ordering: call answerCall before
   resolveGuardianActionRequest so failed delivery leaves request
   pending for retry from another channel (mirrors channel-routes.ts).

2. Persist voice transcripts directly to conversation_store alongside
   notifier fires so transcript history survives without a live
   daemon Session listening on the voice thread.

3. Fix SKILL.md codeLength default documentation (4 → 6) to match
   the actual schema default.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

noanflaherty commented Feb 24, 2026

Follow-up review on eb63e8d73cd4b8b91fc98935bf3fb52d8a140554:

I re-checked the two previously open blockers. The mac guardian-answer ordering fix is now correct (answerCall before resolveGuardianActionRequest). I still see two issues that should be fixed before merge:

  1. Transcript messages are now double-written when a daemon Session is active on the same conversation
  • New direct persistence writes were added in:
    • assistant/src/calls/relay-server.ts (conversationStore.addMessage around lines 459-463)
    • assistant/src/calls/call-orchestrator.ts (conversationStore.addMessage around lines 458-462)
  • But assistant/src/daemon/session-notifiers.ts still persists transcript entries again inside registerCallTranscriptNotifier (around lines 125-129).
  • Result: when a Session is listening on that conversation, each utterance is stored twice (raw + **Live call transcript** ...), and indexed twice.
  1. Completion persistence is still notifier-dependent (can still be dropped with no active Session)
  • Terminal call paths still only fire completion notifiers:
    • assistant/src/calls/relay-server.ts (fireCallCompletionNotifier around line 313)
    • assistant/src/calls/call-orchestrator.ts (fireCallCompletionNotifier around lines 521 and 638)
  • The actual completion message persistence still only happens in assistant/src/daemon/session-notifiers.ts (registerCallCompletionNotifier, around lines 154-158).
  • So if no daemon Session is active for the voice conversation, transcripts now persist, but the call-completed message does not.

Suggested direction:

  • Keep persistence in call-domain/relay/orchestrator paths (source of truth).
  • Convert transcript/completion notifiers to transport-only (emit client events, no conversationStore.addMessage) to avoid double writes and make behavior session-independent.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@noanflaherty noanflaherty merged commit 6c53c08 into main Feb 24, 2026
1 of 4 checks passed
@noanflaherty noanflaherty deleted the feature/voice-cross-guardian branch February 24, 2026 05:04
@noanflaherty
Copy link
Copy Markdown
Contributor Author

The "Send guardian deliveries with gateway bearer auth" feedback was already addressed in the final merged code. The deliverToExternalChannel call passes readHttpToken() ?? undefined as the bearer token (line 150 of guardian-dispatch.ts), and the deliverToExternalChannel function forwards it to deliverChannelReply. No additional PR needed.

noanflaherty added a commit that referenced this pull request Feb 24, 2026
Co-authored-by: Claude <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

All review feedback from Codex and Devin has been verified as addressed in the merged code. The fix commits (#7550, #7552) cover all identified issues:

  1. relay-server.ts — broadcast/assistantId now passed to CallOrchestrator via setRelayBroadcast
  2. guardian-dispatch.ts — bearer token threaded through deliverToExternalChannel via readHttpToken()
  3. channel-routes.ts — answerCall called before resolveGuardianActionRequest (answer-first pattern)
  4. guardian-dispatch.ts + guardian-action-sweep.ts — addMessage uses content block array format
  5. guardian-action-store.ts — expireGuardianActionRequest and cancelGuardianActionRequest expire both 'pending' and 'sent' deliveries
  6. session-process.ts — mac channel guardian answer follows answer-first pattern
  7. SKILL.md — codeLength default corrected to 6
  8. relay-server.ts — verification failure marks status failed immediately before setTimeout
  9. call-conversation-messages.ts — buildCallCompletionMessage uses status-aware labels

No additional code changes required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant