Skip to content

feat(voice): guardian-wait heartbeat, impatience handling & silence suppression#10897

Merged
noanflaherty merged 5 commits into
mainfrom
safe-do/voice-guardian-wait-heartbeat-impatience
Mar 1, 2026
Merged

feat(voice): guardian-wait heartbeat, impatience handling & silence suppression#10897
noanflaherty merged 5 commits into
mainfrom
safe-do/voice-guardian-wait-heartbeat-impatience

Conversation

@noanflaherty
Copy link
Copy Markdown
Contributor

@noanflaherty noanflaherty commented Mar 1, 2026

Summary

  • Add proactive spoken heartbeat updates during guardian approval wait (~5s initial, then jittered 7-10s), with configurable cadence via 4 new config fields
  • Classify caller utterances during wait state (patience check, impatient, callback opt-in/decline, neutral) and respond contextually — including offering callback when caller sounds frustrated
  • Suppress the generic "Are you still there?" silence prompt during guardian wait states (heartbeat handles periodic engagement instead)
  • Use resolved guardian display name/username in all wait-state messages for a more personal experience
  • Add debounce cooldown (3s) to prevent TTS spam from rapid caller interjections

Files changed

  • assistant/src/config/calls-schema.ts — 4 new Zod config fields for heartbeat cadence
  • assistant/src/calls/call-constants.ts — 4 new getter functions for config values
  • assistant/src/calls/relay-server.ts — Heartbeat timer, utterance classifier, wait-state prompt handler, guardian label resolution
  • assistant/src/calls/call-controller.ts — Silence timer suppression during guardian wait
  • assistant/src/__tests__/relay-server.test.ts — 8 new tests + updated existing assertions
  • assistant/src/__tests__/call-controller.test.ts — 2 new silence suppression tests + mock relay enhancements

Test plan

  • Heartbeat timer emits periodic updates during guardian wait
  • Heartbeat stops on approval and on destroy
  • Impatience utterance triggers callback offer
  • Callback opt-in after offer is acknowledged
  • Neutral/empty utterances handled correctly
  • Cooldown prevents rapid-fire responses
  • Silence timeout suppressed during guardian wait
  • Silence timeout still fires normally outside guardian wait
  • Existing call flow tests pass unchanged

Original prompt

/Users/noaflaherty/Repos/vellum-ai/vellum-assistant/.private/plans/voice-guardian-wait-heartbeat-and-impatience-one-pr-plan-2026-02-28.md

🤖 Generated with Claude Code


Open with Devin

…lence suppression

Improve phone-call UX during guardian approval wait states:

- Add proactive spoken heartbeat updates while waiting for guardian
  response (~5s initially, then jittered 7-10s steady state)
- Classify caller utterances during wait (patience check, impatient,
  callback opt-in/decline, neutral) and respond appropriately
- Offer callback path when caller sounds impatient
- Suppress 'Are you still there?' silence prompt during guardian wait
- Use guardian display name/username in wait messages instead of generic
  'your guardian'
- Add cooldown to prevent TTS spam from repeated caller interjections
- Add 4 new config fields for heartbeat cadence tuning

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

@codex review

@noanflaherty
Copy link
Copy Markdown
Contributor Author

@devin review

chatgpt-codex-connector[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

… reset, cross-channel guardian label

- Exempt callback_opt_in and callback_decline from cooldown guard so
  quick callback decisions are never dropped
- Reset callbackOptIn to false on decline so timeout handler respects
  the caller's latest decision
- Fall back to listActiveBindingsByAssistant when no voice-channel
  guardian binding exists, matching the pattern in access-request-helper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

Superseding my previous comment (formatting got mangled by shell escaping). Clean review below.

Review Findings (against agreed PR1+PR2 scope)

I reviewed this against the plan at .private/plans/voice-guardian-wait-heartbeat-and-impatience-one-pr-plan-2026-02-28.md and ran local validation (cd assistant && bunx tsc --noEmit, plus targeted relay/controller tests).

1) [P0] Typecheck is currently broken (build-blocking)

  • assistant/src/calls/relay-server.ts:1592, :1672, :1691, :1699, :1709 emit new call event literals that are not part of CallEventType.
  • assistant/src/calls/types.ts:2 does not include:
    • voice_guardian_wait_heartbeat_sent
    • voice_guardian_wait_prompt_classified
    • voice_guardian_wait_callback_offer_sent
    • voice_guardian_wait_callback_opt_in_set
    • voice_guardian_wait_callback_opt_in_declined
  • assistant/src/__tests__/config-schema.test.ts:579 asserts full calls defaults but does not include the 4 newly-added heartbeat config defaults.

Impact: PR cannot pass strict typecheck as-is.

2) [P1] Explicit callback opt-in can be dropped if caller responds quickly

  • assistant/src/calls/relay-server.ts:1679-1684 applies the 3s cooldown before handling callback_opt_in / callback_decline.
  • Common path: caller says an impatient phrase -> assistant offers callback -> caller immediately says “yes”. That immediate “yes” is suppressed by cooldown and never recorded.

Impact: violates the requirement that explicit caller opt-in should be captured when offered.

3) [P1] Declining callback does not clear prior opt-in state

  • In assistant/src/calls/relay-server.ts:1698-1704 (callback_decline case), callbackOptIn is not reset to false.
  • If caller first opts in, then changes their mind and declines, timeout copy still uses callbackOptIn (assistant/src/calls/relay-server.ts:1381-1386) and may incorrectly state callback was requested.

Impact: stale state can produce incorrect user-facing messaging.

4) [P2] Heartbeat jitter math can produce invalid intervals when misconfigured

  • assistant/src/calls/relay-server.ts:1583-1584 assumes steadyMax >= steadyMin, but schema validates each bound independently.
  • If steadyMax < steadyMin, computed interval can be invalid.

Impact: fragile runtime behavior under bad config; should clamp or add cross-field validation.


Validation Notes

  • cd assistant && bunx tsc --noEmit fails with the issues above.
  • Targeted tests are also noisy/failing after those type-contract issues.

Overall

The UX direction is aligned with the agreed scope (heartbeat + impatience handling + no auto-callback), but I would not merge until the P0/P1 items above are fixed.

…amp jitter math

- Add 5 new voice_guardian_wait_* event types to CallEventType union
- Add 4 new guardianWaitUpdate* config defaults to config-schema test
- Clamp heartbeat jitter to Math.max(0, steadyMax - steadyMin) to
  handle misconfigured steadyMax < steadyMin gracefully

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

@codex review this PR again — the previous issues have been fixed in commit 590f3d0

@noanflaherty
Copy link
Copy Markdown
Contributor Author

@devin review this PR again — the previous issues have been fixed in commit 590f3d0

@noanflaherty
Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review! All 4 findings have been addressed across 2 fix commits:

Fix commit 1 (aaeb5f1): Addressed P1 items #2 and #3 + Devin's cross-channel fallback

Fix commit 2 (590f3d0): Addressed P0 typecheck + P2 jitter

  • P0: Added 5 new voice_guardian_wait_* event types to CallEventType union in types.ts
  • P0: Added 4 new guardianWaitUpdate* config defaults to config-schema.test.ts
  • P2: Clamped heartbeat jitter with Math.max(0, steadyMax - steadyMin) to handle misconfigured bounds gracefully

chatgpt-codex-connector[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

…double TTS

Clear and reschedule the heartbeat timer in impatient, neutral,
callback_opt_in, and callback_decline branches of handleWaitStatePrompt,
matching the existing pattern in patience_check. Prevents a queued
heartbeat from firing immediately after a wait-state response.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

@codex review this PR again — the previous issues have been fixed in commit 6886e1f

@noanflaherty
Copy link
Copy Markdown
Contributor Author

@devin review this PR again — the previous issues have been fixed in commit 6886e1f

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6886e1fd64

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread assistant/src/calls/relay-server.ts
@noanflaherty
Copy link
Copy Markdown
Contributor Author

noanflaherty commented Mar 1, 2026

Second-Pass Review (after aaeb5f1, 590f3d0)

I re-ran typecheck and the targeted test coverage for the heartbeat/impatience scope.

What is now fixed

  • Added missing CallEventType literals.
  • Updated config defaults coverage in config-schema.test.ts for the new cadence keys.
  • Callback opt-in / decline paths now bypass cooldown.
  • callback_decline now clears prior opt-in state.
  • Heartbeat jitter math is clamped for misconfigured ranges.

Remaining blocker

  1. [P1] Silence-path regression test fails consistently
  • Failing test: assistant/src/__tests__/call-controller.test.ts:1738 (silence timeout fires normally when not in guardian wait)
  • Repro:
    • cd assistant && bun test src/__tests__/call-controller.test.ts -t "silence timeout fires normally when not in guardian wait"
  • Observed:
    • Expected one "Are you still there?" prompt.
    • Received zero.
  • Likely cause:
    • CallController uses imported constant SILENCE_TIMEOUT_MS in assistant/src/calls/call-controller.ts:1061.
    • The test mutates a mocked timeout variable at runtime, but that does not affect an already-bound constant in this path.

I recommend fixing this before merge (either make silence timeout runtime/getter-based in the implementation, or update the test strategy so the timeout override is effective).

Replace the constant import SILENCE_TIMEOUT_MS with a getSilenceTimeoutMs()
getter function in call-controller.ts so the test mock can effectively
override the timeout value at runtime. The constant was bound at import
time, making the Object.defineProperty getter in the test ineffective.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@noanflaherty
Copy link
Copy Markdown
Contributor Author

Fixed the silence timeout test regression in 96774831f:

  • Converted SILENCE_TIMEOUT_MS constant to a getSilenceTimeoutMs() getter function (matching the pattern of other timeout constants like getMaxCallDurationMs(), getUserConsultationTimeoutMs())
  • Updated call-controller.ts to import and call the getter instead of using the constant
  • Simplified the test mock from the Object.defineProperty hack back to a plain function mock
  • Verified: all 49 call-controller tests pass, including both new silence suppression tests

@noanflaherty
Copy link
Copy Markdown
Contributor Author

@codex review

@noanflaherty
Copy link
Copy Markdown
Contributor Author

noanflaherty commented Mar 1, 2026

Final Review Pass

Reviewed after commit 96774831f9b45fcb9dc820a5cbd1eb187b42e3fc (plus the follow-up heartbeat timer reset commit on this PR branch).

Result

No remaining blockers from my prior review. This looks merge-ready from the issues I previously raised.

Validation Run

cd assistant && bunx tsc --noEmit

✅ Passed

cd assistant && bun test src/__tests__/call-controller.test.ts -t "silence timeout fires normally when not in guardian wait|silence timeout suppressed during guardian wait: does not say \"Are you still there\?\""

✅ Passed

cd assistant && bun test src/__tests__/relay-server.test.ts -t "guardian wait: heartbeat timer emits periodic updates|guardian wait: heartbeat stops on approval|guardian wait: heartbeat stops on destroy|guardian wait: impatience utterance triggers callback offer|guardian wait: explicit callback opt-in after offer is acknowledged|guardian wait: neutral utterance gets acknowledgment|guardian wait: empty utterance is ignored without response|guardian wait: cooldown prevents rapid-fire responses"

✅ Passed

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96774831f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread assistant/src/calls/relay-server.ts
@noanflaherty noanflaherty merged commit 918a3f4 into main Mar 1, 2026
1 check passed
@noanflaherty noanflaherty deleted the safe-do/voice-guardian-wait-heartbeat-impatience branch March 1, 2026 01:11
@noanflaherty
Copy link
Copy Markdown
Contributor Author

All review feedback has been verified as already addressed in this PR's subsequent commits or in PR #10930 (callback handoff notification).

tkheyfets pushed a commit that referenced this pull request Mar 2, 2026
…uppression (#10897)

* feat(voice): add guardian-wait heartbeat, impatience handling, and silence suppression

Improve phone-call UX during guardian approval wait states:

- Add proactive spoken heartbeat updates while waiting for guardian
  response (~5s initially, then jittered 7-10s steady state)
- Classify caller utterances during wait (patience check, impatient,
  callback opt-in/decline, neutral) and respond appropriately
- Offer callback path when caller sounds impatient
- Suppress 'Are you still there?' silence prompt during guardian wait
- Use guardian display name/username in wait messages instead of generic
  'your guardian'
- Add cooldown to prevent TTS spam from repeated caller interjections
- Add 4 new config fields for heartbeat cadence tuning

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review feedback — cooldown bypass for callbacks, decline reset, cross-channel guardian label

- Exempt callback_opt_in and callback_decline from cooldown guard so
  quick callback decisions are never dropped
- Reset callbackOptIn to false on decline so timeout handler respects
  the caller's latest decision
- Fall back to listActiveBindingsByAssistant when no voice-channel
  guardian binding exists, matching the pattern in access-request-helper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing CallEventType literals, config test defaults, and clamp jitter math

- Add 5 new voice_guardian_wait_* event types to CallEventType union
- Add 4 new guardianWaitUpdate* config defaults to config-schema test
- Clamp heartbeat jitter to Math.max(0, steadyMax - steadyMin) to
  handle misconfigured steadyMax < steadyMin gracefully

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: reset heartbeat timer after all wait-state responses to prevent double TTS

Clear and reschedule the heartbeat timer in impatient, neutral,
callback_opt_in, and callback_decline branches of handleWaitStatePrompt,
matching the existing pattern in patience_check. Prevents a queued
heartbeat from firing immediately after a wait-state response.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: convert SILENCE_TIMEOUT_MS to getter function for test mockability

Replace the constant import SILENCE_TIMEOUT_MS with a getSilenceTimeoutMs()
getter function in call-controller.ts so the test mock can effectively
override the timeout value at runtime. The constant was bound at import
time, making the Object.defineProperty getter in the test ineffective.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant