Skip to content

fix(gateway): steer busy voice follow-ups after STT - #65023

Closed
canorionen wants to merge 1 commit into
NousResearch:mainfrom
canorionen:fix/busy-voice-steer-stt
Closed

fix(gateway): steer busy voice follow-ups after STT#65023
canorionen wants to merge 1 commit into
NousResearch:mainfrom
canorionen:fix/busy-voice-steer-stt

Conversation

@canorionen

@canorionen canorionen commented Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes busy-input steer behavior for voice messages.

A media-only voice follow-up reaches the gateway busy handler before the normal inbound STT pipeline. Its event.text is therefore empty, so running_agent.steer() cannot accept it and the gateway silently falls back to queue mode. That makes voice follow-ups behave differently from text follow-ups even when display.busy_input_mode: steer is configured.

This change transcribes voice-message media inside the busy steer path before calling running_agent.steer(). Successful transcriptions are injected into the active run and are not replayed as a later queued turn. Regular audio-file attachments remain outside automatic STT, matching the existing inbound behavior. If STT fails, any caption is preserved and the existing lossless fallback remains in place.

Related Issue

No existing issue or PR found after searching open and closed items for busy voice/steer/transcription behavior.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Added GatewayRunner._prepare_busy_steer_text() to transcribe voice-message media before steering.
  • Reused the existing STT enrichment and transcript-echo contracts.
  • Kept MessageType.AUDIO attachments as files rather than automatically transcribing them.
  • Added a regression test asserting STT → agent.steer(transcript), no interrupt, and no pending queue replay.

How to Test

  1. Configure display.busy_input_mode: steer and enable inbound STT.
  2. Start a tool-using agent run from a messaging platform, then send a voice message while it is busy.
  3. Verify the voice transcript is injected into the active run and the event is not queued for the next turn.

Targeted automated verification:

pytest -q tests/gateway/test_busy_session_ack.py tests/gateway/test_stt_transcript_echo_config.py
# 33 passed

A full pytest tests/ -q run was also attempted, but exceeded the 600-second local execution limit before completion; the targeted gateway/STT suites pass.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Ubuntu 24.04 / Python 3.11

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A; behavior is internal and documented in the new method docstring
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A; no config keys changed
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A; no architecture/workflow change
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — change uses existing platform-neutral STT/event abstractions
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A; no model tool schema changed

Screenshots / Logs

The regression test covers the original failure mode directly: a voice-only busy event is transcribed, passed to agent.steer(), and absent from the adapter pending queue.

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery tool/tts Text-to-speech and transcription P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Jul 15, 2026
tonydwb
tonydwb reviewed Jul 15, 2026
View reviewed changes

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Looks Good

  • Fix(gateway): steer busy voice follow-ups after STT
  • 95 additions, 1 deletion — targeted fix
  • No issues detected

Reviewed by Hermes Agent

@teknium1

teknium1 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Thanks for the focused regression fix. The premise is confirmed on current main: gateway/platforms/base.py:4799-4802 dispatches active-session messages to GatewayRunner._handle_active_session_busy_message, while the steer path currently reads only event.text at gateway/run.py:5575-5584. A media-only voice event consequently cannot satisfy can_steer and falls back to queueing.

The proposed preprocessing follows the existing inbound voice contract at gateway/run.py:10588-10651: voice media is transcribed, audio-file attachments remain outside automatic STT, and configured transcript echo is preserved. The added regression test verifies the successful steer/no-queue path.

Automated hermes-sweeper review.

@teknium1 teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 16, 2026
@teknium1 teknium1 mentioned this pull request Jul 28, 2026
Merged
teknium1 added a commit that referenced this pull request Jul 28, 2026
@teknium1
fix: route busy-steer voice through the shared out-of-band STT choke …
ea80b55 
…point

Follow-up for salvaged #65023/#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from #67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of #58780. Adds extract_media gating tests for #44826 and the
contributor mapping for chefboyrdave21.
@teknium1

teknium1 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Merged into main via consolidated salvage PR #73518 (merge ea80b557ae). Your _prepare_busy_steer_text fix (transcribe busy voice follow-ups before the steer decision) was cherry-picked as 97f2a91 with your authorship; a follow-up routes it through the shared transcription choke point.

Your contribution is credited to you in git history. Thank you! Closing this PR as merged-via-salvage.

@teknium1 teknium1 closed this Jul 29, 2026
@teknium1 teknium1 mentioned this pull request Jul 29, 2026
Closed
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@teknium1
fix: route busy-steer voice through the shared out-of-band STT choke …
30a6b62 
…point

Follow-up for salvaged NousResearch#65023/NousResearch#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from NousResearch#67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of NousResearch#58780. Adds extract_media gating tests for NousResearch#44826 and the
contributor mapping for chefboyrdave21.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@tonydwb tonydwb tonydwb left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@canorionen @teknium1 @tonydwb @alt-glitch