Skip to content

fix(discord): process concurrent voice utterances - #43003

Open
joelneleber wants to merge 2 commits into
NousResearch:mainfrom
joelneleber:fix/discord-voice-concurrent-stt
Open

joelneleber wants to merge 2 commits into
NousResearch:mainfrom
joelneleber:fix/discord-voice-concurrent-stt

Conversation

@joelneleber

Copy link
Copy Markdown

Summary

  • Keep Discord voice silence polling cheap by scheduling completed utterances into bounded background STT tasks instead of awaiting each speaker inline.
  • Add per-guild in-flight task tracking, global STT concurrency cap (HERMES_DISCORD_VOICE_STT_CONCURRENCY, default 3), and bounded pending backlog (HERMES_DISCORD_VOICE_STT_MAX_PENDING, default 12).
  • Cancel pending voice input tasks on voice leave.
  • Add a regression test proving three simultaneous speaker utterances all enter processing before any one finishes.

Root cause

When receiver.check_silence() returned completed utterances for multiple speakers, _voice_listen_loop() processed them serially with await _process_voice_input(...). PCM->WAV, STT, transcript posting, and optional callback latency for speaker 1 blocked speaker 2/3, and also paused further silence polling while STT was in flight.

Tests

  • RED verified first: new concurrency regression timed out on the old serial implementation.
  • python -m py_compile plugins/platforms/discord/adapter.py
  • python -m pytest tests/gateway/test_voice_command.py::TestUDPKeepalive tests/gateway/test_voice_command.py::TestVoiceListenLoopConcurrency -q -o 'addopts='
  • python -m pytest tests/gateway/test_voice_command.py tests/gateway/test_discord_opus.py tests/integration/test_voice_channel_flow.py -q -o 'addopts='

Result: 168 passed, 22 skipped.

Notes

This builds on the prior Discord voice transcription fix branch/runtime worktree because upstream PR #42611 is still unmerged.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery platform/discord Discord bot adapter labels Jun 9, 2026
@liuhao1024

Copy link
Copy Markdown
Contributor

Verification Review

Reviewed: Well-structured concurrency improvement for Discord voice processing.

Key changes:

  1. Config-gated agent dispatch (voice_transcript_agent_turns): defaults to false — transcripts post to the linked text channel without invoking the agent. This correctly addresses the double-posting issue from earlier iterations (fix(discord): keep voice meetings responsive #42597, fix(discord): keep live voice transcription bounded #42611).
  2. Concurrent STT processing (_schedule_voice_input_processing): fans out completed utterances via asyncio.create_task with a semaphore (default concurrency=3, max pending=12). The listener loop stays cheap.
  3. MAX_UTTERANCE_DURATION = 30.0: caps buffer growth during long monologues.
  4. Voice activity timeout refresh (_note_voice_activity): throttled to once per 30s to avoid timer churn.

Test coverage: 4 new tests covering default no-agent behavior, env opt-in, config opt-in, and concurrent speaker fan-out. Existing tests updated for _reset_voice_timeout mock.

No issues found.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the live-listening bottleneck: current main still awaits each utterance inline at plugins/platforms/discord/adapter.py:3179, so the concurrency direction addresses a real defect.

Problems

  • plugins/platforms/discord/adapter.py:610-611 adds HERMES_DISCORD_VOICE_STT_CONCURRENCY and HERMES_DISCORD_VOICE_STT_MAX_PENDING; the PR also documents HERMES_DISCORD_VOICE_TRANSCRIPT_AGENT_TURNS in .env.example. These are non-secret behavioral settings, while AGENTS.md:102-106 requires such configuration to use config.yaml.
  • Those two direct int(os.getenv(...)) calls make malformed values abort adapter construction instead of using a safe default.
  • The new fan-out test covers simultaneous starts, but not the stated semaphore cap, backlog drop, or leave-time cancellation contracts.

Suggested changes

  • Re-scope the voice knobs to validated discord config.yaml fields and add regression coverage for capacity, backlog, and cancellation behavior.

Automated hermes-sweeper review.

self._voice_listen_tasks: Dict[int, asyncio.Task] = {} # guild_id -> listen loop
self._voice_input_tasks: Dict[int, set[asyncio.Task]] = {} # guild_id -> in-flight STT/callback work
self._voice_input_semaphore: Optional[asyncio.Semaphore] = None
self._voice_input_concurrency = max(1, int(os.getenv("HERMES_DISCORD_VOICE_STT_CONCURRENCY", "3")))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This creates a user-facing non-secret HERMES_* behavior knob and raises ValueError for a malformed value during adapter construction. Please source this from validated discord config.yaml instead; AGENTS.md:102-106 reserves .env for secrets.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/discord Discord bot adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants