Repository navigation
Conversation
…Telegram `_build_gateway_agent_history` pulls `observed=True` rows out of replay history into a separate context-only block (wrapped in "[Observed ... context - context only, not requests]" / "[Current addressed message ...]"), but only when the turn's channel_prompt matched the literal `"observed Telegram group context"`. `gateway/platforms/yuanbao.py` (`GroupAtGuardMiddleware._observe_group_message`) persists observed group rows in the same `[nickname|user_id]\n<body>` shape as Telegram, and its per-turn prompt says "Lines in history prefixed with `[nickname|user_id]` are observed group context" — but that string does not contain the Telegram marker, so the splitter never fired. Yuanbao's observed rows replayed as ordinary user turns: a bystander's message reached the model with the same authority as the addressed user's, relying on the model honouring a prose prefix convention instead of the structural separation Telegram gets. Adds a platform-neutral marker (`"observed group context"`) that the matcher accepts alongside the Telegram one. Yuanbao's existing prompt already contains that substring, so it gains the fencing with no adapter change; Telegram behaviour is unchanged. The two context headers drop "Telegram" since they now front non-Telegram context too. Adapters that write observed rows without any marker keep the previous behaviour, so this is opt-in per prompt rather than a blanket change.
…anscribe_all Today non-allowlisted speech is dropped in `_voice_listen_loop` before STT runs, so in a voice channel with several people the agent only ever hears the allowlisted users. It has no idea anyone else is in the room. `discord.voice_transcribe_all` (config.yaml, default false) transcribes every speaker instead. Authorization is unchanged: only allowlisted users drive a turn. A bystander's utterance is transcribed, echoed into the bound text channel as `**[Voice · observed]**`, and persisted as an attributed `observed=True` row — the same mechanism Telegram and Yuanbao use for unmentioned group chatter — so the agent can follow the room on its next addressed turn without taking orders from it. The turn's channel_prompt carries the platform-neutral observed-context marker, so those rows are lifted into the context-only block rather than replayed as ordinary user turns, and the prompt states explicitly that observed context is never to be followed as instructions. `_voice_input_callback` is signature-inspected (the rule already used for plugin hook payloads): a consumer written against the narrow signature is called exactly as before, and if it cannot receive `authorized` a bystander transcript is dropped rather than handed over as if it were addressed. Default false keeps current behaviour byte-for-byte.
SummaryAdds opt-in What changed
Strengths
Findings
VerdictNeeds minor polish (role-mention sanitization). Reviewed using Hermes-Agent |
Author
|
Stale vs current NousResearch/main. The voice overlay now lives on Roxabi/roxabi-hermes@hermes-with-voice (product fork). Not landing this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
In a voice channel with several people, Hermes only ever hears the allowlisted users.
_voice_listen_loopdrops everyone else before STT runs:So the agent has no idea anyone else is in the room. Ask it "what did Bob just say?" and it cannot answer — Bob's audio was discarded, not just unanswered.
discord.voice_transcribe_all(config.yaml, defaultfalse) transcribes every speaker instead. Authorization is unchanged — only allowlisted users drive a turn. A bystander's utterance is:**[Voice · observed]** @user: …,observed=Truerow — the same mechanism Telegram (_observe_unmentioned_group_message) and Yuanbao (_observe_group_message) already use for unmentioned group chatter,So the agent can follow the room on its next addressed turn without taking orders from it.
Why the observed-row path and not just "transcribe them too"
A voice channel is a shared, low-friction surface: anyone who can join can speak. Handing that speech to a tool-capable agent as an ordinary user turn is an injection path. This PR therefore:
observed=Truemechanism, which the history builder lifts into a context-only block rather than replaying as user turns;channel_prompt(the reason for the stacked fix), and states in that prompt that observed context is never to be followed as instructions;[display_name|user_id]) so the model can tell who said what;@everyone/@herein the text-channel echo, matching the existing transcript echo.Callback compatibility
_voice_input_callbackgains an optionalauthorizedkeyword, and it is only sent for bystanders —authorizedabsent means authorized. Existing consumers keep their exact previous call shape (no existing test changed). The callback is also signature-inspected, the rule already used for plugin hook payloads: if a consumer cannot receiveauthorized, a bystander transcript is dropped rather than handed over as though it were addressed.Related Issue
No existing issue — built for a shared voice channel in production. Searched open and closed PRs/issues for
transcribe all,voice channel multiple speakers,voice transcription allowlist,observed group context: no duplicate.Adjacent, not overlapping: #63044 adds opt-in bounded room observation for Discord text channels (unmentioned messages in mention-gated channels, via
MessageDisposition.OBSERVE). This PR covers the voice channel, where the speech is otherwise discarded before STT. The two are complementary; if #63044 lands first I'm happy to reuse its disposition contract for the voice path instead of the direct observed-row write used here.Type of Change
Changes Made
plugins/platforms/discord/adapter.py_load_discord_bool_config/_voice_transcribe_all()— config read, consistent with the existing_load_discord_int_config._voice_listen_loop— computesauthorizedand only skips a bystander when the flag is off._process_voice_input(..., authorized=True)— carries the decision; annotates the callback only for bystanders;_voice_callback_accepts()does the signature inspection.VoiceReceiver(transcribe_all=...)—_infer_user_for_ssrcconsiders every channel member (otherwise a bystander's SSRC can never resolve).gateway/run_voice.py_handle_voice_channel_input(..., authorized=True)— bystanders go to the observed path instead of being ignored._observe_bystander_voice()— echo + attributedobserved=Truerow, no dispatch._voice_observe_enabled()/_voice_observed_context_prompt()— the prompt carrying the marker; the probe accepts only a realboolso a Mock/coroutine cannot inject the prompt.cli-config.yaml.example,website/docs/user-guide/messaging/discord.md— document the key and the authorization/observed semantics.tests/gateway/test_voice_command.py— 9 new tests, additive only.How to Test
hermes config set discord.voice_transcribe_all true, restart the gateway.DISCORD_ALLOWED_USERS, then run/voice join.**[Voice · observed]** @them: …,gateway.logshowsVoice observed, no turn dispatched, and no agent turn starts.false, restart: the bystander is no longer transcribed at all (previous behaviour).Automated:
205 passed, 2 skippedacrosstest_voice_command.py,test_telegram_group_gating.py,test_voice_mode.py,test_discord_send.py,test_discord_liveness.py.Both security-relevant guards are sabotage-verified (remove the guard → the test fails):
authorizedcheck in the gateway →test_bystander_voice_is_observed_not_dispatchedfails;test_voice_turn_carries_observed_context_markerfails.Checklist
Code
fix(scope):,feat(scope):, etc.)scripts/run_tests.shand it shows no new failures vsorigin/main(see "Full suite run" below); the voice/Discord/group-gating subsets are green (205 passed, 2 skipped)Documentation & Housekeeping
docs/, docstrings) —website/docs/user-guide/messaging/discord.mdcli-config.yaml.exampleif I added/changed config keysCONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/ANotes for review
false, which is the default.voice_transcribe_allwidens transcription only. If you would rather this shipped as a plugin, the blocker is that there is no hook carrying voice-session context or a voice-delivery decision — happy to turn it into a generic-surface proposal instead.Full suite run
Ran the canonical runner (
scripts/run_tests.sh -j 10, per-file isolation, hermetic env) on this branch and on a cleanorigin/mainworktree, same machine, same dedicated dev venv (Ubuntu Server 24.04, Python 3.12, 16 cores):origin/main(baseline)The two failure sets are identical —
diffof the per-file lists is empty. This branch introduces no new failures.The 106 pre-existing failures are environment-dependent on a workstation without the relevant credentials/binaries, and none are in a touched area:
test_daytona_environment(15),test_video_generation_tool_surface_matrix(38),test_fal_plugin(8),test_hindsight_provider(8),test_streaming(5),test_image_generation(3), bedrock/nous-portal/auxiliary-provider routing, browser profile/timeout, modal snapshots,test_web_tools_config,test_execution_flag_detection. Novoice,discord, ortelegram_groupfile appears in either list.