feat(weixin): tag transcribed voice messages and enforce echo on outbound - #27235
feat(weixin): tag transcribed voice messages and enforce echo on outbound#27235aha-lin wants to merge 2 commits into
Conversation
Weixin delivers auto-transcribed voice messages as plain text via the callback, indistinguishable from typed input. Prefixing the transcript lets the agent reliably detect voice messages and apply voice-specific reply conventions (e.g. echo the transcript so the user can verify ASR accuracy).
When the inbound message was an auto-transcribed voice (prefixed with [voice:transcribed]), guarantee the outbound reply starts with the `> 🎤 你说:"…"` echo so the user can verify ASR accuracy. The LLM should add this prefix per the wechat-voice-gateway skill, but enforcing it mechanically removes a recurring drift mode. One-shot: stash on inbound, pop on outbound. Toggleable via WEIXIN_VOICE_ECHO_ENFORCE env or voice_echo_enforce config (default on).
521f6bc to
6eb6c47
Compare
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
|
Thanks for the contribution — the premise still holds on current main: Problems
Suggested changes
This is an automated hermes-sweeper review; human maintainers make the final merge decision. |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the contribution — current main still returns transcribed Weixin voice text without a source marker at gateway/platforms/weixin.py:963-966, so the inbound tagging premise is valid.
Problems
- The pending transcript is stored by
sender_idin this patch (gateway/platforms/weixin.py:1415), while outbound delivery retrieves it bychat_id. Group messages useeffective_chat_idfromroom_id/to_user_id(gateway/platforms/weixin.py:361-367,1453-1458), so group echoes will miss. - The pending map is neither correlated to a specific reply nor expired. Because the hook sits in generic
send(), an unrelated outbound message to the same chat can consume and prepend a stale transcript. WEIXIN_VOICE_ECHO_ENFORCEis a new non-secret behavior setting, contrary to theconfig.yamlpolicy inAGENTS.md:102-107; the option is also absent fromwebsite/docs/user-guide/messaging/weixin.md:111-127.tests/gateway/test_weixin.pyhas no coverage for this inbound-to-outbound behavior.
Suggested changes
- Key state by
effective_chat_idafter_guess_chat_type()and bind it to the specific reply lifecycle, with bounded cleanup. - Keep the setting in
gateway.platforms.weixin.extra, document it, and remove the environment override. - Add DM/group, idempotence, unrelated-send, and cleanup regression tests.
This is an automated hermes-sweeper review.
| if self._voice_echo_enforce and text.startswith("[voice:transcribed] "): | ||
| transcript = text[len("[voice:transcribed] "):].strip() | ||
| if transcript: | ||
| self._pending_voice_echo[sender_id] = transcript |
There was a problem hiding this comment.
This is keyed by sender_id, but send() looks up by chat_id. For groups, current main derives the outbound/source chat ID from room_id or to_user_id in _guess_chat_type() (gateway/platforms/weixin.py:361-367), so this entry cannot be found. Compute effective_chat_id first and key the pending state with it.
| self._dedup = MessageDeduplicator(ttl_seconds=MESSAGE_DEDUP_TTL_SECONDS) | ||
| # Voice echo enforcement: cache last voice transcript per chat_id so the | ||
| # outbound send() can guarantee the `> 🎤 你说:"…"` echo prefix even when | ||
| # the LLM forgets to add it. One-shot: consumed on next outbound send. |
There was a problem hiding this comment.
Please do not add a new user-facing environment-variable setting for this behavior. AGENTS.md:102-107 requires non-secret behavioral configuration to live in config.yaml; retain platforms.weixin.extra.voice_echo_enforce, document it, and remove this fallback.
| # is one-shot regardless). | ||
| if self._voice_echo_enforce and content and content.strip(): | ||
| transcript = self._pending_voice_echo.pop(chat_id, None) | ||
| if transcript and not content.lstrip().startswith("> 🎤 你说"): |
There was a problem hiding this comment.
This consumes process-local state on the next generic nonblank send() for the chat, not necessarily the response to this inbound voice event. An unrelated proactive send can receive the echo, while no reply leaves state unbounded. Correlate this to the inbound request/reply lifecycle or add bounded expiry and explicit cleanup.
Problem
Weixin's iLink callback delivers auto-transcribed voice messages as plain text, indistinguishable from a typed message. Two pain points fall out of that:
Fix
Two small, configurable changes in
gateway/platforms/weixin.py:1. Inbound: tag transcribed voice messages
_extract_text()now returns transcripts with a stable prefix:if item.get("type") == ITEM_VOICE: voice_text = str((item.get("voice_item") or {}).get("text") or "") if voice_text: - return voice_text + return f"[voice:transcribed] {voice_text}"This gives every downstream consumer (agent prompt, tools, hooks) a reliable detection signal. The prefix is plain ASCII so it survives any encoding path, and is distinctive enough to not collide with normal text.
2. Outbound: mechanical echo enforcement (opt-out)
When the inbound message was a voice transcript, the adapter stashes it on a per-
chat_idmap. The outboundsend()checks that map and prepends a> 🎤 你说:"…"blockquote echo if the LLM didn't already include one. One-shot: popped on send so it never bleeds into a later turn.Toggleable via:
voice_echo_enforcein platform configWEIXIN_VOICE_ECHO_ENFORCEenv varOperators who already enforce this in their system prompt and don't want the safety net can set
voice_echo_enforce=false.Why both, and why together
These two commits are paired:
I split them as separate commits for review clarity, but they're a unit — without the prefix, the enforcer has nothing to key on; without the enforcer, the prefix is just metadata. Happy to squash on merge if maintainers prefer.
Behaviour examples
Voice message, agent forgets to echo:
Voice message, agent already echoed:
Typed message:
Edge cases handled
safe = transcript.replace('"', '\\"')) so the blockquote parses cleanly.if content and content.strip()) — wouldn't render usefully.content.lstrip().startswith("> 🎤 你说")so manual echoes by the agent (any whitespace prefix) don't get double-stamped.dict.pop()— never accidentally re-stamps a later unrelated reply.Tests
No new tests — like the surrounding adapter code, this path isn't currently covered by the suite. Existing tests still pass. Happy to add coverage if reviewers want; the natural scaffolding would be a small async test that:
_extract_text→ asserts the prefix._handle_inbound_messagewith a transcribed text → asserts_pending_voice_echopopulated.send()with non-prefixed content → asserts the outbound payload starts with the echo and the map was popped.Manual verification
Running on production gateway for ~2 weeks. The enforcer log line
fires perhaps 10–15% of voice turns when the agent gets distracted by tool use or long context. With it in place, the user-side experience is finally consistent: every voice message comes back with its own transcript echoed, every typed message comes back without one.
Compatibility
send()signature and contract unchanged; prefix is only added when both (a) the feature is enabled and (b) a transcript was stashed for thischat_id.[voice:transcribed]is a new convention — happy to bikeshed the exact string (e.g. some structured tag like<voice transcribed=1>), but it does need to be something distinctive that downstream agents/skills can pattern-match on.