fix(weixin): convert Silk voice messages to WAV via pilk for STT - #42141
fix(weixin): convert Silk voice messages to WAV via pilk for STT#42141annguyenNous wants to merge 1 commit into
Conversation
Fixes NousResearch#42084 WeChat voice messages arrive in Silk audio format, which is not supported by any STT backend (Whisper, faster-whisper). Previously the gateway saved .silk files directly, causing silent STT failures. Also removes the premature return when voice_item has pre-transcribed text — the WeChat API text is unreliable, so we always download and transcribe via STT. Changes: - _download_voice: Always download voice (don't skip when text exists) - After download, convert Silk → WAV using pilk (same as QQBot adapter) - Return WAV path for STT compatibility, fallback to .silk if pilk is not installed or conversion fails - Update media type from 'audio/silk' to 'audio/wav' in caller
|
Same fix as the open #31487 (decode WeChat Silk voice -> WAV so STT can transcribe) and overlaps #11593 (generic |
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
Looks Good
- Fixes WeChat voice message STT: converts Silk format to WAV using
pilk.silk_to_wav()(same approach as QQBot), enabling Whisper/faster-whisper to decode the audio. - Graceful fallback to
.silkifpilkis not installed or conversion fails. - Removes the previous incorrect skip logic that skipped download when
voice_itemhad pre-transcribed text. - No security concerns, no debug artifacts.
Reviewed by Hermes Agent
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tackling the real Weixin STT gap. Current main still caches inbound voice as .silk (gateway/platforms/weixin.py:1660), while transcribe_audio rejects .silk (tools/transcription_tools.py:103,1032-1036), so converting before gateway STT is the right direction.
Problems
- The removal of the text-present branch changes documented behavior:
_extract_text()already usesvoice_item.text(gateway/platforms/weixin.py:963-966), and the Weixin docs say text is used when available (website/docs/user-guide/messaging/weixin.md:198). Please establish and test the intended behavior before always adding an STT attachment. - The new inline
pilkpath duplicates QQBot's existing conversion logic (gateway/platforms/qqbot/adapter.py:2039-2091). This also matches the member comment recommending consolidation with the shared-helper work in #31487. - Please add inbound conversion/fallback/MIME tests; the current Weixin SILK tests are outbound-only (
tests/gateway/test_weixin.py:769-907).
Suggested changes
- Salvage the conversion through shared SILK decoding logic and add mocked inbound regression coverage for WAV success and SILK fallback.
- Preserve or explicitly revise the supplied-transcript contract, including its documentation.
Automated hermes-sweeper review.
| media = voice_item.get("media") or {} | ||
| if voice_item.get("text"): | ||
| return None | ||
| # Don't skip download when voice_item has pre-transcribed text — |
There was a problem hiding this comment.
This removes the current transcript-first behavior: _extract_text() already returns voice_item.text on main (gateway/platforms/weixin.py:963-966), and the docs describe that contract (website/docs/user-guide/messaging/weixin.md:198). Please add a regression test and explicitly decide whether an upstream transcript should be retained, replaced, or accompanied by STT output before changing this branch.
| silk_path = cache_audio_from_bytes(data, ".silk") | ||
| wav_path = silk_path.replace(".silk", ".wav") | ||
| try: | ||
| import pilk |
There was a problem hiding this comment.
QQBot already has pilk conversion with output validation and retry behavior in gateway/platforms/qqbot/adapter.py:2039-2091. Please consolidate this through shared SILK decoding logic, as the member comment on this PR suggests, instead of maintaining a second inline implementation.
|
Verification (Vox Lockin lane 08 — platform adapter voice class): this PR is superseded by main and now conflicts with it; its adapter-level pilk conversion is redundant. Status vs
Evidence: Recommendation: close this PR as superseded (credit to @annguyenNous for identifying the silk problem — the approach was adopted centrally). No duplicate PR will be opened. |
Coordination: same-defect siblingThis PR fixes #42084 — fix(weixin): convert Silk voice messages to WAV via pilk for STT. The same defect was also reported as #32196 (Weixin(WeChat) voice messages in Silk format skip STT — russian transcription broken), which is not in this PR's close set. Please add |
Summary
Fixes #42084
WeChat voice messages arrive in Silk audio format, which is not supported by any STT backend (Whisper, faster-whisper). Previously the gateway saved
.silkfiles directly, causing silent STT failures.Changes
gateway/platforms/weixin.pyRemove premature return: Don't skip voice download when
voice_itemhas pre-transcribed text — the WeChat API text is unreliable, always download and transcribe via STT.Silk → WAV conversion: After downloading, convert the
.silkfile to.wavusingpilk(same approach as the QQBot adapter). Return the WAV path for STT compatibility.Fallback: If
pilkis not installed or conversion fails, return the.silkfile as before.Media type: Update caller to use
audio/wavwhen the conversion succeeds,audio/silkas fallback.Dependency
pip install pilk(optional — graceful fallback if not installed)Testing