Skip to content

fix(weixin): convert Silk voice messages to WAV via pilk for STT - #42141

Open
annguyenNous wants to merge 1 commit into
NousResearch:mainfrom
annguyenNous:fix/weixin-silk-voice-to-wav
Open

fix(weixin): convert Silk voice messages to WAV via pilk for STT#42141
annguyenNous wants to merge 1 commit into
NousResearch:mainfrom
annguyenNous:fix/weixin-silk-voice-to-wav

Conversation

@annguyenNous

Copy link
Copy Markdown
Contributor

Summary

Fixes #42084

WeChat voice messages arrive in Silk audio format, which is not supported by any STT backend (Whisper, faster-whisper). Previously the gateway saved .silk files directly, causing silent STT failures.

Changes

gateway/platforms/weixin.py

  1. Remove premature return: Don't skip voice download when voice_item has pre-transcribed text — the WeChat API text is unreliable, always download and transcribe via STT.

  2. Silk → WAV conversion: After downloading, convert the .silk file to .wav using pilk (same approach as the QQBot adapter). Return the WAV path for STT compatibility.

  3. Fallback: If pilk is not installed or conversion fails, return the .silk file as before.

  4. Media type: Update caller to use audio/wav when the conversion succeeds, audio/silk as fallback.

Dependency

pip install pilk (optional — graceful fallback if not installed)

Testing

  • Verified voice messages download and convert to WAV
  • Verified STT (Whisper) can transcribe the converted WAV files
  • Verified fallback to .silk when pilk is not installed

Fixes NousResearch#42084

WeChat voice messages arrive in Silk audio format, which is not
supported by any STT backend (Whisper, faster-whisper). Previously
the gateway saved .silk files directly, causing silent STT failures.

Also removes the premature return when voice_item has pre-transcribed
text — the WeChat API text is unreliable, so we always download and
transcribe via STT.

Changes:
- _download_voice: Always download voice (don't skip when text exists)
- After download, convert Silk → WAV using pilk (same as QQBot adapter)
- Return WAV path for STT compatibility, fallback to .silk if pilk
  is not installed or conversion fails
- Update media type from 'audio/silk' to 'audio/wav' in caller
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery tool/tts Text-to-speech and transcription labels Jun 8, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Same fix as the open #31487 (decode WeChat Silk voice -> WAV so STT can transcribe) and overlaps #11593 (generic .silk preprocessing via pilk). Fixes #42084. Worth consolidating onto the shared _silk.py helper extracted from the QQ adapter in #31487 rather than inlining a second pilk path in weixin.py.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Looks Good

  • Fixes WeChat voice message STT: converts Silk format to WAV using pilk.silk_to_wav() (same approach as QQBot), enabling Whisper/faster-whisper to decode the audio.
  • Graceful fallback to .silk if pilk is not installed or conversion fails.
  • Removes the previous incorrect skip logic that skipped download when voice_item had pre-transcribed text.
  • No security concerns, no debug artifacts.

Reviewed by Hermes Agent

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling the real Weixin STT gap. Current main still caches inbound voice as .silk (gateway/platforms/weixin.py:1660), while transcribe_audio rejects .silk (tools/transcription_tools.py:103,1032-1036), so converting before gateway STT is the right direction.

Problems

  • The removal of the text-present branch changes documented behavior: _extract_text() already uses voice_item.text (gateway/platforms/weixin.py:963-966), and the Weixin docs say text is used when available (website/docs/user-guide/messaging/weixin.md:198). Please establish and test the intended behavior before always adding an STT attachment.
  • The new inline pilk path duplicates QQBot's existing conversion logic (gateway/platforms/qqbot/adapter.py:2039-2091). This also matches the member comment recommending consolidation with the shared-helper work in #31487.
  • Please add inbound conversion/fallback/MIME tests; the current Weixin SILK tests are outbound-only (tests/gateway/test_weixin.py:769-907).

Suggested changes

  • Salvage the conversion through shared SILK decoding logic and add mocked inbound regression coverage for WAV success and SILK fallback.
  • Preserve or explicitly revise the supplied-transcript contract, including its documentation.

Automated hermes-sweeper review.

media = voice_item.get("media") or {}
if voice_item.get("text"):
return None
# Don't skip download when voice_item has pre-transcribed text —

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This removes the current transcript-first behavior: _extract_text() already returns voice_item.text on main (gateway/platforms/weixin.py:963-966), and the docs describe that contract (website/docs/user-guide/messaging/weixin.md:198). Please add a regression test and explicitly decide whether an upstream transcript should be retained, replaced, or accompanied by STT output before changing this branch.

silk_path = cache_audio_from_bytes(data, ".silk")
wav_path = silk_path.replace(".silk", ".wav")
try:
import pilk

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QQBot already has pilk conversion with output validation and retry behavior in gateway/platforms/qqbot/adapter.py:2039-2091. Please consolidate this through shared SILK decoding logic, as the member comment on this PR suggests, instead of maintaining a second inline implementation.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
@andrexibiza

Copy link
Copy Markdown
Contributor

Verification (Vox Lockin lane 08 — platform adapter voice class): this PR is superseded by main and now conflicts with it; its adapter-level pilk conversion is redundant.

Status vs origin/main @ 70db671:

  • git merge-tree shows a conflict in gateway/platforms/weixin.py — main has since restructured _download_voice (the WeChat voice messages use Tencent Cloud STT which garbles non-Chinese languages — should route through Hermes' own STT pipeline #27300 fix, cb1798b43c) so it always downloads raw audio and surfaces audio/silk media.
  • Main's central STT pipeline already decodes .silk → WAV: f50a7c307a fix(stt): preprocess .silk voice notes before transcription landed in tools/transcription_tools.py::_prepare_audio_for_transcription (lazy pilk==0.2.4 via stt.silk dep, graceful error if pilk missing). This covers the same conversion for every adapter (weixin AND qqbot) at the correct layer — the transcription boundary — instead of duplicating it per-adapter.
  • The PR's second intent (don't trust voice_item.text) is also already on main via cb1798b43c + regression cf6ff8cd30 (TestWeixinVoiceGatewayHandoff).

Evidence: tests/gateway/test_weixin.py -k Voice 7 passed; tests/tools/test_transcription_tools.py -k silk 1 passed, on main pin.

Recommendation: close this PR as superseded (credit to @annguyenNous for identifying the silk problem — the approach was adopted centrally). No duplicate PR will be opened.

@andrexibiza

Copy link
Copy Markdown
Contributor

Coordination: same-defect sibling

This PR fixes #42084fix(weixin): convert Silk voice messages to WAV via pilk for STT. The same defect was also reported as #32196 (Weixin(WeChat) voice messages in Silk format skip STT — russian transcription broken), which is not in this PR's close set.

Please add Closes #32196 to the PR body so the duplicate closes with the fix. The interlock keeps both reports visible and the fix attached to one surface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(gateway): convert WeChat Silk voice messages to WAV via pysilk for STT

5 participants