Conversation
When the iLink API rate-limits a send (ret=-2, errcode=-2), the old code used a fixed 3-second backoff per retry (send_chunk_retry_delay_seconds * 3). With 4 default retries, the total wait window was only ~12 seconds, but WeChat rate-limit windows are typically 15-30 seconds or longer. Messages were silently dropped when all retries were exhausted before the limit cleared. This change replaces the fixed multiplier with exponential backoff (2 ** attempt), giving a progressive wait of 1s, 2s, 4s, 8s, 16s with default settings, which comfortably covers the typical rate-limit windows. NOTE: Due to 100MB GitHub file push limit, this is the complete weixin.py file. The only change is the rate-limit backoff calculation on the `wait = ...` line inside `_send_text_chunk`. Refs: NousResearch#31131
WeCom's WebSocket connections occasionally drop mid-session, causing
send attempts to fail with ercode 846609 ("aibot websocket not subscribed").
When this happens, responses are generated but silently dropped — the user
sees no reply and API tokens are wasted.
This adds a `_send_with_reconnect_retry` wrapper that:
- Checks if the WebSocket is alive before sending
- Detects ercode 846609 in the response
- Automatically reconnects (calls _open_connection) and retries once
- Also handles "not connected" RuntimeErrors similarly
Applied to all outbound send paths: send(), _send_media_message(),
_send_reply_markdown(), and _send_reply_media_message().
Refs: NousResearch#29667
|
LGTM — the _send_with_reconnect_retry wrapper approach is cleaner than patching individual send paths. Closing my earlier #29731 in favor of this PR. |
|
Competing/related PRs for the same fixes: #31132 (weixin exponential backoff), #27697 (weixin reconnect + QQBot close codes), #21135 (weixin backoff + reconnect + chunk delay), #29731 (wecom 846609 retry, now closed in favor of this PR). This PR bundles both weixin backoff and wecom 846609 reconnect. |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for consolidating the related Weixin and WeCom reports. The WeCom lost-subscription premise still holds on current main: plugins/platforms/wecom/adapter.py:1398-1423 returns a failed SendResult for nonzero responses, and _response_error() has no 846609 recovery branch (:1048-1060).
Problems
- The PR modifies
gateway/platforms/wecom.py, which was migrated toplugins/platforms/wecom/adapter.pybyf2a7adba5; this needs a port rather than a clean cherry-pick. gateway/platforms/wecom.py:1350calls_open_connection()directly while the listener remains active. The current equivalent closes the shared socket before reconnecting (plugins/platforms/wecom/adapter.py:272-284), and_listen_loopindependently reconnects after its read fails (:335-360), so the retry can race a second replacement ofself._ws.- No regression tests cover 846609 recovery. Current send-error coverage is only the ordinary error case in
tests/gateway/test_wecom.py:522-552. - The current Weixin circuit breaker opens at the default first rate-limit event and breaks before the fixed-delay branch (
gateway/platforms/weixin.py:1179-1193,:1802-1807), so the exponential-backoff portion requires a current-design decision.
Suggested changes
- Port the WeCom fix to the plugin adapter, coordinate recovery with the listener, and add response/exception retry tests for reply and proactive sends.
- Reconcile the Weixin retry change with the current circuit breaker before salvaging it.
Automated hermes-sweeper review.
| "[%s] ercode 846609 (not subscribed), reconnecting and retrying...", | ||
| self.name, | ||
| ) | ||
| await self._open_connection() |
There was a problem hiding this comment.
_open_connection() replaces the shared websocket while the existing listener is still reading it. Its close causes _listen_loop to enter its own reconnect path, which can replace the retry connection before the retry completes. Please coordinate recovery through a single lifecycle path (or lock/cancel the listener) before retrying.
Summary
修复两个微信/企业微信相关的严重 Bug:
Fix 1: #31131 — WeChat iLink 限流退避不足导致消息静默丢失
问题: 当微信 iLink API 返回 rate limit (
ret=-2, errcode=-2) 时,adapter 使用固定 3 秒退避 × 4 次重试(总计 ~12 秒),但微信限流窗口通常是 15-30 秒。重试过早耗尽 → 消息静默丢弃。修复: 将
_send_text_chunk中的固定* 3改为指数退避* (2 ** attempt)。WEIXIN_SEND_CHUNK_RETRY_DELAY_SECONDS调整基础间隔Fix 2: #29667 — WeCom ercode 846609 WebSocket 断连导致回复静默失败
问题: 企业微信 WebSocket 中途断开后,后续 send 操作返回 ercode 846609 ("aibot websocket not subscribed")。响应已生成但永远无法送达 — 单日出现 173+ 次。
修复: 新增
_send_with_reconnect_retry包装器:_open_connection) + 重试一次send(),_send_media_message(),_send_reply_markdown(),_send_reply_media_message()Files Changed
gateway/platforms/weixin.py— 指数退避 (1 行改动)gateway/platforms/wecom.py— 新增ERRCODE_NOT_SUBSCRIBED常量和_send_with_reconnect_retry方法,修改send和相关发送方法Testing