fix(dingtalk): add AI Card QPS token-bucket throttle - #17365
Conversation
DingTalk's interactive-card PUT API enforces ~20 QPS per tenant; exceeding it returns 403 and drops the card update. - Add _CardTokenBucket — async token-bucket rate limiter (20 QPS) with automatic 2s exponential backoff on 403 responses - Add per-card 800ms minimum interval between non-finalize edits (matches openclaw-connector reply-dispatcher.ts:103) - Per-chat error-send cooldown (60s) to avoid spamming users - Finalize edits are NEVER throttled — dropping them would leave the card stuck in streaming state - Global bucket shared across all adapters in the process to #
alt-glitch
added
type/bug
|
Hi @alt-glitch, thanks for the review and labels! 🙏 You're right — this PR extracts the card QPS throttle component from #14333. To clarify: @PeterGuy326 and @spike2204 are from the same team — the DingTalk (钉钉) Open Platform team at Alibaba. We're the team that builds and maintains the DingTalk stream SDK, AI Card APIs, and Robot OpenAPI that this adapter integrates with. The original #14333 bundled three independent fixes (websockets proxy + card QPS + inbound queue) into a single stacked PR. We've re-submitted them as 3 separate, self-contained PRs (#17364, #17365, #17366) against About this specific fix: The 50 QPS limit on AI Card streaming APIs is an actual platform-enforced rate limit. We've observed HTTP 429 storms in production when 10+ concurrent conversations update cards simultaneously. The token-bucket approach here mirrors what we recommend in our own DingTalk AI Card best practices documentation. Happy to address any feedback — the original #14333 can be closed once these granular PRs are reviewed. 🚀 |
|
The Only at umbrella https://github.com/NousResearch/hermes-agent/pull/12769/changes#diff-5dca76751f29bee741c70b70ed7d84c9040b69771464c4f093596d88af680e35R159 or stacked PR https://github.com/NousResearch/hermes-agent/pull/14333/changes#diff-5dca76751f29bee741c70b70ed7d84c9040b69771464c4f093596d88af680e35R158 |
|
And |
teknium1
left a comment
teknium1
left a comment
There was a problem hiding this comment.
Thanks for splitting out the DingTalk rate-limit work. The current active adapter still calls the streaming update API directly at plugins/platforms/dingtalk/adapter.py:1214, so the underlying concern remains relevant.
Problems
- The PR references
_CARD_EDIT_THROTTLE_MS,_CARD_BUCKET, and_CARD_API_QPS_BACKOFF_MS, but none is defined in its base or diff. The first non-final edit and every streamed update would therefore fail at runtime. - The edited path,
gateway/platforms/dingtalk.py, was migrated toplugins/platforms/dingtalk/adapter.pyby560010547. The current card create, deliver, and stream paths are atadapter.py:1078,:1118, and:1214, respectively. - No tests accompany the limiter; existing lifecycle coverage is in
tests/gateway/test_dingtalk.py:952-1087.
Suggested changes
- Reimplement the complete limiter in the bundled plugin and add hermetic concurrency/backoff/finalize tests there.
Automated hermes-sweeper review.
| now_ms = int(datetime.now(tz=timezone.utc).timestamp() * 1000) | ||
| if not finalize: | ||
| last_ms = self._card_last_edit_ms.get(message_id, 0) | ||
| if now_ms - last_ms < _CARD_EDIT_THROTTLE_MS: |
There was a problem hiding this comment.
_CARD_EDIT_THROTTLE_MS is not defined in this PR or its base. A non-final edit will raise NameError; include the limiter constants and implementation in this self-contained PR.
| await self._card_sdk.streaming_update_with_options_async( | ||
| stream_request, stream_headers, runtime | ||
| ) | ||
| await _CARD_BUCKET.acquire() |
There was a problem hiding this comment.
_CARD_BUCKET is also not defined in this PR or its base, so every streamed card update will raise NameError. Please add the token-bucket implementation and test its acquire/backoff behavior.
alt-glitch
added
type/feature
teknium1
added
sweeper:risk-message-delivery
alt-glitch
added
P3
4 participants
Summary
DingTalk's AI Card streaming API enforces a 50 QPS limit per robot. Without client-side throttling, bursts of concurrent conversations trigger HTTP 429 errors that abort card updates mid-stream.
Changes
_CardTokenBucketclass — async token-bucket rate limiter with configurable QPS, burst capacity, and max wait timeout_CARD_BUCKETshared across all adapter instances_card_create,_card_update,_card_closecall withawait _CARD_BUCKET.acquire()Related
Part of the DingTalk adapter enhancement series — see #12769 for the umbrella PR.