fix(dingtalk-platform): extract ASR recognition and file message text from incoming messages - #50014
Closed
rayjerrywoo wants to merge 2 commits into
Closed
fix(dingtalk-platform): extract ASR recognition and file message text from incoming messages#50014rayjerrywoo wants to merge 2 commits into
rayjerrywoo wants to merge 2 commits into
Conversation
alt-glitch
added
type/feature
rayjerrywoo
force-pushed
the
fix/dingtalk-audio-file-robot
branch
2 times, most recently
from
June 21, 2026 11:31
0f589ce to
b1af1a3
Compare
rayjerrywoo
changed the title
… from incoming messages
3 fixes for the dingtalk-platform plugin (plugins/platforms/dingtalk/adapter.py):
1. _extract_text: parse ASR recognition text from voice messages via
extensions['content']['recognition'], fall back to fileName for
file messages ("[文件] xxx")
2. _extract_media: handle audio/file/image msgtype_str with correct
MIME mapping; exclude audio from media_urls to preserve DingTalk's
built-in ASR result (avoids failed whisper re-transcription)
3. _resolve_media_codes: parse downloadCode from extensions['content']
for file/image message types
All tested with live DingTalk group chat: voice → recognition text,
PDF/Markdown file → filename + download.
teknium1
reviewed
Jul 14, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for targeting the post-plugin-refactor inbound-media gap. The current adapter still lacks handling for extensions.content audio/file/image payloads (plugins/platforms/dingtalk/adapter.py:765-824,1301-1338), so the premise remains valid.
Problems
- The new
msgtype='image'path is grouped with files and unconditionally setsMessageType.DOCUMENTatplugins/platforms/dingtalk/adapter.py:879. Image MIME values map toPHOTOin the normalized adapter contract (plugins/platforms/google_chat/adapter.py:302-316), andPHOTOhas distinct busy-session handling (gateway/platforms/base.py:4763-4769). - Please add regression tests. Existing DingTalk coverage tests rich-text media but not these new
extensions.contentpayload shapes (tests/gateway/test_dingtalk.py:574-623).
Suggested changes
- Split image classification from generic files: retain
DOCUMENTfor files, but emitPHOTOforimage/*attachments. - Cover ASR text, file name/code resolution, and direct-image MIME/type extraction with fixture-like message objects.
Automated hermes-sweeper review.
| mime = EXT_MAP.get(ext, mime) | ||
| media_types.append(mime) | ||
| if msg_type == MessageType.TEXT: | ||
| msg_type = MessageType.DOCUMENT |
Contributor
There was a problem hiding this comment.
This branch also handles msgtype='image', but it always emits DOCUMENT. Please classify image MIME values as PHOTO; PHOTO has separate busy-session album/queue behavior in gateway/platforms/base.py, while the existing MIME-based adapter mappings also treat image/* as PHOTO.
teknium1
added
sweeper:risk-message-delivery
… card/interactiveCard message handling - Promote EXT_MAP to a module-level constant for reuse - Classify DingTalk image messages and image file attachments as PHOTO - Extract DingTalk card/interactiveCard document link content with defensive parsing - Handle None, empty, JSON, and plain-string card content fields - Add coverage: 8 card + 4 interactiveCard + 5 _extract_media = 17 test cases - Remove a shadowed duplicate TestExtractText class (pytest collected only the later one)
rayjerrywoo
force-pushed
the
fix/dingtalk-audio-file-robot
branch
from
July 15, 2026 14:22
b1af1a3 to
d8f667b
Compare
Author
|
@teknium1 Thanks for the review — the feedback has been addressed in the second commit (d8f667b). Summary of the follow-up changes:
17 new test cases covering all the above scenarios are included in the second commit. Please take another look when you get a chance. |
This was referenced Jul 28, 2026
Contributor
|
Merged into main via consolidated salvage PR #73515 (merge Your contribution is credited to you in git history. Thank you! Closing this PR as merged-via-salvage. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
3 fixes for the dingtalk-platform plugin (
plugins/platforms/dingtalk/adapter.py) enabling:All of which were missing after the refactor from
gateway/platforms/dingtalk.pyto the plugin architecture.Changes
_extract_text— Voice ASR and file filenameextensions['content']['recognition']extensions['content']['fileName']for file messages (rendered as[文件] filename)_extract_media— Audio/file/image type detectionmsgtype='audio': setVOICEtype but skip passing media URLs (preserves DingTalk's built-in ASR result, avoids failed whisper re-transcription)msgtype='file'/'image': parse downloadCode, map common file extensions to MIME types_resolve_media_codes— Download code for file/imagedownloadCodefromextensions['content']for file and image message typesNotes
plugins/platforms/dingtalk/adapter.py) — the oldgateway/platforms/dingtalk.pyhas been deleted upstream. Before applying, ensure your Hermes is up to date (post-refactor, commit ≥560010547) and thedingtalk-platformplugin is enabled.