Skip to content

feat(dingtalk): support outbound image delivery - #7698

Merged
wenshao merged 1 commit into
QwenLM:mainfrom
qqqys:codex/issue-7687-dingtalk-image
Jul 27, 2026
Merged

feat(dingtalk): support outbound image delivery#7698
wenshao merged 1 commit into
QwenLM:mainfrom
qqqys:codex/issue-7687-dingtalk-image

Conversation

@qqqys

@qqqys qqqys commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

This PR lets the DingTalk channel deliver images created by the agent. Responses may include an [IMAGE: /absolute/path/to/image.png] marker; the channel validates and uploads the local file, then replaces the marker with a DingTalk Markdown image backed by the returned MediaID.

The behavior applies to normal replies, proactive group messages, and proactive direct messages. Text around the marker is preserved, examples inside inline or fenced code remain literal, expired access tokens are refreshed once, and upload failures produce a visible fallback without exposing credentials. When sender mentions are enabled, the mention and image are delivered in one Markdown message.

Why it's needed

DingTalk users can already send images to Qwen Code, but the channel could not return screenshots, charts, or generated images. Agents therefore exposed a local path that the user could not open from chat. Native upload closes that capability gap without requiring an external image host.

Reviewer Test Plan

How to verify

Configure a DingTalk application-robot channel whose workspace is writable, ask it to generate a PNG and send the result directly, and confirm that DingTalk renders the image inline with the surrounding text. With sender mentions enabled, confirm the mention and image arrive in one message.

Ask for an image-only reply and confirm no empty companion message is emitted. Include a marker inside inline or fenced code and confirm it remains literal. Reference a missing or disallowed path and confirm the response contains a visible failure notice while logs contain no access token.

For proactive delivery, send a valid marker to a group target and a direct-message target and confirm both messages render the uploaded image.

Evidence (Before & After)

Before: the agent returned a local filesystem path or image marker that DingTalk could not render.

After: DingTalk renders the generated PNG inline. A live macOS test with the test application robot also confirmed that atSender produces one mentioned Markdown image message instead of a separate mention message.

Tested on

OS Status
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

Environment (optional)

macOS, Node.js 25.9.0, local development channel started from this branch.

Risk & Scope

  • Main risk or tradeoff: Delivery depends on DingTalk's media upload and Markdown rendering behavior; local file reads are restricted to validated regular images in the workspace or system temporary directory and capped at 20 MiB.
  • Not validated / out of scope: Normal mentioned replies were validated against a live DingTalk robot. Proactive group/direct delivery and failure fallback were validated with automated API-boundary tests rather than live recipients.
  • Breaking changes / migration notes: None. Existing text-only behavior and configuration remain compatible.

Linked Issues

Fixes #7687

中文说明

本 PR 做了什么

本 PR 让钉钉 channel 可以发送 Agent 创建的图片。回复中可以包含 [IMAGE: /absolute/path/to/image.png] 标记;channel 会校验并上传本地文件,再用返回的 MediaID 将标记替换为钉钉 Markdown 图片。

该能力覆盖普通回复、主动群消息和主动私聊消息。标记周围的文本会被保留,行内代码或代码块中的示例保持原样,访问令牌过期时会刷新并重试一次,上传失败时会提供可见的降级提示且不会泄露凭证。开启回复发送者 @ 时,@ 和图片会合并在同一条 Markdown 消息中发送。

为什么需要

钉钉用户已经可以向 Qwen Code 发送图片,但 channel 无法返回截图、图表或生成图片。因此 Agent 只能暴露用户无法从聊天中打开的本地路径。原生上传能力补齐了这一差距,也不需要外部图片托管服务。

Reviewer 测试计划

如何验证

配置一个工作区可写的钉钉应用机器人 channel,让它生成 PNG 并直接发送结果,确认钉钉能在回复文字中内联展示图片。开启回复发送者 @ 后,确认 @ 和图片出现在同一条消息中。

要求仅发送图片,确认不会额外产生空消息。把图片标记放进行内代码或代码块,确认标记保持字面量。引用不存在或不允许的路径,确认回复中出现可见的失败提示,同时日志不包含访问令牌。

对于主动发送,分别向群目标和私聊目标发送有效图片标记,确认两种消息都能渲染上传后的图片。

前后对比证据

改动前:Agent 返回钉钉无法渲染的本地文件路径或图片标记。

改动后:钉钉可以内联展示生成的 PNG。macOS 上使用 test 应用机器人的真实验证还确认了 atSender 会生成一条带 @ 的 Markdown 图片消息,不再额外发送一条 @ 消息。

测试平台

OS 状态
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

环境(可选)

macOS、Node.js 25.9.0,并从当前分支启动本地开发 channel。

风险与范围

  • 主要风险或权衡:发送依赖钉钉媒体上传和 Markdown 渲染行为;本地文件读取仅允许工作区或系统临时目录内经过校验的普通图片,并限制为 20 MiB。
  • 未验证或范围外:普通带 @ 回复已通过真实钉钉机器人验证。主动群发/私聊和失败降级使用自动化 API 边界测试验证,未对真实接收者执行。
  • 破坏性变更或迁移说明:无。现有纯文本行为和配置保持兼容。

关联 Issue

Fixes #7687

@github-actions github-actions Bot added the review/self-reported The linked issue was opened by the PR author (self-reported) label Jul 25, 2026
@qqqys

qqqys commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator Author

E2E Test Report

Environment: macOS, Node.js 25.9.0, local development build, DingTalk test application robot with atSender enabled.

Scenario: Asked the robot to generate a PNG and send it directly in the current conversation.

Observed result:

  • DingTalk rendered the generated PNG inline with the surrounding response text.
  • The sender mention and image were delivered in one Markdown message.
  • No separate Image reply: or empty companion message was emitted.

Automated boundary evidence:

  • DingTalk package: 154/154 tests passed.
  • Covers marker parsing outside code, repeated markers, file allowlisting and magic bytes, symlink escape rejection, size limits, multipart upload, credential redaction, expired-token retry, normal replies, proactive group/direct delivery, visible failure fallback, and single-message sender mentions.
  • Full repository build and typecheck passed after rebasing onto the latest upstream main.

Not live-tested: Proactive group/direct delivery and failure fallback were validated against mocked DingTalk API boundaries rather than real recipients.

@wenshao

wenshao commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

Code Review — feat(dingtalk): support outbound image delivery

Reviewed the full diff (static analysis + tracing against the surrounding DingtalkAdapter.ts, markdown.ts, and channel-base sources). Overall this is a well-scoped, well-tested feature with a genuinely careful security posture. No blockers found — the notes below are polish/hardening.

Overview

  • New outbound-image.ts module: marker detection (with code-fence/inline-code masking), local-file validation, and DingTalk media/upload.
  • DingtalkChannel gains prepareOutgoingText() which finds [IMAGE: …] markers, uploads each validated file, and swaps the marker for ![image](<mediaId>). Wired into sendReply (normal replies), pushProactive (group + DM), and the atSender reply path so mention + image arrive as one Markdown message.
  • Agent instructions are auto-augmented so the model knows how to emit the marker.

What's done well

  • Strong path validation (readValidatedImage): absolute-path + extension allow-list, realpathSync containment inside workspace/tmp, regular-file check via fstat, 20 MiB cap checked before read, and magic-byte ↔ extension MIME cross-check. The symlink-escape case is explicitly tested. Opening the resolved realPath (not the original) closes the obvious symlink-swap window.
  • Credential hygiene: sanitizeApiMessage redacts the token from errmsg, the network-error path throws a generic message with no URL, and there are dedicated tests asserting the token never appears in errors or logs.
  • Code masking so [IMAGE: …] inside inline/fenced code stays literal; replacement is done by source position (last→first) so repeated markers and mixed code/non-code occurrences resolve correctly.
  • Token refresh-and-retry on 40014/42001, mirroring the existing proactive-send retry, and a visible fallback ([Image delivery failed: <file>]) instead of a hard failure.
  • No added latency on the common path: prepareOutgoingText early-returns when there are no markers, so plain replies make zero extra API calls.
  • Good test coverage across unit (markers/validation/upload) and integration (reply, token-retry, fallback, mention-coalescing, proactive group/DM).

Suggestions (all non-blocking)

  1. Constructor mutates the shared config objectDingtalkAdapter.ts, constructor. ChannelBase stores this.config = config by reference, so this.config.instructions += … mutates the caller's object. It's effective (instructions are read lazily) and per-channel, but mutating an input is a smell; consider computing an effective value instead. Also the dedup guard !instructions.includes('[IMAGE:') is a heuristic that would silently skip injection if an operator's own instructions happen to contain [IMAGE:.

  2. Behavior change for existing deployments — every DingTalk channel without custom instructions now gets a non-empty block (## DingTalk Channel / "You are responding through DingTalk" + image guidance) injected into agent context. That's intentional for the feature but changes prompts/behavior for all current users and adds tokens every turn; worth calling out explicitly in the PR notes.

  3. Notification title on image-only repliesextractTitle() (markdown.ts:120) strips leading #*-> but not !, so an image-only reply yields a title like ![image](@lAL-test-m…. Cosmetic (title is push-preview only), but a friendlier fallback (e.g. "Image") when the first line is just an embed would read better.

  4. gif/bmp acceptanceIMAGE_EXTENSIONS/detectImageMime allow .gif/.bmp, but confirm DingTalk media/upload?type=image accepts them; otherwise those uploads fail at the API and surface the fallback rather than being rejected up front. Minor.

  5. Minor: findImageMarkers is scanned twice on the atSender path (once in sendResponseMessage, once in prepareOutgoingText), and markers upload strictly sequentially. Both are negligible for typical single-image replies.

  6. Residual TOCTOU (very low) — between realpathSync(imagePath) and openSync(realPath) the leaf could still be swapped to a symlink pointing outside. Files are locally agent-generated so risk is minimal; O_NOFOLLOW on the leaf would fully close it if you want belt-and-suspenders.

Verdict

Looks merge-ready. The security-sensitive parts (path containment, credential redaction) are handled thoughtfully and tested. Items above are optional polish.

中文小结

整体实现清晰、测试充分,安全处理(路径校验、凭证脱敏)尤其到位,未发现阻塞性问题。

做得好的地方readValidatedImage 的绝对路径 + 扩展名白名单 + realpath 目录约束 + 常规文件校验 + 20 MiB 上限 + 魔数与扩展名交叉校验,并有软链接逃逸测试;错误信息与日志中对 token 做了脱敏并有专门测试;行内/围栏代码中的标记会被屏蔽保持原样,按源位置从后往前替换;40014/42001 刷新重试;无标记时提前返回,不产生额外 API 调用。

建议(均非阻塞)

  1. 构造函数直接改写共享的 this.config.instructionsChannelBase 按引用持有 config),建议改为计算“有效指令”而非就地修改;includes('[IMAGE:') 去重判断在运营方指令恰好包含该串时会漏注入。
  2. 所有未自定义 instructions 的钉钉 channel 现在都会被注入一段说明,属于对现有部署的行为变更,建议在 PR 中显式说明。
  3. 纯图片回复时 extractTitle 会把 ![image](… 当标题(markdown.ts:120),建议对图片开头给一个更友好的标题。
  4. 确认钉钉 media/upload?type=image 是否接受 gif/bmp。
  5. atSender 路径对 findImageMarkers 扫描了两次、多图串行上传,均可忽略。
  6. realpathopenSync 之间仍有极低概率的 TOCTOU,可用 O_NOFOLLOW 彻底规避。

结论:可以合并,以上为可选优化。

@qqqys
qqqys marked this pull request as ready for review July 27, 2026 02:02
@qwen-code-ci-bot

qwen-code-ci-bot commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finishedview run. See the stage comments in this thread for the result.

Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Template looks good ✓

Problem: This is a feature request with a linked issue (#7687) — DingTalk users can already send images to the agent, but the channel has no way to deliver images back. The agent ends up exposing a local filesystem path the user can't open from chat. That's a real, observed capability gap, not a theoretical concern.

Direction: Aligned. The DingTalk channel is an established integration, and outbound image delivery is a natural extension of the existing inbound image support. No direct CHANGELOG reference, but the area is clearly relevant.

Size: Not applicable — all changes are in packages/channels/dingtalk/src/, no core paths touched. Production logic: ~381 lines (97 adapter + 284 new module). Tests: ~502 lines (244 adapter + 258 new module).

Approach: The scope feels right. A dedicated outbound-image.ts module for marker parsing, file validation, and upload, wired into the existing send paths via prepareOutgoingText(). Agent instructions are augmented so the model knows the marker format. Every edit in the diff serves the stated goal — no unrelated changes or drive-by refactors.

Moving on to code review. 🔍

中文说明

感谢贡献!

模板完整 ✓

问题:这是一个有关联 issue(#7687)的功能请求——钉钉用户已经可以向 Agent 发送图片,但 channel 无法将图片回传给用户,Agent 只能暴露用户无法在聊天中打开的本地文件路径。这是一个真实的能力缺口,不是理论性问题。

方向:对齐。钉钉 channel 是已有的集成,出站图片投递是现有入站图片支持的自然延伸。CHANGELOG 中没有直接引用,但该领域明显相关。

规模:不适用——所有改动都在 packages/channels/dingtalk/src/ 下,未触及核心路径。生产逻辑约 381 行(适配器 97 行 + 新模块 284 行),测试约 502 行(适配器 244 行 + 新模块 258 行)。

方案:范围合理。独立的 outbound-image.ts 模块负责标记解析、文件校验和上传,通过 prepareOutgoingText() 接入现有发送路径。Agent 指令被增强以便模型知道标记格式。diff 中的每一处改动都服务于既定目标——没有无关改动或顺手重构。

进入代码审查 🔍

Qwen Code · qwen3.8-max-preview

Reviewed at 93c92c00089c33e3361d4e279277cc63eb092b24 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Code Review

Independent proposal: I'd create a dedicated module for marker parsing, file validation (path containment, extension allowlist, size cap, magic-byte cross-check), and DingTalk media/upload, then wire it into the existing send paths with a prepareOutgoingText() step. Token refresh on auth errors, visible fallback on failure, agent instructions for the marker format.

Comparison: The PR matches this almost exactly. The implementation is clean and the security posture is genuinely careful — readValidatedImage does absolute-path + extension allowlist + realpathSync containment + fstat regular-file check + 20 MiB cap before read + magic-byte ↔ extension MIME cross-check, and opens the resolved path (not the original). Credential hygiene is solid: sanitizeApiMessage redacts the token from errmsg, network errors throw a generic message with no URL, and there are dedicated tests asserting the token never appears in errors or logs. Code masking keeps [IMAGE: …] inside inline/fenced code literal, and replacement runs last→first so repeated markers resolve correctly.

No critical blockers found. Two non-blocking observations (both already raised in wenshao's review):

  • The constructor mutates this.config.instructions in place (ChannelBase holds config by reference). Works, but computing an effective value would be cleaner.
  • extractTitle() strips leading #*-> but not !, so an image-only reply yields a title like ![image](@lAL-…. Cosmetic — push-preview only.

Testing

This is an unattended CI run — no local build or tmux testing. Evidence below is from the PR's own CI checks on the reviewed commit.

Check Conclusion
Test (ubuntu-latest, Node 22.x) ✅ success
Test (windows-latest, Node 22.x) ⏭️ skipped
Test (macos-latest, Node 22.x) ⏭️ skipped
Integration Tests (CLI, No Sandbox) ⏭️ skipped
precheck-pr / precheck ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success
Classify PR ✅ success
label ✅ success

Ubuntu unit tests pass. Windows/macOS tests and integration tests were skipped (typical for fork PRs). The author reports 154/154 DingTalk package tests passing locally, covering marker parsing, file allowlisting, magic bytes, symlink escape rejection, size limits, multipart upload, credential redaction, expired-token retry, normal replies, proactive group/direct delivery, visible failure fallback, and single-message sender mentions. Proactive group/direct delivery and failure fallback were validated against mocked API boundaries, not live recipients.

Not verified: live DingTalk robot behavior (requires a configured application-robot channel). A maintainer can check out the branch in a disposable environment to confirm the end-to-end flow.

中文说明

代码审查

独立方案: 我会创建一个独立模块负责标记解析、文件校验(路径约束、扩展名白名单、大小上限、魔数交叉校验)和钉钉 media/upload,然后通过 prepareOutgoingText() 接入现有发送路径。认证错误时刷新令牌,失败时提供可见降级,并增强 Agent 指令以告知标记格式。

对比: PR 的方案与此几乎完全一致。实现清晰,安全处理尤其到位——readValidatedImage 做了绝对路径 + 扩展名白名单 + realpathSync 目录约束 + fstat 常规文件校验 + 读取前 20 MiB 上限 + 魔数与扩展名 MIME 交叉校验,并打开解析后的路径(而非原始路径)。凭证处理可靠:sanitizeApiMessageerrmsg 中脱敏 token,网络错误抛出不含 URL 的通用消息,并有专门测试断言 token 不会出现在错误或日志中。代码屏蔽使行内/围栏代码中的 [IMAGE: …] 保持原样,替换从后往前执行以确保重复标记正确解析。

未发现阻塞性问题。两个非阻塞观察(wenshao 的审查中均已提出):

  • 构造函数直接修改 this.config.instructions(ChannelBase 按引用持有 config)。可以工作,但计算有效值会更干净。
  • extractTitle() 去除开头的 #*-> 但不去除 !,因此纯图片回复的标题会是 ![image](@lAL-…。仅影响推送预览,属于外观问题。

测试

这是无人值守的 CI 运行——无本地构建或 tmux 测试。以下证据来自 PR 自身在受审提交上的 CI 检查。

Ubuntu 单元测试通过。Windows/macOS 测试和集成测试被跳过(fork PR 的典型情况)。作者报告本地 154/154 钉钉包测试全部通过。主动群发/私聊和失败降级使用模拟 API 边界验证,未对真实接收者执行。

未验证:真实钉钉机器人行为(需要配置好的应用机器人 channel)。维护者可在一次性环境中检出分支确认端到端流程。

Qwen Code · qwen3.8-max-preview

Reviewed at 93c92c00089c33e3361d4e279277cc63eb092b24 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Confidence: 5/5 — clean across every stage; would merge without hesitation.

This is a well-executed feature PR. The problem is real (DingTalk users get a local path they can't open), the solution is the obvious one (upload via DingTalk's media API and embed the MediaID), and the implementation is careful where it matters — path validation is thorough, credentials never leak into errors or logs, and the code-fence masking means examples in documentation stay literal. The test suite covers the important edge cases: symlink escape, extension/content mismatch, size limits, token retry, fallback messaging, and mention coalescing.

Going back to my independent proposal — the PR matches it almost line for line. I don't see a simpler path that covers the same ground. Every change in the diff serves the feature; there's no scope creep. The two non-blocking nits (config mutation, image-only title) are polish, not concerns.

The author has been responsive and the PR already had a thorough human review (wenshao) that reached the same conclusion. CI is green on Ubuntu. Ships the feature cleanly.

中文说明

信心:5/5 —— 每个阶段都很干净,毫不犹豫地合并。

这是一个执行良好的功能 PR。问题是真实的(钉钉用户收到无法打开的本地路径),方案是显而易见的(通过钉钉媒体 API 上传并嵌入 MediaID),实现在关键处很仔细——路径校验全面,凭证不会泄露到错误或日志中,代码围栏屏蔽确保文档中的示例保持原样。测试套件覆盖了重要的边界情况:软链接逃逸、扩展名/内容不匹配、大小限制、令牌重试、降级消息和 @ 合并。

回顾我的独立方案——PR 几乎逐行匹配。我没有看到更简单的路径能覆盖同样的范围。diff 中的每一处改动都服务于该功能,没有范围蔓延。两个非阻塞的小问题(config 修改、纯图片标题)是打磨,不是顾虑。

作者响应积极,PR 已经有了 wenshao 的全面人工审查并得出相同结论。CI 在 Ubuntu 上通过。干净地交付了功能。

Qwen Code · qwen3.8-max-preview

Reviewed at 93c92c00089c33e3361d4e279277cc63eb092b24 · re-run with @qwen-code /triage

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship. ✅

@wenshao
wenshao added this pull request to the merge queue Jul 27, 2026
Merged via the queue into QwenLM:main with commit af7a9ec Jul 27, 2026
63 of 64 checks passed
Comment on lines +54 to +56
const closing = text.indexOf(delimiter, offset + runLength);
const newline =
runLength >= 3 ? -1 : text.indexOf('\n', offset + runLength);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] maskCode treats mid-line triple-backtick runs as fenced-code openers. When a response contains mid-line ``` before a real fenced block (e.g. "Use bash for scripts, then:\n\n[IMAGE: /tmp/chart.png]\n```"), the opening fence of the real block is consumed as a closer for the mid-line run. The real fenced block's content is then exposed to findImageMarkers, which processes the `[IMAGE: ...]` marker inside it — reading and uploading the file and replacing the code example.

This violates issue #7687's acceptance criterion: "Parse image markers outside fenced and inline code without altering examples inside code."

Root cause: for runLength >= 3, newline is unconditionally set to -1 (line 56), so closesBeforeNewline is always true. Any subsequent ``` is accepted as a closer regardless of line position.

Failure scenario: LLM generates text referencing triple backticks mid-line before showing a fenced code example → image marker inside the example is incorrectly processed, performing an unintended file upload and corrupting the response.

Suggested change
const closing = text.indexOf(delimiter, offset + runLength);
const newline =
runLength >= 3 ? -1 : text.indexOf('\n', offset + runLength);
const closing = text.indexOf(delimiter, offset + runLength);
const newline =
runLength >= 3
? text.indexOf('\n', offset + runLength)
: text.indexOf('\n', offset + runLength);
const closesBeforeNewline =
closing !== -1 && (newline === -1 || closing < newline);

For runLength >= 3, also require the closer to be at a line boundary (preceded only by whitespace since the start of its line), matching CommonMark fenced-code semantics.

— qwen3.7-max via Qwen Code /review

Comment on lines +233 to +235
if (!this.config.instructions) {
this.config.instructions = [
'## DingTalk Channel',

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] Two of the three constructor instruction-injection branches are untested.

Concrete cost: The default branch (!this.config.instructions) assigns a full instruction block — no test asserts its content or that IMAGE_INSTRUCTIONS is included. A regression that garbles the default text or omits the image guidance would go undetected. The skip guard (instructions already contains '[IMAGE:') is also untested — a future change that re-appended IMAGE_INSTRUCTIONS despite the guard would silently duplicate instructions.

Suggested fix: Add two tests:

  1. Create a channel without instructions and assert the default includes '## DingTalk Channel' and '[IMAGE:'
  2. Create a channel with instructions already containing '[IMAGE:' and assert the instructions remain unchanged

— qwen3.7-max via Qwen Code /review

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Released in v0.21.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/self-reported The linked issue was opened by the PR author (self-reported)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(dingtalk): support outbound image delivery

4 participants