Skip to content

fix(telemetry): gate request_text/response_text on logPrompts - #11670

Merged
yiliang114 merged 11 commits into
mainfrom
fix/issue-11666-logprompts-request-text
Sep 12, 2026
Merged

fix(telemetry): gate request_text/response_text on logPrompts#11670
yiliang114 merged 11 commits into
mainfrom
fix/issue-11666-logprompts-request-text

Conversation

@yiliang114

Copy link
Copy Markdown
Collaborator

What this PR does

When telemetry.logPrompts is false, the API request/response logging path in LoggingContentGenerator no longer serializes conversation content. request_text and response_text are omitted from ApiRequestEvent / ApiResponseEvent, so they never reach the native OTLP log exporters, the traces-only log-to-span bridge, or telemetry.outfile. It also adds request_text to the bridge's sensitive-attribute denylist so that, when prompt logging is enabled but sensitive span attributes are disabled, the bridge strips request_text consistently with prompt / response_text.

Why it's needed

Fixes #11666. An operator can set telemetry.logPrompts: false and still export full conversation text, tool arguments/results, file content, and opaque reasoning replay material through api_request.request_text (and response_text on native export / outfile). The denylist omission also let the traces-only bridge retain request_text while stripping the other sensitive attributes. This closes the gap between the explicit privacy setting and what actually ships to configured telemetry destinations.

Reviewer Test Plan

How to verify

Unit-level reproduction, no OTLP server required. Two new tests in loggingContentGenerator.test.ts exercise LoggingContentGenerator.generateContent directly and assert on the ApiRequestEvent / ApiResponseEvent handed to the mocked logApiRequest / logApiResponse:

  • logPrompts: falserequest_text and response_text are undefined.
  • logPrompts: truerequest_text contains the request marker and response_text equals the response marker.

The log-to-span-processor.test.ts sensitive-attribute test now includes request_text and asserts it is dropped by default and kept when includeSensitiveSpanAttributes: true.

Evidence (Before & After)

Before the fix, the red tests fail: request_text is "[{\"role\":\"user\",\"parts\":[{\"text\":\"SENSITIVE_REQUEST_MARKER\"}]}]" instead of undefined, and the bridge keeps request_text: "secret request". After the fix, both test files pass.

npx vitest run src/core/loggingContentGenerator/loggingContentGenerator.test.ts src/telemetry/log-to-span-processor.test.ts src/telemetry/loggers.test.ts
# 223 passed (3 files)

Tested on

OS Status
🍏 macOS
🪟 Windows
🐧 Linux

Environment (optional)

N/A — unit tests only.

Risk & Scope

  • Main risk or tradeoff: request_text has no internal consumers (only produced, never read), so omitting it when prompt logging is off is type-safe and breaks nothing in-tree. Same for response_text.
  • Not validated / out of scope: the policy question of whether opaque thoughtSignature replay data (encrypted_content) should be exported as request text even when prompt logging is explicitly enabled — that is a separate decision, not part of this fix. The full monorepo test suite was not run; only the three affected packages/core test files plus tsc --noEmit were exercised.
  • Breaking changes / migration notes: none.

Linked Issues

Fixes #11666

中文说明

这个 PR 做了什么

telemetry.logPromptsfalse 时,LoggingContentGenerator 的 API 请求/响应日志路径不再序列化对话内容。request_textresponse_text 不再写入 ApiRequestEvent / ApiResponseEvent,因此不会流向原生 OTLP 日志导出器、traces-only 的 log-to-span bridge 或 telemetry.outfile。同时将 request_text 加入 bridge 的敏感属性黑名单,使「开启 prompt 日志但关闭敏感 span 属性」时,bridge 对 request_text 的处理与 prompt / response_text 保持一致。

为什么需要

修复 #11666。运营者可显式设置 telemetry.logPrompts: false,却仍通过 api_request.request_text(以及在原生日志导出与 outfile 上的 response_text)导出完整对话文本、工具参数/结果、文件内容以及不透明的推理回放数据。黑名单的遗漏也让 traces-only bridge 在过滤掉其他敏感属性的同时保留了 request_text。本次修复弥合了「显式隐私设置」与「实际导出数据」之间的缺口。

Reviewer 测试计划

如何验证

单元级复现,无需真实 OTLP 服务器。loggingContentGenerator.test.ts 中新增两个测试,直接调用 LoggingContentGenerator.generateContent,并断言交给 mock 的 logApiRequest / logApiResponse 的事件:

  • logPrompts: falserequest_textresponse_text 均为 undefined
  • logPrompts: truerequest_text 包含请求标记、response_text 等于响应标记。

log-to-span-processor.test.ts 的敏感属性测试现在包含 request_text,断言默认被丢弃、开启 includeSensitiveSpanAttributes: true 时被保留。

证据(修复前后)

修复前红测试失败:request_text"[{\"role\":\"user\",\"parts\":[{\"text\":\"SENSITIVE_REQUEST_MARKER\"}]}]" 而非 undefined,bridge 保留了 request_text: "secret request"。修复后两个测试文件均通过。

npx vitest run src/core/loggingContentGenerator/loggingContentGenerator.test.ts src/telemetry/log-to-span-processor.test.ts src/telemetry/loggers.test.ts
# 223 passed (3 files)

测试环境

OS 状态
🍏 macOS
🪟 Windows
🐧 Linux

环境(可选)

N/A —— 仅单元测试。

风险与范围

  • 主要风险或取舍:request_text 无内部消费者(只被生产、从不被读取),因此在关闭 prompt 日志时省略它是类型安全且不影响仓库内任何逻辑的。response_text 同理。
  • 未验证 / 超出范围:关于「即使显式开启 prompt 日志,是否应把不透明的 thoughtSignature 回放数据(encrypted_content)作为请求文本导出」的策略问题——这是单独决策,不属于本次修复。未运行整个 monorepo 测试套件;仅运行了三个受影响的 packages/core 测试文件及 tsc --noEmit
  • 破坏性变更 / 迁移说明:无。

关联 Issue

Fixes #11666

When telemetry.logPrompts is false, skip serializing API request and
response content into telemetry instead of emitting it to every sink.
Also add request_text to the log-to-span bridge's sensitive-attribute
denylist so traces-only telemetry cannot diverge from the native logger.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-issue-patrol/jmtx5dam9y8
@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finishedview run. See the stage comments in this thread for the result.

Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the PR — re-run at head 34c8eb1. Worth saying up front how I pinned that down, because the branch moved three times while I was reviewing it (4db3f55ac00d3e34c8eb1). The last two commits are a test addition and its own revert, so 34c8eb1 and 4db3f55 have the identical tree SHA (4df856e152de30f02601df371f46b3fd203698eb, verified via the compare API — 0 files changed). Everything below is about the content at that tree, not about a commit I only glanced at. The cancelled Lint & Static / Test runs you may see on 4db3f55 were superseded by these pushes, not failures.

Template looks good ✓ — every required heading is present and the Chinese translation is complete rather than abridged.

Problem: observed, not theoretical, and now doubly evidenced. #11666 is open carrying type/bug, category/telemetry, scope/data-privacy, priority/P2, names the commit it was verified against, and reports exact-source probes seeing both markers in request_text. I re-confirmed the mechanism at the base rather than taking the framing on faith: LoggingContentGenerator.logApiRequest did an unconditional JSON.stringify(contents), and logPrompts was consulted in exactly one place in the whole telemetry logger — the prompt attribute in logUserPrompt. Nothing gated the API request payload. On top of that, the sandboxed /verify run reproduced the leak by A/B against the base build on three real destinations (outfile, native OTLP/HTTP logs over real loopback sockets, traces-only bridge), with the base arm leaking and the head arm clean. This is a real defect with a real before/after.

Direction: the gate escalates telemetry to a maintainer mechanically, because what data leaves the product is a privacy contract and that is not a bot's call. I'm keeping that escalation, but it is not doubt about this change — it brings runtime behaviour into line with an already-documented setting rather than setting new policy, the project's own triage of #11666 prescribed precisely this direction, and with logPrompts defaulting to true the default path is untouched.

Size: core paths are touched, so the two-tier gate applies. 7 production logic lines (loggingContentGenerator.ts +4/−2, log-to-span-processor.ts +1/−0) against 188 test lines and 15 docs lines. Nowhere near the 500-line threshold, no large-PR advisory, and the author holds admin, which exempts this from Tier 1 regardless. The title is fix, so the refactor hard-block was never in play. Tier 2 asks for 100% confidence with every downstream consumer named — that audit is in Stage 2, and I have it.

Approach: scope feels right and is close to the minimum this fix could be. Gating inside the two private log helpers is the correct place: I verified there are exactly two production construction sites for ApiRequestEvent / ApiResponseEvent in the entire repo and both live there (every other hit is a test), so a source-level gate is complete by construction rather than by luck, and it covers all three sinks at once instead of chasing them separately. The docs edits aren't scope creep — a denylist whose members are enumerated in two user-facing documents has to move with it. No drive-by refactors.

Two things I'd have flagged last pass are now closed, and I want to record that rather than silently drop them:

Risk: no elevated risk signals — none of the changed files match the high-risk paths from the revert-history analysis (checked mechanically, not by eye).

Moving on to code review. 🔍

中文说明

感谢贡献 —— 本次是在 head 34c8eb1 上的 re-run。先说明我是如何锁定这个 commit 的,因为审查期间分支移动了三次(4db3f55ac00d3e34c8eb1)。最后两个 commit 是一次测试新增及其自身的 revert,因此 34c8eb14db3f55tree SHA 完全相同4df856e152de30f02601df371f46b3fd203698eb,经 compare API 核实,变更文件数为 0)。以下所有内容针对该 tree 的内容,而不是我只扫了一眼的某个 commit。你在 4db3f55 上看到的 Lint & Static / Test 被 cancelled,是被后续推送取代,并非失败。

模板完整 ✓ —— 所有必需标题都在,中文翻译完整、没有省略。

问题: 是已观测到的缺陷,不是理论性加固,且现在有双重证据。#11666 处于 open,带有 type/bugcategory/telemetryscope/data-privacypriority/P2 标签,指明了核查所用的 commit,并报告了 exact-source 探针在 request_text 中实际观测到两个 marker。我没有采信叙述,而是在 base 上独立复核了成因:LoggingContentGenerator.logApiRequest 此前无条件执行 JSON.stringify(contents),而整个 telemetry logger 中只有一处读取 logPrompts —— 即 logUserPrompt 里的 prompt 属性,API 请求载荷从未被管控。此外,沙箱 /verify 已通过与 base 构建的 A/B 对照,在三个真实目的地(outfile、经真实 loopback 套接字的原生 OTLP/HTTP 日志、traces-only bridge)上复现了该泄漏:base 臂泄漏、head 臂干净。这是一个有真实 before/after 的真实缺陷。

方向: 按 gate 规则,telemetry 一律上报给维护者,因为「产品对外输出哪些数据」属于隐私契约,不该由 bot 决定。这一上报我保留,但它并不代表对本改动有疑虑 —— 它是让运行时行为与已有的、已文档化的设置对齐,而非制定新策略;项目自身对 #11666 的分诊也给出了完全相同的方向;且 logPrompts 默认为 true,默认路径不变。

规模: 触及核心路径,适用两级门禁。生产逻辑 7 行loggingContentGenerator.ts +4/−2、log-to-span-processor.ts +1/−0),测试 188 行、文档 15 行。远低于 500 行阈值,不触发大 PR 提示;作者具备 admin 权限,无论如何都豁免于 Tier 1。标题为 fix,因此 refactor 硬阻断本就不适用。Tier 2 要求 100% 确信并点名每一个下游消费者 —— 该核查见 Stage 2,结论是确信的。

方案: 范围合理,基本就是这个修复的最小形态。在两个私有日志辅助方法内加门禁是正确位置:我核实过全仓库构造 ApiRequestEvent / ApiResponseEvent 的生产代码只有两处,且都在这两个方法内(其余命中全部是测试),因此源头门禁是由结构保证完备,而非碰巧完备,并且一次覆盖三个 sink,而不必分别去追。文档改动不属于夹带 —— 一个其成员被两份面向用户文档所枚举的黑名单,本就必须与文档同步。没有顺手重构。

上一轮我会提出的两点现已关闭,我明确记录下来,而不是静默略过:

风险: 无升级风险信号 —— 改动文件均未命中 revert 历史分析得出的高风险路径(机械核查,非目测)。

进入代码审查 🔍

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 34c8eb1cc2373550d5681e9feb48f8b0dec8230f · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Code review

What I'd have written first. Reading only the title and the "why", before opening the diff: gate the two serialisation points inside LoggingContentGenerator on config.getTelemetryLogPromptsEnabled(), emitting undefined rather than an empty string so downstream JSON.stringify drops the key entirely; and add request_text to the log-to-span bridge's SENSITIVE_ATTRIBUTE_KEYS so the traces-only path can't diverge from the native logger a second time. That is what this PR does. I did not find a simpler path it missed — there is only one sensible place to put this gate, and the PR put it there.

No Critical findings, and no AGENTS.md violations. The consumer audit is what makes the 7 production lines safe, so here it is in full rather than as an assertion:

  • request_text has zero production readers. Every hit outside tests is the declaration and constructor assignment in telemetry/types.ts and the producer itself. Nothing can break by it becoming undefined.
  • response_text has one production reader, telemetry/loggers.ts:640-641, and it is truthiness-guarded (if (event.response_text)) — undefined is skipped, not stringified.
  • undefined was already an emitted, already-tested shape for both fields before this PR (isInternal ? undefined : extractResponseText(...) for responses; the existing "omits response_text for … API responses" cases), so no sink is meeting a new value here.
  • Both event types are constructed in exactly two production sites repo-wide, both inside the two helpers this PR gates. The gate is complete by construction.
  • The outfile leg holds. FileExporter.serialize calls safeJsonStringify, which is JSON.stringify with a circular-reference replacer only — undefined-valued keys are dropped, not written as null or "". The new file-exporters.test.ts case drives a real FileLogExporter.export() to a real file and reads it back, and it carries a positive control (expect(out).toContain('"response_text": "visible"')) alongside the absence assertion, so it cannot pass vacuously.
  • Mock-config regression checked, because this is the way a change like this usually breaks something. Adding a new this.config.getTelemetryLogPromptsEnabled() call means every mock Config reaching LoggingContentGenerator must supply it. Every new LoggingContentGenerator in the repo is either in loggingContentGenerator.test.ts (all routed through the createConfig() helper this PR updates) or the single production site in contentGenerator.ts:593, which passes a real Config. And the pre-existing test at loggingContentGenerator.test.ts:791 that reads request_text runs under the default config, where the helper's logPrompts ?? true keeps it populated — so it still passes rather than silently going empty.
  • Docs match the code. The bridge set was {error, error.message, error_message, prompt, function_args, response_text} — 6 keys — and request_text makes 7. Both rewritten enumerations now list all 7, which closes the last two review rounds' findings. I checked the response_text row's new claim that it "carries no content for internal prompt ids or responses with no visible text" against isInternal ? undefined : this.extractResponseText(...): accurate. And log_prompts_enabled in the attribute rows is not an invented name — it's the convention already used at line 620 of the same file for the user_prompt event.

One consistency point worth a follow-up, not a change here. Emitting undefined is what makes the outfile and bridge sinks drop the key, but on the native OTLP log path the key still ships with an empty value — ...event puts request_text: undefined into attributes, undefined is a valid OTel attribute value, and otlp-transformer's toAnyValue(undefined) renders {}, so the receiver sees {"key":"request_text","value":{}}. No content escapes, so this is hygiene rather than a leak — and it is not new, it's how existing optional attributes already serialise. What makes it worth naming is that loggers.ts already states the opposite convention for exactly this shape: normalizeToolCallEvent (lines 168-171) reads "Error fields are deleted (not set to undefined) on success so downstream consumers see key-absent rather than key-present-with-undefined", and implements it with delete. So the file this fix's follow-up would live in has already decided the rule; two lines (if (event.request_text === undefined) delete attributes['request_text'], and the same for response_text — the existing truthiness guard can't do it, because ...event already added the key) would make all three sinks agree with it. I'd leave that out of this PR: it touches a shared logger for a cosmetic wire-shape difference, and this PR is already at the minimum scope that fixes the reported leak.

Relatedly, the PR description says the fields "never reach the native OTLP log exporters" — true of the content, not of the key. The docs text in the diff ("contains request content only when log_prompts_enabled is true") is already accurate, so this is a description nit only, and not worth another push.

One more non-blocking observation. function_args is produced by logToolCall and is not gated on logPrompts, so with prompt logging off, tool arguments (shell commands, file content) still reach the tool_call event. Pre-existing, and outside what #11666 asked for — but see the residual section below, because a maintainer has now measured this and recommends it be tracked.

Test evidence

Per the skill's rules I did not build, run, or execute anything from this PR — the review is static, and test evidence comes from the PR's own CI via the API, the isolated /verify job, and the maintainer's own published rig.

CI at the reviewed head. The branch was pushed three times during this review, which cancelled the previous head's Qwen Code CI run. So at 34c8eb1 the unit suite, lint, and integration jobs are still in progress and have no recorded result yet. I'm reporting that as pending rather than guessing at it — the table below is wrapped in region markers so the finalize job rewrites it in place once CI settles.

Final CI results for 34c8eb1 (auto-updated by the triage finalize job after CI completed):

Check Conclusion
Classify PR ✅ success
Desktop Shell (ubuntu-22.04) ✅ success
Desktop Shell (windows-2022) ✅ success
Integration Tests (no-AK, No Sandbox) ✅ success
Lint & Static (ubuntu-latest, Node 22.x) ✅ success
Test (ubuntu-latest, Node 22.x) ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success

One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。

The cancelled runs on 4db3f55 are worth one line so nobody misreads them: that tree is byte-identical to the current head (shared tree SHA 4df856e152de30f02601df371f46b3fd203698eb), and on it Integration Tests (no-AK, No Sandbox) and Desktop Shell (ubuntu-22.04) both reported success before the push superseded the rest. So the integration signal exists for this exact content; the unit and lint signals do not yet.

Two independent end-to-end verifications now agree, which is more than most privacy fixes get.

Sandboxed /verify: ✅ passed, 110/110 scripted assertions, 0 failed (run). Real node packages/cli/dist/index.js headless processes, real Config, real telemetry SDK, real exporters, real loopback model server; the only difference between arms is the two compiled modules this PR changes. Base arm leaks, head arm clean on all three destinations — outfile (C1/C2), native OTLP/HTTP /v1/logs over real sockets (C5/C6), traces-only bridge (C9/C10, C13/C14) — with logPrompts: true byte-identical between arms (C3/C4, C7/C8, C11/C12), so the no-regression half is pinned too. Mutation matrix: 0 survivors across all three guards, positive control caught. It also discloses that its first run's bridge cells did pass vacuously and were discarded, which is why I trust the rest.

Maintainer verification (wenshao, admin): 18 real qwen -p sessions against a real OTLP/HTTP receiver and a real telemetry.outfile, A/B'd against the merge-base. Same conclusion on all three sinks, plus three things I could not get from a static read:

  • A counterfactual that proves the tests are load-bearing: reverting only the 7 production lines to the merge-base while keeping the PR's tests fails 3 of them; restoring the lines makes all 229 pass. This is the strongest single piece of evidence on the thread — it shows the suite actually pins the change rather than passing either way.
  • Default config is byte-identical (arms P vs O: same request_text lengths, same response_text; residual diff is latency noise), and the QWEN_TELEMETRY_LOG_PROMPTS=false env-var override path works too (R/Q).
  • tsc --noEmit over packages/core produces the identical 12 pre-existing errors on both arms — 0 new type errors.

One correction to how the test count could be over-read, which the maintainer flagged and I'd rather repeat than let stand: the new file-exporters.test.ts case and the two logPrompts: true cases also pass on the merge-base — they're guardrails, not regression tests. That's fine and intentional, but "4 new tests" is not "4 tests that catch this bug"; the counterfactual above is what establishes that. My own praise of the outfile test's positive control stands (it does prevent a vacuous pass), it just isn't the thing that proves the fix.

Does all this still apply at the current head? I checked rather than assumed. /verify ran at d04fb917 and the maintainer at 76de1e23; the compare from either to 34c8eb1 touches only the two .md files and file-exporters.test.tsneither production file changed, and loggingContentGenerator.test.ts / log-to-span-processor.test.ts (the two files whose guards the mutation matrix killed) are unchanged too. The maintainer separately confirmed the three touched source files are byte-identical between the merge-base and current main, so the "before" arm faithfully represents main. Everything transfers.

What is not verified, and what would settle it. The unit suite and lint have no recorded result at 34c8eb1 — that gap is CI completion, which the finalize job handles, not a missing behavioural proof; the behavioural claim is settled twice over on identical production code. ESLint specifically has been run by nobody: the maintainer's local @eslint/eslintrc + ajv is broken (on main too), so that leg rests entirely on the pending Lint & Static check. If you want a /verify pinned to the current head SHA for the record, @qwen-code /verify will produce one — but it would re-prove production code byte-identical to what already passed twice, so I wouldn't spend the runner minutes unless the SHA-pinning itself is what you want.

Residual — measured, pre-existing, and now with a maintainer recommendation attached. With logPrompts: false on the PR head, content still reaches telemetry through attributes this PR doesn't touch: subagent_execution.result carries verbatim model output to the outfile and the traces-only bridge (it is not in SENSITIVE_ATTRIBUTE_KEYS); tool_call.function_args reaches the outfile and native log export (the bridge does deny it); and with includeSensitiveSpanAttributes: true + logPrompts: false, gen_ai.input.messages and the interaction span's new_context still carry the full prompt — so after this PR the bridge requires both flags for request_text while native span attributes require only their own opt-in, and those two sinks disagree in that combination. None of it blocks this PR, whose stated scope is request_text / response_text and which fully delivers that. But #11666 asks for "no request content to any sink", and the maintainer's recommendation — a follow-up issue tracking these three alongside the already-filed #11682 — is the right way to close that loop rather than letting it close by attrition.

Also worth a release-note line, per the same verification: with logPrompts: true + includeSensitiveSpanAttributes: false (an ordinary config), bridge spans carried request_text before this PR and do not after. That is the correct call and it is documented in the diff, but it is a visible dashboard change for anyone on the HTTP traces-only path.

中文说明

代码审查

我原本会怎么写。 只看标题和「为什么需要」、尚未打开 diff 时,我的方案是:在 LoggingContentGenerator 内部两处序列化点上按 config.getTelemetryLogPromptsEnabled() 加门禁,关闭时发出 undefined 而非空字符串,从而让下游 JSON.stringify 直接丢弃该键;同时把 request_text 加入 log-to-span bridge 的 SENSITIVE_ATTRIBUTE_KEYS,避免 traces-only 路径第二次与原生 logger 分叉。本 PR 做的正是这件事。我没有找到它遗漏的更简路径 —— 这个门禁只有一个合理位置,而 PR 就放在那里。

无 Critical 意见,也无 AGENTS.md 违规。 让消费者核查完整呈现(而不是作为一个结论断言),因为这 7 行生产代码的安全性正来自它:

  • request_text 没有任何生产读取方。测试之外的全部命中只有 telemetry/types.ts 中的声明与构造函数赋值,以及生产者本身。它变为 undefined 不会打破任何东西。
  • response_text 只有一个生产读取方 telemetry/loggers.ts:640-641,且带真值判断(if (event.response_text))—— undefined 会被跳过,而不是被序列化。
  • undefined 在本 PR 之前对这两个字段就已是会被发出、且已被测试覆盖的形态(响应侧的 isInternal ? undefined : extractResponseText(...);既有的 "omits response_text for … API responses" 用例),因此没有任何 sink 在此遇到新的取值形态。
  • 全仓库构造这两个 event 的生产代码只有两处,且都在本 PR 加门禁的两个辅助方法内。门禁是由结构保证完备的。
  • outfile 这一支成立。 FileExporter.serialize 调用 safeJsonStringify,它只是带循环引用 replacer 的 JSON.stringify —— undefined 值的键会被丢弃,而不会写成 null""。新增的 file-exporters.test.ts 用例驱动真实的 FileLogExporter.export() 写入真实文件再读回,并且在「不存在」断言之外带有正对照(expect(out).toContain('"response_text": "visible"')),因此不可能空过。
  • 已核查 mock config 回归风险,因为这类改动通常就是这样出问题的。 新增 this.config.getTelemetryLogPromptsEnabled() 调用意味着每一个到达 LoggingContentGenerator 的 mock Config 都必须提供它。全仓库的 new LoggingContentGenerator 要么在 loggingContentGenerator.test.ts(全部经由本 PR 已更新的 createConfig() 辅助方法),要么是 contentGenerator.ts:593 的唯一生产站点,后者传入真实 Config。而 loggingContentGenerator.test.ts:791读取 request_text 的既有测试运行在默认 config 下,辅助方法的 logPrompts ?? true 使其仍被填充 —— 因此它照常通过,而不是静默变空。
  • 文档与代码一致。 bridge 的集合原为 {error, error.message, error_message, prompt, function_args, response_text} —— 6 个键 —— 加上 request_text 成为 7 个。两处重写后的枚举现在都列全了 7 个,这关闭了最近两轮 review 的意见。我把 response_text 行新增的「对内部 prompt id 或无可见文本的响应不携带内容」与 isInternal ? undefined : this.extractResponseText(...) 对照核实:准确。另外,属性行中的 log_prompts_enabled 并非杜撰名称 —— 它是同一文件第 620 行 user_prompt 事件已在使用的约定。

一处值得后续处理、但不必在本 PR 中改的一致性问题。 发出 undefined 正是让 outfile 与 bridge 丢弃该键的原因,但在原生 OTLP 日志路径上,该键仍会以空值发出 —— ...eventrequest_text: undefined 放进了 attributesundefined 是合法的 OTel 属性值,otlp-transformertoAnyValue(undefined) 会渲染成 {},因此接收端看到 {"key":"request_text","value":{}}。没有内容外泄,所以这属于整洁度而非泄漏 —— 而且它不是新问题,既有的可选属性本就是这样序列化的。之所以值得点名,是因为 loggers.ts 已经为完全相同的形态写下了相反的约定normalizeToolCallEvent(168-171 行)的注释是「Error fields are deleted (not set to undefined) on success so downstream consumers see key-absent rather than key-present-with-undefined」,并以 delete 实现。也就是说,这个后续修复所要落的文件本身已经定下了规则;两行代码(if (event.request_text === undefined) delete attributes['request_text']response_text 同理 —— 既有的真值判断做不到这一点,因为 ...event 已经把键加进去了)就能让三个 sink 与该规则一致。我倾向把它留在本 PR 之外:它为了一个外观层面的线路形态差异去改动共享 logger,而本 PR 已经处在能修复所报泄漏的最小范围上。

与此相关,PR 描述中说这两个字段「never reach the native OTLP log exporters」—— 对内容成立,对不成立。diff 中的文档措辞("contains request content only when log_prompts_enabled is true")本身已是准确的,因此这只是描述层面的小瑕疵,不值得再推一次。

另一条不阻塞的观察:function_argslogToolCall 产生,且logPrompts 加门禁,因此在关闭 prompt 日志时,工具参数(shell 命令、文件内容)仍会进入 tool_call 事件。这是既有行为,也超出 #11666 的要求 —— 但请看下方的「残留」一节,维护者已对此做了实测并建议立项跟踪。

测试证据

按 skill 规则,我没有构建、运行或执行本 PR 的任何代码 —— 审查是静态的,测试证据来自 PR 自身 CI(经 API)、隔离的 /verify 任务,以及维护者自己公布的验证环境。

被审查 head 上的 CI。 审查期间分支被推送了三次,导致上一个 head 的 Qwen Code CI 运行被取消。因此在 34c8eb1 上,单元测试、lint 与集成任务仍在进行中,尚无已记录结果。我如实报告为 pending,而不去猜测结果 —— 下方表格以区域标记包裹,CI 结束后由 finalize 任务就地重写。

4db3f55 上那些被取消的运行值得说明一句,以免误读:该 tree 与当前 head 逐字节相同(共享 tree SHA 4df856e152de30f02601df371f46b3fd203698eb),而在它上面 Integration Tests (no-AK, No Sandbox)Desktop Shell (ubuntu-22.04) 都在推送取代其余任务之前报告了 success。因此集成信号对这份完全相同的内容是存在的;单元与 lint 信号尚不存在。

两份彼此独立的端到端验证现已结论一致,这对一个隐私修复来说是超出常规的证据强度。

沙箱 /verify:✅ 通过,110/110 条脚本化断言,0 失败运行记录)。真实的 node packages/cli/dist/index.js headless 进程、真实 Config、真实 telemetry SDK、真实 exporter、真实 loopback 模型服务器;两臂之间唯一差别就是本 PR 改动的那两个已编译模块。base 臂泄漏、head 臂干净,且在全部三个目的地成立 —— outfile(C1/C2)、经真实套接字的原生 OTLP/HTTP /v1/logs(C5/C6)、traces-only bridge(C9/C10、C13/C14);logPrompts: true 时两臂逐字节一致(C3/C4、C7/C8、C11/C12),因此「无回归」这一半同样被钉住。Mutation 矩阵:0 个幸存者,覆盖全部三个 guard,正对照被捕获。报告还披露其第一次运行的 bridge 单元确实空过并被弃用 —— 这正是我信任其余结论的原因。

维护者验证(wenshaoadmin 权限):18 次真实 qwen -p 会话,对接真实 OTLP/HTTP 接收端与真实 telemetry.outfile,并与 merge-base 做 A/B。三个 sink 上结论相同,另外给出了三点我静态阅读无法得到的结果:

  • 一个证明测试确实「吃劲」的反事实实验只把那 7 行生产代码回退到 merge-base、保留 PR 的测试,会有 3 个用例失败;恢复这些行后 229 个用例全绿。这是整个 thread 上最有力的单条证据 —— 它表明测试套件真的钉住了这个改动,而不是改与不改都通过。
  • 默认配置逐字节一致(P 与 O 两臂:request_text 长度相同、response_text 相同,剩余差异仅为耗时噪声),且 QWEN_TELEMETRY_LOG_PROMPTS=false 环境变量覆盖路径同样生效(R/Q)。
  • packages/coretsc --noEmit 在两臂产生完全相同的 12 个既有错误 —— 0 个新增类型错误。

有一处可能被过度解读的地方需要更正,维护者已指出,我宁愿复述也不愿让它留着:新增的 file-exporters.test.ts 用例与两个 logPrompts: true 用例在 merge-base 上同样通过 —— 它们是护栏,不是回归测试。这没有问题、也是有意为之,但「4 个新测试」不等于「4 个能抓住这个 bug 的测试」;真正确立这一点的是上面那个反事实实验。我此前对 outfile 测试正对照的肯定依然成立(它确实能防止空过),只是它并不是证明修复本身的那个东西。

这些结论在当前 head 上是否仍然适用? 我核实了,而不是假定。/verify 运行在 d04fb917、维护者运行在 76de1e23;从任一者到 34c8eb1 的 compare 只涉及两个 .md 文件与 file-exporters.test.ts —— 两个生产文件均未变更,且 loggingContentGenerator.test.ts / log-to-span-processor.test.ts(mutation 矩阵所杀死 guard 所在的两个文件)也未变更。维护者另行确认这三个被改动的源文件在 merge-base 与当前 main 之间逐字节一致,因此 "before" 一侧忠实代表 main。全部结论均可平移。

哪些未验证,以及什么能确立它。 单元测试与 lint 在 34c8eb1 上尚无已记录结果 —— 这一缺口属于 CI 完成度(由 finalize 任务处理),而非缺少行为性证明;行为性主张已在完全相同的生产代码上被确立了两次。特别地,ESLint 至今无人真正跑过:维护者本机的 @eslint/eslintrc + ajv 损坏(main 上同样如此),因此这一支完全依赖仍在进行中的 Lint & Static 检查。如果希望为记录留一份钉在当前 head SHA 上的 /verify@qwen-code /verify 可以产出 —— 但它将重新证明与已两次通过版本逐字节相同的生产代码,因此除非所要的正是 SHA 钉定本身,我不建议为此消耗 runner 时间。

残留 —— 已实测、属既有问题,且现在附有维护者的建议。 在 PR head 上以 logPrompts: false 运行,内容仍会通过本 PR 未触及的属性抵达 telemetry:subagent_execution.result 会把模型输出逐字带到 outfile 以及 traces-only bridge(它不在 SENSITIVE_ATTRIBUTE_KEYS 中);tool_call.function_args 会抵达 outfile 与原生日志导出(bridge 确实会过滤它);而在 includeSensitiveSpanAttributes: true + logPrompts: false 组合下,gen_ai.input.messagesinteraction span 的 new_context 仍携带完整 prompt —— 因此本 PR 之后,bridge 对 request_text 要求两个开关同时开启,而原生 span 属性只要求各自的 opt-in,两者在该组合下出现分歧。这些都不阻塞本 PR:其声明范围是 request_text / response_text,且已完全兑现。但 #11666 要求的是「任何 sink 都不应收到请求内容」,维护者的建议 —— 在已提交的 #11682 之外,再立一个后续 issue 跟踪这三项 —— 是闭合这一缺口的正确方式,而不是让它在消耗中自然消失。

同一份验证还提出一点值得写进 release note:在 logPrompts: true + includeSensitiveSpanAttributes: false(一个很常规的配置)下,bridge span 在本 PR 之前携带 request_text,之后不再携带。这个决定是对的、diff 中也已写明,但对任何走 HTTP traces-only 路径的人来说,这是一次可见的面板变化。

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 34c8eb1cc2373550d5681e9feb48f8b0dec8230f · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Confidence: 3/5 — the change is clean and its central claim is now proven twice end-to-end, including by a maintainer who calls it merge-ready. The cap is purely formal: telemetry is a human call by rule, no human has yet expressed that call as a review, and the merge is blocked by a stale bot artifact. Not doubt about the code.

Stepping back. My independent proposal and this PR landed on the same design, and the reason is that there is only one sensible place to put this gate — I confirmed that rather than assuming it, since two production construction sites for these events exist repo-wide and both are inside the two helpers the PR touches. The gate is therefore complete by construction, not by luck.

What changed my confidence is that this pass stopped relying on argument from source. Two independent end-to-end verifications now agree: the sandboxed /verify A/B (real compiled CLI, real sockets, real files, 110/110 assertions, mutation matrix with zero survivors) and a maintainer's own rig — 18 real qwen -p sessions against a real OTLP receiver and a real outfile, A/B'd against the merge-base. The maintainer's counterfactual is the single most convincing item on the thread: revert only the 7 production lines, keep the PR's tests, and 3 tests fail; restore them and all 229 pass. That is what "the tests pin the change" looks like when it's actually demonstrated rather than asserted. I verified both reports still apply at the current head rather than taking it on faith — neither production file, and neither guard-pinning test file, changed between the verified commits and 34c8eb1. Seven production lines, no reader to break, default path byte-identical, and a real before/after on the wire. If I picked this up in six months I'd thank the author for the denylist entry specifically — that's the part that stops the bridge quietly diverging from the native logger again.

All three of my prior reservations are closed, and I want to record that explicitly rather than let it read as if nothing moved:

  1. The thoughtSignature half of bug(telemetry): API request content is exported despite logPrompts=false #11666 is tracked. Track thoughtSignature/encrypted_content telemetry-export policy decision (#11666 second clause) #11682 is open, carries category/telemetry + scope/data-privacy + need-discussion, and names "bug(telemetry): API request content is exported despite logPrompts=false #11666 second clause" in its title. Fixes #11666 no longer auto-closes a data-privacy bug while silently dropping half its stated expectation. This was my main substantive objection last run.
  2. The Critical is gone. All 19 review threads read resolved (19 total, 19 resolved, 0 open — checked via GraphQL, not by skimming). The one Critical, a Prettier violation in loggers.test.ts, was anchored to a file the author has since dropped from the PR entirely; I confirmed it appears zero times in the current diff. Later rounds were Suggestions about docs enumeration, and the newest commit completed both enumerations to all 7 denylist keys, verified against the actual SENSITIVE_ATTRIBUTE_KEYS set.
  3. The review loop has converged. Five substantive rounds, findings confined to docs wording, every thread resolved. Per AGENTS.md, past roughly five rounds the guidance is to land Critical fixes and defer remaining Suggestions — so I would not open another round over wording, and I'd suggest the same to anyone else reviewing.

Why I am still not approving — and it is now only mechanism, not judgement:

  1. Telemetry is a privacy contract and the gate routes that call to a human — but the human has now spoken, just not in the review state. wenshao (admin) built an independent rig, ran 18 real sessions across all three sinks, and concluded "the fix does what it says on all three sinks, with no default-config change. Merge-ready." Substantively, the direction question I escalated last run is answered, and answered by exactly the person the gate defers to. What's missing is form: that verdict is a comment, not an APPROVE review, and wenshao has submitted zero formal reviews on this PR. I'm not going to convert a maintainer's prose into an approval they chose not to file, and I'm not going to approve a privacy-contract change on the strength of my own reading when the rule says the signature isn't mine to give. So: 3/5, defer, and the ask is narrow — one formal review from someone with CODEOWNERS standing.

  2. A stale bot review is the only thing mechanically blocking the merge. reviewDecision reads CHANGES_REQUESTED and mergeStateStatus reads BLOCKED, but that state comes from the automated /review submitted at 8fb0f8d9 — six commits back — whose only Critical no longer applies and whose every thread is resolved. The main ruleset has dismiss_stale_reviews_on_push: false (verified via the rules API, with required_approving_review_count: 1, require_code_owner_review: true, and no required status checks), so the pushes didn't clear it, and a later COMMENTED review from the same bot doesn't reset a standing vote. The PR is blocked by a review with zero findings behind it. I am deliberately not dismissing it: another flow's artifact, and clearing it is a merge-gate state change this skill doesn't authorise. Dismiss it explicitly, or re-run /review so it lands an opinionated state on the current head.

Lesser: unit suite and lint are still in progress at 34c8eb1 (three pushes mid-review cancelled the prior run), so no recorded unit/lint result exists for this exact tree — though Integration Tests (no-AK, No Sandbox) did report success on the byte-identical 4db3f55. ESLint has now been run by nobody at all, since the maintainer's local install is broken on main too. I have not emitted a deferred-approval instruction on this comment — an escalated PR never carries one — so the finalize job will refresh the CI table but will not approve on my behalf.

Two things I'd ask whoever merges this to carry forward, neither blocking:

What turns this into a merge: one formal approval from someone with CODEOWNERS standing, dismissal (or superseding) of the stale /review, and CI landing green on 34c8eb1. As far as I can tell nothing is waiting on the author — the code is done.

中文说明

Confidence: 3/5 —— 改动干净,其核心主张现已两次被端到端证明,其中一次来自一位称其「可以合并」的维护者。封顶纯属形式原因:按规则 telemetry 由人决定,而这一决定尚未以 review 的形式表达,且合并正被一条过期的 bot 产物阻塞。这不是对代码的疑虑。

退一步看。我独立的方案与本 PR 的设计一致,原因在于这个门禁只有一个合理位置 —— 这一点我是核实过的,不是假设:全仓库构造这两个 event 的生产代码只有两处,且都在本 PR 触及的两个辅助方法内。因此门禁是由结构保证完备,而非碰巧完备。

真正改变我信心的是,本轮不再依赖「从源码推导」。两份彼此独立的端到端验证现已结论一致:沙箱 /verify 的 A/B(真实编译的 CLI、真实套接字、真实文件,110/110 断言,mutation 矩阵 0 幸存者),以及维护者自建的验证环境 —— 18 次真实 qwen -p 会话,对接真实 OTLP 接收端与真实 outfile,并与 merge-base 做 A/B。维护者的反事实实验是整个 thread 上最有说服力的单条证据:回退那 7 行生产代码、保留 PR 的测试,会有 3 个用例失败;恢复后 229 个全绿。当「测试钉住了改动」是被真正演示出来、而不是被断言出来时,它就是这个样子。我核实了两份报告在当前 head 上仍然适用,而不是采信 —— 在被验证的 commit 与 34c8eb1 之间,两个生产文件以及两个钉住 guard 的测试文件均未变更。7 行生产代码,没有读取方会被打破,默认路径逐字节一致,并且在线路上有真实的 before/after。如果半年后由我接手,我会特别感谢作者留下黑名单这一条 —— 正是它防止 bridge 再次悄悄与原生 logger 分叉。

我此前三条保留意见已全部关闭,我明确记录下来,以免读起来像什么都没变:

  1. thoughtSignature 那一半已有跟踪。 Track thoughtSignature/encrypted_content telemetry-export policy decision (#11666 second clause) #11682 处于 open,带有 category/telemetry + scope/data-privacy + need-discussion 标签,标题写明「bug(telemetry): API request content is exported despite logPrompts=false #11666 second clause」。Fixes #11666 不再会在静默丢掉一半陈述期望的情况下关闭一个数据隐私缺陷。这是我上一轮主要的实质性反对意见。
  2. Critical 已消失。 全部 19 条 review thread 均为已解决(共 19 条,19 条已解决,0 条未解决 —— 经 GraphQL 核查,非略读)。唯一的 Critical(loggers.test.ts 中的 Prettier 违规)锚定在作者已完全移出本 PR 的文件上;我确认它在当前 diff 中出现 0 次。后续轮次均为关于文档枚举的 Suggestion,而最新 commit 已把两处枚举补全为全部 7 个黑名单键,我已对照实际的 SENSITIVE_ATTRIBUTE_KEYS 集合核实。
  3. 审查循环已经收敛。 五轮实质性审查,意见局限于文档措辞,每条 thread 均已解决。按 AGENTS.md,超过大约五轮后的指引是只落 Critical、把剩余 Suggestion 推迟 —— 因此我不会因措辞再开一轮,也建议其他审查者同样处理。

为什么仍然没有 approve —— 而且现在只剩机制问题,不是判断问题:

  1. Telemetry 属于隐私契约,按 gate 规则这一决定权在人 —— 但人已经表态了,只是没有落在 review 状态里。 wenshaoadmin 权限)自建了独立验证环境,在全部三个 sink 上跑了 18 次真实会话,结论是「三个 sink 上修复都真实生效,默认配置行为无变化,可以合并」。就实质而言,我上一轮上报的方向问题已经被回答,而且正是由 gate 所要转交的那类人回答的。缺的是形式:该结论是一条评论,而不是一次 APPROVE review,且 wenshao 在本 PR 上提交的正式 review 数为 0。我不会把维护者的文字表述转换成他选择不提交的批准,也不会在规则明确说明「这个签字不该由我给」的情况下,仅凭我自己的阅读去批准一项隐私契约改动。因此:3/5,转交,而所求很窄 —— 只需一位具备 CODEOWNERS 身份的人提交一次正式 review。

  2. 一条过期的 bot review 是机械层面唯一阻塞合并的因素。 reviewDecision 显示 CHANGES_REQUESTEDmergeStateStatus 显示 BLOCKED,但该状态来自自动化 /review8fb0f8d9(六个 commit 之前)提交的审查,而它唯一的 Critical 已不再适用、其每条 thread 均已解决。main 的 ruleset 设置了 dismiss_stale_reviews_on_push: false(已通过 rules API 核实,同时确认 required_approving_review_count: 1require_code_owner_review: true、且无任何必需状态检查),因此推送并未清除它,而同一 bot 之后的 COMMENTED 审查也不会重置既存投票。本 PR 正被一条背后已无任何成立意见的审查所阻塞。我刻意没有去 dismiss 它:它属于另一个流程的产物,清除它是一次影响合并门禁的状态变更,本 skill 未授权。请显式 dismiss,或重跑 /review 让它在当前 head 上落一个有态度的状态。

较小一点:单元测试与 lint 在 34c8eb1 上仍在进行中(审查期间三次推送取消了上一轮运行),因此这份完全相同的 tree 尚无已记录的单元/lint 结果 —— 不过 Integration Tests (no-AK, No Sandbox) 在逐字节相同的 4db3f55 上确实报告了 success。至于 ESLint,至今无人真正跑过,因为维护者本机的安装在 main 上同样是损坏的。我在本评论中没有发出「CI 变绿即批准」的指令 —— 需上报的 PR 从不携带该指令 —— 因此 finalize 任务会刷新 CI 表格,但不会代我批准。

有两件事希望合并者带下去,均不阻塞:

怎样才算可以合并: 一位具备 CODEOWNERS 身份的人提交一次正式批准;dismiss(或取代)那条过期的 /review;CI 在 34c8eb1 上变绿。就我所能判断,没有任何事情在等作者 —— 代码已经完成。

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 34c8eb1cc2373550d5681e9feb48f8b0dec8230f · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

⏸️ Deferring to @zjunothing (assigned owner) and @wenshao (who has already verified this end-to-end and called it merge-ready). The deterministic resolver had nothing to pick from — QWEN_MAINTAINER_HANDLE unset, no area label on the PR so no owner-map match, and zero human reviews on the thread — so these are the PR's own assignee field and the maintainer who actually did the work, not guesses.

Re-run at 34c8eb1. The code side is closed as far as I can tell, and the central claim is now proven twice rather than argued from source.

The sandboxed /verify A/B'd the real compiled CLI against the base build over real sockets and real files: base leaks conversation content on all three telemetry destinations with logPrompts: false, head is clean, both arms byte-identical with logPrompts: true, mutation matrix killed all three guards with zero survivors. Then @wenshao independently built a rig and ran 18 real qwen -p sessions against a real OTLP receiver and a real outfile, A/B'd against the merge-base, and reached the same conclusion — plus the counterfactual that matters most: revert only the 7 production lines, keep the PR's tests, and 3 tests fail; restore them and all 229 pass. Default config byte-identical, env-var override path works, tsc --noEmit shows 0 new type errors. I checked that both reports still apply at the current head: neither production file nor either guard-pinning test file changed between the verified commits and 34c8eb1.

My own static review found no correctness, security, or regression issue: two production construction sites for these events exist repo-wide and both are gated; request_text has zero production readers; the single response_text reader is truthiness-guarded; undefined was already an emitted and tested shape for both fields; the outfile leg holds because safeJsonStringify is JSON.stringify and drops undefined-valued keys; and the new mock-config call can't break any test, since every construction site either routes through the updated createConfig() helper or passes a real Config. Seven production lines, default path unchanged.

All three of my prior reservations are closed. The thoughtSignature half of #11666 is tracked in #11682 (which names "#11666 second clause" in its title), so Fixes #11666 no longer over-closes. All 19 review threads read resolved (19 total, 0 open), including the one Critical — which was anchored to loggers.test.ts, a file the author has since dropped from the PR entirely (0 occurrences in the current diff). The docs enumerations were completed to all 7 denylist keys, verified against the actual set. Five /review rounds have converged on wording; per AGENTS.md's ~5-round guidance I would not open another.

So the ask is now narrow and purely formal — two items, neither a code defect:

  1. One formal review from someone with CODEOWNERS standing. @wenshao, your verification is the strongest evidence on this thread and your "merge-ready" verdict answers the direction question I escalated last run. But it's a comment, not an APPROVE review, and you have zero formal reviews filed here. I'm not going to convert your prose into an approval you chose not to file, and the gate says the signature on a telemetry privacy contract isn't mine to give. If your verdict stands, filing it as a review is the one thing that moves reviewDecision.

  2. Dismiss (or supersede) the stale bot review. reviewDecision is CHANGES_REQUESTED and mergeStateStatus is BLOCKED because of the automated /review submitted at 8fb0f8d9 — six commits back — whose sole Critical no longer applies and whose every thread is resolved. The main ruleset has dismiss_stale_reviews_on_push: false, so the pushes didn't clear it, and a later COMMENTED review doesn't reset a standing vote. The PR is blocked by a review with nothing behind it. I did not dismiss it: another flow's artifact, and clearing it is a merge-gate change this skill doesn't authorise.

Smaller notes, none blocking:

  • Unit suite and lint are still in progress at 34c8eb1 (three pushes mid-review cancelled the prior run), so no recorded unit/lint result exists for this exact tree; Integration Tests (no-AK, No Sandbox) did report success on the byte-identical 4db3f55. ESLint has been run by nobody — @wenshao's local @eslint/eslintrc + ajv is broken on main too — so that leg rests entirely on the pending check. No deferred-approval instruction on the Stage 3 comment: an escalated PR never carries one, so the finalize job will refresh the CI table but won't approve on my behalf.
  • Worth filing before bug(telemetry): API request content is exported despite logPrompts=false #11666 auto-closes: three attributes still carry content with logPrompts: falsesubagent_execution.result (verbatim model output, to outfile and the bridge), tool_call.function_args, and gen_ai.input.messages / new_context under includeSensitiveSpanAttributes: true. Out of this PR's scope and non-blocking, but bug(telemetry): API request content is exported despite logPrompts=false #11666 asks for "no request content to any sink", so a follow-up issue alongside Track thoughtSignature/encrypted_content telemetry-export policy decision (#11666 second clause) #11682 stops the gap closing untracked. I flagged function_args from a static read; @wenshao then measured it — two independent passes on the same residual.
  • A release-note line for the traces-only change: logPrompts: true + includeSensitiveSpanAttributes: false used to carry request_text on bridge spans and no longer does. Correct and documented, but visible on dashboards.
  • Non-blocking wire-shape nit: on the native OTLP log path the key still ships with an empty value ({"key":"request_text","value":{}}) — no content, and consistent with how existing optional attributes serialise. loggers.ts already states the opposite convention for tool-call errors ("deleted (not set to undefined) … so downstream consumers see key-absent"), so a two-line follow-up there would make all three sinks agree. Leave it out of this PR.

Nothing is waiting on the author. Needs a human click, not more code.

中文说明

⏸️ 转交 @zjunothing(PR 的 assignee)与 @wenshao(已对本 PR 做端到端验证并判定可以合并)。确定性解析器没有可选项 —— QWEN_MAINTAINER_HANDLE 未设置、PR 上无 area 标签因此 owner map 无匹配、且 thread 上没有任何人类 review —— 因此这两位分别来自 PR 自身的 assignee 字段和实际完成验证的维护者,而非猜测。

本次是在 34c8eb1 上的 re-run。就我所能判断,代码层面已经收尾,且核心主张现在是被证明了两次,而不是从源码推导出来的。

沙箱 /verify 将真实编译后的 CLI 与 base 构建经真实套接字、真实文件做了 A/B 对照:logPrompts: false 时 base 在全部三个 telemetry 目的地泄漏对话内容,head 干净;logPrompts: true 时两臂逐字节一致;mutation 矩阵杀死了全部三个 guard、0 个幸存者。随后 @wenshao 独立搭建了验证环境,跑了 18 次真实 qwen -p 会话,对接真实 OTLP 接收端与真实 outfile,并与 merge-base 做 A/B,得出相同结论 —— 外加最关键的那个反事实实验:回退那 7 行生产代码、保留 PR 的测试,会有 3 个用例失败;恢复后 229 个全绿。默认配置逐字节一致,环境变量覆盖路径生效,tsc --noEmit 显示 0 个新增类型错误。我核实了两份报告在当前 head 上仍然适用:在被验证的 commit 与 34c8eb1 之间,两个生产文件以及两个钉住 guard 的测试文件均未变更。

我自己的静态审查未发现任何正确性、安全性或回归问题:全仓库构造这两个 event 的生产代码只有两处且均已加门禁;request_text 无任何生产读取方;唯一的 response_text 读取方带真值判断;undefined 对这两个字段本就是会被发出且已被测试覆盖的形态;outfile 这一支成立,因为 safeJsonStringify 就是 JSON.stringify,会丢弃 undefined 值的键;而新增的 mock-config 调用不会打破任何测试,因为每一个构造站点要么经由已更新的 createConfig() 辅助方法,要么传入真实 Config。7 行生产代码,默认路径不变。

我此前三条保留意见已全部关闭。 #11666thoughtSignature 那一半由 #11682 跟踪(标题写明「#11666 second clause」),因此 Fixes #11666 不再过度关闭。全部 19 条 review thread 均为已解决(共 19 条,0 条未解决),包括唯一的 Critical —— 它锚定在 loggers.test.ts,而作者已将该文件完全移出本 PR(当前 diff 中 0 次出现)。文档枚举已补全为全部 7 个黑名单键,我已对照实际集合核实。五轮 /review 已收敛于措辞;按 AGENTS.md 的大约五轮指引,我不会再开一轮。

因此所求现在很窄、且纯属形式 —— 两项,均非代码缺陷:

  1. 一位具备 CODEOWNERS 身份的人提交一次正式 review。 @wenshao,你的验证是这个 thread 上最有力的证据,你「可以合并」的结论回答了我上一轮上报的方向问题。但它是一条评论,而不是一次 APPROVE review,而你在此提交的正式 review 数为 0。我不会把你的文字表述转换成你选择不提交的批准,且按 gate 规则,telemetry 隐私契约上的签字不该由我给。如果你的结论不变,把它落成一次 review 就是唯一能推动 reviewDecision 的动作。

  2. dismiss(或取代)那条过期的 bot review。 reviewDecisionCHANGES_REQUESTEDmergeStateStatusBLOCKED,原因是自动化 /review8fb0f8d9(六个 commit 之前)提交的审查,而它唯一的 Critical 已不再适用、其每条 thread 均已解决。main 的 ruleset 设置了 dismiss_stale_reviews_on_push: false,因此推送并未清除它,而之后的 COMMENTED 审查也不会重置既存投票。本 PR 正被一条背后已无任何成立意见的审查所阻塞。我没有 dismiss 它:它属于另一个流程的产物,清除它是一次影响合并门禁的变更,本 skill 未授权。

较小几点,均不阻塞:

  • 单元测试与 lint 在 34c8eb1 上仍在进行中(审查期间三次推送取消了上一轮运行),因此这份完全相同的 tree 尚无已记录的单元/lint 结果;Integration Tests (no-AK, No Sandbox) 在逐字节相同的 4db3f55 上确实报告了 success。ESLint 至今无人跑过 —— @wenshao 本机的 @eslint/eslintrc + ajvmain 上同样损坏 —— 因此这一支完全依赖仍在进行中的检查。Stage 3 评论上没有「CI 变绿即批准」的指令:需上报的 PR 从不携带,因此 finalize 任务会刷新 CI 表格,但不会代我批准。
  • 值得在 bug(telemetry): API request content is exported despite logPrompts=false #11666 自动关闭前立项:logPrompts: false 下仍有三个属性携带内容 —— subagent_execution.result(逐字模型输出,抵达 outfile 以及 bridge)、tool_call.function_args,以及 includeSensitiveSpanAttributes: true 时的 gen_ai.input.messages / new_context。超出本 PR 范围且不阻塞,但 bug(telemetry): API request content is exported despite logPrompts=false #11666 要求「任何 sink 都不应收到请求内容」,因此在 Track thoughtSignature/encrypted_content telemetry-export policy decision (#11666 second clause) #11682 之外再建一个后续 issue,才能避免这一缺口在无人跟踪的情况下关闭。function_args 是我从静态阅读中提出的,随后 @wenshao 对它做了实测 —— 两次独立审查落在同一个残留项上。
  • 一行 release note,说明 traces-only 的变化:logPrompts: true + includeSensitiveSpanAttributes: false 此前会在 bridge span 上携带 request_text,现在不再携带。这个决定是对的、也已写入文档,但在面板上可见。
  • 不阻塞的线路形态小问题:在原生 OTLP 日志路径上,该键仍会以空值发出({"key":"request_text","value":{}})—— 无内容,且与既有可选属性的序列化方式一致。loggers.ts 已为 tool-call 错误字段写下了相反的约定(「deleted (not set to undefined) … so downstream consumers see key-absent」),因此在那里做两行的后续修改即可让三个 sink 一致。建议留在本 PR 之外。

没有任何事情在等作者。需要的是人点一下,而不是更多代码。

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 34c8eb1cc2373550d5681e9feb48f8b0dec8230f · re-run with @qwen-code /triage

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Plan (not a blocker): src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25527, 2054, 30788, 1016, 2000, 569, 7430 passed.

中文说明

Test Plan(非阻断):src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25527, 2054, 30788, 1016, 2000, 569, 7430 passed

— qwen3.8-max via Qwen Code /review (v0.23.3)

Comment thread packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts Outdated
Comment thread packages/core/src/telemetry/log-to-span-processor.ts
yiliang114 and others added 2 commits September 12, 2026 03:46
Prettier-formats the two new generateContent logPrompts tests so the
Lint & Static gate passes, and adds two streaming counterparts driving
generateContentStream so the request_text/response_text stripping
invariant is pinned on the path that carries interactive traffic.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmtxcihx2yk
Adding request_text to SENSITIVE_ATTRIBUTE_KEYS makes the retained set on
log-to-span bridge spans larger than what two doc statements enumerate.
Update both enumerations to four fields, carry the log_prompts_enabled
condition into the api_request/api_response attribute reference, and widen
the logPrompts descriptions to cover API request/response text.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmtxcihx2yk
@yiliang114

Copy link
Copy Markdown
Collaborator Author

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 110 passed · 0 failed · 110 total

Flakiness gate: ✅ 2 changed test file(s) x 5 identical rounds, no divergence

中文 — 判定:✅ 通过 · 可合入(agent 判定)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:110 通过 · 0 失败 · 110 总计

抖动门:✅ 2 changed test file(s) x 5 identical rounds, no divergence

Verification report

PR #11670 — fix(telemetry): gate request_text/response_text on logPrompts

Verdict: merge-ready — 110/110 scripted assertions passed, 0 failed (assertions.json). Verified head: d04fb917cd80a6f1f9e4538e94105c7891b3eb6d (git rev-parse HEAD^2). Base of the A/B: 28df8b8a7897b0a8490220d00280c1c17d5ad002 (HEAD^1). Diff: 6 files, +168/−9 — of which 7 changed production lines (4+/2− in loggingContentGenerator.ts, 1+ in log-to-span-processor.ts); the rest are tests and docs.

The central claim is proven load-bearing on three real telemetry destinations (outfile, native OTLP/HTTP logs over real sockets, traces-only log-to-span bridge): with telemetry.logPrompts: false, base ships the full serialized conversation as api_request.request_text / api_response.response_text and head does not, on every destination; with logPrompts: true both arms are byte-identical. All three guards the PR introduces are pinned by tests (mutation matrix: 0 survivors, positive control caught). The four findings below are pre-existing or informational, none introduced by this PR, none blocking — they bound what logPrompts: false still does not stop.

中文摘要
  • 结论: merge-ready。110/110 条脚本化断言全部通过,0 失败。验证的 head 为 d04fb917
  • A/B 结论(核心主张成立): 在三个真实遥测目的地(telemetry.outfile、原生 OTLP/HTTP 日志(真实 loopback 套接字)、traces-only 的 log-to-span bridge)上,telemetry.logPrompts: false 时 base 会把完整序列化对话作为 api_request.request_text / api_response.response_text 发出去,head 不会;logPrompts: true 时两臂输出逐字节一致(无回归)。见下表 "Central claim A/B" 与证据图 01-outfile-ab-base-leaks-head-clean.png02-bridge-ab-base-leaks-head-strips.png03-otlp-logs-wire-ab.png
  • Mutation 矩阵: 三个 guard 全部被各自的断言杀死(M1/M2/M3),组合行 M4 恰为 M1+M3 的并集(guard 相互独立、无层叠),正对照 M5 被捕获,0 个幸存者。见 05-mutation-matrix.png
  • Findings(均为既有问题或信息性,非本 PR 引入,均不阻塞):
    1. logPrompts: falsetool_call.function_args 仍逐字导出工具参数(含 shell 命令、文件内容)——同一 bug 类的相邻门,见 04-sibling-tool-args-ungated.png
    2. subagent_execution.result 仍导出模型输出文本(三个目的地都测到,base/head 一致)。
    3. OTLP/HTTP JSON 会以 {"key":"request_text","value":{}} 形式保留空键(无内容泄漏;与既有 undefined 可选属性序列化方式一致)。
    4. traces-only bridge 只有在显式清空 otlpEndpoint 时才会被注册(否则回落到默认 http://localhost:4317 使 logsUrl 为真)——既有行为,但决定了本 PR 第二个主张实际可达的配置范围。
  • 未覆盖范围: 逐 commit 归因(shallow clone,本地仅 1/3 个 commit 可达);与当前 main 的 trial merge(snapshot 的 baseRefOid 本地不存在且与 HEAD^1 不同,无网络);gRPC OTLP 路径;全仓测试套件;--resumeui_telemetry 镜像的回放;交互式 TUI 模式。

Central claim + A/B

Central claim. When telemetry.logPrompts is false, conversation content no longer reaches any telemetry destination via api_request.request_text / api_response.response_text.
Secondary claims. (a) request_text joins the bridge's sensitive-attribute denylist, so the traces-only bridge strips it when prompt logging is on but sensitive span attributes are off, and keeps it when they are on. (b) No in-tree consumer depends on the two fields, so omitting them breaks nothing.

Each cell is one real node packages/cli/dist/index.js headless process (real Config, real telemetry SDK, real exporters) against a real loopback OpenAI-compatible model server; the only difference between arms is the two compiled modules the PR changes. Base-arm cells are expected to leak, so a leaking base cell is a pass (the control validated the probe).

cell arm destination logPrompts sensitive oracle (what the destination actually received) result
C1 base outfile false api_request.request_text carries user marker ×2, resp ×1; api_response.response_text resp ×2 LEAKS (control)
C2 head outfile false no request_text/response_text key at all; no marker in any api_* attribute clean
C3/C4 base/head outfile true request_text + response_text present, identical both arms no regression
C5 base OTLP/HTTP /v1/logs false wire carries request_text/response_text values with markers LEAKS (control)
C6 head OTLP/HTTP /v1/logs false no value-bearing request_text/response_text; no marker in any api_* attribute clean
C7/C8 base/head OTLP/HTTP /v1/logs true values retained, identical both arms no regression
C9 base traces-only bridge true false BRIDGE qwen-code.api_request :: request_text :: USER (x2) (14 bridge spans) LEAKS (control)
C10 head traces-only bridge true false 14 bridge spans incl. 2 api_request; no request_text value stripped
C11/C12 base/head traces-only bridge true true request_text retained, identical both arms docs claim holds
C13 head traces-only bridge false no request_text value (producer gate already removed it) clean
C14 base traces-only bridge false request_text carries user marker ×2 — base got zero protection from logPrompts: false on this path LEAKS (control)

Witnesses: 01-outfile-ab-base-leaks-head-clean.png, 03-otlp-logs-wire-ab.png, 02-bridge-ab-base-leaks-head-strips.png (the last also shows the one attribute that still carries content on every cell — finding 2). Every bridge cell asserts a validity control (bridge produced api_request spans, 14 spans / 2 api_request spans on both arms), so "absent" cannot pass vacuously — my first run's bridge cells did pass vacuously and were discarded (see Methodology).

Secondary claim (b), census. Grepping request_text / response_text across packages/** (excluding tests) finds only producers, the OTLP attribute writer (loggers.ts:640-641), the bridge denylist (log-to-span-processor.ts:64), and tokenUsageService.test.ts:130 asserting the token-usage record does not carry response_text. No reader. Corroborated end-to-end: 225/225 unit tests and the full CLI run green at head.

Corrections

  • Test count in the PR body is stale. The body reports # 223 passed (3 files); measured at the verified head: 225 passed (3 files) (loggingContentGenerator.test.ts 83, log-to-span-processor.test.ts 55, loggers.test.ts 87). The two extra are the streaming counterparts added by the PR's own second commit (d04bb9e9), so the body's number predates them. Description accuracy only — the tests themselves pass.
  • log_prompts_enabled is not an attribute. The docs edits write `request_text` (string, excluded if `log_prompts_enabled` is false); no attribute or setting by that name exists anywhere in packages/** (grep: docs only). This is a pre-existing docs convention (already used for prompt at docs/developers/development/telemetry.md:620) that this PR extends consistently, so matching it is defensible — flagged only so the next reader does not grep for a field that does not exist.
  • Confirming, not correcting: the body's claim that the two fields "never reach the native OTLP log exporters, the traces-only log-to-span bridge, or telemetry.outfile" is true for those two attributes on all three destinations (C2, C6, C10, C13), and the claim that request_text has no internal consumers holds (census above).

Findings

Ordered by severity. None is a regression introduced by this PR; none blocks it.

1. Suggestion (pre-existing, scope): tool_call.function_args still exports verbatim tool arguments when logPrompts: false

Same bug class, adjacent door. On the head build with logPrompts: false, a tool call's arguments reach the outfile (and the native OTLP log exporter) verbatim — including shell commands and, for write_file, file content:

qwen-code.tool_call :: attributes.function_args :: TOOL_ARG  (x1)
tool_call record: function_name="run_shell_command" status="success"
  function_args="{\n  \"command\": \"printf '%s\\n' 'SENSITIVE_TOOL_OUTPUT_MARKER_zz9'\",\n  \"description\": \"SENSITIVE_TOOL_ARG_MARKER_zz9\"\n}"

while in the same run api_request has no request_text key at all (the PR's fix working). Cause: logToolCall (packages/core/src/telemetry/loggers.ts:302, attributes at 323-328) spreads the normalized event and sets function_args: safeJsonStringify(...) with no shouldLogUserPrompts(config) gate; shouldLogUserPrompts has exactly one call site (loggers.ts:271, inside logUserPrompt). function_args is in the bridge denylist, so only the native log / outfile paths carry it. Pre-existing: identical on base arms.

Repro: node tmp/pr11670-verify-20260911-205902/sibling-probe.mjs head false (see 04-sibling-tool-args-ungated.png).

Why it matters to this PR specifically: the description motivates the fix with "an operator can set telemetry.logPrompts: false and still export full conversation text, tool arguments/results, file content …", and the Risk & Scope section names one out-of-scope item (opaque thoughtSignature replay) but not this one. After this PR that content still exports — through tool_call.function_args instead of request_text. The docs wording itself is narrow and accurate, so this is a completeness note for whoever closes #11666, not a doc defect. (The codebase already redacts one specific case for exactly this reason — STRUCTURED_OUTPUT_REDACTED_ARGS in ToolCallEvent — so the hazard is known.)

2. Suggestion (pre-existing, scope): subagent_execution.result still exports model output text when logPrompts: false, on all three destinations

Measured in every logPrompts: false cell on both arms — e.g. C2 (outfile), C6 (OTLP wire), C10/C13 (bridge spans):

qwen-code.subagent_execution :: attributes.result :: RESP (x1)

The value is the subagent's full result string (here the auto-memory extractor's output, which echoed the model's response). Cause: logSubagentExecution (loggers.ts:1020, spread at 1029) has no logPrompts gate, and result is not in SENSITIVE_ATTRIBUTE_KEYS, so the bridge passes it even with includeSensitiveSpanAttributes: false. Pre-existing and untouched by this PR; visible in 02-bridge-ab-base-leaks-head-strips.png as the single surviving line in the head cells.

3. Informational: OTLP/HTTP JSON keeps the attribute key with an empty AnyValue

On the OTLP wire at head with logPrompts: false, the record still contains {"key":"request_text","value":{}} and {"key":"response_text","value":{}} (measured: empty-key residue: request_text=true response_text=true in C6). No content crosses — the census shows no marker in any api_request/api_response attribute — and this matches how other undefined optional attributes already serialize (subagent_name, error_message in the same payload). The outfile sink drops the key entirely. Practical consequence only: a wire audit that greps for the key name still sees it. This is precisely the false positive my own first oracle produced (see Methodology); noting it so a future auditor is not misled. No action needed.

4. Informational (pre-existing): the traces-only bridge is unreachable unless otlpEndpoint is explicitly cleared

Measured A/B of the configuration, same head build (bridge-reachability.log):

settings SDK's own endpoint resolution bridge spans on the wire
only otlpTracesEndpoint set traces=…/v1/traces, logs=http://localhost:4317/v1/logs, metrics=http://localhost:4317/v1/metrics 0
same + otlpEndpoint: '' traces=…/v1/traces, logs=none, metrics=none 14

Cause: Config.getTelemetryOtlpEndpoint() returns telemetrySettings.otlpEndpoint ?? DEFAULT_OTLP_ENDPOINT (packages/core/src/config/config.ts:7795-7796, DEFAULT_OTLP_ENDPOINT = 'http://localhost:4317' at packages/core/src/telemetry/index.ts:13), and the bridge is created only in the else if (tracesUrl) branch of createHttpExporters (packages/core/src/telemetry/sdk-exporters-http.ts:56-62), i.e. only when logsUrl is falsy. So an operator who follows the docs ("bridge spans … used when HTTP traces are exported without a logs endpoint") and sets only otlpTracesEndpoint gets a native log exporter pointed at a default localhost endpoint instead of the bridge — and, pre-PR, would have been shipping request_text there. This bounds how much of the PR's second claim is reachable in practice; the denylist fix itself is correct and now measured (C9–C14). Pre-existing behavior, not caused by this PR.

Minor observation (non-blocking, no behavior impact)

responseText is still computed (extractResponseText, capped at 4096 chars) and the streaming path still consolidates responses before _logApiResponse discards the value. Gating one step earlier would skip that work when logPrompts: false; bounded memory either way. Noted only for completeness.

Not covered

  • Per-commit attribution. Shallow clone (depth 2): git rev-list HEAD^1..HEAD^2 returns 1 commit while the metadata snapshot lists 3 (22aec43a, d04bb9e9, d04fb917), so only the PR head is locally reachable. Verified the aggregate HEAD^1..HEAD diff; per-commit claims were not individually exercised.
  • Trial merge into current main. The snapshot's baseRefOid (78bbd9f55c5a8ba9aab96bad91f42e8c83025737) is not present locally (git cat-file -t fails) and differs from the merge ref's base (HEAD^1 = 28df8b8a…), so main has moved since the merge ref was created. No network/token in this job → could not fetch, could not confirm a conflict-free merge or re-run the suite on merged main.
  • gRPC OTLP path (otlpProtocol: 'grpc'): not exercised; the bridge does not exist on that path.
  • Repo-wide test suite: not run; targeted gates only (3 affected test files, typecheck, prettier, eslint on the changed files).
  • --resume replay of the persisted ui_telemetry mirror after response_text is omitted: not exercised. The no-reader claim rests on the census; the mirror still carries token counts, which is what the replay consumes per usageHistoryService.
  • Interactive TUI mode: all cells are headless (-p). The gates sit in a shared producer, so mode should not matter, but it was not measured.
  • The thoughtSignature / encrypted_content policy question the PR explicitly declares out of scope: not evaluated.
  • Windows/macOS: Linux container only.
  • Two discarded harness iterations, kept as raw evidence: matrix-v1-flawed-oracle.log (byte-substring oracle over OTLP payloads, which false-positives on the empty-AnyValue key residue of finding 3) and matrix-v2-c14-mislabelled.log (C14 expectation mislabelled bridge-clean instead of bridge-leak). Both were corrected and re-run; only the final run feeds assertions.json.

Methodology

Environment: the CI verify container (node:22-bookworm), merge-ref checkout at depth 2, npm ci + npm run build already done at head. Artifact dir tmp/pr11670-verify-20260911-205902/ holds every harness (.mjs), raw logs (matrix.log, mutation.log, typecheck.log, bridge-reachability.log, sibling-head-lp*.log, mutation-*.json, results.json, gates.json, assertions-detail.json) and evidence/*.png.

Control construction. Base sources were taken with git show HEAD^1:<path> for exactly the two changed production files and compiled with the real npm run build -w packages/core (53 s). The head rebuild reproduced the CI-built head bytes exactly (diff -q identical for both modules), and .d.ts output is byte-identical across arms, so swapping only the two compiled .js modules is an exact arm switch; every cell re-asserts the landed arm by grepping the compiled module before spawning. Module sha256 (base 69d2f61c…/e3973cca…, head d5222129…/90937df9…, full values plus the realpath and build-reproduction checks in provenance.txt). Internal workspace links were asserted before trusting any control: readlink -f node_modules/@qwen-code/qwen-code-core/__w/qwen-code/qwen-code/packages/core (the head tree), which is why the base arm swaps compiled modules in place rather than using a second worktree whose node_modules would resolve into the head tree.

How the harnesses drove the code. Each cell spawns the real CLI (node packages/cli/dist/index.js --no-chat-recording --yolo … --auth-type openai --openai-base-url …) with an isolated QWEN_HOME/cwd settings file, against a loopback OpenAI-compatible server that returns marker content (and, in the sibling probe, a tool call whose arguments carry a distinct marker). Destinations are real: telemetry.outfile (FileLogExporter JSON), a loopback OTLP/HTTP receiver for /v1/logs, and a traces-only receiver for /v1/traces feeding the log-to-span bridge. OTLP payloads here are JSON, so oracles parse resourceLogs…attributes / resourceSpans…attributes and judge values, not key substrings. Bridge cells hold the process alive past the bridge's 5 s unref'd flush tick (model-side delay) and assert that bridge spans actually reached the wire.

Vacuity/mutation. Six mutations reverted one guard at a time (plus a 1+3 combination row and a same-file positive control) in the TypeScript source and ran the two affected test files via vitest's JSON reporter; every red is quoted with its expected-vs-actual message in mutation.log / 05-mutation-matrix.png. Gate liveness was proven by planting a formatting break (prettier exit 1) and a type error (tsc exit 2) and restoring; the unit-test gate's liveness is the mutation matrix itself (M1/M2/M5 land in the mutated file). The working tree and packages/core/dist were verified clean / at head afterwards (git status --porcelain empty; diff -q against the head build).

Flakiness gate log

rounds=5 files=2 skipped=0
file packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/loggingContentGenerator/loggingContentGenerator.test.ts
file packages/core/src/telemetry/log-to-span-processor.test.ts: (cd packages/core) npx --no-install vitest run ./src/telemetry/log-to-span-processor.test.ts


per-file results (P=pass F=fail I=infra-exit, one letter per run):
  packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: PPPPP
  packages/core/src/telemetry/log-to-span-processor.test.ts: PPPPP

verdict: pass
summary: 2 changed test file(s) x 5 identical rounds, no divergence

--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 1 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 2 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 2 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 3 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 3 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 4 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 4 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 5 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 5 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)

Evidence images

01-outfile-ab-base-leaks-head-clean

02-bridge-ab-base-leaks-head-strips

03-otlp-logs-wire-ab

04-sibling-tool-args-ungated

05-mutation-matrix

06-targeted-gates

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not explored to full depth (tool budget reached): "agent 1a": runtime confirmation that an undefined log attribute reaches the OTLP payload as { key: 'request_text', value: {} } — two node --input-type=module -e prob…; "agent 6c": confirming that @opentelemetry/core 's addAttribute drops undefined attribute values (i.e. that "excluded" is literally key-absent on the OTLP wire rather ….

Test Plan (not a blocker): src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25528, 2054, 30788, 1016, 2000, 569, 7430 passed.

中文说明

未探索到全部深度(达到工具调用预算):"agent 1a"runtime confirmation that an undefined log attribute reaches the OTLP payload as { key: 'request_text', value: {} } — two node --input-type=module -e prob…"agent 6c"confirming that @opentelemetry/core 's addAttribute drops undefined attribute values (i.e. that "excluded" is literally key-absent on the OTLP wire rather …

Test Plan(非阻断):src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25528, 2054, 30788, 1016, 2000, 569, 7430 passed

— qwen3.8-max via Qwen Code /review (v0.23.3)

Comment thread docs/developers/development/telemetry.md Outdated
Comment thread docs/developers/development/telemetry.md Outdated
Comment thread docs/developers/development/telemetry.md
Comment thread docs/users/configuration/settings.md Outdated
The prior "excluded if log_prompts_enabled is false" phrasing over-promised
presence for response_text (it is also absent for internal prompt ids and for
thought-only/tool-call-only turns) and misdescribed request_text (the key ships
empty on native OTLP log export, and is only absent from telemetry.outfile and
log-to-span bridge spans). Qualify the includeSensitiveSpanAttributes bridge-span
list so prompt / request_text / response_text note they additionally require
telemetry.logPrompts.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmtxixz7myu

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partially reviewed — gaps disclosed.

Not explored to full depth (tool budget reached): "agent 3a": verifying telemetry.md:645/648's claim that request_text / response_text are "empty on native OTLP log export" when logPrompts is false — node_modules/@ope…; "agent 3c": could not directly verify the "key is empty on native OTLP log export" sub-claim for request_text / response_text ( telemetry.md:645,648 ) — node_modules is…; "agent 2": could not execute the OTLP log exporter to observe the empty- AnyValue wire shape directly — node_modules is absent from this review worktree, so that one li…; "agent reverse-audit (round 1)": did not read the three remaining non-streaming response converters (packages/core/src/core/openaiContentGenerator/converter.ts:1294 and :1426, packages/core/src….

Not reviewed: reverse audit — stopped before round 4 by the review time budget.

Test Plan (not a blocker): src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory.

Deferred under the convergence posture (round 3, not a blocker) — recorded, not requested in this round:

  • docs/users/configuration/settings.md:664 — [probe] logPrompts description widened in two of five user-facing places; settings.md:805 (the env-var row), settings.md:881 and the --telemetry-log-prompts help string still scope it to user promp…

Convergence: round 3 posted 4 inline comment(s), 3 of them reported for the first time; the previous round posted 4 (4 new). Findings keep coming back to the same files: docs/developers/development/telemetry.md (findings in rounds 1, 2; 3 more now). A cluster that keeps producing siblings usually means the fixes are treating instances of a shared root cause — triaging that cause before the next round, or splitting an independent cluster into its own pull request, tends to end the loop faster than fixing them one at a time. (Observation only — nothing was withheld from this review because of this observation.)

中文说明

仅完成部分审查,审查缺口已披露。

未探索到全部深度(达到工具调用预算):"agent 3a"verifying telemetry.md:645/648's claim that request_text / response_text are "empty on native OTLP log export" when logPrompts is false — node_modules/@ope…"agent 3c"could not directly verify the "key is empty on native OTLP log export" sub-claim for request_text / response_text ( telemetry.md:645,648 ) — node_modules is…"agent 2"could not execute the OTLP log exporter to observe the empty- AnyValue wire shape directly — node_modules is absent from this review worktree, so that one li…"agent reverse-audit (round 1)"did not read the three remaining non-streaming response converters (packages/core/src/core/openaiContentGenerator/converter.ts:1294 and :1426, packages/core/src…

未审查:反向审计——评审时间预算不足,未能开始第 4 轮。

Test Plan(非阻断):src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory

收敛姿态下延后(第 3 轮,非阻断)——已记录,本轮不要求修改:共 1 条(原文未翻译,列表见上方英文部分)。

收敛情况:第 3 轮发布了 4 条行内评论,其中 3 条是首次提出;上一轮发布了 4 条(其中 4 条首次提出)。发现反复回到同一批文件:docs/developers/development/telemetry.md(第 1、2 轮已出过发现,本轮又有 3 条)。一个不断再生兄弟发现的簇,通常意味着逐条修复只在处理同一根因的实例——先定位并处理该根因,或把独立的簇拆成单独的 PR,通常比逐条修复更快结束循环。(仅为观察——本轮评审未因此扣留任何内容。)

— qwen3.8-max via Qwen Code /review (v0.23.3)

Comment thread docs/developers/development/telemetry.md Outdated
Comment thread docs/developers/development/telemetry.md Outdated
Comment thread docs/developers/development/telemetry.md Outdated
Comment thread docs/developers/development/telemetry.md
yiliang114 and others added 2 commits September 12, 2026 09:42
…t findings

Address four review findings on the logPrompts gating of request_text/response_text:

- R1-1/R2-1: rewrite the api_request/api_response attribute docs so the three suppression causes (internal prompt id, no visible text, logPrompts off) all collapse to "key present but empty on native OTLP log export, absent from telemetry.outfile and log-to-span bridge spans", and note that bridge spans additionally require includeSensitiveSpanAttributes for both keys.
- R2-2: record that opaque thoughtSignature provider payload is exported verbatim with request_text, with the policy decision tracked in #11682.
- R3-1: add loggers.test.ts assertions that request_text/response_text are present-but-undefined keys when logPrompts is off, and a FileLogExporter test that undefined-valued keys are dropped from the outfile JSON.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmtxpdgjtz5

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not explored to full depth (tool budget reached): "agent 1a": end-to-end OTLP wire probe — a LoggerProvider harness against a local collector to observe whether request_text: undefined really arrives as a present-but-emp…; "agent 5": end-to-end verification that the OTLP exporter renders an undefined-valued log attribute as a present-but-empty key (the @opentelemetry/otlp-transformer AnyV…; "agent 1c": the doc clause "opaque thoughtSignature provider payload is exported verbatim as part of request_text " is unverified — I confirmed request_text is JSON.s…; "agent 1c": the three new tests were not executed (no node_modules in this review worktree); their pass/fail is reasoned from source only.; "agent 1c": did not verify that GitHub issue #11682 exists or concerns thoughtSignature policy, as the doc line asserts., and 2 more.

Test Plan (not a blocker): src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25530, 2054, 30788, 1016, 2000, 569, 7430 passed.

Convergence: round 4 posted 5 inline comment(s), 5 of them reported for the first time; the previous round posted 4 (3 new). Findings keep coming back to the same files: docs/developers/development/telemetry.md (findings in rounds 1, 2, 3; 2 more now). The rate of new findings is not falling. A cluster that keeps producing siblings usually means the fixes are treating instances of a shared root cause — triaging that cause before the next round, or splitting an independent cluster into its own pull request, tends to end the loop faster than fixing them one at a time. Batching the remaining fixes and verifying them before the next push, or dropping this PR's reviews to --severity-floor critical, keeps the loop from re-deriving the same set. (Observation only — nothing was withheld from this review because of this observation.)

中文说明

未探索到全部深度(达到工具调用预算):"agent 1a"end-to-end OTLP wire probe — a LoggerProvider harness against a local collector to observe whether request_text: undefined really arrives as a present-but-emp…"agent 5"end-to-end verification that the OTLP exporter renders an undefined-valued log attribute as a present-but-empty key (the @opentelemetry/otlp-transformer AnyV…"agent 1c"the doc clause "opaque thoughtSignature provider payload is exported verbatim as part of request_text " is unverified — I confirmed request_text is JSON.s…"agent 1c"the three new tests were not executed (no node_modules in this review worktree); their pass/fail is reasoned from source only."agent 1c"did not verify that GitHub issue #11682 exists or concerns thoughtSignature policy, as the doc line asserts.,另有 2 条。

Test Plan(非阻断):src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25530, 2054, 30788, 1016, 2000, 569, 7430 passed

收敛情况:第 4 轮发布了 5 条行内评论,其中 5 条是首次提出;上一轮发布了 4 条(其中 3 条首次提出)。发现反复回到同一批文件:docs/developers/development/telemetry.md(第 1、2、3 轮已出过发现,本轮又有 2 条)。新发现的产出速度没有下降。一个不断再生兄弟发现的簇,通常意味着逐条修复只在处理同一根因的实例——先定位并处理该根因,或把独立的簇拆成单独的 PR,通常比逐条修复更快结束循环。把剩余修复攒成一批、验证后再推送,或将本 PR 的评审降到 --severity-floor critical,可以避免循环反复推导同一组发现。(仅为观察——本轮评审未因此扣留任何内容。)

— qwen3.8-max via Qwen Code /review (v0.23.3)

Comment thread packages/core/src/telemetry/loggers.test.ts Outdated
Comment thread packages/core/src/telemetry/loggers.test.ts Outdated
Comment thread packages/core/src/telemetry/file-exporters.test.ts Outdated
Comment thread docs/developers/development/telemetry.md Outdated
Comment thread docs/developers/development/telemetry.md Outdated
@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Historical-head review — head moved to 3e2fd8e373482ae9ad0fbb994f2eaa22bf2946f7 while this review was in flight (past the salvage threshold), so the run finished and posted against the head it reviewed: 8fb0f8d91a3764ab2e746edaa746256b60a1f70c (#10110). The next automatic review covers the delta from that anchor. Full log in the workflow run.

中文说明

历史 head 评审 —— 本次评审进行中 head 移动到了 3e2fd8e373482ae9ad0fbb994f2eaa22bf2946f7(已过 salvage 阈值),因此评审跑完并针对其实际评审的 head 8fb0f8d91a3764ab2e746edaa746256b60a1f70c 发布(#10110)。下一次自动评审将从该锚点起评审增量。完整日志见 workflow 运行

yiliang114 and others added 2 commits September 12, 2026 12:44
…ge gating docs

- loggers.test.ts: drop the inert getTelemetryLogPromptsEnabled overrides; the
  two present-but-empty tests now name and pin the native OTLP log export leg
  (key present with an empty value) instead of a flag neither function reads.
- file-exporters.test.ts: drive FileLogExporter.export() with a real
  ReadableLogRecord rather than poking the inherited private serialize.
- telemetry.md: log-to-span bridge spans keep function_args under
  includeSensitiveSpanAttributes alone, plus prompt/request_text/response_text
  only when logPrompts is also enabled.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmtxvsxv1ze

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Suggestions are inline.

Not explored to full depth (tool budget reached): "agent 1a": whether the OTel logs SDK keeps undefined -valued attributes on the emitted ReadableLogRecord (i.e. whether the OTLP/JSON exporters omit request_text entir…; "agent 1a": a full packages/core unit-test run — I ran only the four touched test files plus contentGenerator.test.ts and loggers.test.ts , so an unrelated core test t….

Test Plan (not a blocker): src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25525, 2054, 30787, 1016, 2000, 569, 7430 passed.

Deferred under the convergence posture (round 5, not a blocker) — recorded, not requested in this round:

  • docs/users/configuration/settings.md:666 — [probe] Two rows describing telemetry.includeSensitiveSpanAttributes now disagree: this diff qualified the env-var row at :807 on logPrompts but left the settings row at :666 promising bridge-span …

Convergence: round 5 posted 3 inline comment(s), 3 of them reported for the first time; the previous round posted 5 (5 new). Findings keep coming back to the same files: docs/developers/development/telemetry.md (findings in round 4; 2 more now). A cluster that keeps producing siblings usually means the fixes are treating instances of a shared root cause — triaging that cause before the next round, or splitting an independent cluster into its own pull request, tends to end the loop faster than fixing them one at a time. (Observation only — nothing was withheld from this review because of this observation.)

中文说明

已审查。 建议见行内评论。

未探索到全部深度(达到工具调用预算):"agent 1a"whether the OTel logs SDK keeps undefined -valued attributes on the emitted ReadableLogRecord (i.e. whether the OTLP/JSON exporters omit request_text entir…"agent 1a"a full packages/core unit-test run — I ran only the four touched test files plus contentGenerator.test.ts and loggers.test.ts , so an unrelated core test t…

Test Plan(非阻断):src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory; 223 passed — this review observed 25525, 2054, 30787, 1016, 2000, 569, 7430 passed

收敛姿态下延后(第 5 轮,非阻断)——已记录,本轮不要求修改:共 1 条(原文未翻译,列表见上方英文部分)。

收敛情况:第 5 轮发布了 3 条行内评论,其中 3 条是首次提出;上一轮发布了 5 条(其中 5 条首次提出)。发现反复回到同一批文件:docs/developers/development/telemetry.md(第 4 轮已出过发现,本轮又有 2 条)。一个不断再生兄弟发现的簇,通常意味着逐条修复只在处理同一根因的实例——先定位并处理该根因,或把独立的簇拆成单独的 PR,通常比逐条修复更快结束循环。(仅为观察——本轮评审未因此扣留任何内容。)

— qwen3.8-max via Qwen Code /review (v0.23.3)

Comment thread docs/developers/development/telemetry.md
Comment thread docs/developers/development/telemetry.md Outdated
Comment thread docs/users/configuration/settings.md Outdated
@wenshao

wenshao commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 131 passed · 0 failed · 131 total

Flakiness gate: ✅ 3 changed test file(s) x 5 identical rounds, no divergence

中文 — 判定:✅ 通过 · 可合入(agent 判定)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:131 通过 · 0 失败 · 131 总计

抖动门:✅ 3 changed test file(s) x 5 identical rounds, no divergence

Verification report

PR #11670 — fix(telemetry): gate request_text/response_text on logPrompts

Verdict: merge-ready — 131/131 scripted assertions passed, 0 failed (assertions.json). Verified head: 76de1e239874cb1c1622eeb1df240c4a1e561c0e (git rev-parse HEAD^2). Base of the A/B: 05a54fc32adc455d3eb6c09a6416de5f8897085f (HEAD^1). Diff: 7 files, +200/−10 — of which 7 changed production lines (4+/2− in loggingContentGenerator.ts, 1+ in log-to-span-processor.ts); the rest is tests and docs.

This is a follow-up round. The previous round verified head d04fb917 (commit 3 of 8); this round verifies 76de1e2398 (commit 8). Every carried-forward measurement below was re-run at the new head, not diffed from the old report. Two facts make the delta cheap to bound and are themselves measured, not assumed: the compiled head modules are byte-identical to the previous round's (d5222129… / 90937df9…), and the compiled base modules are byte-identical too (69d2f61c… / e3973cca…) even though main moved (28df8b8a05a54fc3). So commits 4–8 changed only tests and docs, and the two files this PR touches did not change on main between the two bases. The production A/B was re-run in full anyway.

The central claim is proven load-bearing on three real telemetry destinations at the new head. The sharpest single cell, on the real OTLP/HTTP wire with telemetry.logPrompts: false: base ships request_text carrying 6,829 and 9,541 characters of serialized conversation — including the system-reminder block — while head ships an empty AnyValue and nothing else (01-otlp-wire-ab-base-leaks-head-empty.png). No finding below is a regression introduced by this PR, and none blocks it.

中文摘要
  • 结论: merge-ready。131/131 条脚本化断言全部通过,0 失败。验证的 head 为 76de1e2398(第 8 个 commit),A/B 的 base 为 05a54fc3HEAD^1)。
  • 本轮是复验轮。上一轮验证到 d04fb917(第 3 个 commit)。所有承接的测量都在新 head 上重新执行,未沿用旧报告结论。增量(第 4–8 个 commit)只改了测试与文档:编译产物与上一轮的 head 逐字节相同(d5222129…/90937df9…),base 侧也相同(69d2f61c…/e3973cca…),尽管 main 已从 28df8b8a 前进到 05a54fc3。生产代码 A/B 仍完整重跑。
  • A/B 结论(核心主张成立): 在三个真实遥测目的地(telemetry.outfile、原生 OTLP/HTTP 日志(真实 loopback 套接字)、traces-only 的 log-to-span bridge)上,logPrompts: false 时 base 把完整序列化对话作为 request_text/response_text 发出(实测 6,829 与 9,541 字符,含 system-reminder),head 只留一个空 AnyValuelogPrompts: true 时两臂属性键集与内容一致(无回归)。见 "Central claim + A/B" 各表与证据图 01-otlp-wire-ab-base-leaks-head-empty.png02-outfile-and-bridge-ab.png
  • Mutation 矩阵: 三个 guard 全部被各自的断言杀死(M1/M2/M3),组合行 M4 恰为 M1+M3 的并集(3 = 2+1,说明两个 guard 相互独立、无层叠隐藏),两个同文件正对照 M5/M6 均变红,0 个幸存者。见 03-mutation-matrix-zero-survivors.png
  • 增量唯一新增测试非空洞: 对 safeJsonStringify 做定向变异(undefined → null)后,file-exporters.test.ts 的新测试确实变红并报出 not to contain 'request_text'
  • 上一轮 findings 状态: 全部 stands(均为既有问题或信息性,非本 PR 引入),其中第 4 条被测得更准也更严重——见下方状态表。
  • 未覆盖范围: 逐 commit 归因(shallow clone,8 个 commit 中本地仅 1 个可达);与当前 main 的 trial merge(snapshot 的 baseRefOid 本地不存在,无网络);gRPC OTLP 路径;全仓测试套件;--resume 端到端回放(仅做了读取方普查);交互式 TUI;thoughtSignature 策略问题(PR 自述超范围)及 #11682 引用本身(离线不可核验)。

Previous-finding status at the new head

Re-measured, not diffed. "Stands" means the behaviour was reproduced at 76de1e2398 by a fresh run.

# previous finding severity status at 76de1e2398
1 tool_call.function_args still exports verbatim tool arguments (shell command, file content) when logPrompts: false Suggestion (pre-existing, scope) stands — re-measured on both arms: head's api_request has no request_text key while function_args in the same run still carries {"command": "printf …", "description": "SENSITIVE_TOOL_ARG_MARKER"}; base identical for function_args, so pre-existing (04-sibling-tool-args-still-ungated.png)
2 subagent_execution.result still exports model output text when logPrompts: false Suggestion (pre-existing, scope) stands — the residual-carrier scan finds exactly qwen-code.subagent_execution.result on every head lpOFF cell, and the same carrier on the matching base cell (asserted per cell)
3 OTLP/HTTP JSON keeps the attribute key with an empty AnyValue Informational stands, and now unpinned — measured request_text present=2 empty=2, response_text present=2 empty=2 at head lpOFF. The delta removed the loggers.test.ts assertions that commit 8fb0f8d9 had added for exactly this, so no test now pins it (see "The delta" below — I agree with the removal)
4 traces-only bridge unreachable unless otlpEndpoint is explicitly cleared Informational (pre-existing) stands, and sharper/worse than reported — with the definitive log.bridge oracle: documented traces-only shape yields 0 bridge spans of 5 and delivers 0 log records anywhere; clearing otlpEndpoint yields 14 bridge spans of 19. The traces-only operator does not merely lose the bridge, they lose the logs (05-bridge-reachability-log-bridge-oracle.png)
5 responseText still computed then discarded when logPrompts: false Minor, non-blocking stands — production modules byte-identical to the previous round, so unchanged by construction
C1 Correction: PR body's 223 passed (3 files) is stale Correction stands — measured 225 passed (3 files) at the new head (loggingContentGenerator 83, log-to-span-processor 55, loggers 87)
C2 Correction: log_prompts_enabled is not a real attribute Correction stands, widened — still docs-only, now in 3 places (was 2); the delta extended the same pre-existing convention to request_text and response_text

Central claim + A/B

Central claim. When telemetry.logPrompts is false, conversation content no longer reaches any telemetry destination via api_request.request_text / api_response.response_text.
Secondary claims. (a) request_text joins the bridge's sensitive-attribute denylist, so the traces-only bridge strips it when prompt logging is on but sensitive span attributes are off, and keeps it when they are on. (b) No in-tree consumer depends on the two fields, so omitting them breaks nothing.

Each cell is one real node packages/cli/dist/index.js headless process (real Config, real telemetry SDK, real exporters) against a real loopback OpenAI-compatible model server, driven through the documented settings path (telemetry.logPrompts in settings.json, not the deprecated flag). The only difference between arms is the two compiled modules the PR changes. Base-arm cells are expected to leak, so a leaking base cell is a pass.

Destination 1 — telemetry.outfile (FileLogExporter)

cell arm logPrompts oracle (what the file actually contains) result
OF-base-lpOFF base false request_text PRESENT, response_text PRESENT; 71,250 B LEAKS (control)
OF-head-lpOFF head false both keys ABSENT; 54,303 B clean
OF-base-lpON / OF-head-lpON both true both keys present, attribute key sets byte-identical no regression

The 16,947-byte drop between the two lpOFF cells is the removed conversation content.

Destination 2 — native OTLP/HTTP /v1/logs (real loopback socket)

cell arm logPrompts request_text value on the wire response_text value
OT-base-lpOFF base false [{"role":"user","parts":[{"text":"<system-reminder>…6,829 chars (2nd record 9,541) SENSITIVE_RESPONSE_MARKER_OT-base-lpOFF
OT-head-lpOFF head false <EMPTY AnyValue> <EMPTY AnyValue>
OT-base-lpON base true 6,826 / 9,531 chars marker present
OT-head-lpON head true 6,826 / 9,531 chars — identical to base marker present

Witness: 01-otlp-wire-ab-base-leaks-head-empty.png. Note the base leak is not just "conversation text" — it includes the system-reminder block, i.e. prompt scaffolding the operator never typed.

Destination 3 — traces-only log-to-span bridge

Oracle is the log.bridge attribute the processor itself sets (log-to-span-processor.ts:191), so bridge spans cannot be confused with native trace spans. Every cell asserts a validity control (bridge spans reached the wire = 14, bridge produced api_request spans = 2) on both arms, so "absent" cannot pass vacuously.

cell arm logPrompts sensitive api_request bridge spans carrying request_text result
BR-base-lpON-sensOFF base true false 2 of 2 LEAKS (control) — the denylist gap this PR closes
BR-head-lpON-sensOFF head true false 0 of 2 stripped — secondary claim (a) proven
BR-base-lpON-sensON base true true 2 of 2 retained
BR-head-lpON-sensON head true true 2 of 2 retained — docs claim holds
BR-base-lpOFF-sensON base false true 2 of 2 LEAKS (control) — base got zero protection from logPrompts: false here
BR-head-lpOFF-sensON head false true 0 of 2 clean (producer gate)

Witness: 02-outfile-and-bridge-ab.png.

Secondary claim (b), census — widened past the previous round

Grepping every property access of the two fields across packages/*/src finds no non-test reader: only the producers (types.ts:290, types.ts:412), the OTLP attribute writer (loggers.ts:640-641), and the bridge denylist (log-to-span-processor.ts:64). Every other hit is a test file.

The previous round left the persisted ui_telemetry mirror uncovered, so I closed it: logApiResponse does hand the event to recordUiTelemetryEventToChatchatRecordingService.recordUiTelemetryEvent (chatRecordingService.ts:2458), which persists { uiEvent } verbatim into the chat record. That mirror does have readers — but the only one touching uiEvent is the session-id remap at sessionService.ts:4587, which reads prompt_id alone and spreads the rest through unchanged. Nothing reads response_text off disk. Claim (b) holds; this is a census, not an end-to-end --resume run (see Not covered).

The delta since the previous round (commits 4–8)

Scoped new probes to what actually changed: one new test file and docs wording.

The delta's only new test is not vacuous. file-exporters.test.ts gained FileLogExporter omits undefined-valued attributes (logPrompts off). Mutating the mechanism its comment names — making safeJsonStringify's replacer return null for undefined — turns exactly that test red with expected '{\n "body": "api response",…' not to contain 'request_text' (V1, 03-mutation-matrix-zero-survivors.png). The unmutated control is green (4/4).

The delta also removed tests, so I checked for lost coverage. Commit 8fb0f8d9 added loggers.test.ts assertions pinning "present-but-undefined keys"; head commit 76de1e23 ("avoid pinning exporter empty-key internals") removed them — loggers.test.ts is absent from the final diff. Measured consequence: the mutation matrix still has 0 survivors and both positive controls still fire, so no guard lost coverage. What was lost is a pin on finding 3's informational empty-key residue. I agree with the removal: {"key":"request_text","value":{}} is an OTLP serialization detail of every undefined optional attribute (subagent_name behaves identically in the same payload), and pinning it would freeze a wire-format accident. Recording it here so the unpinned behaviour is a known choice rather than a silent gap.

Docs claims in the delta, tested rather than read.

  • "opaque thoughtSignature provider payload is included" — true by construction and confirmed: request_text is JSON.stringify(this.toContents(req.contents)), and thoughtSignature is an own enumerable part property, so it serializes verbatim (JSON.stringify of a part carrying it reproduces the payload).
  • bridge bullet: "keep their existing function_args field, plus prompt, request_text, and response_text when logPrompts is also enabled" — accurate, and the separation is the subtle part: prompt is gated (loggers.ts:271), function_args is not (logToolCall has no gate, confirmed by finding 1's measurement). The sentence correctly puts function_args outside the logPrompts condition.
  • "carries no content for internal prompt ids" — consistent with the call sites: logApiRequest runs only inside if (!isInternal) (loggingContentGenerator.ts:436, :602).
  • "tracked in Track thoughtSignature/encrypted_content telemetry-export policy decision (#11666 second clause) #11682" — an issue reference; not verifiable without network (see Not covered).

Suggestion (nit, test naming). The new test's name promises a logPrompts scenario but its fixture is a hand-built ReadableLogRecord with request_text: undefined — no Config, no getTelemetryLogPromptsEnabled, no LoggingContentGenerator. The comment above it is honest about this ("when logPrompts is off the producer emits…"), and the assertion is genuinely load-bearing, so this costs nothing functionally; the name just buys a little more confidence than the fixture pays for. A name like omits undefined-valued attributes (as produced when logPrompts is off) would match. Not blocking.

Mutation matrix

One row per guard, plus a combination row, plus two positive controls landed in the same file as the mutant, plus the vacuity check on the delta's new test. Every red is the intended assertion with a behavioural expected-vs-actual message — no red came from a broken import or fixture.

id what was reverted / perturbed suite red quoted failure
M0 (none — unmutated control) all 3 files 0 / 142 green, as required
M1 guard #1: request_text producer gate lcg 2 / 83 expected '[{"role":"user","parts":[{"text":"SEN…' to be undefined
M2 guard #2: response_text producer gate lcg 2 / 83 expected 'SENSITIVE_RESPONSE_MARKER' to be undefined
M3 guard #3: 'request_text' denylist entry l2s 1 / 55 expected { Object (request_text, …) } to not have property "request_text"
M4 combination M1 + M3 together both 3 / 138 union of M1 and M3 — the guards are independent, no layered defence hiding either
M5 positive control, same file as M3: drop the pre-existing 'response_text' entry l2s 1 / 55 expected { …(4) } to not have property "response_text"
M6 positive control, same file as M1/M2: MAX_RESPONSE_TEXT_LENGTH 4096 → 4095 lcg 1 / 83 expected 'xxx…' to have a length of 4096 but got 4095
V1 vacuity check on the delta's new test: safeJsonStringify replacer undefinednull fx 1 / 4 expected '{\n "body": "api response",…' not to contain 'request_text'

Survivors: 0. Both positive controls went red, so the harness can make these suites fail; the unmutated control is green, so the kills mean something. M1's message reproduces the PR body's own claimed "Before" evidence verbatim. Working tree verified clean after every restore (git status --porcelain empty), and packages/core/dist left at head (d5222129…).

Corrections

  • The PR body's test count is stale (carried forward). The body reports # 223 passed (3 files); measured at the verified head: 225 passed (3 files). Description accuracy only — the tests pass.
  • log_prompts_enabled is not an attribute (carried forward, now wider). The delta's docs wording writes contains request content only when `log_prompts_enabled` is true; no attribute or setting by that name exists anywhere in packages/** (grep: docs only, now 3 occurrences). This is a pre-existing docs convention — already used for prompt at telemetry.md:621 — that the PR extends consistently, so matching it is defensible. Flagged so the next reader does not grep for a field that does not exist.
  • Confirming, not correcting: the body's claim that the two fields "never reach the native OTLP log exporters, the traces-only log-to-span bridge, or telemetry.outfile" is true on all three destinations at the new head, and the no-internal-consumer claim holds (census above, now including the persisted ui_telemetry mirror).
  • One correction to the previous round's framing of finding 4, in the direction of worse: it reported the consequence as "gets a native log exporter pointed at a default localhost endpoint instead of the bridge". Measured with the log.bridge oracle, the traces-only shape yields 0 bridge spans and 0 log records delivered to any reachable receiver — the log telemetry is silently dropped, not merely rerouted. Still pre-existing, still not caused by this PR, but it bounds the PR's second claim more tightly than the previous round stated.

Findings

Ordered by severity. None is a regression introduced by this PR; none blocks it. Findings 1, 2 and 4 are re-measurements of carried-forward items and are tabulated above; the substance is restated here only where the new head changed it.

1. Suggestion (pre-existing, scope): logPrompts: false still exports verbatim tool arguments via tool_call.function_args

Same bug class, adjacent door — and the door the PR's own motivation names ("tool arguments/results, file content"). At head with logPrompts: false, in a single run where api_request correctly has no request_text key:

tool_call record: function_name="run_shell_command"
  function_args="{\n  \"command\": \"printf '%s\\n' 'SENSITIVE_TOOL_OUTPUT_MARKER'\",\n  \"description\": \"SENSITIVE_TOOL_ARG_MARKER\"\n}"

Cause: logToolCall (loggers.ts:302) spreads the normalized event and sets function_args: safeJsonStringify(...) with no gate; shouldLogUserPrompts has exactly one call site (loggers.ts:271). function_args is in the bridge denylist, so only the native-log and outfile paths carry it. Base arm is identical → pre-existing. Repro: node tmp/pr11670-verify-20260912-073825/probes.mjs (section A), witness 04-sibling-tool-args-still-ungated.png. Worth naming for whoever closes #11666, since the issue's stated symptom survives through a different key.

2. Suggestion (pre-existing, scope): subagent_execution.result still exports model output text

The residual-carrier scan reports, for every head logPrompts: false cell, exactly which attributes still carry the model's response marker: qwen-code.subagent_execution.result on the outfile and OTLP legs, and additionally the native GenAI span attributes (interaction.gen_ai.output.messages, llm_request.gen_ai.input.messages, llm_request.gen_ai.output.messages) on the bridge leg with includeSensitiveSpanAttributes: true. Each is asserted present on the matching base cell too, so none is introduced here. Cause: logSubagentExecution has no logPrompts gate and result is not in SENSITIVE_ATTRIBUTE_KEYS.

3. Informational: the native OTLP leg keeps an empty-valued key, and nothing pins it any more

At head with logPrompts: false: request_text present=2 empty=2, response_text present=2 empty=2{"key":"request_text","value":{}}. No content crosses (the census finds no marker in any api_request/api_response value), and this matches how every undefined optional attribute already serializes (subagent_name in the same payload). The outfile sink drops the key entirely, which is what the delta's new test pins. Practical consequence only: a wire audit grepping for the key name still sees it. Now unpinned by any test after 76de1e23 — deliberately, and I agree (see "The delta").

4. Informational (pre-existing): in the documented traces-only configuration, log telemetry is silently dropped

settings bridge spans (log.bridge) native spans log records delivered
only otlpTracesEndpoint set 0 5 0
same + otlpEndpoint: '' 14 5 n/a (bridged)

Cause, read from source and confirmed by the table: Config.getTelemetryOtlpEndpoint() returns telemetrySettings.otlpEndpoint ?? DEFAULT_OTLP_ENDPOINT (DEFAULT_OTLP_ENDPOINT = 'http://localhost:4317', telemetry/index.ts:13), so logsUrl stays truthy and the bridge is never constructed — it lives only in the else if (tracesUrl) branch of createHttpExporters (sdk-exporters-http.ts:56-62). An operator following the docs ("used when HTTP traces are exported without a logs endpoint") who sets only otlpTracesEndpoint gets log records aimed at an unreachable default and loses them without any signal. Pre-existing; it bounds how much of this PR's second claim is reachable in practice. Witness 05-bridge-reachability-log-bridge-oracle.png.

Minor observation (non-blocking, unchanged)

responseText is still computed (extractResponseText, capped at 4096) and the streaming path still consolidates responses before _logApiResponse discards the value. Gating one step earlier would skip that work when logPrompts: false; bounded memory either way.

Not covered

  • Per-commit attribution. Shallow clone (git rev-parse --is-shallow-repositorytrue): git rev-list HEAD^1..HEAD^2 returns 1 commit while the metadata snapshot lists 8 (22aec43a, d04bb9e9, d04fb917, 7f9f68b0, 8fb0f8d9, 3e2fd8e3, b51e0af0, 76de1e23), and the previous round's head d04fb917 is not present locally (git cat-file -t fails). Verified the aggregate HEAD^1..HEAD diff; the delta's contents were inferred from the final diff plus commit messages, not from per-commit diffs. No per-commit table is presented.
  • Trial merge into current main. The snapshot's baseRefOid (78bbd9f5…) is not present locally and differs from the merge ref's base (HEAD^1 = 05a54fc3…), so main has moved since the merge ref was created. No network/token in this job → could not fetch, could not confirm a conflict-free merge, could not re-run on merged main. Mitigating evidence: the two changed production files compile to bytes identical to the previous round's base, i.e. main did not touch them between 28df8b8a and 05a54fc3.
  • #11682, referenced by the new docs wording for the thoughtSignature policy decision: an issue number, unverifiable offline. The thoughtSignature serialization claim was verified; the tracking issue's existence and content were not.
  • gRPC OTLP path (otlpProtocol: 'grpc'): not exercised; the bridge does not exist on that path.
  • Repo-wide test suite: not run. Targeted gates only — the 3 PR-named files (225), the new test file (4), the whole src/telemetry suite (1002 passed / 30 files), tsc --noEmit for packages/core, prettier and eslint on the changed files. packages/cli was not run.
  • --resume end-to-end replay of the persisted ui_telemetry mirror: closed by census only (the sole reader of systemPayload.uiEvent is sessionService.ts:4587, which reads prompt_id and spreads the rest). No resume was actually driven.
  • Interactive TUI mode: all cells are headless (-p). The gates sit in a shared producer, so mode should not matter, but it was not measured.
  • The thoughtSignature / encrypted_content policy question the PR explicitly declares out of scope: not evaluated.
  • Windows/macOS: Linux container only.
  • Six superseded assertions, kept as raw evidence. The first probes.mjs run scored 11/14; the 3 reds were my harness's fault, not the PR's — a crude "total span count" oracle that counted native trace spans as bridge spans, plus a debug-line oracle (QWEN_DEBUG_LOG_FILE) that never produced a file. Corrected to the log.bridge oracle and re-run as reachability.mjs (9/9); the 6 superseded entries are excluded from assertions.json by build-assertions.mjs and the exclusion count is recorded in assertions-detail.json. An earlier matrix.mjs iteration likewise scored 70/86 because otlpLogRecords keyed name off the record body and OTLP AnyValue objects were compared as strings; both were fixed and the matrix re-run clean at 92/92. Only the final runs feed the counts.

Methodology

Environment: the CI verify container (node:22-bookworm, no zstd, $RUNNER_TEMP unset in-shell), merge-ref checkout at depth 2, npm ci + npm run build already done at head. Artifact dir tmp/pr11670-verify-20260912-073825/ holds every harness (harness.mjs, matrix.mjs, probes.mjs, reachability.mjs, mutations.mjs, wire-summary.mjs, build-assertions.mjs, gates.sh), raw logs (matrix.log, probes.log, reachability.log, mutation.log, gates.log, wire-summary.log, build-*.log), per-cell JSON (cells/*.json), machine-readable results (matrix-results.json, probes.json, reachability.json, mutation-rows.json, assertions.json, assertions-detail.json, provenance.txt) and evidence/*.png.

Control construction. Internal workspace links were asserted before trusting any control: readlink -f node_modules/@qwen-code/qwen-code-core/__w/qwen-code/qwen-code/packages/core, i.e. the head tree — so a second worktree would have silently loaded head code. Base sources were therefore taken with git show HEAD^1:<path> for exactly the two changed production files and compiled with the real npm run build -w packages/core (31 s); the arm switch swaps the two compiled .js modules in place, and every cell re-asserts the landed arm by grepping the compiled bytes (gateCount, denylisted) and recording both sha256s in the cell JSON before spawning. The head rebuild reproduced the CI-built head bytes exactly (diff -q identical for both modules), so swapping modules is an exact arm switch. The PR leaves package.json/package-lock.json untouched, so reusing the root node_modules for both arms is a clean control.

How the harnesses drove the code. Each cell spawns the real CLI with an isolated HOME/cwd settings.json against a loopback OpenAI-compatible server returning marker content (and, in the sibling probe, a tool call whose arguments carry a distinct marker). Destinations are real: telemetry.outfile, a loopback OTLP/HTTP receiver for /v1/logs, and a traces-only receiver for /v1/traces feeding the bridge. Oracles judge values, not key substrings: OTLP AnyValues are unwrapped and an empty one is distinguished from an absent key, which is exactly the false positive a substring oracle produces (finding 3). Bridge cells use a 6 s model-side delay to outlive the bridge's unref'd flush tick and assert on the processor's own log.bridge marker. The outfile is safeJsonStringify(data, 2) — pretty-printed concatenated objects, not JSONL — so it is parsed with a string-aware brace scanner. Universal validity controls (CLI exited 0, model was actually called, api_request records/spans were produced) run on both arms, so no "absent" result can pass vacuously.

Counts. assertions.json is generated by build-assertions.mjs from the result files, never hand-counted: matrix 92, sibling probe 8, reachability 9, mutation rows 9, gates 13 → 131 pass / 0 fail. Gate liveness was proven by planting a formatting break (prettier exit 1), a lint violation (no-explicit-any + no-console, eslint exit 1) and a type error (TS2322, tsc exit 2), then restoring; the unit-test gate's liveness is the mutation matrix itself (M5/M6 land in the mutated files). Working tree and packages/core/dist verified clean/at-head afterwards.

Flakiness gate log

rounds=5 files=3 skipped=0
file packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/loggingContentGenerator/loggingContentGenerator.test.ts
file packages/core/src/telemetry/file-exporters.test.ts: (cd packages/core) npx --no-install vitest run ./src/telemetry/file-exporters.test.ts
file packages/core/src/telemetry/log-to-span-processor.test.ts: (cd packages/core) npx --no-install vitest run ./src/telemetry/log-to-span-processor.test.ts


per-file results (P=pass F=fail I=infra-exit, one letter per run):
  packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: PPPPP
  packages/core/src/telemetry/file-exporters.test.ts: PPPPP
  packages/core/src/telemetry/log-to-span-processor.test.ts: PPPPP

verdict: pass
summary: 3 changed test file(s) x 5 identical rounds, no divergence

--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 1 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 1 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 2 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 2 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 2 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 3 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 3 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 3 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 4 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 4 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 4 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 5 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 5 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 5 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)

Evidence images

01-otlp-wire-ab-base-leaks-head-empty

02-outfile-and-bridge-ab

03-mutation-matrix-zero-survivors

04-sibling-tool-args-still-ungated

05-bridge-reachability-log-bridge-oracle

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@yiliang114

yiliang114 commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator Author

Resolve-comments scope ledger

Scope corrected: production scope stays at two files, +5/-2, identical to the first reviewed head. Existing review rounds expanded tests from 78 to 188 changed lines and added two documentation files. This audit removed 29 lines pinning exporter internals; the remaining tests cover generation, bridge filtering and real file export. 229 focused tests pass and current-head CI is green. Further policy coverage belongs to #11682; the final documentation-only commit 4db3f55 corrects the existing bridge-key inventory without changing behavior. Prettier and diff checks pass; the new-head CI snapshot is pending. Triage has been requested.

@yiliang114

Copy link
Copy Markdown
Collaborator Author

@qwen-code /triage

@wenshao

wenshao commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

Maintainer verification — real CLI × real telemetry sinks

I built a local end-to-end rig and ran 18 real qwen -p sessions against a real OTLP/HTTP receiver and a real telemetry.outfile, A/B'd against the merge-base. This targets exactly the leg the earlier review flagged as unverified ("runtime emission through a live OTLP or outfile sink").

Verdict: the fix does what it says on all three sinks, with no default-config change. Merge-ready. Two non-blocking observations and one follow-up suggestion below.

Verified at 76de1e23. Head has since moved to 4db3f55a; git diff 76de1e23 4db3f55a touches only two .md files, so every runtime result carries over. The three touched source files are byte-identical between the merge-base 78bbd9f5 and current main (3b2283ee), so the "before" arm faithfully represents main.

Harness

real `qwen -p` (dev entry, streaming path)
  → fake OpenAI-compatible provider on 127.0.0.1  (records every request body)
  → real OTel SDK → { telemetry.outfile | OTLP/HTTP logs+traces | OTLP/HTTP traces-only (bridge) }
  → real OTLP receiver on 127.0.0.1 (dumps decoded /v1/logs and /v1/traces payloads)
isolated QWEN_HOME per arm; marker text: SENSITIVE_REQUEST_MARKER / SENSITIVE_RESPONSE_MARKER

Results

sink matrix

arm sink logPrompts merge-base 78bbd9f5 PR head
A / B telemetry.outfile false request_text 6787 + 9507 chars, response_text = marker both keys absent
E / F OTLP/HTTP logs (native) false stringValue = full conversation JSON no content (key present, empty value — see 1.)
G / H OTLP/HTTP traces-only (bridge) false request_text on bridge span absent
I / J OTLP/HTTP traces-only (bridge) true request_text kept dropped (new denylist entry — see 2.)
K OTLP/HTTP traces-only (bridge) true + includeSensitive: true request_text + response_text + prompt kept, matches the new docs
C / D telemetry.outfile true content present content present, unchanged
P / O telemetry.outfile unset (default) 6787 / 9507 chars 6787 / 9507 chars — identical
R / Q telemetry.outfile env QWEN_TELEMETRY_LOG_PROMPTS=false leaks both keys absent

The bug reproduces on the merge-base for all three sinks, and is gone on the PR head for all three. The env-var override path (R/Q) works too. Default config is byte-identical (P vs O: same request_text lengths, same response_text, remaining diff is latency noise only).

What actually leaked pre-fix is worth seeing — it is not just the prompt: system reminders, the working directory, the folder listing, every prior turn, and the managed-memory subagent instruction block, 6.8 KB on the first request and 9.5 KB on the second, with logPrompts: false explicitly set.

OTLP wire bytes

Tests are load-bearing, and typecheck is clean

Reverting only the 7 production lines to the merge-base while keeping the PR's tests fails 3 of them; restoring the production lines makes all 229 pass. tsc --noEmit over packages/core produces the identical 12 pre-existing errors on both arms (@types/node Dirent drift on my box) — 0 new type errors.

counterfactual

Note the new file-exporters.test.ts case and the two logPrompts: true cases pass on the merge-base too — they are guardrails, not regression tests. That is fine and intentional; just don't read "4 new tests" as "4 tests that catch this bug".

I could not run ESLint locally (@eslint/eslintrc + ajv are broken in my environment, on main too), so I'm relying on the green Lint & Static check.

Two observations (neither blocks)

1. On the native OTLP log exporter the key still ships, with an empty value. undefined is a valid OTel log attribute value, so attributes['request_text'] = undefined survives into the exported record and otlp-transformer's toAnyValue(undefined) emits {}:

// arm F, PR head, logPrompts: false, received by the OTLP receiver
{"key":"request_text","value":{}}
{"key":"response_text","value":{}}

No content escapes (grep SENSITIVE_REQUEST_MARKER → 0 hits), and the outfile and bridge sinks both drop the key entirely, so this is hygiene, not a leak. But loggers.ts — the file where the fix below would go — already states the rule for exactly this shape (normalizeToolCallEvent, ~L168):

"Error fields are deleted (not set to undefined) on success so downstream consumers see key-absent rather than key-present-with-undefined."

A two-line follow-up in loggers.ts would make all three sinks agree:

// logApiRequest
if (event.request_text === undefined) delete attributes['request_text'];

and the same for response_text in logApiResponse — note the existing if (event.response_text) guard there cannot remove the key, because ...event already added it. Also worth a small wording tweak: the PR description says the fields "never reach the native OTLP log exporters", which is true of the content but not of the key. The docs text in the diff ("contains request content only when log_prompts_enabled is true") is already accurate.

2. Arms I → J are a deliberate behaviour change for existing traces-only users. With logPrompts: true and includeSensitiveSpanAttributes: false — a perfectly ordinary config — bridge spans carried request_text before this PR and do not after it. That is the correct call (it is now consistent with prompt and response_text) and it is documented in the diff, but it is a visible dashboard change for anyone on the HTTP traces-only path, so it may be worth a release-note line. Also note the denylist entry is only load-bearing in this combination: with logPrompts: false the producer gate already makes the bridge skip the attribute before the denylist is consulted (the processor drops undefined first).

Residual: same class of gap, outside this PR's scope

With logPrompts: false on the PR head, model-generated and user-supplied content still reaches telemetry through other attributes:

residual

  • qwen-code.subagent_executionresult carried verbatim model output ("result": "SENSITIVE_RESPONSE_MARKER") to the outfile and to the traces-only bridge (arms B and H) — it is not in SENSITIVE_ATTRIBUTE_KEYS.
  • qwen-code.tool_callfunction_args reaches the outfile and native log export (arms M/N). Unchanged by this PR; the bridge does deny it.
  • includeSensitiveSpanAttributes: true + logPrompts: false (arm L): gen_ai.input.messages and the interaction span's new_context still carry the full prompt. After this PR the bridge requires both flags for request_text while native span attributes require only their own opt-in, so those two sinks now disagree in that combination.

None of this blocks the PR — its stated scope is request_text / response_text, and it fully delivers that. But issue #11666 asks for "no request content to any sink", so a follow-up issue tracking these three (alongside the already-filed #11682 for thoughtSignature) would close the loop.

Coverage note

The real-CLI arms exercise the streaming path, which is what qwen -p uses (stream: true on every captured provider request). The non-streaming path shares the same single gate in logApiRequest / _logApiResponse and is covered by the PR's unit tests, which I also ran.

Reproduce
git worktree add head <PR head> && git worktree add --detach base 78bbd9f55c
# per arm: isolated QWEN_HOME with
#   {"security":{"auth":{"selectedType":"openai"}},
#    "telemetry":{"enabled":true,"logPrompts":false,
#                 "outfile":"…"                                  # sink 1
#                 /* or */ "otlpProtocol":"http","otlpEndpoint":"http://127.0.0.1:PORT"   # sink 2
#                 /* or */ "otlpProtocol":"http","otlpTracesEndpoint":"http://127.0.0.1:PORT/v1/traces","otlpEndpoint":""  # sink 3 = bridge
#   }}
OPENAI_BASE_URL=http://127.0.0.1:FAKE/v1 OPENAI_API_KEY=x OPENAI_MODEL=fake-model \
  node <arm>/scripts/dev.js --approval-mode yolo -p "Say the phrase SENSITIVE_REQUEST_MARKER and nothing else"
jq -c '..|.attributes?|arrays|.[]|select(.key=="request_text" or .key=="response_text")' otlp/logs.jsonl

Sink 3 (the bridge) needs a traces endpoint and no logs endpoint — set otlpTracesEndpoint and "otlpEndpoint": "", otherwise getTelemetryOtlpEndpoint() falls back to the default endpoint and you get the native log exporter instead.

中文版

维护者验证 —— 真实 CLI × 真实遥测 sink

我在本地搭建了端到端验证环境,用真实 OTLP/HTTP 接收端与真实 telemetry.outfile,跑了 18 次真实 qwen -p 会话,并与 merge-base 做 A/B。这正是之前 review 明确指出「未验证」的那一环("runtime emission through a live OTLP or outfile sink")。

结论:三个 sink 上修复都真实生效,默认配置行为无变化,可以合并。 下面是两点不阻塞的观察和一条后续建议。

验证基于 76de1e23。之后 head 变为 4db3f55agit diff 76de1e23 4db3f55a 只涉及两个 .md 文件,因此运行时结论完全适用。三个被改动的源文件在 merge-base 78bbd9f5 与当前 main3b2283ee)之间字节一致,所以 "before" 一侧忠实代表 main

验证环境

真实 qwen -p(dev 入口,走流式路径)
  → 127.0.0.1 上的假 OpenAI 兼容服务(记录每次请求体)
  → 真实 OTel SDK → { telemetry.outfile | OTLP/HTTP logs+traces | OTLP/HTTP 仅 traces(bridge) }
  → 127.0.0.1 上的真实 OTLP 接收端(落盘解码后的 /v1/logs、/v1/traces 负载)
每个 arm 使用独立 QWEN_HOME;标记文本:SENSITIVE_REQUEST_MARKER / SENSITIVE_RESPONSE_MARKER

结果

arm sink logPrompts merge-base 78bbd9f5 PR head
A / B telemetry.outfile false request_text 6787 + 9507 字符,response_text = marker 两个 key 均不存在
E / F OTLP/HTTP logs(原生) false stringValue = 完整会话 JSON 无内容(key 仍在,值为空 —— 见观察 1)
G / H OTLP/HTTP 仅 traces(bridge) false bridge span 上带 request_text 不存在
I / J OTLP/HTTP 仅 traces(bridge) true 保留 request_text 被丢弃(新增黑名单项 —— 见观察 2)
K OTLP/HTTP 仅 traces(bridge) true + includeSensitive: true request_text/response_text/prompt 均保留,与新文档一致
C / D telemetry.outfile true 内容存在 内容存在,无变化
P / O telemetry.outfile 未设置(默认) 6787 / 9507 字符 6787 / 9507 字符 —— 完全一致
R / Q telemetry.outfile 环境变量 QWEN_TELEMETRY_LOG_PROMPTS=false 泄漏 两个 key 均不存在

三个 sink 在 merge-base 上都能复现该缺陷,在 PR head 上都已修复;环境变量覆盖路径(R/Q)同样生效;默认配置逐字节一致(P vs O:request_text 长度相同、response_text 相同,差异仅剩耗时噪声)。

修复前实际泄漏的内容值得一看,远不止用户 prompt:system reminder、当前工作目录、目录结构、此前每一轮对话,以及 managed-memory 子代理的完整指令块 —— 在显式设置 logPrompts: false 的情况下,第一次请求 6.8 KB、第二次 9.5 KB。

测试确实「吃劲」,类型检查干净

只把 7 行生产代码回退到 merge-base、保留 PR 的测试,会有 3 个用例失败;恢复生产代码后 229 个用例全绿。packages/coretsc --noEmit 在两侧产生完全相同的 12 个既有错误(我这台机器上 @types/nodeDirent 漂移)—— 0 个新增类型错误

需要说明:新增的 file-exporters.test.ts 用例与两个 logPrompts: true 用例在 merge-base 上同样通过,它们是护栏而非回归测试。这没有问题,只是不要把「4 个新测试」理解成「4 个能抓住这个 bug 的测试」。

我这里跑不了 ESLint(@eslint/eslintrc + ajv 在本机损坏,main 上同样如此),因此这一项依赖 CI 的 Lint & Static 绿灯。

两点观察(都不阻塞)

1. 在原生 OTLP 日志导出上,key 仍然会发出,只是值为空。 undefined 是合法的 OTel 日志属性值,因此 attributes['request_text'] = undefined 会一路进入导出记录,otlp-transformertoAnyValue(undefined) 产出 {}

// arm F,PR head,logPrompts: false,OTLP 接收端收到的内容
{"key":"request_text","value":{}}
{"key":"response_text","value":{}}

没有任何内容外泄(grep SENSITIVE_REQUEST_MARKER → 0 命中),outfile 与 bridge 两个 sink 也都完全丢弃了该 key,所以这属于整洁性问题而非泄漏。但 loggers.ts(也就是下面这个建议改动所在的文件)本身已经为这一形态写明了约定(normalizeToolCallEvent,约 L168):

"Error fields are deleted (not set to undefined) on success so downstream consumers see key-absent rather than key-present-with-undefined."

loggers.ts 里加两行即可让三个 sink 行为一致:

// logApiRequest
if (event.request_text === undefined) delete attributes['request_text'];

logApiResponse 中的 response_text 同理 —— 注意那里现有的 if (event.response_text) 判断无法移除该 key,因为 ...event 展开时已经把它加进去了。另外 PR 描述里「never reach the native OTLP log exporters」的措辞可以微调:对内容成立,对 key 不成立;diff 中的文档表述("contains request content only when log_prompts_enabled is true")本身是准确的。

2. arm I → J 对现有 traces-only 用户是一次有意的行为变更。logPrompts: true + includeSensitiveSpanAttributes: false 这一相当常见的配置下,bridge span 此前带 request_text,此后不再带。这个取舍是对的(与 promptresponse_text 保持一致),diff 里也写进了文档,但对走 HTTP traces-only 的用户来说是看板上的可见变化,值得在 release note 里提一句。另外,这条黑名单项只在这一组合下真正起作用:当 logPrompts: false 时,生产者侧的门禁已经让 bridge 在查黑名单之前就跳过了该属性(processor 先过滤 undefined)。

残留:同类问题,但不在本 PR 范围内

在 PR head 上、logPrompts: false 时,模型生成内容与用户输入仍会经由其他属性进入遥测:

  • qwen-code.subagent_executionresult 原样携带模型输出("result": "SENSITIVE_RESPONSE_MARKER"),同时进入 outfile traces-only bridge(arm B 与 H)—— 它不在 SENSITIVE_ATTRIBUTE_KEYS 中。
  • qwen-code.tool_callfunction_args 进入 outfile 与原生日志导出(arm M/N)。本 PR 未改动此处;bridge 确实会拦截它。
  • includeSensitiveSpanAttributes: true + logPrompts: false(arm L):gen_ai.input.messagesinteraction span 的 new_context 仍携带完整 prompt。本 PR 之后,bridge 上的 request_text 需要两个开关同时打开,而原生 span 属性只需自己的开关,因此这两个 sink 在该组合下出现了不一致。

这些都不阻塞本 PR —— 它声明的范围就是 request_text / response_text,并且完全兑现了。但 #11666 的目标是「任何 sink 都不导出请求内容」,因此建议再开一个 follow-up issue 跟踪上述三项(与已存在的 #11682 thoughtSignature 一并收口)。

覆盖范围说明

真实 CLI 的所有 arm 走的是流式路径,也就是 qwen -p 实际使用的路径(抓到的每个 provider 请求都是 stream: true)。非流式路径与之共用 logApiRequest / _logApiResponse 里的同一处门禁,并由本 PR 的单元测试覆盖 —— 这些单测我也一并跑过。

yiliang114 and others added 2 commits September 12, 2026 15:55
Add a regression test asserting the thoughtSignature provider payload
survives toContents serialization into request_text.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmty28f6zzp
…t_text"

R5-1 is declined and deferred to #11682, which owns the enabled-policy
thoughtSignature decision. This PR only changes the logPrompts=false
privacy gate; a test that freezes the opposite policy widens scope.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmty28f6zzp
@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 189 passed · 0 failed · 189 total

Flakiness gate: ✅ 3 changed test file(s) x 5 identical rounds, no divergence

中文 — 判定:✅ 通过 · 可合入(agent 判定)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:189 通过 · 0 失败 · 189 总计

抖动门:✅ 3 changed test file(s) x 5 identical rounds, no divergence

Verification report

PR #11670 — fix(telemetry): gate request_text/response_text on logPrompts

Verdict: merge-ready — 189/189 scripted assertions passed, 0 failed (assertions.json). Verified head: 34c8eb1cc2373550d5681e9feb48f8b0dec8230f (git rev-parse HEAD^2). Base of the A/B: 3b2283ee0d3b87592c15269feac7603169c2a1d9 (HEAD^1). Diff: 7 files, +200/−10 — of which 7 changed production lines (4+/2− in loggingContentGenerator.ts, 1+ in log-to-span-processor.ts); the rest is tests and docs.

This is a follow-up round. The previous round verified head 76de1e2398; the snapshot now lists 11 commits, so the delta is commits 9–11: 4db3f55a (docs: list error attributes retained by the bridge), ac00d3ee (test: pin opaque thoughtSignature survival in request_text), 34c8eb1c (revert of ac00d3ee, declining review finding R5-1 and deferring it to #11682). Every carried-forward measurement below was re-run at the new head, not diffed from the old report.

Two facts bound the delta cheaply, and both are measured rather than assumed. The compiled head modules are byte-identical to the previous round's (d5222129… / 90937df9…), and the compiled base modules are byte-identical too (69d2f61c… / e3973cca…) even though main moved (05a54fc33b2283ee0d, which landed an auto-retry feature in core). So commits 9–11 changed no production code, main did not touch either changed file between the two bases, and commit 10's test is a clean net-zero against commit 11's revert — thoughtSignature survives in the final diff only inside one docs sentence. The full 14-cell A/B was re-run anyway, because the rest of the CLI build did change under the harness.

The central claim is proven load-bearing on three real telemetry destinations at the new head. The sharpest number: with telemetry.logPrompts: false, base ships 16,282 characters of serialized conversation as request_text on the outfile leg and 16,373 on the bridge leg; head ships none (02-ab-side-by-side-head-vs-base.png). No finding below is a regression introduced by this PR, and none blocks it.

中文摘要
  • 结论: merge-ready。189/189 条脚本化断言全部通过,0 失败。验证的 head 为 34c8eb1c(快照中第 11 个 commit),A/B 的 base 为 3b2283ee0dHEAD^1)。
  • 本轮是复验轮。上一轮验证到 76de1e2398(第 8 个 commit)。增量(第 9–11 个 commit)为:一条文档改动(bridge 保留的 error 属性枚举)、一个新增测试、以及对该测试的 revert(作者以此婉拒评审意见 R5-1,转交 Track thoughtSignature/encrypted_content telemetry-export policy decision (#11666 second clause) #11682)。所有承接的测量都在新 head 上重新执行。增量未改动任何生产代码:head 侧编译产物与上一轮逐字节相同(d5222129…/90937df9…),base 侧也相同(69d2f61c…/e3973cca…),尽管 main 已从 05a54fc3 前进到 3b2283ee0d。commit 10 与 commit 11 相互抵消,thoughtSignature 在最终 diff 中只剩一句文档。14 格 A/B 仍完整重跑。
  • A/B 结论(核心主张成立): 见 "Central claim + A/B" 三张表与证据图 01-…02-…logPrompts: false 时 base 在 outfile 与 bridge 上分别发出 16,282 / 16,373 字符的 request_text,head 一个字符都不发;logPrompts: true 时两臂载荷完全一致(无回归)。
  • 增量唯一的新主张已被行为验证,不是读文档: commit 9 声称 bridge 保留 function_argserrorerror.messageerror_message,外加 logPrompts 开启时的 prompt/request_text/response_text。用真实编译产物驱动 LogToSpanProcessor行为推导出的敏感键集合与文档枚举双向吻合(完整性 + 正确性各一条断言),且 error_type/model/duration_ms 等对照键未被误判。
  • Mutation 矩阵: 3 个 guard 全部被各自断言杀死,组合行 M4 = M1+M3 的并集(3 = 2+1,两个 guard 相互独立、无层叠隐藏),两个同文件正对照 M5/M6 均变红,vacuity 检查 V1 命中新测试,0 个幸存者;每行都验证过工作树已还原干净。见 06-mutation-matrix-zero-survivors.png
  • 被婉拒的 R5-1 重新测过,且我同意婉拒: 不透明 thoughtSignature 载荷确实原样进入 request_text(实测出现 2 次)。但这是 logPrompts: true 下的策略问题,本 PR 只改 logPrompts: false 的隐私门;且文档已如实披露该行为,属于"记录在案"而非"被测试冻结"。
  • 上一轮 findings 状态: 全部 stands;其中第 4 条被测得更准(见 Corrections 与状态表),第 1、2 条在两臂上逐字节相同,确认为既有问题。
  • 未覆盖范围: 逐 commit 归因(shallow clone,快照 11 个 commit 中本地仅 1 个可达);与当前 main 的 trial merge(快照的 baseRefOid 本地不存在且无网络);#11682 引用本身;gRPC OTLP 路径;全仓测试套件与 packages/cli--resume 端到端回放;交互式 TUI;Windows/macOS。

Previous-finding status at the new head

Re-measured, not diffed. "Stands" means the behaviour was reproduced at 34c8eb1c by a fresh run of the same probe.

# previous finding severity status at 34c8eb1c
1 tool_call.function_args still exports verbatim tool arguments when logPrompts: false Suggestion (pre-existing, scope) stands — re-measured on both arms with a real tool call: function_args is byte-identical on base and head, carrying {"command": "printf '%s\\n' 'SENSITIVE_TOOL_OUTPUT_MARKER'", "description": "SENSITIVE_TOOL_ARG_MARKER"} in the same run where head's api_request has no request_text key (04-sibling-tool-args-still-ungated.png)
2 subagent_execution.result still exports model output text when logPrompts: false Suggestion (pre-existing, scope) stands — the residual-carrier scan reports exactly qwen-code.subagent_execution:result on every head lpOFF cell, with result = "SENSITIVE_RESPONSE_MARKER_<cell>" verbatim, and the same carrier on the matching base cell
3 native OTLP/HTTP keeps the attribute key with an empty AnyValue, unpinned Informational stands, still unpinned — measured at head lpOFF: request_text present=2 empty=2 withContent=0, response_text present=2 empty=2 withContent=0. The outfile leg drops the key entirely (that is what file-exporters.test.ts pins)
4 traces-only bridge unreachable unless otlpEndpoint is explicitly cleared Informational (pre-existing) stands, now measured more precisely — documented traces-only shape: 0 bridge spans on the configured endpoint, 5 native spans; and the log records are not lost in-process, they are delivered to the unconfigured default http://localhost:4317 (14 records observed by a receiver bound there). Clearing otlpEndpoint: 14 bridge spans of 19. See Corrections for why this refines the previous round's wording
5 responseText still computed then discarded when logPrompts: false Minor, non-blocking stands — head production modules byte-identical to the previous round, so unchanged by construction
C1 Correction: PR body's 223 passed (3 files) is stale Correction stands — measured 225 passed (3 files) again (loggingContentGenerator 83, log-to-span-processor 55, loggers 87)
C2 Correction: log_prompts_enabled is not a real attribute Correction stands, unchanged — 0 hits in packages/**/*.ts, 3 hits in telemetry.md (lines 621, 646, 649), same count as the previous round
R5-1 Declined by the author this round: opaque thoughtSignature payload exported as request_text even when logPrompts: true declined → #11682 stands as behaviour; I agree with the decline — measured through the real compiled toContents/toPart: the signature appears verbatim in the serialized request_text. The decline rationale ("this PR only changes the logPrompts=false privacy gate; a test that freezes the opposite policy widens scope") holds, and commit 9's docs sentence now discloses the behaviour, so it is documented rather than silently pinned

Central claim + A/B

Central claim. When telemetry.logPrompts is false, conversation content no longer reaches any telemetry destination via api_request.request_text / api_response.response_text.
Secondary claims. (a) request_text joins the bridge's sensitive-attribute denylist, so the traces-only bridge strips it when prompt logging is on but sensitive span attributes are off, and keeps it when they are on. (b) No in-tree consumer depends on the two fields.

Each cell is one real node packages/cli/dist/index.js -p … process (real Config, real telemetry SDK, real exporters) against a real loopback OpenAI-compatible model server, driven through the documented settings path (telemetry.logPrompts in settings.json, not the deprecated flag). Cell HOME/QWEN_HOME/cwd are isolated outside the repo so no project-level .qwen can leak in. The only difference between arms is the two compiled modules the PR changes. Base-arm cells are expected to leak, so a leaking base cell is a pass (encoded as expectLeak: true). Full witness: 01-ab-matrix-14-cells-base-leaks-head-clean.png (119 assertions as printed); side-by-side: 02-ab-side-by-side-head-vs-base.png.

Destination 1 — telemetry.outfile (FileLogExporter)

cell arm logPrompts oracle (what the file actually contains) result
OF-base-lpOFF base false request_text PRESENT, lengths 6,772 and 9,510 chars; response_text present LEAKS (control)
OF-head-lpOFF head false both keys ABSENT clean
OF-base-lpON / OF-head-lpON both true both present, lengths 6,769 / 9,500 on both arms, carrier sets identical no regression

Δ at logPrompts: false = −16,282 characters. That delta is the removed conversation content.

Destination 2 — native OTLP/HTTP /v1/logs (real loopback socket)

cell arm logPrompts request_text on the wire response_text result
OT-base-lpOFF base false 2 present, 2 with content, lengths 6,772 / 9,510 2 with content LEAKS (control)
OT-head-lpOFF head false 2 present, 2 EMPTY AnyValue, 0 with content 2 present, 2 empty, 0 with content clean
OT-base-lpON / OT-head-lpON both true 2 with content, lengths 6,769 / 9,500 on both arms identical no regression

Oracles judge values, not key substrings: AnyValues are unwrapped and an empty one is distinguished from an absent key — exactly the false positive a substring oracle produces (finding 3).

Destination 3 — traces-only log-to-span bridge

Oracle is the log.bridge attribute the processor itself sets (log-to-span-processor.ts:191), so bridge spans cannot be confused with native trace spans. Every cell asserts validity controls (bridge spans reached the wire = 14, bridge produced api_request spans = 2) on both arms, so "absent" cannot pass vacuously.

cell arm logPrompts sensitive api_request bridge spans carrying request_text result
BR-base-lpON-sensOFF base true false 2 of 2 LEAKS (control) — the denylist gap this PR closes
BR-head-lpON-sensOFF head true false 0 of 2 stripped — secondary claim (a) proven
BR-base-lpON-sensON base true true 2 of 2 (6,790 / 9,570 chars) retained
BR-head-lpON-sensON head true true 2 of 2, identical lengths and carrier set retained — docs claim holds
BR-base-lpOFF-sensON base false true 2 of 2 (6,793 / 9,580) LEAKS (control) — base got zero protection from logPrompts: false here
BR-head-lpOFF-sensON head false true 0 of 2 clean (producer gate)

The last pair is the cleanest isolation of the producer gate: includeSensitiveSpanAttributes is true on both arms, so the denylist cannot explain the difference — only the new gate can. Δ = −16,373 characters.

Secondary claim (b), census

Re-run at the new head: every property access of the two fields across packages/*/src resolves to a producer (types.ts), the OTLP attribute writer (loggers.ts), or the bridge denylist (log-to-span-processor.ts:64). No non-test reader. The persisted ui_telemetry mirror is the one path with on-disk readers, and the only reader touching uiEvent reads prompt_id alone and spreads the rest through unchanged — so nothing reads response_text off disk. Claim (b) holds; this remains a census, not an end-to-end --resume run (see Not covered).

The delta since the previous round (commits 9–11)

New probes were scoped to what actually changed: one docs sentence and one test that was added then reverted.

Commit 9's docs claim, tested rather than read. The bridge bullet now enumerates the retained set as function_args, error, error.message, error_message, plus prompt, request_text, response_text when logPrompts is also enabled. That is a completeness claim about SENSITIVE_ATTRIBUTE_KEYS, so it was checked behaviourally against the real compiled LogToSpanProcessor, not by reading the Set literal: a log record carrying all 7 candidate keys plus 4 controls was driven through at includeSensitiveSpanAttributes: false and true (03-delta-docs-census-and-thoughtsignature.png).

  • sensitive OFF → dropped exactly ["error","error.message","error_message","function_args","prompt","request_text","response_text"]; kept exactly ["error_type","model","duration_ms","safe_control"].
  • sensitive ON → all 11 retained verbatim (11/11), so nothing is lost on the accept path.
  • The docs enumeration and the behaviourally-derived set are equal, and the census runs both directions: no key the bridge drops is missing from the docs list (completeness), and no docs-listed key is actually retained when sensitive is off (soundness). Commit 9's sentence is accurate and complete.
  • The logPrompts qualifier is also accurate, and the reason is worth stating because it is not in the processor: the bridge filter is key-only and has no logPrompts awareness, so prompt/request_text/response_text disappear at logPrompts: false purely because the producer omits them. The matrix shows this directly — qwen-code.user_prompt:prompt is a residual carrier in every lpON cell and in no lpOFF cell, while function_args is present at lpOFF (finding 1). The sentence correctly places function_args outside the logPrompts condition.

Commits 10 + 11 are a clean net-zero. thoughtSignature appears once in the final diff, inside commit 9's docs sentence; no test file carries it. Confirmed by the byte-identical head modules and by the mutation matrix, which still has 0 survivors — so the revert removed a test that was pinning behaviour nothing else needed pinned, and lost no guard coverage.

The declined R5-1, re-measured. toPart is a pass-through for non-thought parts and a spread ({...part}) for thought parts, so every own enumerable property survives — including an opaque thoughtSignature. Driving the real compiled toContents (the methods are stateless, so a prototype-created instance exercises real code with no Config) with two parts carrying the signature yields a request_text serialization containing it 2×, verbatim. The behaviour stands exactly as the declined finding said.

I agree with the decline, for a reason the revert message does not state: the PR's docs sentence now discloses it ("opaque thoughtSignature provider payload is included, with its policy decision tracked in #11682"). A passing test would freeze a policy the author is explicitly not deciding here, whereas the docs make the behaviour discoverable and #11682 owns the decision. That is the better of the two outcomes for a PR whose scope is the logPrompts: false gate. Note the boundary: at logPrompts: false this payload does not ship — the gate removes it, which is the third delta assertion.

Carried nit (unchanged, non-blocking). file-exporters.test.ts's FileLogExporter omits undefined-valued attributes (logPrompts off) still names a logPrompts scenario while its fixture is a hand-built ReadableLogRecord with request_text: undefined — no Config, no LoggingContentGenerator. The comment above it is honest about this and the assertion is genuinely load-bearing (V1 kills it), so the cost is only that the name buys slightly more confidence than the fixture pays for.

Mutation matrix

One row per guard, a combination row for layered defence, two positive controls landed in the same file as the mutant, and the vacuity check on the PR's newest test file. Every red is the intended behavioural assertion with an expected-vs-actual message — no red came from a broken import or fixture. Ground truth is vitest's own exit code, cross-checked against parsed counts (countsAgree=true on all 8 rows). Witness: 06-mutation-matrix-zero-survivors.png.

id what was reverted / perturbed suite red quoted failure
M0 (none — unmutated control) all 3 files 0 / 142 green, as required
M1 guard #1: request_text producer gate lcg 2 / 83 expected '[{"role":"user","parts":[{"text":"SEN…' to be undefined
M2 guard #2: response_text producer gate lcg 2 / 83 expected 'SENSITIVE_RESPONSE_MARKER' to be undefined
M3 guard #3: 'request_text' denylist entry l2s 1 / 55 expected { Object (request_text, error_type, …) } to not have property "request_text"
M4 combination M1 + M3 together both 3 / 138 union of M1 (2) and M3 (1) — the guards are independent, no layered defence hiding either
M5 positive control, same file as M3: drop the pre-existing 'response_text' entry l2s 1 / 55 expected { …(4) } to not have property "response_text"
M6 positive control, same file as M1/M2: MAX_RESPONSE_TEXT_LENGTH 4096 → 4095 lcg 1 / 83 expected 'xxx…' to have a length of 4096 but got 4095
V1 vacuity check: safeJsonStringify replacer returns null for undefined fx 1 / 4 expected '{\n "body": "api response",…' not to contain 'request_text'

Survivors: 0. Both positive controls went red, so the harness can make these suites fail; the unmutated control is green (142), so the kills mean something. M1's message reproduces the PR body's own claimed "Before" evidence verbatim. git status --porcelain verified empty after every row (asserted per row, not just at the end), and packages/core/dist left at head (d5222129… / 90937df9…).

Corrections

  • Refinement of the previous round's finding-4 wording, in the direction of more precise, not worse. The previous round reported that in the documented traces-only configuration the operator "loses the logs", with 0 log records delivered to any reachable receiver. Measured with a receiver bound to the default OTLP port, the mechanism is narrower and more useful: the 14 log records are produced and are delivered — to http://localhost:4317, the unconfigured default that getTelemetryOtlpEndpoint() falls back to. Nothing was emitted in-process and dropped; the records were routed to an endpoint the operator never named. In production, where nothing listens on 4317, the observable outcome is the same silence the previous round described, so its conclusion stands — but the cause is routing, not production, and that distinction matters to whoever fixes it: the fix is the logsUrl/tracesUrl branch condition in createHttpExporters, not anything in the logging path. Still pre-existing, still not caused by this PR.
  • The PR body's test count is stale (carried forward). The body reports # 223 passed (3 files); measured at the verified head: 225 passed (3 files). Description accuracy only — the tests pass.
  • log_prompts_enabled is not an attribute (carried forward, count unchanged). Commit 9's docs wording again writes only when `log_prompts_enabled` is true. No attribute or setting by that name exists anywhere in packages/** (0 TypeScript hits); it appears only in telemetry.md, 3 times. This is a pre-existing docs convention — already used for prompt at line 621 — that the PR extends consistently, so matching it is defensible. Flagged so the next reader does not grep for a field that does not exist.
  • Confirming, not correcting: the body's claim that the two fields "never reach the native OTLP log exporters, the traces-only log-to-span bridge, or telemetry.outfile" is true on all three destinations at the new head, and the no-internal-consumer claim holds.

Findings

Ordered by severity. None is a regression introduced by this PR; none blocks it. Findings 1, 2, 4 and R5-1 are re-measurements of carried-forward or newly-declined items and are tabulated above; the substance is restated here only where it changed.

1. Suggestion (pre-existing, scope): logPrompts: false still exports verbatim tool arguments via tool_call.function_args

Same bug class, adjacent door — and the door the PR's own motivation names ("tool arguments/results, file content"). At head with logPrompts: false, in a single run where api_request correctly has no request_text key:

tool_call record: function_name="run_shell_command"
  function_args="{\n  \"command\": \"printf '%s\\n' 'SENSITIVE_TOOL_OUTPUT_MARKER'\",\n  \"description\": \"SENSITIVE_TOOL_ARG_MARKER\"\n}"

The base arm produced a byte-identical function_args, so this is pre-existing and not attributable to the PR. Cause: logToolCall spreads the normalized event and sets function_args: safeJsonStringify(…) with no gate; shouldLogUserPrompts has exactly one call site. function_args is in the bridge denylist, so only the native-log and outfile legs carry it. Repro: PROBE_SET=siblings node tmp/pr11670-verify-20260912-082150/harness.mjs, witness 04-sibling-tool-args-still-ungated.png. Worth naming for whoever closes #11666, since the issue's stated symptom survives through a different key.

2. Suggestion (pre-existing, scope): subagent_execution.result still exports model output text

The residual-carrier scan reports, for every head logPrompts: false cell, exactly one surviving carrier of the model's response marker: qwen-code.subagent_execution:result, whose value is the marker verbatim ("SENSITIVE_RESPONSE_MARKER_OF-head-lpOFF"). The same carrier appears on the matching base cell, asserted per cell, so it is not introduced here. On the bridge leg with includeSensitiveSpanAttributes: true the native GenAI span attributes join it. Cause: logSubagentExecution has no logPrompts gate and result is not in SENSITIVE_ATTRIBUTE_KEYS.

3. Informational: the native OTLP leg keeps an empty-valued key, and nothing pins it

At head with logPrompts: false: request_text present=2 empty=2 withContent=0, response_text present=2 empty=2 withContent=0 — i.e. {"key":"request_text","value":{}}. No content crosses (the carrier scan finds no marker in any api_request/api_response value), and this matches how every undefined optional attribute already serializes. The outfile sink drops the key entirely, which is what file-exporters.test.ts pins. Practical consequence only: a wire audit grepping for the key name still sees it. Unpinned by design since 76de1e23, and I agree with that choice — pinning it would freeze an OTLP serialization accident.

4. Informational (pre-existing): in the documented traces-only configuration, log telemetry is routed to an endpoint the operator never configured

settings bridge spans (log.bridge) native spans on the configured endpoint log records at the default :4317
only otlpTracesEndpoint set 0 5 14 (incl. api_request)
same + otlpEndpoint: '' 14 of 19 5 n/a (bridged)

Cause, read from source and confirmed by the table: getTelemetryOtlpEndpoint() returns telemetrySettings.otlpEndpoint ?? DEFAULT_OTLP_ENDPOINT ('http://localhost:4317'), so logsUrl stays truthy and the bridge is never constructed — it lives only in the else if (tracesUrl) branch of createHttpExporters. An operator following the docs ("used when HTTP traces are exported without a logs endpoint") who sets only otlpTracesEndpoint gets log records aimed at a default they never named. Pre-existing; it bounds how much of this PR's second claim is reachable in practice, since the bridge — and therefore the denylist hunk — does not exist on that path. Repro: PROBE_SET=reach node …/harness.mjs, witness 05-bridge-reachability-default-endpoint-oracle.png.

Minor observation (non-blocking, unchanged)

responseText is still computed (extractResponseText, capped at 4096) and the streaming path still consolidates responses before the value is discarded by the gate. Gating one step earlier would skip that work when logPrompts: false; bounded memory either way.

Not covered

  • Per-commit attribution. Shallow clone (git rev-parse --is-shallow-repositorytrue): git rev-list HEAD^1..HEAD^2 returns 1 commit while the metadata snapshot lists 11 (22aec43a, d04bb9e9, d04fb917, 7f9f68b0, 8fb0f8d9, 3e2fd8e3, b51e0af0, 76de1e23, 4db3f55a, ac00d3ee, 34c8eb1c). The previous round's head 76de1e23 and the reverted ac00d3ee are both unreachable locally, so commit 10's test content could not be read — its net-zero effect was established from the final diff and the byte-identical modules instead. Verified the aggregate HEAD^1..HEAD diff; no per-commit table is presented.
  • Trial merge into current main. The snapshot's baseRefOid (78bbd9f5…) is not present locally and differs from the merge ref's base (HEAD^1 = 3b2283ee0d…), so main has moved since the merge ref was created. No network/token in this job → could not fetch, could not confirm a conflict-free merge, could not re-run on merged main. Mitigating evidence: the base-side compiled modules are byte-identical to the previous round's, i.e. main did not touch either changed file between 05a54fc3 and 3b2283ee0d.
  • #11682, referenced by commit 9's docs wording for the thoughtSignature policy decision: an issue number, unverifiable offline. The thoughtSignature serialization claim was verified behaviourally; the tracking issue's existence and content were not.
  • gRPC OTLP path (otlpProtocol: 'grpc'): not exercised; the bridge does not exist on that path.
  • Repo-wide test suite: not run. Targeted gates only — the 3 PR-named files (225), file-exporters.test.ts (4), the whole src/telemetry suite (1002 passed / 30 files), tsc --noEmit for packages/core, prettier and eslint on the 7 changed files. packages/cli was not run.
  • --resume end-to-end replay of the persisted ui_telemetry mirror: closed by census only. No resume was driven.
  • Interactive TUI mode: all cells are headless (-p). The gates sit in a shared producer, so mode should not matter, but it was not measured.
  • The thoughtSignature / encrypted_content policy question itself — whether that payload should ship when prompt logging is enabled — is explicitly out of scope for this PR and was not evaluated as a policy matter. Only the behaviour and the accuracy of the docs sentence describing it were measured.
  • Windows/macOS: Linux container only.
  • Ten superseded harness failures, kept as raw evidence and excluded from assertions.json. All ten were my harness's fault, not the PR's, and each was fixed and re-run clean: (a) 2 from the outfile oracle keying the event name off the record body instead of attributes["event.name"], which the api_request records produced validity control caught; (b) 1 from the docs-census regex sweeping up `logPrompts` — a setting name in that sentence, not an attribute key; (c) 7 from two mutation-harness bugs: git checkout HEAD -- <rel> used paths relative to packages/core from the repo root, so restores silently failed and mutations stacked (the run aborted, the tree was cleaned and its source hashes re-verified against the pre-mutation values before re-running), and vitest summary counts were parsed without stripping ANSI, reporting failed=0 for rows that had actually gone red. Ground truth for the matrix is now vitest's exit code, cross-checked by a counts-agree assertion. Exclusions and reasons are recorded in assertions-detail.json; only the final runs feed the counts.

Methodology

Environment: the CI verify container (node:22-bookworm, node v22.23.2, $RUNNER_TEMP unset in-shell), merge-ref checkout at depth 2, npm ci + npm run build already done at head. Artifact dir tmp/pr11670-verify-20260912-082150/ holds every harness (harness.mjs, model-server.mjs, otlp-receiver.mjs, delta-probes.mjs, mutations.mjs, summary-table.mjs, build-assertions.mjs, gates.sh), raw logs (matrix.log, delta-probes.log, mutation.log, mutation-summary.txt, mutation-raw/*.log, gates.log, summary-table.log, build-base.log, gates-stdout.log), per-cell JSON and raw sinks (cells/<id>/{cell.json,outfile.json,otlp.jsonl,otlp-default-4317.jsonl,cli.out,cli.err}), machine-readable results (matrix-results{,-main,-siblings,-reach}.json, delta-probes.json, mutation-rows.json, assertions.json, assertions-detail.json, provenance.txt) and evidence/*.png.

Control construction. Internal workspace links were asserted before trusting any control: readlink -f node_modules/@qwen-code/qwen-code-core/__w/qwen-code/qwen-code/packages/core, i.e. the head tree — so a second worktree would have silently loaded head code. Base sources were therefore taken with git show HEAD^1:<path> for exactly the two changed production files and compiled with the real npm run build -w packages/core; the arm switch swaps the two compiled .js modules in place, and every cell re-asserts the landed arm two ways — by grepping the compiled bytes for the gate and the denylist entry (gateCount, denylisted) and by recording both sha256s in the cell JSON before spawning. The head rebuild reproduced the CI-built head bytes exactly (d5222129… / 90937df9…), so swapping modules is an exact arm switch. packages/cli/dist/index.js is a 245-byte shim, not a bundle, which is what makes the swap observable at all. The PR leaves package.json/package-lock.json untouched, so reusing the root node_modules for both arms is a clean control.

How the harnesses drove the code. Each cell spawns the real CLI with an isolated HOME/QWEN_HOME/cwd outside the repo against a loopback OpenAI-compatible server returning per-cell marker content (and, in the sibling probe, a tool call whose arguments carry a distinct marker, emitted on the first request only so the CLI does not loop). Destinations are real: telemetry.outfile, a loopback OTLP/HTTP receiver for /v1/logs, a loopback traces receiver feeding the bridge, and — for the reachability probe — a second receiver bound to the default port 4317 so the fate of diverted records is observed, not inferred. Bridge cells use a 6 s model-side delay to outlive the bridge's unref'd flush tick and assert on the processor's own log.bridge marker. The outfile is safeJsonStringify(data, 2) — pretty-printed concatenated objects, not JSONL — so it is parsed with a string-aware brace scanner. Universal validity controls (CLI exited 0, model was actually called, api_request records/spans were produced, bridge spans reached the wire) run on both arms, so no "absent" result can pass vacuously — and in one case did exactly its job, catching my own broken oracle before it could produce a confident negative. The delta probes import the real compiled LogToSpanProcessor and LoggingContentGenerator from dist/ with only a capturing span exporter and a prototype-created instance as collaborators, so the units under test are never stubbed.

Counts. assertions.json is generated by build-assertions.mjs from the result files, never hand-counted: main matrix 119, sibling probe 20, reachability probe 14, delta probes 16, mutation rows 8 + final tree clean 1, gates 11 → 189 pass / 0 fail. Gate liveness was proven by planting a formatting break (prettier exit 1), a lint violation (no-explicit-any, eslint exit 1) and a type error (TS2322, tsc exit 2), then restoring; the unit-test gate's liveness is the mutation matrix itself (M5/M6 land in the mutated files). Working tree verified clean after every mutation row and after all gates; packages/core/dist left at head; scratch run dirs and the smoke cell removed; git worktree list shows only the main checkout.

Flakiness gate log

rounds=5 files=3 skipped=0
file packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/loggingContentGenerator/loggingContentGenerator.test.ts
file packages/core/src/telemetry/file-exporters.test.ts: (cd packages/core) npx --no-install vitest run ./src/telemetry/file-exporters.test.ts
file packages/core/src/telemetry/log-to-span-processor.test.ts: (cd packages/core) npx --no-install vitest run ./src/telemetry/log-to-span-processor.test.ts


per-file results (P=pass F=fail I=infra-exit, one letter per run):
  packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: PPPPP
  packages/core/src/telemetry/file-exporters.test.ts: PPPPP
  packages/core/src/telemetry/log-to-span-processor.test.ts: PPPPP

verdict: pass
summary: 3 changed test file(s) x 5 identical rounds, no divergence

--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 1 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 1 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 2 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 2 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 2 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 3 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 3 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 3 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 4 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 4 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 4 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)
round 5 · packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts: P (exit 0)
round 5 · packages/core/src/telemetry/file-exporters.test.ts: P (exit 0)
round 5 · packages/core/src/telemetry/log-to-span-processor.test.ts: P (exit 0)

Evidence images

01-ab-matrix-14-cells-base-leaks-head-clean

02-ab-side-by-side-head-vs-base

03-delta-docs-census-and-thoughtsignature

04-sibling-tool-args-still-ungated

05-bridge-reachability-default-endpoint-oracle

06-mutation-matrix-zero-survivors

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Triage re-run completed without a new review.

⚠️ The bot has neither a verdict nor a deferral on 34c8eb1cc2373550d5681e9feb48f8b0dec8230f — no APPROVED, CHANGES_REQUESTED, or COMMENTED review of its own. A DISMISSED one does not count: dismiss_stale_reviews voids the bot's approval on every push, which is exactly when a fresh one is needed. If this re-run was meant to review or approve, it did not, and an approval left by another account is a separate vote that does not count as the bot's own.

⚠️ 机器人在 34c8eb1cc2373550d5681e9feb48f8b0dec8230f既没有裁决也没有 defer —— 没有属于它自己的 APPROVEDCHANGES_REQUESTEDCOMMENTED 评审。DISMISSED 不算:dismiss_stale_reviews 会在每次推送时作废机器人的批准,而那恰恰是需要一次新批准的时刻。如果这次重跑本应评审或批准,那么它没有做到;而其他账号留下的批准是另一张票,不能算作机器人自己的。

The stage comments above were updated with the latest result. View workflow run.

上方各阶段评论已更新为最新结果。查看工作流运行

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. LGTM! ✅

Not explored to full depth (tool budget reached): "agent reverse-audit (round 1)": enumerating every telemetry event type's fields for a content-bearing key outside the 7-key denylist (e.g. a spread result attribute reaching bridge spans ung…; "agent 6c": whether OTLP log export renders the undefined -valued request_text / response_text attributes that ...event spreads onto api_request/api_response records (….

Test Plan (not a blocker): src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory.

中文说明

未发现问题。LGTM!✅

未探索到全部深度(达到工具调用预算):"agent reverse-audit (round 1)"enumerating every telemetry event type's fields for a content-bearing key outside the 7-key denylist (e.g. a spread result attribute reaching bridge spans ung…"agent 6c"whether OTLP log export renders the undefined -valued request_text / response_text attributes that ...event spreads onto api_request/api_response records (…

Test Plan(非阻断):src/core/loggingContentGenerator/loggingContentGenerator.test.tsno such file or directory; src/telemetry/log-to-span-processor.test.tsno such file or directory; src/telemetry/loggers.test.tsno such file or directory

— qwen3.8-max via Qwen Code /review (v0.23.3)

@qwen-code-dev-bot qwen-code-dev-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

已核对 head 34c8eb1cc2373550d5681e9feb48f8b0dec8230f(vs merge-base 78bbd9f55c)。产代码只有三处:logApiRequestgetTelemetryLogPromptsEnabled() 为假时不再 JSON.stringify(contents) 而是传 undefinedloggingContentGenerator.ts:238-240)、_logApiResponse 同口径处理 responseText:273)、log-to-span-processor.tsSENSITIVE_ATTRIBUTE_KEYS 补上 request_text

独立复查未发现 Critical

  • 发布点收敛完整:new ApiRequestEvent(new ApiResponseEvent( 在全仓只有这两处(loggingContentGenerator.ts:243/266),没有绕过开关的第三条构造路径;ApiErrorEvent:307)不带正文。types.tsrequest_text?: string 本就是可选字段,undefined 是合法值。
  • 另一条会写完整请求/响应的通道是 logOpenAIInteraction:1083-1108),它只在 generatorConfig.enableOpenAILogging 为真时才存在(:200),是独立的显式开关与本 PR 语义无关,不属于这里要收的门。
  • 三sink契约与文案一致:文件导出会把 undefined 值的属性丢掉(FileLogExporter omits undefined-valued attributes (logPrompts off))、OTLP 原生日志侧键存在但为空、log-to-span 桥接跨度按 denylist 丢弃。

本地验证log-to-span-processor.test.ts + file-exporters.test.ts 在该 head Tests 59 passed (59),并做了一处变异——从 SENSITIVE_ATTRIBUTE_KEYS 删掉新增的 'request_text' —— LogToSpanProcessor > drops sensitive attributes before exporting bridged spans 立即变红,说明这一行的效果有见证而非靠文案。
说明一处环境限制:loggingContentGenerator.test.tsloggers.test.ts 在本机无法收集(本地 node_modules 缺 @modelcontextprotocol/clientsrc/tools/mcp-client.ts 在导入链上,与本 PR 无关),因此那 4 条「开关 × 一元/流式」四象限用例的绿色以 CI 的 Test (ubuntu-latest, Node 22.x)(本 head 已 success)为准;我逐行读过这 4 条用例,断言的是 request_text/response_texttoBeUndefined()toContain('KEEP_*_MARKER'),方向与开关一致。

CI:required 全部 success(Test (ubuntu-latest, Node 22.x)Lint & StaticIntegration Tests (no-AK, No Sandbox)web-shell E2E Smoke),无失败无 pending。

历史项:第 1~5 轮共 19 条线程全部关闭,其中三条 Prettier/Critical(R1-4、R1-1 fix-induced、R4-1)在当前 head 均已落地(3e2fd8e 之后格式检查转绿,文档矩阵改写为「三个抑制原因 + present-but-empty」口径);R5-1thoughtSignature 的导出策略)按作者说明移交 #11682,本 head 的最新一笔正是该断言的回退,与移交一致。

一条留给后续的事实性提醒(不阻塞):两处文档现在枚举的是 4 个键(prompt/function_args/request_text/response_text),而 SENSITIVE_ATTRIBUTE_KEYS 实际有 7 个(还含 errorerror.messageerror_message),R5-2/R5-3 的这一点成立;只是「桥接跨度保留哪几个敏感键」的说明,按现状会少列 3 个,可在后续一并补齐。

@qqqys qqqys left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

核对基线:head 34c8eb1cc2373550d5681e9feb48f8b0dec8230f

历史阻塞问题:已确认消除

本 PR 历史上有两条 Critical,我在当前 head 上分别核对:

  • R1-1(telemetry.mdresponse_text 行同时发布两种互斥的线上形状)—— 已关闭。 第 4 轮 review 的 ledger 自身已把它记为 closed,且当前 head 上该文件的对应线程为 isResolved: true
  • R4-1(新增测试块未经 Prettier 格式化,会让仓库格式检查失败)—— 已消除。 该问题锚定的 packages/core/src/telemetry/loggers.test.ts 已不在当前 PR 的变更文件列表里;同时当前 head 的 Lint & Static (ubuntu-latest, Node 22.x) 为 pass,格式门禁实际通过。

第 5 轮 review 只留下 3 条 Suggestion(文档中 denylist 键枚举不完整等),按本渠道策略不作为合入门禁。当前 PR 上所有 review thread 均为 isResolved: true

本轮独立扫描:未发现 Critical

这是一个隐私门禁改动,我按「关闭状态下是否还有内容外泄」这条主线核了完整调用链:

  1. 门禁本身正确。 loggingContentGenerator.ts 现在把 requestTextresponseText 都改成 getTelemetryLogPromptsEnabled() ? ... : undefined。该访问器定义在 config.ts:7452,与 telemetry/loggers.ts:149telemetry/qwen-logger/qwen-logger.ts:1100 用的是同一个门禁,口径一致,没有引入第二套判断。
  2. 类型安全。 ApiRequestEvent 构造器的 request_text?: stringtelemetry/types.ts:283)与 ApiResponseEventresponse_text?: string:396)本来就是可选参数,传 undefined 合法;当前 head 的 Lint & Static(含类型检查)pass 也印证了这一点。
  3. span 侧的补漏是真实缺口,不是防御性冗余。 log-to-span-processor.ts:176-190 的属性拷贝循环只在 includeSensitiveSpanAttributes 为真时才放行 SENSITIVE_ATTRIBUTE_KEYS 中的键,而该开关默认是 falseconfig.ts:2606:7461)。改动前 request_text 不在这个集合里(response_text 已在),因此默认配置下带 request_text 的 bridge 日志记录会被原样拷进 span —— 这正是本 PR 补上的那条。补入后与 promptfunction_argsresponse_text 同级处理,语义一致。
  4. 两层门禁叠加后没有残留写入口。 关闭 logPrompts 时字段为 undefined,拷贝循环的 value !== undefined 前置条件(:180)会直接跳过该键,因此既不会写出空字符串,也不会留下一个值为 undefined 的键。

CI:Test (ubuntu-latest, Node 22.x)Lint & StaticIntegration Tests (no-AK, No Sandbox)web-shell E2E Smokereview-pr 在当前 head 上均为 pass,没有由本 PR 引入的失败。

结论:历史阻塞问题已确认消除,本轮未发现可证明的 Critical,提交 APPROVE。

@yiliang114
yiliang114 added this pull request to the merge queue Sep 12, 2026
Merged via the queue into main with commit 908452b Sep 12, 2026
79 checks passed

@qwen-code-dev-bot qwen-code-dev-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM。已核对 head 34c8eb1c(vs origin/main merge-base),CI 全绿。

logPrompts 关闭时 request_text / response_text 不再进入任何 sink:生产侧只有两个产出点,都过了 gate(packages/core/src/core/loggingContentGenerator/loggingContentGenerator.ts:238-240:273);bridge 侧 request_text / response_text 已进 SENSITIVE_ATTRIBUTE_KEYSpackages/core/src/telemetry/log-to-span-processor.ts:58-66),默认不再复制。

一条非阻塞观察(不影响 approve):gate 是把字段设成 undefined,不是删除键。OTLP 序列化时 toAnyValue(undefined) 不产出任何分支(@opentelemetry/otlp-transformercommon/internal.js:31-51),属性键仍会留在记录里、只是值为空,而不是 key-absent——仓库既有约定恰好相反,packages/core/src/telemetry/loggers.ts:169-172 的注释写明 error 字段是 delete 而不是 set undefined,为的就是让下游看到 key-absent。要不要对齐可以另开一个,不阻塞本次。

@chiga0 chiga0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings. Approval blockers: none.

Scope: loggingContentGenerator.ts, log-to-span-processor.ts, loggers.ts, types.ts. NOT reviewed — the OTLP exporter internals, FileLogExporter runtime behaviour (only its test pinning undefined-key serialization was checked), and the full monorepo test suite.

Checked:

  • Symmetric gating: Both request_text and response_text are gated via getTelemetryLogPromptsEnabled() at the producer in loggingContentGenerator.ts (non-streaming and streaming paths both route through logApiRequest / _logApiResponse, so there is no bypass).
  • Defense-in-depth layering: producer (undefined when off) → bridge onEmit short-circuits on undefined/nullSENSITIVE_ATTRIBUTE_KEYS denylist now includes request_textJSON.stringify in file exporter drops undefined keys. No single layer failure exposes the content.
  • request_text consumer claim verified: assigned in ApiRequestEvent constructor, spread into OTel attribute map in loggers.ts, read nowhere for business logic. The PR description is accurate.
  • Denylist addition is correct: request_text now treated consistently with prompt / response_text when includeSensitiveSpanAttributes=false.
  • Tests: four new test cases cover non-streaming omit, non-streaming keep, streaming omit, and streaming keep — all four branches directly exercised.

Not covered: whether opaque thoughtSignature (encrypted_content) should be exported as request text even when prompt logging is explicitly enabled — the PR description correctly flags this as a separate policy question, not a defect in this change.

Reviewed with AI assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(telemetry): API request content is exported despite logPrompts=false

7 participants