Skip to content

refactor(core): shrink the content generator interface - #9676

Merged
DragonnZhang merged 7 commits into
QwenLM:mainfrom
DragonnZhang:dragon/shrink-content-generator-interface
Aug 24, 2026
Merged

refactor(core): shrink the content generator interface#9676
DragonnZhang merged 7 commits into
QwenLM:mainfrom
DragonnZhang:dragon/shrink-content-generator-interface

Conversation

@DragonnZhang

@DragonnZhang DragonnZhang commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

This removes unused token-counting and summarized-thinking capabilities from the shared content-generation contract, its lazy and logging decorators, and the provider implementations. It also removes the corresponding dead tests and mock members while leaving embedding support unchanged because it still has an active call path.

Why it's needed

The shared provider contract required every backend and test double to implement operations that no production caller used. That widened the Google SDK type surface, duplicated provider-specific code, and made the abstraction look more capable than its consumers require. Narrowing the contract reduces provider coupling and makes later protocol work smaller without changing generation behavior.

Reviewer Test Plan

How to verify

  1. Confirm the OpenAI-compatible, Anthropic, Gemini, and Qwen generators satisfy the narrowed contract and that lazy provider creation still happens exactly once on first real use.
  2. Confirm the logging decorator still forwards generation and embedding unchanged.
  3. Run npm run build and npm run typecheck; both should complete successfully.
  4. Run the affected core tests from packages/core and the session-picker/AppContainer tests from packages/cli; the verified result is 14 core files with 1,628 passing tests and 2 CLI files with 175 passing tests.

Evidence (Before & After)

N/A — this is a non-user-visible contract cleanup.

Tested on

OS Status
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

Environment (optional)

Local workspace with Node.js v24.18.0 and npm 11.16.0; no sandbox.

Risk & Scope

  • Main risk or tradeoff: Out-of-repository TypeScript consumers that relied on the removed public contract members will no longer compile against the narrowed interface.
  • Not validated / out of scope: No UI E2E run was performed because generation behavior is unchanged; embedding remains in the contract, and broader protocol-type migration is intentionally deferred.
  • Breaking changes / migration notes: External implementations and callers must remove countTokens and useSummarizedThinking from their ContentGenerator usage. There are no remaining production callers in this repository.

Linked Issues

Refs #4063

中文说明

这个 PR 做了什么

本 PR 从共享内容生成协议、懒加载与日志装饰器以及各 provider 实现中移除了未使用的 token 计数和 summarized-thinking 查询能力,同时清理了对应的无效测试和 mock 成员。Embedding 支持仍然保留,因为目前还有真实调用链路。

为什么需要这个改动

共享 provider 协议此前要求每个后端和测试替身都实现生产代码从未调用的操作。这扩大了 Google SDK 类型的影响范围,造成了 provider 侧重复代码,也让抽象层看起来承担了超出实际消费者需求的职责。收窄协议可以降低 provider 耦合,并让后续协议改造范围更小,同时不改变内容生成行为。

Reviewer 测试计划

如何验证

  1. 确认 OpenAI-compatible、Anthropic、Gemini 和 Qwen generator 都满足收窄后的协议,并且懒加载 provider 仍然只在第一次真实调用时创建一次。
  2. 确认日志装饰器仍然原样转发内容生成和 embedding。
  3. 运行 npm run buildnpm run typecheck,两者都应成功完成。
  4. packages/core 运行受影响的 core 测试,并在 packages/cli 运行 session picker 与 AppContainer 测试;本地验证结果为 14 个 core 测试文件共 1,628 个用例通过,以及 2 个 CLI 测试文件共 175 个用例通过。

证据(改动前后)

N/A — 这是一个无用户可见变化的协议清理。

测试平台

OS 状态
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

环境(可选)

本地工作区,Node.js v24.18.0、npm 11.16.0,未使用 sandbox。

风险与范围

  • 主要风险或取舍:仓库外依赖已移除公共协议成员的 TypeScript 消费者,在升级后将无法通过编译。
  • 未验证 / 不在范围内:由于内容生成行为没有变化,本次未运行 UI E2E;embedding 仍保留在协议中,更广泛的协议类型迁移有意留待后续处理。
  • 破坏性变更 / 迁移说明:外部实现和调用方需要从 ContentGenerator 用法中移除 countTokensuseSummarizedThinking。本仓库生产代码中已经没有这些成员的调用方。

关联 Issue

Refs #4063

@DragonnZhang
DragonnZhang marked this pull request as ready for review August 21, 2026 15:20
@DragonnZhang
DragonnZhang enabled auto-merge August 21, 2026 15:20
@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finishedview run. See the stage comments in this thread for the result.

Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Re-run at the updated head (main merge + the follow-up cleanup requested by the last triage round). Gate outcome unchanged — passes, with one size note below that deserves maintainer visibility.

Template still fine ✓

Problem: still real dead code, re-verified at the new head. countTokens and useSummarizedThinking have zero production callers — only the interface, the implementations themselves, the decorator forwards, and one CLI preview stub — and the request-tokenizer estimator cluster this PR now deletes was imported only by the two countTokens implementations being removed (I checked the import graph at current main myself). Keeping embedContent is defensible as exported surface even though generateEmbedding has no production callers today (see finding F1 in the verification round below).

Direction: unchanged — narrowing a contract that every backend and test double must implement but nothing consumes is the right direction. ContentGenerator is exported from the core package, so this remains a compile-time break for out-of-repo implementors; the tradeoff is disclosed in the PR body and was made with maintainer awareness — the approval in this thread is that call being made.

Size: 1,089 production-logic lines (interface shrink 157, orphaned estimator-cluster deletion ~922, remainder comment/doc-reference cleanup) vs 932 test lines, 32 doc lines, and 2 lines of CI bookkeeping. To be straight about it: this is above the 500-production-line threshold at which a fresh fork refactor touching core gets hard-blocked as needing maintainer initiation. This re-run doesn't apply that block for two reasons — the growth is precisely the follow-up cleanup the previous triage asked the author to track, and, decisively, the maintainer decision that rule exists to guarantee is already on the record in this thread: @wenshao deep-verified this exact head in isolation (75/75 scripted assertions, A/B wire-identical request bodies) and formally approved the commit. Requesting changes from the bot against a standing maintainer approval would invert what the gate is for, so the review proceeds and the final call stays where the policy puts it — with the maintainers. If another maintainer reads the size differently, say so here.

Approach: scope remains minimal. The new delta is exactly the follow-up: the estimator cluster deleted root and branch (zero residual references), stale references in two design docs and one code comment updated with dated sync notes rather than silent edits, and .size-baseline corrected to the real 42,519 bytes of cd-cua-driver.yml (the gap is pre-existing main-side growth; the record fix lets the size gate pass after the merge). No drive-by churn.

Risk: openaiContentGenerator.ts still matches a high-risk path from the repo's revert history — the change there remains a pure removal of the unreachable method, reviewed at full depth with CI evidence below. The fork-refactor approval guardrail still applies: this bot will not approve this PR regardless of review outcome.

Re-review complete. 🔍

中文说明

基于更新后的 head(合入 main + 上一轮 triage 要求的后续清理)重新运行。门禁结论不变——通过,但下面有一条应当让维护者知悉的规模说明。

模板仍然完整 ✓

问题:仍是真实的死代码,已在新 head 上复核。countTokensuseSummarizedThinking 没有任何生产调用方——仅有接口、实现本身、装饰器转发和一个 CLI 预览 stub;本 PR 新删除的 request-tokenizer 估计器簇也只被本次移除的两个 countTokens 实现导入(导入图我已在当前 main 上亲自核查)。保留 embedContent 作为导出接口是站得住的,尽管 generateEmbedding 目前没有生产调用方(见下文验证结论 F1)。

方向:不变——收窄每个后端和测试替身都必须实现、却无消费者的协议,方向正确。ContentGenerator 从 core 包导出,对仓库外实现方仍是编译期破坏;该取舍已在 PR 正文披露,且维护者是在知情的情况下做出决定——本线程中的批准就是该决定本身。

规模:1,089 行生产逻辑(接口收窄 157 行、孤儿估计器簇删除约 922 行、其余为注释/文档引用清理),测试 932 行、文档 32 行、CI 记账 2 行。坦率说明:这已超过 500 行生产逻辑阈值——一个全新的、触及核心路径的 fork refactor 在该规模下会被硬拦截,要求由维护者发起。本次 re-run 不执行该拦截,原因有二:规模增长恰恰是上一轮 triage 要求作者跟进的清理;更关键的是,该规则所要保证的维护者决策已在本线程记录在案:@wenshao 对完全相同的 head 做了隔离深度验证(75/75 脚本断言、A/B wire 请求逐字节一致)并正式批准。对已存在的维护者批准再发 bot request-changes,是把门禁的目的反过来用,因此审查继续进行,最终决定权留在政策指定的位置——维护者手中。如果其他维护者对规模的解读不同,请在此指出。

方案:范围仍然最小。新增改动恰为所需:估计器簇连根删除(零残留引用),两个设计文档与一处代码注释中的过期引用以带日期的同步注记更新而非静默改写,.size-baseline 修正为 cd-cua-driver.yml 的实际 42,519 字节(差额为 main 侧既有增长,修正记录使合入后 size 门禁可通过)。无顺手改动。

风险:openaiContentGenerator.ts 仍命中仓库 revert 历史的高风险路径——该处改动仍是不可达方法的纯移除,已按完整深度审查并要求 CI 证据(见下)。fork-refactor 审批护栏仍然适用:无论审查结果如何,本机器人不会批准此 PR。

复审完成 🔍

Qwen Code · qwen3.8-max

Reviewed at cf82579ac44c96a41fc42466b973d25e0b984a68 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Code review (re-run at the merged head)

My independent baseline for this round, written before reading the new delta: confirm the estimator cluster has no importers beyond the two countTokens implementations being removed, delete it root and branch, sweep every stale reference (docs, comments, inert mocks), and fix the size-baseline record after the main merge. The PR does exactly that — nothing missed, nothing extra.

What I verified myself against the current main base and the diff:

  • The cluster was orphaned exactly as claimed. The only import edges into utils/request-tokenizer/ are supportedImageFormats (fileUtils.ts) and textTokenizer (pdf.ts) — the two files this PR keeps. The estimator classes were imported only from the openai/anthropic countTokens implementations (removed here), one inert vi.mock in client.test.ts (removed), and a stale comment in review/lib/assets.ts (updated). node-repl mentions the cluster in a comment only.
  • Head-side exhaustiveness is compiler-enforced. Every ContentGenerator implementor lives in this repo, so repo-wide typecheck — green on this head's CI — is the oracle that nothing still calls the removed members.
  • The surviving files' edits are comment-only. supportedImageFormats.ts keeps its code and gets an accurate doc rewrite (token accounting now uses DEFAULT_IMAGE_TOKEN_ESTIMATE from compactionInputSlimming.ts — verified present); assets.ts drops its cross-reference to the deleted imageTokenizer.ts.
  • .size-baseline records reality. cd-cua-driver.yml is 42,519 bytes at this head; the old 29,715 entry was pre-existing main-side growth, and the corrected record is what lets the size gate pass after the merge (also confirmed in the maintainer's verify round).
  • Lazy-creation, preload, and single-flight semantics stay pinned — re-pointed from countTokens to embedContent with the same assertions, not deleted.

No blockers. The two non-blocking findings from @wenshao's deep-verify round both stand, and I re-checked them against the base: F1 — the PR body's "embedding still has an active call path" is inaccurate (generateEmbedding has zero production callers; pre-existing, not caused by this PR) — a description-accuracy note, not a code issue. F2LoggingContentGenerator.embedContent() forwarding is not pinned by a test (that mutation survived the suite; behavior was verified correct against the compiled artifact in the verify round). A one-line follow-up test would close F2.

Files changed (30 of 34 shown)
File What changed
packages/core/src/core/contentGenerator.ts Interface drops both members; lazy wrapper loses the summarized-thinking flag and forwards
packages/core/src/core/openaiContentGenerator/openaiContentGenerator.ts Removes character-estimation countTokens and false useSummarizedThinking
packages/core/src/core/anthropicContentGenerator/anthropicContentGenerator.ts Removes tokenizer-based countTokens and false useSummarizedThinking
packages/core/src/core/geminiContentGenerator/geminiContentGenerator.ts Removes SDK countTokens forward and true useSummarizedThinking
packages/core/src/qwen/qwenContentGenerator.ts Removes the pass-through countTokens override
packages/core/src/core/loggingContentGenerator/loggingContentGenerator.ts Decorator stops forwarding the two removed members
packages/cli/src/ui/components/StandaloneSessionPicker.tsx Preview config stub no longer fakes getContentGenerator
packages/cli/src/commands/review/lib/assets.ts Comment drops the stale cross-reference to deleted imageTokenizer
packages/core/src/utils/request-tokenizer/requestTokenizer.ts Deleted — text estimator with no remaining importer
packages/core/src/utils/request-tokenizer/imageTokenizer.ts Deleted — image dimension parser with no remaining importer
packages/core/src/utils/request-tokenizer/types.ts Deleted — the cluster's result/metadata types
packages/core/src/utils/request-tokenizer/index.ts Deleted — directory barrel
packages/core/src/utils/request-tokenizer/supportedImageFormats.ts Kept — doc comment rewritten to post-removal reality
docs/design/2026-07-22-lazy-google-genai-loading.md Synced to the three-operation interface with a dated note
docs/design/web-shell/web-shell-image-drag-and-drop.md ImageTokenizer BMP references replaced with a dated sync note
.github/workflows/.size-baseline Records the real 42,519 bytes of cd-cua-driver.yml
packages/core/src/core/contentGenerator.test.ts Lazy/preload/single-flight tests re-pointed to embedContent
packages/core/src/core/openaiContentGenerator/openaiContentGenerator.test.ts Drops countTokens describe and tokenizer mock
packages/core/src/core/anthropicContentGenerator/anthropicContentGenerator.test.ts Same removal for the Anthropic side
packages/core/src/qwen/qwenContentGenerator.test.ts Removes countTokens tests including the no-auth case
packages/core/src/core/geminiContentGenerator/geminiContentGenerator.test.ts Removes delegation test and SDK mock member
packages/core/src/core/geminiChat.test.ts Seven mock generators drop the dead members
packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts Wrapped-generator mock drops the dead members
packages/core/src/core/client.test.ts Removes the inert RequestTokenizer vi.mock and countTokens mock member
packages/core/src/config/config.test.ts Four mock generators drop countTokens
packages/core/src/utils/request-tokenizer/requestTokenizer.test.ts Deleted with its implementation
packages/core/src/utils/request-tokenizer/imageTokenizer.test.ts Deleted with its implementation
packages/core/src/core/tests/openaiTimeoutHandling.test.ts Deleted — redundant timeout case that spied on countTokens
packages/cli/src/ui/AppContainer.test.tsx getContentGenerator mock now returns an empty object
packages/cli/src/ui/components/StandaloneSessionPicker.test.tsx Test stub drops the useSummarizedThinking fake
…and 4 more files agent-headless.test.ts, both promptHook tests, nextSpeakerChecker.test.ts — same mock-member cleanup

Test evidence (this PR's own CI — I never run PR code)

All PR-CI runs on the reviewed commit have completed, and the unit suite is fully green: from the Test (ubuntu-latest, Node 22.x) job log — Test Files 875 passed (875), Tests 23486 passed | 25 skipped (23511), with zero failures anywhere in the log. The single red from the previous head (src/serve/acp-http/transport.test.ts) is gone — it lived in code this PR never touches, and the merge from main brought in exactly that area's fixes, matching the author's diagnosis.

One correction to the previous run's wording: the macOS and Windows test legs and the CLI integration job are skipped here by design, not fail-fast — ci.yml gates all three on github.event_name == 'merge_group', so ubuntu is the PR-level signal and the skipped named jobs still satisfy the required contexts.

Check Conclusion
Test (ubuntu-latest, Node 22.x) ✅ success (875/875 files, 23,486 tests)
Test (macos-latest, Node 22.x) ⏭️ skipped (merge-queue-only by design)
Test (windows-latest, Node 22.x) ⏭️ skipped (merge-queue-only by design)
Integration Tests (CLI, No Sandbox) ⏭️ skipped (merge-queue-only by design)
Desktop Shell (ubuntu-22.04) ✅ success
Desktop Shell (windows-2022) ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success
precheck-pr / precheck ✅ success
Secret scan (TruffleHog) ✅ success
Dependency CVE audit ✅ success

Real-scenario testing: N/A — unattended run, and the change has no user-visible surface to drive. The central claim is a negative one (zero behavior change), and it is substantiated: repo-wide typecheck as the exhaustiveness oracle, a fully green 23,486-test suite on this head, and @wenshao's local A/B round showing byte-identical wire requests between base and head bundles. A sandboxed CI-side verification run for this head is also in flight and will post its own report in this thread — belt and braces rather than an open gap.

中文说明

代码审查(合入后 head 的复审)

本轮在读新改动之前写下的独立基线:确认估计器簇除本次移除的两个 countTokens 实现外无任何导入方,连根删除,清扫全部过期引用(文档、注释、无效 mock),并在合入 main 后修正 size-baseline 记录。PR 的做法与此完全一致——没有遗漏,也没有多余。

我亲自对照当前 main 基线与 diff 核查的内容:

  • 该簇确实已成孤儿。 进入 utils/request-tokenizer/ 的导入边只有 supportedImageFormats(fileUtils.ts)与 textTokenizer(pdf.ts)——正是本 PR 保留的两个文件。估计器类此前只被 openai/anthropic 的 countTokens 实现(本次移除)、client.test.ts 里一个无效 vi.mock(已删除)以及 review/lib/assets.ts 一处过期注释(已更新)引用。node-repl 仅在注释中提及。
  • head 侧的完备性由编译器强制保证。 所有 ContentGenerator 实现都在本仓库内,全仓库 typecheck——在本 head 的 CI 中为绿——就是"无任何残留调用方"的判定依据。
  • 保留文件的改动仅为注释。 supportedImageFormats.ts 代码不变,文档注释被准确改写(token 计数现使用 compactionInputSlimming.tsDEFAULT_IMAGE_TOKEN_ESTIMATE——已核实存在);assets.ts 删除了指向已删 imageTokenizer.ts 的交叉引用。
  • .size-baseline 记录的是事实。 本 head 上 cd-cua-driver.yml 为 42,519 字节;旧的 29,715 是 main 侧既有增长,修正记录后 size 门禁才能在合入后通过(维护者验证轮亦确认)。
  • 懒构造、preload、single-flight 语义继续被锚定——从 countTokens 改指向 embedContent,断言不变,未被删除。

无阻塞项。@wenshao 深度验证轮的两条非阻塞发现均成立,且我已对照基线复核:F1——PR 正文"embedding 仍有活跃调用链路"不准确(generateEmbedding 无生产调用方;既有状况,非本 PR 引入)——属描述准确性问题,非代码问题。F2——LoggingContentGenerator.embedContent() 转发无测试锚定(该 mutation 在套件中存活;验证轮已对编译产物确认行为正确)。一条单行后续测试即可补上 F2。

测试证据(来自本 PR 自己的 CI——审查过程不运行 PR 代码)

受审 commit 上的所有 PR-CI 运行均已完成,单测套件全绿Test (ubuntu-latest, Node 22.x) job 日志——Test Files 875 passed (875)Tests 23486 passed | 25 skipped (23511),整份日志中零失败。上一 head 的唯一红项(src/serve/acp-http/transport.test.ts)已消失——它位于本 PR 从不触碰的代码中,合入 main 恰好带入了该区域的修复,与作者的诊断一致。

对上一轮措辞的一处更正:macOS/Windows 测试腿与 CLI 集成 job 在此为设计上跳过,而非 fail-fast——ci.yml 将三者都门禁在 github.event_name == 'merge_group',因此 ubuntu 是 PR 级信号,被跳过的具名 job 仍满足必需检查上下文。

真实场景测试:不适用——无人值守运行,且改动无用户可见界面可驱动。中心主张是否定性主张(零行为变化),且已被证实:全仓库 typecheck 作为完备性判定、本 head 上 23,486 个测试全绿、@wenshao 的本地 A/B 轮显示 base 与 head 构建的 wire 请求逐字节一致。针对本 head 的 CI 侧沙箱验证运行也在进行中,会在本线程另行发布报告——属于双保险,而非未闭合缺口。

Qwen Code · qwen3.8-max

Reviewed at cf82579ac44c96a41fc42466b973d25e0b984a68 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Confidence: 3/5 — clean re-review at the final head; the fork-refactor guardrail keeps the approval with the maintainer, who has already exercised it on this commit (policy cap, not doubt).

Stepping back: this PR grew since the last run along exactly the axis that run asked for, and it's better for it. The author took the orphaned-utility note and deleted the estimator cluster root and branch, updated every stale reference with dated sync notes instead of silent edits, merged main to clear the unrelated red leg, and kept the diff free of drive-by churn. The result: every provider, decorator, and test double now implements only what something actually calls, and the follow-up the previous comment tracked as "worth a cleanup issue" simply no longer exists. My independent baseline for the follow-up round and the PR's approach are the same plan — I found no simpler path it missed.

All three original defer reasons are resolved:

  1. Fork-refactor guardrail — this is a cross-repository refactor, which this bot never approves. That's now been handled the right way: a human maintainer looked. @wenshao ran an isolated deep verification on this exact head (75/75 scripted assertions, base-vs-head wire requests byte-identical, dist-level flip confirming the removal) and formally approved the commit.
  2. The exported-interface semver decision — disclosed in the PR body, examined in the verify round, and settled by that approval. It was always a maintainer call; it has been made.
  3. Green CI — 875/875 test files (23,486 tests) green on the reviewed commit; the previous head's single red was an unrelated flake that the main merge resolved.

So: no approval from this account — the guardrail is absolute for fork refactors — and no change request, because nothing blocking was found. The bot side of the gate is complete on this head; the merge decision rests with the maintainers.

中文说明

置信度:3/5 —— 在最终 head 上的复审是干净的;fork-refactor 护栏把批准权留在维护者手中,而维护者已在此 commit 上行使了该批准(这是政策上限,而非对代码有疑虑)。

退一步看:自上一轮以来,这个 PR 恰恰沿着上一轮要求的方向生长,并且因此变得更好。作者接到孤儿工具的提示后,把估计器簇连根删除,用带日期的同步注记(而非静默修改)更新了每一处过期引用,合入 main 清除了无关的红色测试腿,且 diff 中没有顺手改动。结果是:现在每个 provider、装饰器和测试替身只实现真正被调用的东西,上一条评论里记为"值得开一个清理 issue"的后续事项已不复存在。我为本轮后续工作独立写下的基线方案与 PR 的做法就是同一个方案——我没有找到被它遗漏的更简路径。

最初转交的三个原因已全部解决:

  1. fork-refactor 护栏 —— 这是一个来自 fork 的 refactor,本机器人从不批准。现在它以正确的方式被处理了:人类维护者亲自把关。@wenshao 对完全相同的 head 运行了隔离深度验证(75/75 脚本断言、base 与 head 的 wire 请求逐字节一致、dist 级翻转证实移除生效),并正式批准了该 commit。
  2. 导出接口的 semver 决策 —— 已在 PR 正文披露、在验证轮中审视,并由该批准落定。这始终是维护者的决定;现在已经做出。
  3. 全绿 CI —— 受审 commit 上 875/875 个测试文件(23,486 个用例)全绿;上一 head 的唯一红项是无关 flake,已随合入 main 解决。

因此:本账号不会给出批准——护栏对 fork refactor 是绝对的——也不会请求修改,因为没有发现任何阻塞项。门禁的机器人一侧在此 head 上已完成;合入决定权在维护者。

Qwen Code · qwen3.8-max

Reviewed at cf82579ac44c96a41fc42466b973d25e0b984a68 · re-run with @qwen-code /triage

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partially reviewed — gaps disclosed. Suggestions are inline.

Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.

Test Plan (not a blocker): 628 passing — this review observed 22866, 20658, 1661, 1638, 495, 4026, 595 passed; 175 passing — this review observed 22866, 20658, 1661, 1638, 495, 4026, 595 passed.

中文说明

仅完成部分审查,审查缺口已披露。 建议见行内评论。

未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。

Test Plan(非阻断):628 passing — this review observed 22866, 20658, 1661, 1638, 495, 4026, 595 passed; 175 passing — this review observed 22866, 20658, 1661, 1638, 495, 4026, 595 passed

— qwen3.8-max via Qwen Code /review (v0.21.15)

Comment thread packages/core/src/core/contentGenerator.ts
Removing countTokens from both providers deleted the last production
consumers of RequestTokenEstimator. Delete the orphaned cluster:
requestTokenizer.ts (330), imageTokenizer.ts (534), types.ts (36), the
directory barrel (11), and both test files (608 lines). Also drop the
inert vi.mock of requestTokenizer.js left in client.test.ts and the
stale dimension-extractor cross-reference in review/lib/assets.ts.

textTokenizer.ts and supportedImageFormats.ts stay: converter.ts, pdf.ts,
and fileUtils.ts still consume them and the core barrel re-exports them.
…face

countTokens and useSummarizedThinking no longer exist on ContentGenerator,
so the design record for the lazy-wrapper architecture must not keep
advertising them: list the three remaining shared async operations, drop
the useSummarizedThinking sentence and the summarized-thinking item from
the consumer audit and Verification section, and add a dated note
recording the interface shrink from PR QwenLM#9676.
@DragonnZhang

Copy link
Copy Markdown
Collaborator Author

Status update (maintenance pass after stall) — new head 1871bb5b81 (was 68d3bb4cbf).

1. Red CI diagnosed. The only failing check (Test ubuntu-latest, Node 22.x) failed exactly 1 of 22,883 tests: packages/cli/src/serve/acp-http/transport.test.ts › "keeps a shared session/list scan alive when one connection is destroyed" — a shared session/list scan resolved with -32603 Internal error while a sibling connection was destroyed. This PR touches no serve code (there is no path from removing two never-called interface members to an ACP connection-teardown race), and the branch was 65 commits behind main, which has since changed exactly that area (dispatch.ts, session-list.ts, 372 changed lines in transport.test.ts itself, plus the archive-race fixes from #9513). Resolution: merged origin/main (clean merge, no conflicts), which syncs the branch and supersedes the stale failure. The red leg was unrelated to this PR's diff.

2. All three review suggestions verified and actioned (per-thread replies posted, threads resolved):

  • The orphaned request-tokenizer estimator cluster is removed root and branch in 0ee17632c7 (zero importers verified at the merged head): requestTokenizer.ts (330), imageTokenizer.ts (534), types.ts (36), the directory barrel (11), both test files (608) — 1,519 lines — plus the inert vi.mock in client.test.ts and the stale cross-reference in review/lib/assets.ts. textTokenizer.ts / supportedImageFormats.ts stay, as they are still consumed (converter.ts, pdf.ts, fileUtils.ts, core barrel).
  • docs/design/2026-07-22-lazy-google-genai-loading.md is updated in 1871bb5b81 to match the shrunk interface: three shared async operations, the useSummarizedThinking sentence deleted, consumer audit and Verification sections corrected, and a dated note recording the interface shrink.

3. Verification evidence. Repo-wide typecheck on the merged head: core tsc --noEmit and cli tsc --noEmit report zero errors in any file this PR touches — every diagnostic emitted sits in files this PR never modifies and is an artifact of the shared build host's corrupted node_modules (disk-full-truncated third-party .d.ts files: @xterm/headless, fzf, web-tree-sitter, etc.) plus unbuilt workspace dists, not of this diff. A repo-wide sweep finds zero remaining references to countTokens, useSummarizedThinking, or any deleted cluster file. Full behavioral confirmation is left to this PR's CI at 1871bb5b81: the shared host's disk was at 100% during this pass (which is also what corrupted its node_modules), so local vitest runs could not be completed reliably here; the affected suites are the content-generator/provider/logging/lazy-wrapper tests plus AppContainer/StandaloneSessionPicker, exactly what CI runs.

4. Maintainer items (policy, unchanged): fork-refactor approval guardrail and the exported-ContentGenerator semver call (disclosed in the PR body) still need a human decision — see the stage-3 triage comment.

Same latent main-side violation as fixed in QwenLM#9682: QwenLM#9587 grew the
workflow without a baseline update; record the new size as the check
message directs (precedent QwenLM#9747).

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partially reviewed — gaps disclosed. Suggestions are inline.

Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.

Not explored to full depth (tool budget reached): "agent 1c": none — though I did not execute the affected test suites (cross-file verification was static; runtime behavior of the edited mocks is attested by the author's C…; chunk 3: typecheck/unit-test run for the modified core test files (worktree lacks node_modules and dist; install+build exceeds budget).

Deferred under the convergence posture (round 2, not a blocker) — recorded, not requested in this round:

  • packages/core/src/core/__tests__/openaiTimeoutHandling.test.ts:352 — [review] Suite titled 'token estimation on timeout' outlives the deleted token-estimation test
中文说明

仅完成部分审查,审查缺口已披露。 建议见行内评论。

未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。

未探索到全部深度(达到工具调用预算):"agent 1c"none — though I did not execute the affected test suites (cross-file verification was static; runtime behavior of the edited mocks is attested by the author's C…;chunk 3:typecheck/unit-test run for the modified core test files (worktree lacks node_modules and dist; install+build exceeds budget)

收敛姿态下延后(第 2 轮,非阻断)——已记录,本轮不要求修改:共 1 条(原文未翻译,列表见上方英文部分)。

— qwen3.8-max via Qwen Code /review (v0.22.0)

Comment thread docs/design/2026-07-22-lazy-google-genai-loading.md
Comment thread packages/cli/src/commands/review/lib/assets.ts
…emoval

Follow-up to 0ee1763/1871bb5b81 (review round 2):
- supportedImageFormats.ts header and getSupportedImageFormatsString doc
  no longer describe a tokenizer decode/metadata-extraction stage; the
  list is now documented as the vision-input acceptance list, with token
  accounting noted as the flat DEFAULT_IMAGE_TOKEN_ESTIMATE.
- web-shell-image-drag-and-drop.md's BMP rationale no longer claims
  ImageTokenizer parses BMP dimensions; dated sync note added stating
  BMP support rests on SUPPORTED_IMAGE_MIME_TYPES plus converter
  passthrough since PR QwenLM#9676.

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no blockers. Suggestions are inline.

中文说明

已审查——无阻断问题。 建议见行内评论。

— qwen3.8-max via Qwen Code /review (v0.22.0)

Comment thread docs/design/web-shell/web-shell-image-drag-and-drop.md
Follow-up to 18f08c0: the test plan still required converter/tokenizer
focused tests for image paths; the image-tokenizer estimator cluster was
removed in PR QwenLM#9676 (text tokenizer is unaffected and out of scope here).

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Downgraded from Approve to Comment: CI failing: Test (ubuntu-latest, Node 22.x). Reviewed.

Deferred under the convergence posture (round 4, not a blocker) — recorded, not requested in this round:

  • docs/design/web-shell/web-shell-image-drag-and-drop.md:512 — [probe] Sync-note blockquote absorbs the following paragraph — missing blank line
中文说明

⚠️ 已从批准降级为评论:CI failing: Test (ubuntu-latest, Node 22.x)。 已审查。

收敛姿态下延后(第 4 轮,非阻断)——已记录,本轮不要求修改:共 1 条(原文未翻译,列表见上方英文部分)。

— qwen3.8-max via Qwen Code /review (v0.22.0)

@wenshao

wenshao commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

🔬 Deep verification: merge-ready — 75/75 scripted assertions passed, 0 unexpected failures

Local maintainer round against verified head cf82579a (base 7385b278, main): two scratch worktrees, fresh npm ci + npm run build + npm run bundle per arm, mock-free wire harnesses on the compiled bundles. Full report and harnesses archived locally (tmp/pr9676-verify-20260824-131256/).

Central claim — proven by A/B

"Removing countTokens + useSummarizedThinking and the orphaned request-tokenizer cluster is dead-code removal with zero generation-behavior change."

Identical headless scenario (2 prompt runs, loopback OpenAI-compatible server, isolated HOME/settings) against the base bundle vs head bundle — twice, including a fresh build && bundle artifact:

oracle base head
wire requests to fake server 4 (2 main + 2 memory-extraction) 4, same shape
normalized request bodies (~422KB total) byte-identical to base
CLI exit codes / stdout 0 / canned text 0 / canned text, identical

A/B wire comparison

Supporting evidence

  • Census 9/9: zero residual refs at head (only vendored @types/vscode mentions remain); zero production callers of the removed members at base outside the implementing chain; kept textTokenizer/supportedImageFormats still consumed; the estimator cluster's only external importer (review/lib/assets.ts) is exactly what this PR cleans.
  • Mutation matrix (witness below): breaking lazy construct-once → 6 tests red (pinned); breaking lazy-layer embedContent forwarding → 14 tests red (pinned); breaking decorator-layer embedContent forwarding → survives (coverage gap, see F2). The killed rows double as the positive control.

Mutation matrix

  • Artifact-level flip: on the base dist countTokens/useSummarizedThinking are live functions; on the head dist both are undefined, and embedContent/generateContent forwarding verified correct against the compiled LoggingContentGenerator (4/4).
  • Gates: root typecheck exit 0 · PR-touched core suite 13 files / 1632 tests green (the 14th file, openaiTimeoutHandling.test.ts, was deleted wholesale by this PR — 19 lines) · CLI suite 2 files / 175 tests green (matches the plan's 175 exactly) · .size-baseline correctly records the latent main-side growth (29715 → actual 42519; Δ12,804 from feat(cua-driver): add versioned Computer Use SDK and release pipeline #9587), size gate exit 0.

Findings (both non-blocking)

  • F1 (Low, description accuracy — pre-existing): "embedding support unchanged because it still has an active call path" is not accurate at head: embedContent's only production call site is baseLlmClient.generateEmbedding(), which itself has zero production callers (whole-tree census; referenced only by its own definition and test file). Keeping embedding is still defensible (public surface / future use), but the stated premise should be corrected in the PR text, not the code.
  • F2 (Low, coverage gap — behavior verified correct this round): no test drives LoggingContentGenerator.embedContent(); the vi.fn() in its test file only satisfies the narrowed interface, so a forwarding break survives the suite (mutation M2 above). A one-line follow-up asserting the forwarded request would pin it — the lazy layer already has exactly that style of test.

Not covered

Per-provider wire harnesses for Anthropic/Gemini/Qwen (covered by typecheck + unit suites only); per-commit attribution (aggregate diff verified, 7 commits); UI E2E (matches the PR's own out-of-scope note); the merit of #9587's workflow growth (only the baseline bookkeeping was checked).

📄 Full report

PR 9676 deep verification — VERDICT: merge-ready (75/75 scripted assertions passed, 0 unexpected failures)

  • Verified head: cf82579ac44c96a41fc42466b973d25e0b984a68 · base: 7385b278b2017a0b6bfeff59380d23b57848fd4a (main)
  • Mode: maintainer-local round — scratch worktrees under tmp/, fresh npm ci + npm run build + npm run bundle per arm, no credentials inside the run dirs

Central claim + A/B table

cell environment oracle result
base bundle of 7385b278 (postinstall + fresh build) 4 wire requests recorded by loopback fake OpenAI server; CLI exit codes; stdout 8/8 arm assertions PASS
head bundle of cf82579a (postinstall build) same 8/8 PASS
head (rebuilt) bundle of cf82579a (fresh build + bundle) same 8/8 PASS
A/B compare base vs head fingerprints byte-identical normalized request bodies 11/11 PASS — IDENTICAL (~422KB: 2 main requests ≈156KB each + 2 managed-memory-extraction requests ≈55KB each)

Scenario: 2 headless --prompt runs per arm against the same fake server (SSE streaming, canned chunks, fixed usage), isolated HOME + project settings (security.auth.selectedType: "openai", telemetry off, sandbox off), identical absolute run-dir per arm. Each run issues exactly 2 requests — the main prompt request, then the session-end managed-memory extraction subagent request, which embeds the assistant turn (deeper fingerprint).

Controls that held: no package.json/lockfile changes in the PR; per-tree npm ci; node_modules/@qwen-code/qwen-code-core resolves inside each tree; wires compared after UUID normalization only.

Census (9/9 scripted assertions)

See summary above; raw log logs/census.log.

Mutation matrix (5/5)

mutation expectation suite observed
control (unmutated) green contentGenerator / loggingContentGenerator tests 21 passed / 74 passed
M1: break LazyContentGenerator construct-once (??==) KILLED (positive control) contentGenerator.test.ts 6 failed ✓ pinned
M2: LoggingContentGenerator.embedContent body → throw SURVIVES (coverage gap) loggingContentGenerator.test.ts 74 passed — survived ✓ as predicted
M3: LazyContentGenerator.embedContent body → throw KILLED contentGenerator.test.ts 14 failed ✓ pinned

Dist-level behavior checks (compiled head artifact)

HEAD 4/4 (embedContent returns wrapped response unchanged + forwards identical request object; generateContent forwards req + userPromptId; removed members undefined). BASE control arm: 3 green + 4th red exactly as intended (both members are function on base) — the A/B flip proving the removal load-bearing at the artifact level. Also confirmed: StandaloneSessionPicker.tsx has zero remaining getContentGenerator refs, so the deleted preview-stub member was dead weight.

Targeted gates

gate result
root npm run typecheck (incl. example package) exit 0
PR-touched core suite 13 files / 1632 tests passed, 0 failed
PR-touched CLI suite 2 files / 175 tests passed, 0 failed
check-workflow-size.sh at head exit 0
size-baseline consistency head records 42519 cd-cua-driver.yml = actual bytes; base recorded 29715 vs actual 42519 (latent Δ12,804 from #9587)

Corrections: the Reviewer Test Plan's "14 core files, 1,628 passing" reads 13 files / 1632 at the verified head — the PR itself deleted the 14th file; +4 tests from the main merge. CLI numbers match exactly.

Findings (both non-blocking)

F1 (Low — description accuracy; pre-existing): "embedding … still has an active call path" — generateEmbedding() has zero production callers at head (identical on base; the PR neither caused nor worsened it). Suggested fix is to the PR text, not the code; removing embedding too is a follow-up scope decision.

F2 (Low — coverage gap on surviving behavior, verified correct): no test drives LoggingContentGenerator.embedContent() (mutation M2 survives). This round's dist harness verified the forwarding correct; a one-line follow-up test would pin it.

Not covered

Per-provider wire harnesses for Anthropic/Gemini/Qwen (typecheck + unit suites only); per-commit attribution (aggregate diff verified); UI E2E (matches the PR's own note); the merit of #9587's workflow growth; embedding end-to-end reachability (no production caller exists by construction — decorator- and lazy-layer forwarding verified against the compiled artifact instead).

Methodology

Maintainer-local, macOS (darwin), Node v24.18.1. Two detached scratch worktrees (tmp/pr9676-base-tree @ base OID, tmp/pr9676-head-tree @ head OID — explicit OIDs from gh pr view, never HEAD^). Wire A/B: self-contained .mjs loopback OpenAI-compatible server + spawned node dist/cli.js --no-chat-recording --yolo --prompt …, env recipe from the repo's own integration tests; arms ran the identical scenario in an identical absolute run-dir (cleaned per arm); bundles additionally frozen as dist snapshots. Mutations applied to head sources only after builds completed, reverted with git checkout -- (tree verified clean). Gate suites ran in-package via npx vitest run <files>.

中文摘要

结论:可以合并(merge-ready),75/75 脚本断言全部通过,0 个意外失败。

  • A/B 主张成立:base 与 head 两个真实构建的 CLI 在相同 headless 场景(2 轮对话、假 OpenAI 兼容服务器)下产生的 4 个 wire 请求(约 422KB 请求体,含主请求与会话结束的 memory 提取子代理请求)经 UUID 归一化后逐字节一致;新跑的 npm run build && npm run bundle 产物复验同样一致。中心主张「移除 countTokens/useSummarizedThinking 与孤儿 tokenizer 集群不影响生成行为」在 OpenAI 兼容全链路(lazy → logging decorator → openai generator → wire)上被证明。
  • 死代码前提成立:base 上两个成员在生产代码中只有实现链自身引用;head 上残留引用为零(仅剩 vendored @types/vscode,与本仓库无关)。保留的 textTokenizer/supportedImageFormats 仍有真实消费方。
  • mutation 矩阵:破坏 lazy construct-once → 6 个测试变红(被钉住);破坏 lazy 层 embedContent 转发 → 14 个测试变红(被钉住);破坏 decorator 层 embedContent 转发 → 全绿存活(覆盖缺口),但用 dist 级 harness 对真实编译产物验证了该转发行为正确;base 对照臂上 countTokens/useSummarizedThinking 仍为 function、head 上为 undefined(A/B 翻转证据)。
  • 门禁全绿:typecheck exit 0;PR 触及的 core 测试 13 个文件 1632 用例全过(作者称 14 文件/1628:第 14 个文件 openaiTimeoutHandling.test.ts 被 PR 整体删除,+4 用例为合入 main 的漂移);CLI 2 个文件 175 用例与作者声称完全一致。.size-baseline 修正了 main 侧既有的未记录增长(29715→42519),size gate exit 0。
  • 两条非阻塞发现:① PR 描述称 embedding「仍有活跃调用链路」不准确——generateEmbedding() 在 head 全树无生产调用方(既有情况,非本 PR 引入),建议修正描述措辞;② decorator 层 embedContent 转发无测试钉住(本轮已用 harness 验证行为正确),建议后续补一条断言。
  • 未覆盖:Anthropic/Gemini/Qwen 三个 provider 仅以 typecheck+单测覆盖(未做逐 provider wire harness);按提交逐个归因未做(验证的是聚合 diff);UI E2E 未跑(与作者自述一致)。

@wenshao

wenshao commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ❌ not passed — the PR could not be built - workflow run

The PR could not be built because npm ci failed twice in a row before any verification started. This is treated as a PR failure verdict rather than an infrastructure failure.

中文 — 判定:❌ 不通过 · PR 构建失败

由于 npm ci 在验证开始前失败(连续两次),无法构建该 PR。判定为 PR 问题而非基础设施故障;安装日志见下方折叠块。

Install/build log

$ npm ci --prefer-offline --no-audit --progress=false --cache "$RUNNER_TEMP/npm-cache"
npm warn deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported
npm warn deprecated prebuild-install@7.1.3: No longer maintained. Please contact the author of the relevant native addon; alternatives are available.
npm warn deprecated node-domexception@1.0.0: Use your platform's native DOMException instead
npm warn deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated @humanwhocodes/object-schema@2.0.3: Use @eslint/object-schema instead
npm warn deprecated @humanwhocodes/config-array@0.13.0: Use @eslint/config-array instead
npm warn deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
npm warn deprecated eslint@8.57.1: This version is no longer supported. Please see https://eslint.org/version-support for other options.
npm warn deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported

> @qwen-code/qwen-code@0.22.0 postinstall
> patch-package

patch-package 8.0.1
Applying patches...
ink@7.0.3 ✔

> @qwen-code/qwen-code@0.22.0 prepare
> node scripts/prepare.js


> @qwen-code/qwen-code@0.22.0 build
> cross-env NODE_OPTIONS="--max-old-space-size=3072" node scripts/build.js


> @qwen-code/qwen-code@0.22.0 generate
> node scripts/generate-git-commit-info.js


> @qwen-code/qwen-code-core@0.22.0 build
> node ../../scripts/build_package.js

src/core/contentGenerator.test.ts(776,21): error TS2339: Property 'countTokens' does not exist on type 'ContentGenerator'.
node:internal/errors:983
  const err = new Error(message);
              ^

Error: Command failed: tsc --build
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at checkExecSyncError (node:child_process:916:11)
    at execSync (node:child_process:988:15)
    at file:///__w/qwen-code/qwen-code/scripts/build_package.js:38:1
    at ModuleJob.run (node:internal/modules/esm/module_job:343:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:681:26)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:117:5) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 7246,
  stdout: null,
  stderr: null
}

Node.js v22.23.2
npm error Lifecycle script `build` failed with error:
npm error code 1
npm error path /__w/qwen-code/qwen-code/packages/core
npm error workspace @qwen-code/qwen-code-core@0.22.0
npm error location /__w/qwen-code/qwen-code/packages/core
npm error command failed
npm error command sh -c node ../../scripts/build_package.js
node:internal/errors:983
  const err = new Error(message);
              ^

Error: Command failed: npm run build --workspace=packages/core
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at checkExecSyncError (node:child_process:916:11)
    at execSync (node:child_process:988:15)
    at file:///__w/qwen-code/qwen-code/scripts/build.js:89:3
    at ModuleJob.run (node:internal/modules/esm/module_job:343:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:681:26)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:117:5) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 7226,
  stdout: null,
  stderr: null
}

Node.js v22.23.2
prepare: npm run build exited with status 1
npm error code 1
npm error path /__w/qwen-code/qwen-code
npm error command failed
npm error command sh -c node scripts/prepare.js
npm error A complete log of this run can be found in: /__w/_temp/npm-cache/_logs/2026-08-24T07_28_07_376Z-debug-0.log

npm ci failed with exit code 1; retrying once.
$ npm ci --prefer-offline --no-audit --progress=false --cache "$RUNNER_TEMP/npm-cache"
npm warn deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
npm warn deprecated eslint@8.57.1: This version is no longer supported. Please see https://eslint.org/version-support for other options.

> @qwen-code/qwen-code@0.22.0 postinstall
> patch-package

patch-package 8.0.1
Applying patches...
ink@7.0.3 ✔

> @qwen-code/qwen-code@0.22.0 prepare
> node scripts/prepare.js


> @qwen-code/qwen-code@0.22.0 build
> cross-env NODE_OPTIONS="--max-old-space-size=3072" node scripts/build.js


> @qwen-code/qwen-code@0.22.0 generate
> node scripts/generate-git-commit-info.js


> @qwen-code/qwen-code-core@0.22.0 build
> node ../../scripts/build_package.js

src/core/contentGenerator.test.ts(776,21): error TS2339: Property 'countTokens' does not exist on type 'ContentGenerator'.
node:internal/errors:983
  const err = new Error(message);
              ^

Error: Command failed: tsc --build
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at checkExecSyncError (node:child_process:916:11)
    at execSync (node:child_process:988:15)
    at file:///__w/qwen-code/qwen-code/scripts/build_package.js:38:1
    at ModuleJob.run (node:internal/modules/esm/module_job:343:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:681:26)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:117:5) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 7489,
  stdout: null,
  stderr: null
}

Node.js v22.23.2
npm error Lifecycle script `build` failed with error:
npm error code 1
npm error path /__w/qwen-code/qwen-code/packages/core
npm error workspace @qwen-code/qwen-code-core@0.22.0
npm error location /__w/qwen-code/qwen-code/packages/core
npm error command failed
npm error command sh -c node ../../scripts/build_package.js
node:internal/errors:983
  const err = new Error(message);
              ^

Error: Command failed: npm run build --workspace=packages/core
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at checkExecSyncError (node:child_process:916:11)
    at execSync (node:child_process:988:15)
    at file:///__w/qwen-code/qwen-code/scripts/build.js:89:3
    at ModuleJob.run (node:internal/modules/esm/module_job:343:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:681:26)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:117:5) {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 7469,
  stdout: null,
  stderr: null
}

Node.js v22.23.2
prepare: npm run build exited with status 1
npm error code 1
npm error path /__w/qwen-code/qwen-code
npm error command failed
npm error command sh -c node scripts/prepare.js
npm error A complete log of this run can be found in: /__w/_temp/npm-cache/_logs/2026-08-24T07_29_51_773Z-debug-0.log

npm ci failed with exit code 1 after 2 attempts.

Qwen Code · sandboxed verification

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Triage re-run completed without a new review.

⚠️ The bot's only review on cf82579ac44c96a41fc42466b973d25e0b984a68 is a COMMENTED one, which carries no vote — so it has no verdict of its own on this commit, and main needs two approving reviews: an approval left by another account is a separate vote and does not count as the bot's own. Two different things look like this, and the stage-3 comment above says which: the triage skill deferring on purpose at 3/5 — a fork refactor hitting the approval guardrail, or a core change escalated for maintainer awareness, both normal outcomes — or an earlier approval that a push dismissed, leaving only the comment behind, which needs a fresh review.

⚠️ 机器人在 cf82579ac44c96a41fc42466b973d25e0b984a68 上唯一的评审是 COMMENTED不带票 —— 因此它在该 commit 上没有自己的裁决,而 main 需要两个批准(其他账号的批准是另一张票)。有两种情况长这样,上方的 stage-3 评论会说明是哪一种:triage skill 在 3/5 时有意 defer(fork refactor 命中审批护栏,或核心改动被升级交由维护者把关,两者都是正常结果);或者更早的批准被一次推送作废、只剩下这条评论,此时需要重新评审。

The stage comments above were updated with the latest result. View workflow run.

上方各阶段评论已更新为最新结果。查看工作流运行

@yiliang114 yiliang114 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Verified the removal is complete at head cf82579:

  • No residual production callers: a grep over the head tree finds zero countTokens( call sites in packages and no useSummarizedThinking code references (only historical mentions in the 2026-07-22 design doc, which this PR updates). The LazyContentGenerator constructor loses the summarizedThinking flag along with the interface member, and the StandaloneSessionPicker stub that faked useSummarizedThinking: () => false is removed with it.
  • Embedding stays intact as claimed: embedContent remains on the contract and on every provider implementation.
  • The request-tokenizer pruning is surgical: only the orphaned estimator cluster (imageTokenizer/requestTokenizer/index/types) is deleted; textTokenizer (still used by pdf.ts and the OpenAI converter) and supportedImageFormats (still used by fileUtils' PIPELINE_IMAGE_MIME_TYPES) stay, and the supportedImageFormats doc comment is updated to record the removal and the flat DEFAULT_IMAGE_TOKEN_ESTIMATE replacement.
  • Typecheck-level contract compliance is covered by CI, which is green at approval time (14 checks passing, none failing).

@DragonnZhang
DragonnZhang added this pull request to the merge queue Aug 24, 2026
Merged via the queue into QwenLM:main with commit 43d46be Aug 24, 2026
104 of 105 checks passed

@yiliang114 yiliang114 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the head at cf82579ac4. The contract narrowing is correct and complete — no blockers.

Verified on the head:

  • countTokens / useSummarizedThinking: zero residual references across production, tests, and types.
  • RequestTokenizer / ImageTokenizer were only consumed by the anthropic/openai countTokens implementations, so removing them as orphaned (along with request-tokenizer/index.ts + types.ts) is correct.
  • textTokenizer / supportedImageFormats and embedContent are retained with live call sites (converter.ts, pdf.ts, fileUtils.ts, baseLlmClient.ts).
  • LoggingContentGenerator still forwards generation + embedding unchanged.

Linked this PR to #4063 (item 1) in the body.

One observation, non-blocking: on this fork PR build-cli and precheck-pr (typecheck) report skipped, so CI only exercised Test (ubuntu-latest, Node 22.x) (green). The body states build+typecheck pass locally and reference consistency checks out statically, but letting precheck-pr run green once before merge would close that gap.

@yu-xin-c

Copy link
Copy Markdown
Contributor

Opened follow-up #9899 for the post-merge build failure already captured by sandboxed verification: the independently added Vertex ADC test still called the removed ContentGenerator.countTokens. The patch moves that one lazy-construction check to embedContent; clean npm ci/build, 27 focused tests, and workspace + integration typecheck pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants