fix(scripts): slim release-note model prompts and log request timing - #7941
Merged
Conversation
Collaborator
|
✅ Qwen Triage finished — CI landed green on ✅ Qwen Triage 已完成 —— |
Collaborator
Author
Test reportUnit testsThe new regression test pins the summaries payload to exactly Prompt-content experiment behind the 700-char / title+body decisionTo check that dropping file paths and size counts doesn't degrade output quality, I generated release notes for the full v0.20.1..v0.21.0 range (135 PRs) using only each PR's title and truncated body — the exact fields this PR keeps. The result below has complete Highlights and per-category coverage of all 135 PRs; nothing in it needed the file-list or line-count signal, and the category grouping matches what For scale: a 12-PR batch at 3,000 chars/body serializes to roughly 40KB of prompt; at 700 chars it is ~10KB. The per-request timing lines added here will show whether the remaining timeouts (#7523) correlate with prompt size or are flat 60s hangs. Generated v0.21.0 release notes (title + 700-char body only, 135 PRs)Highlights
Features
Bug Fixes
Performance
Documentation
Internal Changes
中文说明单元测试 52/52 通过，新增回归测试钉住模型载荷仅含 number、title、700 字符 body 摘录和 category。为验证去掉文件列表不影响质量，用 v0.20.1..v0.21.0 全量 135 个 PR 仅凭标题 + 截断描述生成了完整 release notes（见上方折叠内容）：Highlights 和分类覆盖完整，没有任何内容依赖文件路径或行数信号。单批 prompt 体积从约 40KB 降到约 10KB；新增的逐请求耗时日志将用于判断 #7523 的剩余超时是否与 prompt 大小相关。 |
Collaborator
|
Thanks for the PR! Template looks good ✓ Problem: observed bug with evidence — #7523 documents repeated timeouts in the AI release-notes step, and the PR includes a manual experiment over 135 PRs confirming titles + bodies are sufficient for classification. Direction: aligned. This is a CI-only script, the payload reduction directly addresses the timeout, and the timing logs fill a real observability gap. No CHANGELOG signal needed for internal tooling. Size: not applicable (no core paths touched). Approach: scope feels right — every edit serves the stated goal (drop unused metadata from the model prompt, add per-request timing). No drive-by changes. The 700-char body cap is a reasonable tradeoff backed by the experiment. Risk: no elevated risk signals. Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题：已观测到的 bug，有证据——#7523 记录了 AI release-notes 步骤反复超时，PR 包含对 135 个 PR 的人工实验，确认仅标题+描述即可完成分类。 方向：对齐。这是 CI 专用脚本，载荷缩减直接解决超时问题，耗时日志填补了真实的可观测性空白。 规模：不适用（未触及核心路径）。 方案：范围合理——每处改动都服务于目标（移除模型 prompt 中未使用的元数据、添加每请求耗时）。无顺手改动。700 字符描述上限有实验支撑。 风险：无升级风险信号。 进入代码审查 🔍 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
Code ReviewIndependent proposal for this problem: remove the unused The PR matches this exactly. Specific observations:
No correctness bugs, security issues, or convention violations found. CI Test EvidenceFinal CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 The ubuntu unit test suite is still running. macOS/Windows and integration tests are skipped (not triggered for this PR's paths). No failures so far. Testing: N/A for real-scenario testing — this is a CI-only script with no user-visible TUI surface. The unit test suite (52 tests per the PR description) covers the payload shape and enrichment logic. 中文说明代码审查独立方案：移除 GraphQL 查询和模型载荷中未使用的 PR 与独立方案完全一致：
未发现正确性、安全性或规范问题。 CI 测试证据Ubuntu 单元测试仍在运行中，暂无失败。macOS/Windows 和集成测试未触发。 真实场景测试：不适用——这是 CI 专用脚本，无用户可见 TUI 界面。 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
|
Confidence: 5/5 — focused, well-evidenced fix for a real CI timeout; would merge without hesitation. This is exactly the kind of PR that's easy to review and easy to maintain in six months. The problem is real (repeated timeouts in #7523), the experiment validates the approach (135 PRs classified correctly from titles + bodies alone), and the implementation is the minimal change: drop four unused fields, cap the body, add two log lines. The regression test pins the payload shape so nobody accidentally bloats it again. No reservations. The timing logs are a nice touch for the next timeout investigation without adding noise to normal operation (stderr only). Approval deferred until CI lands green on 中文说明置信度：5/5 — 聚焦、有证据支撑的 CI 超时修复；毫不犹豫即可合并。 问题真实（#7523 反复超时），实验验证了方案（135 个 PR 仅凭标题+描述即可正确分类），实现是最小改动：移除四个未使用字段、限制描述长度、添加两行日志。回归测试钉住载荷形状，防止未来意外膨胀。 无保留意见。批准延迟至 CI 在该提交上全绿后执行。 — Qwen Code · qwen3.8-max-preview Reviewed at |
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
approved these changes
Jul 28, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
qwen-code-ci-bot
approved these changes
Jul 28, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
No issues found. LGTM! ✅
中文说明
未发现问题。LGTM！✅
— qwen3.7-max via Qwen Code /review
github-actions
Bot
added
the
review/self-reported
yiliang114
enabled auto-merge
July 28, 2026 15:32
qqqys
approved these changes
Jul 28, 2026
Collaborator
Maintainer verification — local build + real end-to-end runI built and ran this locally against real production data rather than fixtures, to check the two claims the PR makes: that the payload really shrinks to roughly a quarter, and that the output is unaffected. Verdict: LGTM — safe to merge. Both claims reproduce exactly. Everything below is non-blocking. Verified at head How I testedI could not use the real model endpoint, so I built a harness that exercises the actual script end to end:
Note the two 1. The payload claim reproduces
The summary payload is 4.55× smaller — 22.0 % of before. The PR says "roughly a quarter"; measured on the real range it is slightly better than that. The GraphQL query also drops four fields per PR, so the metadata fetch gets cheaper against the GitHub API too — a nice side effect the PR does not claim. 2. The output is unchanged
3. The diagnostics do what they promiseSame run, first two model calls forced to HTTP 500:
4. Tests, lint, and non-vacuity
52/52 pass; prettier and eslint clean. I then checked the new regression test is not vacuous by reverting each behaviour one at a time:
The payload half of this PR is properly pinned. The logging half has no coverage at all — see finding A. 5. Is 700 characters still enough?This is the part with real judgement in it, so I measured it rather than trusting the sample.
Every one of the 129 bodies exceeds 700 characters (p50 is 4 727), so the cut is universal — but it lands well: 116 open directly with Findings (all non-blocking)A. The new timing logs ship with no test. Proven above by mutation: deleting either B. After this PR the highlights request becomes the largest single prompt in the run — and it is the one this PR does not shrink. The highlights call serialises all entries in one request ( C. Suggestion: log D. Nit: the excerpt window can be spent on the PR template. 3 of 129 bodies open with the maintainer-facing HTML comment ( E. Nit: PR description wording. It says each request logs 中文版本维护者验证 —— 本地构建 + 真实端到端运行我在本地构建并针对真实生产数据（而非 fixture）运行了这个 PR，重点核验它的两个主张：载荷是否真的缩到约四分之一，以及输出是否不受影响。 结论：LGTM，可以合并。 两个主张都精确复现。下面所有内容都不阻塞合并。 验证提交 测试方式我没有模型凭据，所以搭了一套真正跑通脚本的验证环境：
两行 1. 载荷缩减主张成立
摘要载荷缩小 4.55 倍，为原来的 22.0 %。PR 说「约四分之一」，实测比这还略好一些。 GraphQL 查询同时少取了每个 PR 的四个字段，因此对 GitHub API 的元数据抓取也更便宜——这是 PR 没有提及的额外收益。 2. 输出没有变化两次运行的 3. 诊断日志确实有用同一次运行，前两个模型请求强制返回 HTTP 500：
4. 测试、lint 与非空洞性52/52 通过；prettier 与 eslint 均干净。我又逐项回退行为，检验新增回归测试是否真的钉住了它声称的东西：
载荷瘦身这一半被测试牢牢钉住；日志这一半完全没有覆盖，见发现 A。 5. 700 字符够不够？这是真正需要判断的地方，所以我做了实测而不是只看样本。 129 个 PR 描述全部超过 700 字符（中位数 4 727），所以截断是普遍发生的——但截得很准：116 个直接以 发现（均不阻塞）A. 新增的耗时日志没有任何测试。 上面的变异测试已证明：删掉任一个 B. 这个 PR 之后，highlights 请求变成了整轮最大的单个 prompt——而它恰恰是本 PR 没有瘦身的那个。 highlights 调用把全部条目放进一个请求（ C. 建议：把 D. 小问题：摘录窗口可能被 PR 模板吃掉。 129 个描述里有 3 个以面向维护者的 HTML 注释开头（ E. 小问题：PR 描述用词。 描述里写日志是 |
Collaborator
|
Released in v0.21.1. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Shrinks the payload the AI release-notes generator sends per pull request to the title, a 700-character body excerpt, and the locally computed category. File paths, line counts, and label lists are no longer serialized into the model prompt, and the metadata query stops fetching the fields that no longer have a consumer. Every model request now also logs its elapsed time and prompt size on both success and failure.
Why it's needed
The AI-assisted step for recent stable releases timed out on nearly every request (#7523). Each summary batch packed 12 PRs with up to 3,000 characters of description plus up to 40 file paths and size counts into a single non-streaming request, and the run logs offered no way to tell whether a request was slowly generating or silently hung. A manual experiment over the full v0.20.1..v0.21.0 range (135 PRs) produced complete, well-categorized notes and highlights from titles and bodies alone — the classification the extra metadata supported is already derived locally from labels and conventional-commit prefixes before the prompt is built. Slimming the payload cuts each batch request to roughly a quarter of its former size, and the new per-request timing lines give the next timeout investigation the elapsed/size evidence it currently lacks.
Reviewer Test Plan
How to verify
Run the script suite:
npx vitest run scripts/tests/generate-release-notes.test.js --config scripts/tests/vitest.config.ts(52 tests). The new regression test pins the model payload to exactly number, title, a 700-character body excerpt, and category. With model credentials you can also run the generator with--dry-runfor any tag pair and confirm the stderr now carries oneModel summaries request succeeded/failed in Nms (prompt N chars).line per request.Evidence (Before & After)
N/A (CI-only script; output format is unchanged, only the model input and stderr diagnostics change).
Tested on
Environment (optional)
Unit tests only (vitest via the scripts project config).
Risk & Scope
Linked Issues
Closes #7523.
中文说明
本 PR 做了什么
将 AI release notes 生成器发送给模型的每个 PR 载荷缩减为标题、700 字符的描述摘录和本地计算的分类。文件路径、增删行数和标签列表不再序列化进模型 prompt，元数据查询也不再抓取这些已无消费者的字段。每次模型请求现在会在成功和失败时都记录耗时与 prompt 大小。
为什么需要
最近几个 stable release 的 AI 步骤几乎每个请求都超时（#7523）。每个摘要批次把 12 个 PR、每个最多 3000 字符描述外加最多 40 个文件路径和行数统计塞进一个非流式请求，且日志无法区分请求是在缓慢生成还是静默挂起。用 v0.20.1..v0.21.0 全量 135 个 PR 做的人工实验表明，仅凭标题和描述就能产出完整、分类准确的 notes 和 highlights——这些额外元数据支撑的分类本来就在构建 prompt 之前由标签和 conventional commit 前缀在本地推出。瘦身后每个批次请求约为原来的四分之一，新增的耗时日志为后续超时排查提供了目前缺失的耗时/大小证据。
评审验证
运行
npx vitest run scripts/tests/generate-release-notes.test.js --config scripts/tests/vitest.config.ts（52 个测试）。新增回归测试钉住模型载荷仅含 number、title、700 字符 body 摘录和 category。有模型凭据时也可对任意 tag 区间--dry-run，确认 stderr 每个请求输出一行耗时日志。风险与范围