Skip to content

feat(release): generate AI-assisted release notes - #6756

Merged
wenshao merged 7 commits into
QwenLM:mainfrom
yiliang114:codex/ai-release-notes-main
Jul 12, 2026
Merged

feat(release): generate AI-assisted release notes#6756
wenshao merged 7 commits into
QwenLM:mainfrom
yiliang114:codex/ai-release-notes-main

Conversation

@yiliang114

@yiliang114 yiliang114 commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

This PR adds AI-assisted release notes for stable releases. GitHub-generated notes remain the authoritative pull request set; the generator enriches those pull requests with their descriptions, labels, changed paths, and diff statistics, then produces concise user-facing summaries and up to six highlights.

The model cannot add, remove, or renumber pull requests. The generator validates the complete pull request set, output length, and plain-text formatting. An invalid summary falls back independently to its original pull request title, while a generator-level failure leaves the release workflow on GitHub's existing --generate-notes path. Nightly and preview releases are unchanged.

Generated notes carry a versioned marker so the changelog generator can reuse the same content with adjusted heading levels instead of producing a second, divergent summary.

Why it's needed

Current release notes and the aggregated changelog are primarily built from commit or pull request titles. They are complete and auditable, but larger releases are difficult to scan because they do not explain user impact or identify the most important changes.

This keeps the full linked pull request list while adding a non-blocking editorial layer. AI does not participate in version calculation, pull request inclusion, or release publication decisions.

Reviewer Test Plan

How to verify

  1. Configure OPENAI_API_KEY, OPENAI_BASE_URL, and OPENAI_MODEL, run the generator, and confirm the result contains Highlights, Breaking Changes, and a Complete Change List.
  2. Confirm every pull request from GitHub-generated notes appears exactly once in the Complete Change List and retains its pull request link, author attribution, and New Contributors credit.
  3. Remove the model configuration or return an invalid model response and confirm affected summaries safely fall back to the original pull request titles.
  4. Simulate an unsupported GitHub-generated-notes entry format and confirm generation fails, allowing the release workflow to retain its --generate-notes fallback.
  5. Confirm marked release notes are reused in CHANGELOG.md with headings demoted below the release version heading.
  6. Confirm nightly and preview releases continue to use the existing GitHub-generated-notes path.

Local verification:

  • npm run lint
  • npm run typecheck
  • npx vitest run scripts/tests/generate-release-notes.test.js scripts/tests/generate-changelog.test.js scripts/tests/ai-release-notes-workflow.test.js — 53/53 passed
  • actionlint -ignore SC2129 .github/workflows/release.yml
  • npm run build
  • Real v0.19.8...v0.19.9 smoke test with qwen3-coder-flash: 87/87 pull requests retained in the Complete Change List and six highlights generated

Evidence (Before & After)

N/A — this changes release automation and generated Markdown, not runtime UI.

Tested on

OS Status
🍏 macOS ✅ tested
🪟 Windows ⚠️ not tested
🐧 Linux ⚠️ not tested

Environment (optional)

Node.js 22, GitHub API, and an OpenAI-compatible qwen3-coder-flash provider.

Risk & Scope

  • Main risk or tradeoff: Summary and highlight quality depends on the configured model. Classification remains deterministic from pull request labels and Conventional Commit prefixes, so maintainers should still scan the generated highlights before publication.
  • Not validated / out of scope: This does not rewrite historical releases, let AI decide inclusion or classification, or send full diffs to the model. The AI path requires three repository secrets; without them, generation safely falls back.
  • Breaking changes / migration notes: None. Nightly releases, preview releases, and version calculation are unchanged.

Linked Issues

N/A

中文说明

What this PR does

这个 PR 为稳定版本增加 AI 辅助的 Release Notes 生成流程。GitHub generated notes 仍然是完整 PR 集合的权威来源;生成器读取 PR 描述、labels、变更路径和 diff 统计,为每个 PR 生成简短的用户视角摘要,并从完整列表中选择最多六条 Highlights。

模型不能增加、删除或重新编号 PR。生成器会验证完整 PR 集合、输出长度和纯文本格式;单条摘要不合规时只回退该条,生成器整体失败时则让发布 workflow 继续使用 GitHub 原有的 --generate-notes 路径。Nightly 和 preview 发布流程保持不变。

生成后的 Release Notes 带有版本化标记,CHANGELOG 生成器会复用同一正文并调整标题层级,避免 GitHub Release 和 CHANGELOG 产生两套不同内容。

Why it's needed

当前 Release Notes 和汇总 CHANGELOG 主要由 commit 或 PR 标题组成。列表虽然完整且可审计,但较大的版本很难快速阅读,因为它们没有说明用户影响,也没有突出最重要的变化。

这次调整保留完整、带链接的 PR 列表,同时增加一个非阻塞的编辑层。AI 不参与版本计算、PR 收录或发布决策。

Reviewer Test Plan

How to verify

  1. 配置 OPENAI_API_KEYOPENAI_BASE_URLOPENAI_MODEL,运行生成器并确认输出包含 Highlights、Breaking Changes 和 Complete Change List。
  2. 确认 Complete Change List 中每个 GitHub generated notes PR 恰好出现一次,并保留 PR 链接、author attribution 和 New Contributors credit。
  3. 移除模型配置或模拟无效模型响应,确认受影响的摘要安全回退到原始 PR 标题。
  4. 模拟不支持的 GitHub generated notes 条目格式,确认生成失败并由发布 workflow 保留 --generate-notes fallback。
  5. 确认带标记的 Release Notes 被 CHANGELOG.md 复用,并正确降低标题层级。
  6. 确认 nightly 和 preview 发布仍使用原有 GitHub generated notes 路径。

本地验证:

  • npm run lint
  • npm run typecheck
  • npx vitest run scripts/tests/generate-release-notes.test.js scripts/tests/generate-changelog.test.js scripts/tests/ai-release-notes-workflow.test.js — 53/53 passed
  • actionlint -ignore SC2129 .github/workflows/release.yml
  • npm run build
  • 使用真实 v0.19.8...v0.19.9 数据和 qwen3-coder-flash 完成 smoke test:Complete Change List 保留 87/87 个 PR,并生成六条 Highlights

Evidence (Before & After)

N/A — 这是发布自动化和 Markdown 内容生成变更,不涉及运行时 UI。

Tested on

OS Status
🍏 macOS ✅ tested
🪟 Windows ⚠️ not tested
🐧 Linux ⚠️ not tested

Environment (optional)

Node.js 22、GitHub API 和 OpenAI-compatible qwen3-coder-flash provider。

Risk & Scope

  • Main risk or tradeoff: 摘要和 Highlights 的质量取决于配置的模型。分类仍由 PR labels 和 Conventional Commit 前缀确定,因此发布前维护者仍应快速检查生成的 Highlights。
  • Not validated / out of scope: 不修改历史 Release,不让 AI 决定 PR 收录或分类,也不向模型发送完整 diff。AI 路径需要配置三个仓库 secret;未配置时会安全降级。
  • Breaking changes / migration notes: 无。Nightly、preview 和版本计算流程不变。

Linked Issues

N/A

@yiliang114

Copy link
Copy Markdown
Collaborator Author

Here is a concrete preview from a real v0.19.8...v0.19.9 dry run using qwen3-coder-flash. The input contained 87 pull requests; all 87 remained in the Complete Change List.

Before: title-only entries

- feat(core): render PDF pages to images when text extraction overflows or fails (#6585)
- fix(session): detect and mark broken history chains instead of silently truncating (#6502)
- feat: add `qwen update` and `/update` commands with auto-update support (#5780)

After: generated structure and summaries

## Highlights

- Sessions now clearly mark broken history chains and prevent silent truncation (#6502)
- Workspace runtimes support session ownership indexing for cross-runtime session loading (#6540)
- Scheduled tasks can now use preconditions to control isolated run timing (#6619)

## Breaking Changes

No known breaking changes.

## Complete Change List

### Features

- PDF reading now falls back to image rendering when text extraction fails or exceeds token limits, improving handling of scanned or dense PDFs. (#6585)
- Adds 'qwen update' and '/update' commands with auto-update functionality. (#5780)

### Bug Fixes

- Sessions with broken history chains now show a clear 'history lost' marker instead of silently truncating. (#6502)

The exact wording and highlight selection remain model-dependent, so maintainers should still scan that short section before publishing. The important invariant is deterministic: AI cannot add or omit pull requests, and invalid output falls back to the original title without blocking the release.

@qwen-code-ci-bot

qwen-code-ci-bot commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the PR! (Re-run after review fixes.)

Template looks good ✓

Problem: This is a feature, not a bug fix — adds AI-assisted release note generation for stable releases. The problem is real: current release notes are flat PR title lists that are hard to scan for large releases. The author provided a concrete before/after preview from a real v0.19.8...v0.19.9 dry run with 87/87 PRs retained.

Direction: Aligned with the project's release automation tooling — same family as scripts/generate-changelog.js. The design is non-blocking by construction: continue-on-error: true in the workflow, graceful fallback to --generate-notes, AI never controls PR inclusion or version calculation. Touches the release workflow and introduces three new repository secrets (OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_MODEL) — maintainer awareness required.

Size: Not applicable — no core module paths are touched. Files changed: scripts/ (new script + changelog modification), .github/workflows/. Production logic: ~792 lines (generate-release-notes.js: 707, generate-changelog.js: 20, release.yml: 22+25, main-ci-failure-issue.yml: 85). Test code: ~712 lines.

Approach: The scope feels right for the release notes feature itself. However, this PR also includes main-ci-failure-issue.yml (85 lines) + its test file (62 lines) — a separate workflow that auto-creates GitHub issues on main CI failures. This is a self-contained feature that isn't related to AI release notes. It works fine and is well-tested, but ideally would have been a separate PR for clarity. Not blocking — flagging for the author's awareness on future PRs. The release notes script follows existing conventions (scripts/release-script-utils.js, ESM, parseArgs, isMainModule, same validateRepo pattern). The changelog integration is minimal — a marker check and heading demotion in formatRelease().

Moving on to code review. 🔍

中文说明

感谢贡献!(review 修复后的重新审查。)

模板完整 ✓

问题:这是新功能而非 bug 修复——为稳定版本添加 AI 辅助 Release Notes 生成。问题是真实的:当前 Release Notes 是扁平的 PR 标题列表,大版本难以快速阅读。作者提供了 v0.19.8...v0.19.9 真实 dry run 的 before/after 预览,87/87 个 PR 完整保留。

方向:与项目现有的发布自动化工具对齐,和 scripts/generate-changelog.js 属于同一系列。设计上是非阻塞的:workflow 中 continue-on-error: true、回退到 --generate-notes、AI 不参与 PR 收录或版本计算。触及发布 workflow并引入三个新仓库 secret,需维护者关注。

规模:不涉及核心模块路径。变更文件:scripts/(新脚本 + changelog 修改)、.github/workflows/。生产代码约 792 行,测试代码约 712 行。

方案:Release Notes 功能本身范围合理。但此 PR 还包含了 main-ci-failure-issue.yml(85 行)及其测试文件(62 行)——一个在 main CI 失败时自动创建 issue 的独立 workflow。功能完整且测试充分,但与 AI Release Notes 无关,理想情况下应拆为独立 PR。不阻塞——仅提醒作者未来注意。Release Notes 脚本遵循现有约定。Changelog 集成简洁。

进入代码审查 🔍

Qwen Code · qwen3.7-max

@qwen-code-ci-bot

qwen-code-ci-bot commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Code Review (re-run)

The implementation is well-structured. generate-release-notes.js follows the same conventions as generate-changelog.js — ESM, shared release-script-utils.js for parseArgs/isMainModule, execFileSync for subprocess calls, same validateRepo pattern.

Security posture is solid. Model output is treated as untrusted and validated at multiple levels: validateModelText() rejects HTML entities, GFM autolinks (www.), email patterns, @mentions, #issue refs, Markdown formatting (**, __, `), and enforces single-line + length bounds. Prompts explicitly delimit data boundaries. The OpenAI completer uses AbortSignal.timeout for request timeouts and response_format: { type: 'json_object' } — no tools exposed.

Review fixes from the last round are clean:

  • parseModelJson() now strips fenced JSON wrappers before parsing — fixes silent degradation when models return json ... blocks.
  • Breaking Changes are excluded from the Complete Change List's category loop (if (category === 'Breaking Changes') continue) — fixes duplicate rendering.
  • CLI error handler checks error.message.startsWith('ERROR: ') before prefixing — fixes ERROR: ERROR: duplication.

Changelog integration is minimal and backward-compatible. toReleaseModel propagates body: raw.body || '', formatRelease checks for a curated marker with heading demotion (#####). The ENTRY_RE update to support with @collaborator credits matches GitHub's current output format. Historical releases flow through the existing parser unchanged.

main-ci-failure-issue.yml is self-contained — no code checkout, deduplicates by commit SHA marker, applies autofix labels. Clean workflow, no concerns.

No correctness bugs, security issues, or AGENTS.md violations found.

Testing

Unit Tests (61/61 passed)

$ npx vitest run scripts/tests/generate-release-notes.test.js scripts/tests/generate-changelog.test.js scripts/tests/ai-release-notes-workflow.test.js scripts/tests/main-ci-failure-issue-workflow.test.js

 RUN  v3.2.4 /home/github-runner/actions-runner-3/_work/qwen-code/qwen-code/.qwen/worktrees/triage

 ✓  scripts  tests/ai-release-notes-workflow.test.js (2 tests) 2ms
 ✓  scripts  tests/main-ci-failure-issue-workflow.test.js (4 tests) 2ms
 ✓  scripts  tests/generate-changelog.test.js (27 tests) 8ms
 ✓  scripts  tests/generate-release-notes.test.js (28 tests) 116ms

 Test Files  4 passed (4)
      Tests  61 passed (61)
   Start at  20:51:56
   Duration  424ms

Lint

$ npx eslint scripts/generate-release-notes.js scripts/generate-changelog.js
(clean — no output, exit 0)

$ npx eslint scripts/tests/generate-release-notes.test.js scripts/tests/generate-changelog.test.js scripts/tests/ai-release-notes-workflow.test.js scripts/tests/main-ci-failure-issue-workflow.test.js
(clean — no output, exit 0)

Script Help Output

$ node scripts/generate-release-notes.js --help

Generate AI-assisted release notes with a complete PR list.

Usage:
  node scripts/generate-release-notes.js --tag=<tag> --previous-tag=<tag> [options]

Options:
  --repo=<owner/name>            Repository (default: $GITHUB_REPOSITORY or QwenLM/qwen-code).
  --tag=<tag>                    Release tag to generate.
  --previous-tag=<tag>           Previous release tag.
  --target=<ref>                 Target commitish (default: HEAD).
  --output=<path>                Output file (default: release-notes.md).
  --dry-run                      Print Markdown instead of writing a file.
  -h, --help                     Show this help.

Real-Scenario Testing

This PR modifies release infrastructure (GitHub Actions workflow + changelog script), not user-facing CLI behavior. There is no interactive CLI command or TUI to drive in tmux. The --dry-run mode requires live GitHub API access (repos/.../releases/generate-notes + GraphQL metadata query), which times out in CI environments (>60s). The unit tests comprehensively cover the fixture-file path with injected metadata, including edge cases for fenced JSON, invalid model responses, overlong text, HTML injection, unknown PR references, collaborator credits, and curated changelog marker propagation. The author reports a successful real-world smoke test with 87 PRs in the PR description.

Qwen Code · qwen3.7-max

@qwen-code-ci-bot

qwen-code-ci-bot commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

This is a clean, well-thought-out addition to the release tooling. The code follows the project's existing script conventions (reuses release-script-utils.js, same ESM/parseArgs/isMainModule pattern), the security posture is solid (model output treated as untrusted and validated at multiple levels — HTML entities, autolinks, mentions, Markdown formatting all rejected), and the workflow integration is genuinely non-blocking: continue-on-error: true, file existence check before --notes-file, --generate-notes fallback for all failure/preview/nightly paths.

The review fixes from the last round are clean: fenced JSON stripping, Breaking Changes dedup in the Complete Change List, and ERROR: ERROR: prefix fix are all correct one-line fixes with corresponding tests.

61/61 unit tests pass, lint is clean on both production and test code.

The one scope note: main-ci-failure-issue.yml is bundled with the release notes feature. It's self-contained, well-tested, and works — but it's a separate feature that ideally belongs in its own PR. Not blocking.

The live dry-run against real GitHub API data still can't be verified in this CI environment (GraphQL metadata query times out at >60s), but the unit tests cover the fixture-file path thoroughly, and the author reports a successful 87-PR smoke test.

Approving. The maintainer should confirm the three new secrets (OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_MODEL) are configured in the repo before expecting AI-enriched output on the next stable release — without them, the fallback to categorized title-only output still works.

中文说明

这是一个干净、考虑周全的发布工具增强。代码遵循项目现有的脚本约定(复用 release-script-utils.js),安全性到位(模型输出在多个层面进行验证——HTML entity、autolink、mention、Markdown 格式均被拒绝),workflow 集成是非阻塞的——出问题时发布会继续使用 GitHub 标准的 --generate-notes

上一轮 review 的修复干净利落:fenced JSON 解析、Breaking Changes 在 Complete Change List 中的去重、ERROR: ERROR: 前缀修复都是正确的一行修复并配有对应测试。

61/61 单元测试通过,生产代码和测试代码 lint 均干净。

范围备注:main-ci-failure-issue.yml 与 Release Notes 功能捆绑在一起。它是自包含的、测试充分的、功能正常——但作为独立功能理想应放在单独的 PR 中。不阻塞。

CI 环境中仍无法验证对真实 GitHub API 数据的 dry-run(GraphQL 查询超时),但单元测试充分覆盖了 fixture 文件路径,作者报告了 87 个 PR 的真实 smoke test 成功。

批准。维护者应在下次稳定发布前确认三个新 secret 已配置——未配置时回退到分类标题列表仍然有效。

Qwen Code · qwen3.7-max

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship. ✅

@qwen-code-ci-bot qwen-code-ci-bot added scope/ci-cd Continuous integration/deployment type/enhancement Non-bug improvement or optimization labels Jul 12, 2026
wenshao
wenshao previously approved these changes Jul 12, 2026
@wenshao

wenshao commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /review

@github-actions

Copy link
Copy Markdown
Contributor
_Qwen Code review request accepted. Review is queued in [workflow run](https://github.com/QwenLM/qwen-code/actions/runs/29186990128)._

Comment thread scripts/generate-release-notes.js
Comment thread .github/workflows/release.yml
Comment thread scripts/generate-release-notes.js Outdated
Comment thread scripts/generate-changelog.js
Comment thread scripts/generate-release-notes.js

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no blockers. Suggestions are inline.

Comment thread scripts/generate-release-notes.js
Comment thread scripts/tests/generate-release-notes.test.js
Comment thread scripts/tests/generate-release-notes.test.js

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no blockers. Suggestions are inline.

Comment thread scripts/generate-release-notes.js
Comment thread scripts/generate-release-notes.js
Comment thread scripts/generate-release-notes.js
Comment thread scripts/generate-release-notes.js
Comment thread scripts/tests/main-ci-failure-issue-workflow.test.js
Comment thread scripts/tests/generate-changelog.test.js
Comment thread scripts/generate-release-notes.js
@yiliang114

Copy link
Copy Markdown
Collaborator Author

本轮按最小范围处理 review:

  • 当前 head 已覆盖 plain-text 安全边界:validateModelText() 会拒绝 HTML entity、GFM autolink、email 和 mention 形式,测试已覆盖 www.example.comsecurity@example.com&#x40;octocat
  • 已修复 fenced JSON 响应导致 AI summaries/highlights 无声降级的问题。
  • 已修复 Breaking Changes 在专门章节和 Complete Change List 中重复渲染的问题。
  • 已修复 CLI 参数错误输出 ERROR: ERROR: ... 的重复前缀问题。
  • workflow timeout-minutes: 5 在当前 head 已存在,确认后作为已处理关闭。

未纳入本轮的 suggestion:raw body 传播测试、CLI fake-gh 全链路增强、renderReleaseNotes() 额外 repo 校验、测试命名、OpenAI error 分支、main CI issue workflow 断言、curated changelog 额外合同测试、AI completer orchestration 覆盖。这些是覆盖/诊断/边界增强,不是当前 closeout 必须修复的 correctness 缺陷;为避免扩大 PR,本轮不继续加测试矩阵。

验证:

  • ./node_modules/.bin/vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/generate-release-notes.test.js
  • ./node_modules/.bin/prettier --check scripts/generate-release-notes.js scripts/tests/generate-release-notes.test.js
  • node --check scripts/generate-release-notes.js
  • git diff --check

@yiliang114
yiliang114 requested a review from wenshao July 12, 2026 12:07
@wenshao

wenshao commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship. ✅

@wenshao
wenshao added this pull request to the merge queue Jul 12, 2026
Merged via the queue into QwenLM:main with commit ae9e778 Jul 12, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope/ci-cd Continuous integration/deployment type/enhancement Non-bug improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants