Skip to content
Merged
218 changes: 218 additions & 0 deletions docs/design/2026-08-15-user-facing-release-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
# User-Facing Release Notes

## Problem

Stable release notes are a developer-facing PR list. `finalize-release.yml`
runs `scripts/generate-release-notes.js`, which buckets every merged PR into
commit-type sections (Features / Bug Fixes / Performance / Documentation /
Internal Changes) and rewrites each entry with a one-sentence model summary.
For users this reads as a wall of PRs:

- Entries are grouped by change _type_, not by the area a user cares about
(Web Shell, Desktop, multi-agent, model support).
- Styles mix: model sentences ("Adds standard OpenTelemetry…") sit next to
raw conventional-commit titles ("feat(serve): bound daemon ACP NDJSON
buffers") whenever a summary fell back, which reads as unedited tooling
output.
- Highlights repeat full-list entries nearly verbatim, adding length without
a second level of abstraction.
- No Chinese version, despite a large Chinese-speaking user base.
- UI changes ship without visuals even when the PR body already carries
Before/After screenshots.

Measured context (2026-08-15): v0.21.11 listed 49 PRs; only 2 of those PR
bodies contain images (~4%), and 3 of the last 60 merged PRs overall. Image
support is therefore best-effort decoration, never structure.

## Goals

1. Replace the type-bucketed PR list with a **themed digest**: model groups
changes into user-facing themes, each with a short intro and items.
2. Add a **Chinese digest** mirroring highlights and themes (PR-level list
stays English; PR titles are English by convention).
3. **Attach screenshots** from PR bodies to digest items when available,
degrade silently when not.
4. Lose no information and no robustness: the full PR list remains as a
collapsed appendix, and every model failure path keeps today's output.

## Non-Goals

- Translating the full PR list into Chinese.
- Changing nightly/preview notes (they never run the AI path).
- Sourcing images from anywhere other than the merged PR body.
- Editing the GitHub Release creation step in `release.yml` (it still
publishes GitHub-generated notes immediately; finalize rewrites later).

## Pipeline Recap

1. `release.yml` → `gh api …/releases/generate-notes` anchored at the
previous tag → `cap-release-notes.mjs` → `gh release create`.
2. `finalize-release.yml` → `generate-release-notes.js` parses the
GitHub-generated bullets, fetches PR bodies/labels via GraphQL, calls the
model (summaries in batches of 8, then highlights), renders Markdown, and
`gh release edit`s it in place. Marker: `<!-- qwen-release-notes:v1 -->`.
3. `npm run changelog` (`generate-changelog.js`) rebuilds CHANGELOG.md from
the GitHub Releases API; bodies starting with the marker are embedded
verbatim (headings demoted one level).

## Proposed Changes

### 1. Model content: summaries gain Chinese; new themes call

`scripts/generate-release-notes.js` keeps the batched summaries call and the
highlights call, and adds one **themes** call:

- Summaries response becomes
`{"summaries":[{"pr","summary","summaryZh"}]}`. English rules unchanged
(≤180 chars, plain text). `summaryZh` is Simplified Chinese, ≤120 chars,
technical identifiers (commands, settings, product names) stay English.
An invalid `summaryZh` falls back to the English summary for that entry
with a warning — the Chinese section never drops wholesale.
- Highlights response gains `textZh` (same limits as `summaryZh`).
- New themes call input: every entry's number, category, English and Chinese
summary. Response:

```json
{
"themes": [
{
"title": "Web Shell",
"titleZh": "Web Shell",
"intro": "…≤200 chars, optional…",
"introZh": "…",
"items": [8780, 8973]
}
]
}
```

Validation mirrors the existing summary/highlight guards: ≤8 themes,
title ≤40 chars, items reference known PRs, a PR appears in at most one
theme. PRs the model leaves unassigned are collected into a deterministic
catch-all theme rendered last ("Other Changes" / "其他变更").

All three calls share the existing retry/backoff/deadline machinery.
The themes call scales `max_tokens` with the entry count (capped at 8192);
summaries and highlights keep the fixed 4096 budget, which leaves headroom
for every reachable summaries batch (at most 8 entries × English + Chinese).

### 2. Rendering: v2 layout

```
<!-- qwen-release-notes:v2 -->

## Highlights

## Breaking Changes ← bilingual when present: English item plus an
indented Chinese line ("No known breaking
changes." stays English-only)

## <Theme title> ← intro + items; screenshots under items
## <Theme title> …

---

## 中文摘要

### 亮点 ← Chinese highlights
### <theme titleZh> ← introZh + Chinese items

<details><summary>Complete Change List (N pull requests)</summary>

### Features
- web-shell: improve compact tool activity ([#8973](…)) by @ytahdn
</details>

## New Contributors
**Full Changelog**: …compare/v0.21.11...v0.21.12
```

Decisions:

- **Block layout, not interleaved**: English digest on top, one `---`
divider, then `## 中文摘要`. Each audience reads one contiguous block;
GitHub's TOC and release page stay scannable.
- **Themes use `##`**, matching today's section weight; Chinese themes use
`###` under the `## 中文摘要` umbrella.
- **Appendix uses normalized raw titles**, not model summaries: strip the
`type(scope):` prefix to `scope: description` (same rule as
`generate-changelog.js` `formatEntry`), keep ` by @author` and co-author
Comment on lines +138 to +140

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R7-3: the committed design doc claims appendix title normalization uses the "same rule as generate-changelog.js formatEntry", but the two strip sets differ: formatEntry strips feat/refactor/revert/fix/perf/docs (6 types via TYPE_TO_SECTION), while normalizeAppendixTitle strips only APPENDIX_STRIP_TYPES = feat/fix/perf/docs (4). Probe at HEAD: refactor(core): rework session storage keeps its prefix in the appendix but is stripped in CHANGELOG.md; revert: likewise diverges (feat/fix agree in both — presumably how the wording slipped through). The 4-type set is deliberate in code (TYPE_CATEGORIES comment: Internal Changes headings don't name refactor/revert) — the doc, not the code, is wrong. — Concrete cost: a maintainer or agent aligning the code to this doc would strip type info from refactor/revert appendix entries under Internal Changes, the one place no heading conveys it.

Suggested change
- **Appendix uses normalized raw titles**, not model summaries: strip the
`type(scope):` prefix to `scope: description` (same rule as
`generate-changelog.js` `formatEntry`), keep ` by @author` and co-author
- **Appendix uses normalized raw titles**, not model summaries: strip the
`type(scope):` prefix to `scope: description` (same shape as
`generate-changelog.js` `formatEntry`, but the strip set is limited to
feat/fix/perf/docs — refactor/revert keep their prefix because the
Internal Changes heading does not name them), keep ` by @author` and co-author
中文说明

R7-3:已提交的设计文档声称附录标题归一化与 generate-changelog.js formatEntry 采用"相同规则",但两者的剥离集合并不相同:formatEntry 剥离 feat/refactor/revert/fix/perf/docs(经 TYPE_TO_SECTION 共 6 类),而 normalizeAppendixTitle 只剥离 APPENDIX_STRIP_TYPES = feat/fix/perf/docs(4 类)。HEAD 探针:refactor(core): rework session storage 在附录中保留前缀、在 CHANGELOG.md 中被剥离;revert: 同样出现分歧(feat/fix 两边一致——措辞大概因此溜过)。代码中的 4 类集合是有意为之(TYPE_CATEGORIES 注释:Internal Changes 标题不点名 refactor/revert)——错的是文档而非代码。— 具体代价:若有人按此文档对齐代码,会把 Internal Changes 下 refactor/revert 附录条目的类型信息剥掉,而那是唯一没有标题传达类型的位置。修复见 suggestion。

— qwen3.8-max via Qwen Code /review (v0.21.12)

credits. This kills the mixed-style problem deterministically and makes
the appendix independent of model availability. Category sub-headings
(Features / Bug Fixes / …) remain — the appendix is the developer view.
- **Highlights** keep the v1 shape (text + PR links); no bolding tricks,
since highlight text already names the capability.
- Author attribution stays in the appendix only; digest items show just the
text + PR link, keeping lines short.

### 3. Images from PR bodies

Deterministic extraction, no model involvement:

- Sources in the PR body (already fetched by the GraphQL query): Markdown
`![alt](url)`, `<img src="url">`, and bare image URLs.
- Host allowlist (https only): `github.com/user-attachments/`,
`user-images.githubusercontent.com`,
`private-user-images.githubusercontent.com`, and `raw.githubusercontent.com`
pinned to a 40-hex commit-SHA ref — a branch ref stays mutable after
publication, so its owner could swap the image in a shipped release.
Anything else is ignored — the release body must never become a hotlinking
vector. The camo image proxy is deliberately not allowed even though GitHub
serves it: its HMAC signs arbitrary external URLs without repository
binding, so admitting it would re-admit every excluded host.
- First two matches per entry; first eight images per release; images render
only under digest items (never in the collapsed appendix).

Measured coverage is ~4% of release PRs, so the extractor must be cheap and
its absence invisible: no images → identical output to the image-less case.

### 4. Fallback ladder

| Failure | Result |
| ------------------------------ | ------------------------------------- |
| No model config | Today's v1 render (titles only) |
| Summaries batch fails | Circuit breaker as today; titles used |
| Highlights call fails | Digest without a highlights section |
| Themes call fails | Whole note falls back to v1 render |
| One `summaryZh` invalid | That item shows English in 中文摘要 |
| A theme intro invalid | Intro dropped; theme itself kept |
| No Chinese produced anywhere | 中文摘要 block omitted entirely |
| Image extraction finds nothing | No image lines |

Every rung emits the existing `::warning::` annotations, so degradation is
visible in the Actions run without failing the release.

### 5. CHANGELOG.md handling

`generate-changelog.js` accepts markers `v1` and `v2`. For v2 bodies it:

- unwraps `<details><summary>…</summary>` into a heading and drops the
closing tag (a text changelog has no collapse affordance); the heading is
emitted at `##` so the demotion lands it at `###`, the same sibling rank
v1's `## Complete Change List` reaches, keeping one skeleton across v1/v2
releases in the same file,
- drops image lines and the `---` divider that precedes the Chinese
digest (release-page chrome),
- otherwise applies the existing heading demotion.

v1 bodies keep today's verbatim embedding.

## Files Affected

| File | Change |
| ---------------------------------------------- | ------------------------------------------- |
| `scripts/generate-release-notes.js` | prompts, themes call, extraction, v2 render |
| `scripts/generate-changelog.js` | v2 marker + details/image transform |
| `scripts/tests/generate-release-notes.test.js` | new coverage |
| `scripts/tests/generate-changelog.test.js` | v2 embedding coverage |

No workflow, package.json, or `cap-release-notes.mjs` changes: the body
size stays far below the 120,000-char cap, and the script's CLI contract is
unchanged.

## Open Questions

None blocking. Chinese phrasing quality is prompt-controlled and reviewed
per release; if it disappoints, tightening the summaries prompt is a
follow-up, not a design change.
39 changes: 36 additions & 3 deletions scripts/generate-changelog.js
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ const SECTION_ORDER = SECTIONS.map((section) => section.name);

/** Matches a stable `vX.Y.Z` tag (no `-preview` / `-nightly` suffix). */
const STABLE_TAG_RE = /^v?(\d+)\.(\d+)\.(\d+)$/;
const CURATED_RELEASE_MARKER = '<!-- qwen-release-notes:v1 -->';
/** Marker comment AI-assisted release notes start with (`v1`, `v2`, …). */
const CURATED_RELEASE_MARKER_RE = /^<!-- qwen-release-notes:v(\d+) -->/;

/**
* Matches a GitHub "What's Changed" bullet, e.g.
Expand Down Expand Up @@ -143,6 +144,34 @@ export function formatEntry(entry, cat = categorize(entry.title)) {
return `- ${text} ([#${entry.prNumber}](${entry.prUrl}))`;
}

/**
* Per-line adjustments applied before the heading demotion. v1 notes embed
* verbatim; v2 notes are a digest with a collapsed appendix and inline
* screenshots, and the changelog keeps the text while unwrapping the
* collapse and dropping the images and the Chinese-digest divider.
*/
export function transformCuratedLine(line, version) {
if (version < 2) {
return [line];
}
if (/^\s*!\[[^\]]*\]\(/.test(line)) {
return [];
}
Comment on lines +157 to +159

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R6-9: this v2 image-drop matches any line merely beginning with the image-open prefix — strictly broader than the renderer's actual image lines (two-space-indented, closed paren, bracket-free alt). validateModelText rejects inline links only when the paren closes, so a malformed theme intro ![x](y passes every gate; the release page then shows it as literal text while this transform drops it from CHANGELOG.md — the two artifacts diverge with no ::warning:: (this script has no warning machinery at all). — Failure scenario (executed end-to-end at HEAD): a malformed model intro ![x](y (rare — but malformed model output is exactly what this machinery handles) renders verbatim in the published release, silently absent from CHANGELOG.md.

Witness (probe at HEAD): transformCuratedLine('![x](y', 2)[] while the true image line ' ![alt](https://u)'[] too; generateAiContent keeps the intro with warnings: []. Flip: the tightened regex keeps ![x](y) in the changelog while still dropping every real image line.

Suggested change
if (/^\s*!\[[^\]]*\]\(/.test(line)) {
return [];
}
if (/^\s*!\[[^\]]*\]\([^)]*\)\s*$/.test(line)) {
return [];
}
中文说明

R6-9:该 v2 图片剥离规则匹配任何以图片起始前缀开头的行——比渲染器实际输出的图片行(两空格缩进、闭合括号、无中括号的 alt)严格更宽。validateModelText 仅在括号闭合时拒绝行内链接,因此畸形主题导语 ![x](y 能通过全部门禁;release 页面把它当字面文本展示,而该转换把它从 CHANGELOG.md 中丢弃——两个产物出现分歧且没有任何告警(本脚本根本没有告警机制)。— 失败场景(已在 HEAD 端到端执行):畸形模型导语 ![x](y 原样出现在发布的 release 中,却静默缺席于 CHANGELOG.md。建议:收紧为匹配渲染器真实输出的形状(闭合括号、整行)。

— qwen3.8-max via Qwen Code /review (v0.21.12)

if (/^\s*---\s*$/.test(line)) {
return [];
}
if (/^\s*<\/?details>\s*$/.test(line)) {
return [];
}
Comment on lines +163 to +165

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R8-26: the cross-file contract transformCuratedLine depends on — renderReleaseNotesV2 emitting <details> and <summary>…</summary> as two SEPARATE lines — is pinned by zero tests on either side, while the design doc's layout sketch draws the combined one-line form. — Failure scenario: the changelog arms match only standalone lines; the renderer-side tests assert only unanchored substrings (toContain('<details>')), and the changelog-side test uses a hand-written two-line fixture. If a maintainer aligns the renderer with the doc sketch, every test on both sides stays green, but in production the combined line matches no arm, skips heading demotion, and leaks raw <details><summary>… HTML into CHANGELOG.md with the appendix never unwrapped.

Suggested fix: pin the contract on one side: build the changelog v2 test's fixture body by calling renderReleaseNotesV2(...), or add a release-notes assertion anchored on the two-line shape (e.g. toContain('<details>\n<summary>')).

中文说明

R8-26:transformCuratedLine 依赖的跨文件契约——renderReleaseNotesV2<details><summary>…</summary> 作为两个独立行输出——两侧都没有测试固化,而设计文档的布局草图画的却是合并的单行形态。— 失败场景:changelog 的各分支只匹配独立成行的内容;渲染器侧测试只做未锚定的子串断言(toContain('<details>')),changelog 侧测试使用手写的双行夹具。若 maintainer 按文档草图把渲染器改成单行,两侧所有测试仍全绿,但生产中合并行匹配不到任何分支、跳过标题降级,原始 <details><summary>… HTML 泄漏进 CHANGELOG.md,附录也不会被展开。建议:在其中一侧固化契约:让 changelog v2 测试的夹具正文直接调用 renderReleaseNotesV2(...) 生成,或在 release-notes 侧补锚定双行形态的断言(例如 toContain('<details>\n<summary>'))。

— qwen3.8-max via Qwen Code /review (v0.21.12)

const summary = /^\s*<summary>([\s\S]*?)<\/summary>\s*$/.exec(line);
if (summary) {
// Emit at ## so the heading demotion below lands the unwrapped appendix
// at ### — the same sibling rank v1's "Complete Change List" reaches.
return [`## ${summary[1]}`];
}
Comment on lines +166 to +171

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-7: the unwrapped appendix heading is emitted at ###, which formatRelease's #{2,5} demotion pushes to #### — one level deeper than the section's true rank: in CHANGELOG.md the Complete Change List nests under the preceding section (### 中文摘要, or the last theme) instead of being a sibling like v1's ## Complete Change List (which demotes to ###). Failure scenario: probe-confirmed A/B through the real formatRelease — v1 skeleton: ### Complete Change List (sibling); v2 skeleton: #### Complete Change List (2 pull requests) at the same rank as #### 亮点, nested under ### 中文摘要. Outline parsers attribute the entire change list to the preceding section; the same logical section sits at different ranks across v1/v2 releases within one file. The new changelog test pins #### Complete Change List, codifying rather than arguing the behavior; the design doc argues the unwrap, not the level. Suggested fix: emit at the rank that lands as a sibling after demotion (coordinated with the #### Features nesting beneath it) — or, if the deeper rank is intended, state it in the comment/design doc.

中文说明

R1-7:展开后的附录标题以 ### 输出,而 formatRelease#{2,5} 降级把它推到 ####——比该章节的真实层级深一级:在 CHANGELOG.md 中,Complete Change List 嵌套在前一个章节(### 中文摘要 或最后一个主题)之下,而不是像 v1 的 ## Complete Change List(降级为 ###)那样平级。触发场景:对真实 formatRelease 的 A/B 探针——v1 骨架:### Complete Change List(平级);v2 骨架:#### Complete Change List (2 pull requests)#### 亮点 同级、嵌套在 ### 中文摘要 之下。大纲解析器会把整个变更列表归入前一章节;同一逻辑章节在同一文件的 v1/v2 release 中层级不一致。新的 changelog 测试把 #### Complete Change List 固化下来,是“记录”而非“论证”该行为;设计文档只论证了展开、未论证层级。建议修复:按降级后能落到平级的层级输出(与其下 #### Features 的嵌套协调)——或者,如果更深层级是有意为之,请在注释/设计文档中写明。

— qwen3.8-max via Qwen Code /review (v0.21.12)

return [line];
}

/** Render one release as a Markdown block. */
export function formatRelease(release) {
const lines = [];
Expand All @@ -151,10 +180,14 @@ export function formatRelease(release) {
: `## [${release.version}] - ${release.date}`;
lines.push(heading, '');

if (release.body?.trimStart().startsWith(CURATED_RELEASE_MARKER)) {
const marker = CURATED_RELEASE_MARKER_RE.exec(release.body?.trimStart());
if (marker) {
const version = Number(marker[1]);
const curated = release.body
.trimStart()
.split(/\r?\n/)
.filter((line) => line.trim() !== CURATED_RELEASE_MARKER)
.filter((line) => !CURATED_RELEASE_MARKER_RE.test(line))
.flatMap((line) => transformCuratedLine(line, version))
.map((line) => line.replace(/^(#{2,5})(\s+)/, '#$1$2'))
.join('\n')
.trim();
Expand Down
Loading
Loading