Skip to content

fix(core): record one merged turn after MAX_TOKENS output recovery - #8980

Open
harjothkhara wants to merge 27 commits into
QwenLM:mainfrom
harjothkhara:oss-find/qwen-code-2026-08-08
Open

fix(core): record one merged turn after MAX_TOKENS output recovery#8980
harjothkhara wants to merge 27 commits into
QwenLM:mainfrom
harjothkhara:oss-find/qwen-code-2026-08-08

Conversation

@harjothkhara

@harjothkhara harjothkhara commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Keeps the durable transcript aligned with in-memory history after MAX_TOKENS output recovery. The recovery flow now defers assistant records, coalesces successful continuations into the one surviving model turn, and writes exactly one record from that turn once recovery settles. Deferred records stay bound to the exact history object they belong to, so retries, abandonment, redaction, and concurrent history replacement cannot persist or remove the wrong turn.

The same recovery boundary now preserves text, thought-block boundaries and signatures, media/code parts, tool calls, usage, and per-attempt Goal billing. Send admission is reserved synchronously, canceled queued sends release their reservation without allowing later sends to overtake, and terminal handling uses a bounded drain whose first terminal result wins.

Why it's needed

The transcript used by --resume and --continue was written once per successful stream attempt, while live history merged MAX_TOKENS continuations into one model turn. Resuming therefore rehydrated split, overlapping assistant turns that no longer matched the live session. Fixing the write side preserves the existing reconstruction contract and extends the record-vs-history invariant established for transport continuations in #8624 to output-token recovery.

Reviewer Test Plan

How to verify

From packages/core, run:

../../node_modules/.bin/vitest run src/core/llm-chat.test.ts src/core/turn.test.ts src/services/chatRecordingService.test.ts
../../node_modules/.bin/tsc --noEmit -p tsconfig.json

Confirm that escalation and non-escalation recovery, tool-call skip paths, failed and abandoned continuations, history replacement, signed and unsigned thought boundaries, non-text output, missing or delayed usage, terminal races, queued-send cancellation, and per-attempt Goal billing each leave one durable assistant record matching the surviving history turn. The tests also round-trip the produced records through the production transcript preparation and history reconstruction path.

Exact rebased head: 636 tests passed; package typecheck, targeted ESLint, Prettier, and git diff --check passed. The required grogu+mario private review was rerun on the exact head after all correctness fixes; both reviewers found the core recovery mechanics sound, and every in-scope correctness finding was resolved with a regression test.

A fresh-workspace npm run preflight completed install, formatting, lint, build, and typecheck successfully. Its aggregate test phase exited non-zero with 26,281 CLI tests passing and 19 unrelated failures, plus 22,525 core tests passing and 108 unrelated failures. The failures were confined to sandboxed-home EPERM, watcher/file-descriptor EMFILE, and existing UI timing paths; the affected production areas are not changed here. The exact touched suites above remain fully green.

Evidence (Before & After)

N/A — no UI layout changes. Under the old eager-recording behavior, the regression cases produce multiple assistant records for one live coalesced turn; with this change they produce exactly one record that reconstructs to the same ordered history content.

Tested on

OS Status
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

Environment (optional)

Node 24.16.0 with the repository toolchain and mocked content generators. A global qwen executable was not installed, so the requested live-CLI baseline could not be run; Linux and Windows validation is left to CI.

Risk & Scope

  • Main risk or tradeoff: this is a non-refactor core correctness fix with 1,274 additions and 205 deletions of production code (5,981 additions and 784 deletions across the full six-file diff), so it exceeds the 1,000-line core advisory and needs maintainer awareness. The 13 commits are intentionally separated by invariant for commit-by-commit review. After the existing review rounds, subsequent changes were restricted to correctness findings.
  • Deferral trades immediate persistence of a completed truncated attempt for one correct merged record. Cooperative cancellation flushes the surviving turn, but a hard process crash during the short recovery window can lose that partial turn.
  • If a continuation emits visible, persistable output before a provider error, that output is now retained in the surviving turn and the stream receives a synthetic terminal event instead of losing content by propagating the provider error. Citation metadata remains ephemeral, as on ordinary turns, because resumable history stores model content parts rather than response metadata.
  • Terminal draining is bounded to one second and requests cancellation when the bound wins; a non-cooperative provider is not guaranteed to terminate. Usage metadata arriving after that bound is intentionally not retained.
  • Goal spend remains accumulated per model attempt even though the durable transcript contains one coalesced turn. The record keeps the final resume-usable usage boundary, or the latest usable boundary when the final response omits one; unknown fields remain absent rather than being coerced to zero.
  • Recorded projection preserves text, signed and unsigned thought blocks, media/code parts, tool calls, and part metadata. Structured-output arguments remain redacted under the existing transcript privacy policy.
  • Existing malformed transcripts are not rewritten. A pre-existing headless-subagent consumer also clears accumulated output on continuation retry; that file is unchanged here and should be handled separately rather than widening this already-large fix.
  • Breaking changes / migration notes: none.

Linked Issues

Closes #8979

Related invariant: #8624

中文说明

本 PR 的作用

在 MAX_TOKENS 输出恢复后,使持久化 transcript 与内存 history 保持一致。恢复流程现在会延迟写入 assistant 记录,将成功的续写合并到最终保留的唯一 model 轮次中,并在恢复结束后只从该轮次写入一条记录。延迟记录会绑定到其所属的精确 history 对象,因此重试、中途放弃、脱敏或并发替换 history 都不会持久化或删除错误的轮次。

同一恢复边界现在会保留文本、思考块边界与签名、媒体/代码 part、工具调用、usage,以及按每次尝试累计的 Goal 计费。每次发送会同步预留队列位置;取消排队中的发送会释放其位置且不会让后续发送越过仍在运行的发送;终止处理采用有界排空,并以第一个终止结果为准。

为什么需要

--resume--continue 使用的 transcript 原先会为每次成功的流尝试各写一条记录,而实时 history 会把 MAX_TOKENS 续写合并成一个 model 轮次。因此恢复会话时会重新水合出拆分且重叠的 assistant 轮次,与实时会话不一致。修复写入侧可以保留现有重建契约,并把 #8624 为传输中断续写建立的“记录与 history 一致”不变式扩展到输出 token 恢复。

审阅测试计划

验证方法

packages/core 运行:

../../node_modules/.bin/vitest run src/core/llm-chat.test.ts src/core/turn.test.ts src/services/chatRecordingService.test.ts
../../node_modules/.bin/tsc --noEmit -p tsconfig.json

请确认升级与不升级恢复、工具调用跳过路径、失败和被放弃的续写、history 替换、带签名和不带签名的思考边界、非文本输出、缺失或延迟的 usage、终止竞态、排队发送取消,以及按尝试累计的 Goal 计费,都会留下与最终保留 history 轮次一致的一条持久化 assistant 记录。测试还会让生成的记录经过生产环境的 transcript 准备和 history 重建路径进行往返验证。

最终变基后的精确提交:636 个测试通过;包级 typecheck、目标 ESLint、Prettier 和 git diff --check 均通过。所有正确性修复完成后,又在精确提交上重新运行了必需的 grogu+mario 私有评审;两位评审都认为核心恢复机制可靠,所有范围内的正确性问题均已通过回归测试解决。

在全新 workspace 中运行的 npm run preflight 成功完成了安装、格式检查、lint、build 和 typecheck。其汇总测试阶段因环境问题以非零状态退出:CLI 有 26,281 个测试通过、19 个无关失败;core 有 22,525 个测试通过、108 个无关失败。失败仅涉及沙箱 home 目录的 EPERM、watcher/文件描述符 EMFILE 及既有 UI 时序路径,本 PR 未改动这些生产区域。上述精确受影响测试套件仍全部通过。

前后证据

N/A —— 没有 UI 布局改动。在旧的立即写入行为下,回归用例会为一个实时合并轮次生成多条 assistant 记录;修复后只生成一条记录,并可重建为相同的有序 history 内容。

测试平台

OS 状态
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

环境

Node 24.16.0、仓库自带工具链和 mocked content generator。系统未安装全局 qwen 可执行文件,因此无法运行要求的实时 CLI 基线;Linux 和 Windows 验证交由 CI。

风险与范围

  • 主要风险或取舍:这是 core 中的非重构正确性修复,生产代码新增 1,274 行、删除 205 行(完整六文件 diff 新增 5,981 行、删除 784 行),因此超过 1,000 行的 core 提示阈值,需要维护者知悉。13 个提交按不变式拆分,以便逐提交评审。现有多轮评审后,后续改动仅限正确性问题。
  • 延迟写入以“不立即持久化已完成的截断尝试”换取一条正确的合并记录。协作式取消会写入最终保留轮次,但若进程在短暂恢复窗口内硬崩溃,该部分轮次可能丢失。
  • 如果续写在 provider 报错前已经输出可见且可持久化的内容,该内容现在会保留在最终轮次中,流会收到合成的终止事件,而不会因传播 provider 错误而丢失内容。citation metadata 与普通轮次一样保持临时状态,因为可恢复 history 存储的是 model content part,而不是 response metadata。
  • 终止排空最多等待一秒,并在达到边界时请求取消;不保证不配合的 provider 一定终止。超过该边界才到达的 usage metadata 会按设计不予保留。
  • 虽然持久化 transcript 只保留一个合并轮次,Goal 花费仍按每次模型尝试累计。记录保留最终可用于恢复的 usage 边界;若最终响应缺少该边界,则使用最近一次可用边界。未知字段会保持缺失,而不会被强制填零。
  • 记录投影会保留文本、带签名和不带签名的思考块、媒体/代码 part、工具调用和 part 元数据。structured-output 参数仍按既有 transcript 隐私策略脱敏。
  • 已存在的异常 transcript 不会被重写。一个既有的 headless-subagent consumer 也会在续写重试时清除累计输出;本 PR 未改动该文件,应单独处理,而不是继续扩大这个已经较大的修复。
  • 破坏性变更/迁移说明:无。

关联 Issue

Closes #8979

相关不变式:#8624

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/self-reported The linked issue was opened by the PR author (self-reported)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(core): MAX_TOKENS output recovery leaves the durable transcript disagreeing with history, so --resume rehydrates duplicated turns

4 participants