fix(core): record one merged turn after MAX_TOKENS output recovery - #8980
Open
harjothkhara wants to merge 27 commits into
Open
fix(core): record one merged turn after MAX_TOKENS output recovery#8980harjothkhara wants to merge 27 commits into
harjothkhara wants to merge 27 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Keeps the durable transcript aligned with in-memory history after MAX_TOKENS output recovery. The recovery flow now defers assistant records, coalesces successful continuations into the one surviving model turn, and writes exactly one record from that turn once recovery settles. Deferred records stay bound to the exact history object they belong to, so retries, abandonment, redaction, and concurrent history replacement cannot persist or remove the wrong turn.
The same recovery boundary now preserves text, thought-block boundaries and signatures, media/code parts, tool calls, usage, and per-attempt Goal billing. Send admission is reserved synchronously, canceled queued sends release their reservation without allowing later sends to overtake, and terminal handling uses a bounded drain whose first terminal result wins.
Why it's needed
The transcript used by
--resumeand--continuewas written once per successful stream attempt, while live history merged MAX_TOKENS continuations into one model turn. Resuming therefore rehydrated split, overlapping assistant turns that no longer matched the live session. Fixing the write side preserves the existing reconstruction contract and extends the record-vs-history invariant established for transport continuations in #8624 to output-token recovery.Reviewer Test Plan
How to verify
From
packages/core, run:Confirm that escalation and non-escalation recovery, tool-call skip paths, failed and abandoned continuations, history replacement, signed and unsigned thought boundaries, non-text output, missing or delayed usage, terminal races, queued-send cancellation, and per-attempt Goal billing each leave one durable assistant record matching the surviving history turn. The tests also round-trip the produced records through the production transcript preparation and history reconstruction path.
Exact rebased head: 636 tests passed; package typecheck, targeted ESLint, Prettier, and
git diff --checkpassed. The required grogu+mario private review was rerun on the exact head after all correctness fixes; both reviewers found the core recovery mechanics sound, and every in-scope correctness finding was resolved with a regression test.A fresh-workspace
npm run preflightcompleted install, formatting, lint, build, and typecheck successfully. Its aggregate test phase exited non-zero with 26,281 CLI tests passing and 19 unrelated failures, plus 22,525 core tests passing and 108 unrelated failures. The failures were confined to sandboxed-homeEPERM, watcher/file-descriptorEMFILE, and existing UI timing paths; the affected production areas are not changed here. The exact touched suites above remain fully green.Evidence (Before & After)
N/A — no UI layout changes. Under the old eager-recording behavior, the regression cases produce multiple assistant records for one live coalesced turn; with this change they produce exactly one record that reconstructs to the same ordered history content.
Tested on
Environment (optional)
Node 24.16.0 with the repository toolchain and mocked content generators. A global
qwenexecutable was not installed, so the requested live-CLI baseline could not be run; Linux and Windows validation is left to CI.Risk & Scope
Linked Issues
Closes #8979
Related invariant: #8624
中文说明
本 PR 的作用
在 MAX_TOKENS 输出恢复后,使持久化 transcript 与内存 history 保持一致。恢复流程现在会延迟写入 assistant 记录,将成功的续写合并到最终保留的唯一 model 轮次中,并在恢复结束后只从该轮次写入一条记录。延迟记录会绑定到其所属的精确 history 对象,因此重试、中途放弃、脱敏或并发替换 history 都不会持久化或删除错误的轮次。
同一恢复边界现在会保留文本、思考块边界与签名、媒体/代码 part、工具调用、usage,以及按每次尝试累计的 Goal 计费。每次发送会同步预留队列位置;取消排队中的发送会释放其位置且不会让后续发送越过仍在运行的发送;终止处理采用有界排空,并以第一个终止结果为准。
为什么需要
--resume和--continue使用的 transcript 原先会为每次成功的流尝试各写一条记录,而实时 history 会把 MAX_TOKENS 续写合并成一个 model 轮次。因此恢复会话时会重新水合出拆分且重叠的 assistant 轮次,与实时会话不一致。修复写入侧可以保留现有重建契约,并把 #8624 为传输中断续写建立的“记录与 history 一致”不变式扩展到输出 token 恢复。审阅测试计划
验证方法
在
packages/core运行:请确认升级与不升级恢复、工具调用跳过路径、失败和被放弃的续写、history 替换、带签名和不带签名的思考边界、非文本输出、缺失或延迟的 usage、终止竞态、排队发送取消,以及按尝试累计的 Goal 计费,都会留下与最终保留 history 轮次一致的一条持久化 assistant 记录。测试还会让生成的记录经过生产环境的 transcript 准备和 history 重建路径进行往返验证。
最终变基后的精确提交:636 个测试通过;包级 typecheck、目标 ESLint、Prettier 和
git diff --check均通过。所有正确性修复完成后,又在精确提交上重新运行了必需的 grogu+mario 私有评审;两位评审都认为核心恢复机制可靠,所有范围内的正确性问题均已通过回归测试解决。在全新 workspace 中运行的
npm run preflight成功完成了安装、格式检查、lint、build 和 typecheck。其汇总测试阶段因环境问题以非零状态退出:CLI 有 26,281 个测试通过、19 个无关失败;core 有 22,525 个测试通过、108 个无关失败。失败仅涉及沙箱 home 目录的EPERM、watcher/文件描述符EMFILE及既有 UI 时序路径,本 PR 未改动这些生产区域。上述精确受影响测试套件仍全部通过。前后证据
N/A —— 没有 UI 布局改动。在旧的立即写入行为下,回归用例会为一个实时合并轮次生成多条 assistant 记录;修复后只生成一条记录,并可重建为相同的有序 history 内容。
测试平台
环境
Node 24.16.0、仓库自带工具链和 mocked content generator。系统未安装全局
qwen可执行文件,因此无法运行要求的实时 CLI 基线;Linux 和 Windows 验证交由 CI。风险与范围
关联 Issue
Closes #8979
相关不变式:#8624