feat(core): checkpoint long-running Goal evidence - #8465
Conversation
|
Re-run on the current head — gate re-checked after the latest fix round. Template ✓ — all sections present, bilingual body. Problem: observed, not theoretical. #8430 (merged) stops the retry loop after catalog truncation but deliberately leaves the Goal revision uncompletable, and open #8458 is a live user report of exactly that dead end (Goal completion channel permanently locked by a truncated evidence catalog). This PR is the declared second stage of that fix. Direction: aligned. It completes the two-stage design already endorsed by merging #8430, and stays inside the Goal system's existing fail-closed model rather than inventing a new one. Compaction of long-running agent state is also a live product area upstream — Claude Code's CHANGELOG carries repeated auto-compaction and post-compaction recovery entries — so this is not a one-off concern. Size: touches core paths. 1,508 production logic lines (1,374 added + 134 deleted) vs 3,030 test lines; no generated/schema files. That clears both the 500-line maintainer-awareness bar and the 1,000-line large-PR advisory, so it is flagged for maintainer awareness. It is a Approach: the scope looks right — the diff does one thing (the checkpoint lifecycle) plus the minimum display plumbing it forces: a cause-aware bookkeeping-suppression predicate shared by ACP replay and session resume, and hiding the new Risk: the changed set touches Moving on to code review. 🔍 中文说明在当前 head 上重跑——最新一轮修复后重新过 gate。 模板 ✓ ——各节齐全，中英双语。 **问题：**已观测到，不是理论问题。#8430（已合并）在目录截断后停止了重试循环，但有意未让该 Goal revision 恢复可完成性；仍开启的 #8458 正是这个死路的真实用户报告（Goal 完成通道被 truncated 的证据目录永久锁定）。本 PR 是该修复声明的第二阶段。 **方向：**对齐。它补全了合并 #8430 时已认可的两阶段设计，并沿用 Goal 系统既有的 fail-closed 模型而非另起炉灶。长时运行 agent 状态的压缩在上游也是活跃方向——Claude Code 的 CHANGELOG 中有多条 auto-compaction 与压缩后恢复的记录——因此这不是孤立问题。 **规模：**触及核心路径。生产逻辑 1,508 行（+1,374 / −134），测试 3,030 行；无生成/schema 文件。同时超过 500 行维护者关注线与 1,000 行大 PR 建议线，因此标记供维护者知悉。它是 **方案：**范围合理——diff 只做一件事（checkpoint 生命周期），加上它必需的最小展示层管道：一个 cause 感知的 bookkeeping 抑制谓词（ACP replay 与 session resume 共用），以及在 CLI 滚屏中隐藏新的 **风险：**改动集触及 进入代码审查。🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code reviewBefore reading the diff I sketched my own solution: compact verified evidence with the existing tool-free side-query mechanism before the cap, persist a pending-checkpoint marker in the journal itself (no new storage), validate every claim against known sources, fail closed, and keep the three-turn raw lineage for repeated blockers. This PR's approach matches that baseline closely, and goes further where it matters — crash recovery by replaying No critical blockers at this commit. The earlier review rounds' Criticals are all verifiably fixed here, not just claimed: the replay/resume bookkeeping suppression is now cause-aware through the shared Two things worth maintainer attention, neither blocking. First, the The remaining deferred Suggestion-level items from the autofix rounds (default-timeout pinning, The flow below is the piece a reviewer should sanity-check first: sequenceDiagram
participant P1 as Goal worker
participant P2 as Goal runtime
participant P3 as Journal
participant P4 as Checkpoint verifier
P1->>P2: finishTurn
P2->>P3: persist turn_finished with checkpointPending
P2->>P2: build evidence window
alt window truncated or request too large
P2->>P3: recordCheckpointFailure, usage_limited
else below threshold or transient verifier failure
P2->>P3: bookkeeping record, goal stays active
else at threshold
P2->>P4: compress evidence into bounded claims
P4-->>P2: validated claims
P2->>P3: recordCheckpoint, advance cursor
P2->>P2: queued user turn wins, else auto-continue
end
Files changed (all 28 shown)
Test evidence (the PR's own CI, read via API — no PR code executed)All checks on the reviewed commit are settled. The unit suite — the only test lane that runs on PR pushes — is green.
The standing Sandboxed verification would settle the remaining behavioural gap: Not verified: runtime behaviour on Windows and Linux (author tested macOS only); merge-queue lanes (structurally unreachable from a PR push). 中文说明代码审查读 diff 之前我先独立构思了方案：在上限前用既有的无工具 side-query 机制压缩已验证证据、把 pending checkpoint 标记持久化在 journal 本身（不新增存储层）、逐条校验 claim 来源、fail closed、并为 repeated blocker 保留三个原始 turn 的 lineage。本 PR 的方案与该基线高度一致，并在关键处更进一步—— **当前 commit 无致命阻塞项。**此前各轮的 Critical 均已被可验证地修复（而非仅声称）：replay/resume 的 bookkeeping 抑制改为 cause 感知（共享 两点提请维护者注意，均不阻塞。其一，autofix 轮留下的 autofix 轮遗留的 Suggestion 级事项（默认超时固化、 测试证据（PR 自身 CI，经 API 读取——未执行任何 PR 代码）被审 commit 上所有 check 已出结果。PR push 通道唯一会跑的测试通道——单元套件——为绿。macOS/Windows 单元与 CLI 集成测试显示 skipped，但这是结构性的：
沙箱验证可以补齐剩余的行为缺口： 未验证：Windows 与 Linux 上的运行行为（作者仅测 macOS）；merge queue 通道（PR push 结构上不可达）。 — Qwen Code · qwen3.8-max Reviewed at |
|
Confidence: 3/5 — clean review at every stage on this head, but the core-size awareness escalation caps the gate's score by policy, so the withheld approval is the rule working, not doubt. Stepping back: this is the rare large PR where the size is honest. The approach matches the one I'd have proposed before reading the diff — journal-native pending-checkpoint persistence, tool-free bounded verification, fail-closed semantics — and the 3,000+ test lines are mostly lifecycle assertions (crash recovery, queued-user priority, verifier-reject recovery), not padding. Every Critical from the earlier review rounds is fixed in a way I could verify at this commit, including the subtle ones (trim-equivalent truncation probe, cause-aware bookkeeping suppression, symmetric byte accounting between materialization and the persisted validator). If I had to maintain this in six months, the comments explaining why each failure mode settles the way it does would save me, and the fail-closed defaults mean the worst case is a halted Goal a user can resume or edit, never corrupted evidence. Reservations, stated plainly: this is ~1,500 production lines in Core from a fork, which is exactly the shape the two-tier gate says a bot must not sign off on alone — hence the cap and the defer, regardless of how clean the passes were. The residual ⏸️ Deferring to @wenshao — the core-size awareness escalation requires a human call, and yours is the most recent review on this PR. For context: you already approved this exact commit ( 中文说明置信度：3/5 ——当前 head 上各阶段审查均为干净，但核心规模知会升级按政策为 gate 评分设了上限，因此保留批准是规则使然，并非存疑。 退一步看：这是一个规模诚实的大型 PR。方案与我在读 diff 之前独立构思的一致——journal 原生的 pending checkpoint 持久化、无工具的有界校验、fail-closed 语义——且 3,000 多行测试多为生命周期断言（崩溃恢复、排队用户优先、verifier 拒绝后的恢复），并非充数。此前各轮的所有 Critical 在本 commit 上均已被可验证地修复，包括微妙的那些（trim 等价的截断探测、cause 感知的 bookkeeping 抑制、materialize 与持久化校验器之间对称的字节口径）。如果六个月后由我维护，那些解释每种失败模式为何如此收敛的注释会帮上大忙；而 fail-closed 的默认行为意味着最坏情况只是 Goal 被暂停、用户可 resume 或编辑，证据永远不会损坏。 坦率地说，保留意见在于：这是来自 fork 的约 1,500 行 Core 生产代码，正是双层 gate 规定机器人不得单独背书的形态——因此无论各轮多么干净，都要设上限并转交。 ⏸️ 转交 @wenshao ——核心规模知会升级需要人工决断，而你是本 PR 最近的评审者。供参考：你已批准过这个确切的 commit（ — Qwen Code · qwen3.8-max Reviewed at |
|
Qwen Code review timed out. Qwen review timed out after 21600 seconds (of the 360-minute budget). This run already used the maximum 360 minute timeout. See workflow logs. |
|
已修复：证据目录 truncated 时，blocked 终态 proposal 不再绕过 fail-closed 或进入 terminal verifier，而是直接转为 usage_limited，并停止 continuation。验证证据：独立 public-runtime 复现确认旧行为会错误进入 blocked；修复后 323 个 Goal 测试、Core typecheck、repository build、Prettier/ESLint 均通过。 |
|
Qwen Code review timed out. Qwen review timed out after 21600 seconds (of the 360-minute budget). This run already used the maximum 360 minute timeout. See workflow logs. |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline. Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
中文说明
已审查。 建议见行内评论。 未审查：build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| evidenceCursor: copyCursor(transition.cursor), | ||
| evidenceCheckpoint: undefined, | ||
| lastReason: undefined, |
There was a problem hiding this comment.
[Suggestion] The edit branch's evidenceCheckpoint: undefined clearing has no paired test — deleting this line keeps the whole suite green (verified by mutant). The reducer is the only enforcement point that clears the checkpoint when a Goal's objective is edited.
Failure scenario: if this line regresses away, editing an objective that has a checkpoint leaves stale claims (injected into the new revision's catalog by checkpointCatalogEntries, which has no revision guard) and persisted state with evidenceCheckpoint.checkpointId !== evidenceCursor.recordId — parseGoalRecord rejects that combination wholesale, so the goal becomes unrecoverable after a restart.
Suggested fix: add a reducer test that starts from a goalRecord carrying an evidenceCheckpoint (cursor matching the checkpoint id), dispatches edit, and expects next.evidenceCheckpoint to be undefined.
中文说明
edit 分支中 evidenceCheckpoint: undefined 的清理没有配套测试——删掉这一行后整个测试套件仍然全绿（已用变异验证）。reducer 是编辑 Goal 目标时清理 checkpoint 的唯一强制点。
失败场景：若该行回归丢失，编辑带有 checkpoint 的目标会留下过期 claims（被无 revision 防护的 checkpointCatalogEntries 注入新 revision 的目录），且持久化状态出现 evidenceCheckpoint.checkpointId !== evidenceCursor.recordId——parseGoalRecord 会整体拒绝该组合，导致 goal 在重启后无法恢复。
建议修复：新增一个 reducer 测试：从携带 evidenceCheckpoint（cursor 与 checkpoint id 匹配）的 goalRecord 出发，派发 edit，断言 next.evidenceCheckpoint 为 undefined。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| claim: { | ||
| type: 'string', | ||
| minLength: 1, | ||
| maxLength: GOAL_CHECKPOINT_CLAIM_MAX_CHARACTERS, | ||
| }, |
There was a problem hiding this comment.
[Suggestion] The response schema licenses up to 32 claims × 2000 chars (~64 KB ASCII, more for multibyte) and the system prompt presses "retain every still-relevant fact", while materializeGoalEvidenceCheckpoint fail-closes at 16,000 cumulative bytes — no layer in between communicates or checks that budget.
Failure scenario: a faithful cumulative compression exceeding 16 KB (easiest for a multibyte-heavy Goal, where 16,000 bytes is only ~5.3k characters across all claims) is schema-conformant, passes validateGoalCheckpointVerifierText (it never checks bytes), then throws at materialize → resumable usage_limited. The cursor never moved and the window is unchanged, so the temperature-0 verifier reproduces the same oversized output — every resume re-fails identically until the Goal is edited or replaced.
Suggested fix: state the 16 KB cumulative claim budget in the system prompt and/or lower the schema maxItems/maxLength so conforming output cannot exceed the budget severalfold; optionally enforce the byte budget inside validate so the failure surfaces with a clear message at the verifier.
中文说明
响应 schema 允许最多 32 条 claim × 2000 字符（ASCII 约 64 KB，多字节语言更多），系统提示词又要求「保留所有仍相关的事实」，而 materializeGoalEvidenceCheckpoint 在累计 16,000 字节处 fail-closed——中间没有任何一层告知或检查该预算。
失败场景：一次忠实的累计压缩超过 16 KB（多字节语言 Goal 最容易触发：16,000 字节合计仅约 5.3k 字符）时，输出符合 schema、能通过 validateGoalCheckpointVerifierText（它从不检查字节数），随后在 materialize 抛错 → 可 resume 的 usage_limited。由于游标未推进、窗口未变化，temperature 0 的 verifier 会复现同样的超尺寸输出——每次 resume 都以相同方式再次失败，直到 Goal 被编辑或替换。
建议修复：在系统提示词中写明 16 KB 累计 claim 预算，和/或调低 schema 的 maxItems/maxLength 使合规输出不会数倍超出预算；也可在 validate 中强制执行字节预算，让失败在 verifier 层就以清晰信息暴露。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| const timeoutController = new AbortController(); | ||
| const timer = setTimeout(() => { | ||
| timeoutController.abort( | ||
| new Error(`Goal checkpoint verifier timed out after ${timeoutMs}ms`), | ||
| ); | ||
| }, timeoutMs); |
There was a problem hiding this comment.
[Suggestion] The 30 s timeout abort and the timeoutMs option are untested — deleting the timer keeps all tests green (verified by mutant). runSideQuery has no internal timeout of its own, so this timer is the sole time bound on the checkpoint request; the sibling goal verifier has an equivalent timeout test.
Failure scenario: if the timer regresses and the provider stalls, nothing aborts the attempt — the runtime stays at activity verifying with checkpointAttempt set, beginTurn and continuations stay gated, and the goal is frozen until disposal.
Suggested fix: add a test using createGoalCheckpointVerifier(config, { timeoutMs: 1 }) where the mocked generateText rejects when its abort signal fires, and assert the verifier rejects with the timeout error.
中文说明
30 秒超时中止与 timeoutMs 选项没有测试覆盖——删除该 timer 后所有测试仍为绿（已用变异验证）。runSideQuery 自身没有内部超时，因此该 timer 是 checkpoint 请求唯一的时间边界；同级的 goal verifier 有对应的超时测试。
失败场景：若 timer 回归丢失且 provider 挂起，没有任何机制中止该请求——runtime 停留在 activity verifying 且 checkpointAttempt 已置位，beginTurn 与 continuation 持续被门控，goal 被冻结直到 dispose。
建议修复：新增测试：createGoalCheckpointVerifier(config, { timeoutMs: 1 })，mock 的 generateText 在 abort signal 触发时 reject，断言 verifier 以超时错误 reject。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| if (checkpointBytes > GOAL_CHECKPOINT_CLAIM_MAX_BYTES) { | ||
| throw new InvalidGoalCheckpointError( | ||
| `Goal checkpoint exceeds the ${GOAL_CHECKPOINT_CLAIM_MAX_BYTES}-byte claim limit`, | ||
| ); | ||
| } |
There was a problem hiding this comment.
[Suggestion] This cumulative byte gate has no test — deleting it keeps all tests green (verified by mutant). The verifier schema bounds per-claim characters and claim count but never the total size, so this gate is the sole pre-persistence byte enforcement.
Failure scenario: without the gate, a schema-legal oversized result (32 × 2,000 multibyte chars ≫ 16,000 bytes) is persisted by recordCheckpoint; on the next restore isGoalEvidenceCheckpoint rejects the payload, so recovery fails — the goal is lost across a restart.
Suggested fix: add a materializeGoalEvidenceCheckpoint test whose claims exceed GOAL_CHECKPOINT_CLAIM_MAX_BYTES (e.g. 16 claims of ~2,000 bytes each) and expect InvalidGoalCheckpointError.
中文说明
该累计字节上限没有测试——删除后所有测试仍为绿（已用变异验证）。verifier schema 限制单条 claim 字符数与条数，但从不限制总大小，因此该门禁是持久化前唯一的字节约束。
失败场景：若无此门禁，schema 合法的超尺寸结果（32 × 2,000 多字节字符 ≫ 16,000 字节）会被 recordCheckpoint 持久化；下次 restore 时 isGoalEvidenceCheckpoint 拒绝该负载，恢复失败——goal 在重启后丢失。
建议修复：新增 materializeGoalEvidenceCheckpoint 测试，claims 超过 GOAL_CHECKPOINT_CLAIM_MAX_BYTES（例如 16 条各约 2,000 字节），断言抛出 InvalidGoalCheckpointError。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| if (checkpoint && !checkpoint.controller.signal.aborted) { | ||
| checkpoint.controller.abort(new Error(reason)); | ||
| } |
There was a problem hiding this comment.
[Suggestion] Mutant survived: deleting checkpoint.controller.abort(new Error(reason)) in invalidateAttempts leaves all tests green — no test gates checkpoint-invalidation abort on cancel/pause/dispose.
Failure scenario: without the abort, an in-flight checkpoint verifier request (an independent model call over up to ~19.2 KB of evidence) runs to completion after the goal is cancelled/paused/disposed — network/compute spent on a result nothing consumes, and async work continues past dispose(). The persisted-state guards (isCurrentCheckpointAttempt) prevent a corrupt commit, but nothing cancels the request.
Suggested fix: add a runtime test that starts a checkpoint attempt, cancels/disposes the goal mid-flight, and asserts the checkpoint controller's signal is aborted (e.g. the verifier promise rejects on the abort signal / no verifier result is consumed).
中文说明
变异存活：删除 invalidateAttempts 中的 checkpoint.controller.abort(new Error(reason)) 后所有测试仍为绿——没有测试保障 cancel/pause/dispose 时 checkpoint 的中止行为。
失败场景：若无该中止，goal 被 cancel/pause/dispose 后，进行中的 checkpoint verifier 请求（针对最多约 19.2 KB 证据的独立模型调用）仍会跑完——网络/算力浪费在无人消费的结果上，且异步工作在 dispose() 之后继续。持久化状态防护（isCurrentCheckpointAttempt）能防止损坏的提交，但没有任何机制取消该请求。
建议修复：新增 runtime 测试：启动 checkpoint，中途 cancel/dispose goal，断言 checkpoint controller 的 signal 已被中止（例如 verifier promise 因 abort signal reject / verifier 结果未被消费）。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| await restored.restore([journal.records.at(-1)!]); | ||
|
|
||
| expect(restoredHost.inputs[0]?.verifierFeedback).toBe('More work remains'); |
There was a problem hiding this comment.
[Suggestion] No test restores from a verifier_reject journal record that also carries checkpointPending (a crash between the reject record and the checkpoint record); the two halves are tested separately — 'recovers a durable pending checkpoint…' covers turn_finished + pending, and this test covers verifier_reject without pending. Verified by mutant: gating restore's feedback recovery on !checkpointAttempt survives the whole suite.
Failure scenario: terminal verifier rejects → the runtime durably writes the verifier_reject record with checkpointPending → crash before the checkpoint record. On restart, restore must both re-run the checkpoint AND recover nextVerifierFeedback from lastReason so the post-checkpoint continuation is told why the completion was rejected; under the mutant the re-checkpointed goal's continuation starts without the feedback and is likely to re-propose the same rejected terminal, wasting another verification cycle.
Suggested fix: add a restore test whose single recovery record has cause: 'verifier_reject' and checkpointPending, asserting the checkpoint is re-verified (checkpointVerifier called once, restored journal cause 'verifier_reject') and the resulting host turn input carries verifierFeedback: 'More work remains'.
中文说明
没有测试从「同时携带 checkpointPending 的 verifier_reject 日志记录」恢复（即拒绝记录与 checkpoint 记录之间崩溃）；两半是分开测试的——'recovers a durable pending checkpoint…' 覆盖 turn_finished + pending，本测试覆盖不带 pending 的 verifier_reject。已用变异验证：把 restore 的反馈恢复用 !checkpointAttempt 门禁后整个套件仍为绿。
失败场景：terminal verifier 拒绝 → runtime 持久化写入带 checkpointPending 的 verifier_reject 记录 → checkpoint 记录之前崩溃。重启时 restore 必须既重跑 checkpoint、又从 lastReason 恢复 nextVerifierFeedback，让 checkpoint 后的续跑知道完成提议被拒的原因；在该变异下，重新 checkpoint 后的续跑拿不到反馈，很可能再次提交同一被拒终态，浪费又一个验证周期。
建议修复：新增 restore 测试：唯一恢复记录为 cause: 'verifier_reject' 且带 checkpointPending，断言 checkpoint 被重新验证（checkpointVerifier 被调用一次、恢复后日志 cause 为 'verifier_reject'），且宿主轮输入携带 verifierFeedback: 'More work remains'。
— qwen3.8-max via Qwen Code /review (v0.21.5)
There was a problem hiding this comment.
已修复。验证证据：commit 9a9fb0a353；packages/core focused tests 92/92 通过，恢复用例会重新执行 checkpoint、保留 verifier_reject，并向续跑传递 More work remains。
| expect(payload).toMatchObject({ | ||
| goal: { objective: 'Ship the requested change' }, | ||
| previousClaims: [{ id: 'checkpoint-1:1' }], | ||
| evidence: [{ uuid: 'tool-1', content: '18 tests passed' }], | ||
| }); |
There was a problem hiding this comment.
[Suggestion] This test pins the prompt's proofKind rule (systemInstruction must contain 'never change a source proofKind') but the toMatchObject subset never pins proofKind pass-through in the payload — for either evidence or previousClaims. Verified by mutant: stripping proofKind from both mappings in verifierContents keeps all goal tests green while the prompt rules 'never change a source proofKind' / 'do not combine sources with different proofKind values' become unfulfillable by construction.
Failure scenario: blind to proof kinds, the verifier can combine mixed-kind sources into one claim; materializeGoalEvidenceCheckpoint then throws 'changes the proof kind of source …' → recordCheckpointFailure → the goal flips to usage_limited (resumable, cursor unmoved, so it re-fails each turn until the window changes shape). The production cost depends on model behavior, but the unpinned gap itself is demonstrated.
Suggested fix: extend the assertion to pin the data the instruction depends on, e.g. previousClaims: [{ id: 'checkpoint-1:1', proofKind: 'user_input', claim: …, sourceRefs: […] }] and evidence: [{ uuid: 'tool-1', proofKind: 'external_fact', content: '18 tests passed' }].
中文说明
该测试钉住了提示词中的 proofKind 规则（systemInstruction 必须包含 'never change a source proofKind'），但 toMatchObject 子集从未钉住 payload 中的 proofKind 透传——evidence 与 previousClaims 都没有。已用变异验证：把 verifierContents 两处映射中的 proofKind 剥掉后所有 goal 测试仍为绿，而提示词规则「永不改变来源 proofKind」「不要合并不同 proofKind 的来源」从此在构造上不可能被满足。
失败场景：对 proof kind 失明后，verifier 可能把不同 kind 的来源合并进同一条 claim；materializeGoalEvidenceCheckpoint 随即抛出 'changes the proof kind of source …' → recordCheckpointFailure → goal 转为 usage_limited（可 resume、游标未动，因此每轮重复失败直到窗口形状变化）。生产代价取决于模型行为，但未钉住的缺口本身已被证明。
建议修复：扩展断言以钉住该指令所依赖的数据，例如 previousClaims: [{ id: 'checkpoint-1:1', proofKind: 'user_input', claim: …, sourceRefs: […] }] 与 evidence: [{ uuid: 'tool-1', proofKind: 'external_fact', content: '18 tests passed' }]。
— qwen3.8-max via Qwen Code /review (v0.21.5)
There was a problem hiding this comment.
已修复。验证证据：commit 9a9fb0a353；删除两处 proofKind 映射的 mutation probe 会使新断言按预期失败，恢复后 packages/core focused tests 92/92 通过。
| const oversized = input(); | ||
| oversized.evidence[0]!.content = 'x'.repeat(256_000); |
There was a problem hiding this comment.
[Suggestion] The oversized-input probe uses ASCII only, so the 256 KB gate's UTF-8 byte measurement (Buffer.byteLength(text, 'utf8') in verifierContents) is unpinned. Verified by mutant: replacing it with text.length (UTF-16 units) keeps all existing tests green — and a multibyte probe fails under the mutant while passing under this code.
Failure scenario: a future refactor swapping byte length for string length is invisible under the ASCII probe ('x'.repeat(256_000) is identical in both units). For a multibyte-heavy Goal — plausible in this project's user base — a window of ~100k UTF-16 units of CJK transcript content (~300 KB UTF-8) would then pass the mutated gate and be sent to the provider, converting today's deterministic local fail-closed rejection into a provider-dependent oversized-request outcome.
| const oversized = input(); | |
| oversized.evidence[0]!.content = 'x'.repeat(256_000); | |
| const oversized = input(); | |
| oversized.evidence[0]!.content = '中'.repeat(90_000); |
(Or keep the ASCII probe and add this multibyte one alongside it — ~270 KB UTF-8, 90k UTF-16 units, same GoalCheckpointVerifierInputTooLargeError expectation.)
中文说明
超尺寸输入探针只使用 ASCII，因此 256 KB 门禁的 UTF-8 字节度量（verifierContents 中的 Buffer.byteLength(text, 'utf8')）未被钉住。已用变异验证：将其替换为 text.length（UTF-16 单元）后现有测试全部仍为绿——而多字节探针在该变异下失败、在当前代码下通过。
失败场景：未来把字节长度换成字符串长度的重构在 ASCII 探针下不可见（'x'.repeat(256_000) 两种单位下相同）。对多字节语言居多的 Goal（本项目用户群中很常见），约 10 万 UTF-16 单元的 CJK 转录内容（约 300 KB UTF-8）会穿过变异后的门禁被发给 provider，把今天确定性的本地 fail-closed 拒绝变成依赖 provider 的超尺寸请求结果。
— qwen3.8-max via Qwen Code /review (v0.21.5)
There was a problem hiding this comment.
已修复。验证证据：commit 9a9fb0a353；将 Buffer.byteLength 替换为 text.length 的 mutation probe 会使多字节用例按预期失败，恢复后 packages/core focused tests 92/92 通过。
| case 'turn_finished': | ||
| case 'checkpoint': | ||
| case 'verifier_accept': |
There was a problem hiding this comment.
[Suggestion] The new per-turn checkpoint goal_state record is hidden in the CLI display path, but ACP transcript replay emits it: projectSystemRecord in packages/acp-bridge/src/transcript-replay.ts emits a session update for every parseable goal_state record with no cause filter, and the checkpoint broadcast snapshot always differs (fresh updatedAt/activeTimeMs/evidenceCursor/evidenceCheckpoint), so the adapter's exact-signature dedup never fires.
Failure scenario: replaying or resuming a post-change Goal session over ACP yields two near-identical 'checking' goal-status updates per turn — the second carrying the full evidenceCheckpoint.claims payload (up to the 16 KB claim budget) — where pre-change there was one; automation keyed on goal_state events sees a spurious event per checkpoint, and the TUI and ACP frontends report different streams for the same session.
Suggested fix: skip replay emission for bookkeeping-only goal records the same way the CLI display path does (reuse/export shouldDisplayGoalStateCause in the replay projection), or emit only when the projected goal status materially changes.
中文说明
新增的每轮 checkpoint goal_state 记录在 CLI 显示路径被隐藏，但 ACP 转录回放会发出它：packages/acp-bridge/src/transcript-replay.ts 的 projectSystemRecord 对每条可解析的 goal_state 记录都发出 session update、没有 cause 过滤，且 checkpoint 广播快照总是不同（全新的 updatedAt/activeTimeMs/evidenceCursor/evidenceCheckpoint），因此适配器的精确签名去重永远不会命中。
失败场景：通过 ACP 回放或恢复改动后的 Goal 会话时，每轮会得到两条几乎相同的 'checking' 状态更新——第二条还携带完整 evidenceCheckpoint.claims 负载（最多 16 KB claim 预算）——而改动前只有一条；依赖 goal_state 事件的自动化会看到每个 checkpoint 多出一条伪事件，TUI 与 ACP 前端对同一会话报告不同的事件流。
建议修复：像 CLI 显示路径一样跳过纯簿记 goal 记录的回放发送（在回放投影中复用/导出 shouldDisplayGoalStateCause），或仅在投影出的 goal 状态发生实质变化时发送。
— qwen3.8-max via Qwen Code /review (v0.21.5)
| case 'checkpoint': | ||
| case 'verifier_accept': | ||
| return false; |
There was a problem hiding this comment.
[Suggestion] Hiding cause 'checkpoint' here does not suppress the checkpoint record on session resume when it follows a verifier rejection: the runtime persists that follow-up record under cause 'verifier_reject' (persistedCause = nextVerifierFeedback === undefined ? 'checkpoint' : 'verifier_reject' in both finishCheckpointCheck and recordCheckpoint) — a cause this function returns true for. The PR's own test pins the resulting journal sequence ['create', 'turn_finished', 'verifier_reject', 'verifier_reject'].
Failure scenario: on resume, resumeHistoryUtils replays goal_state records through this filter and renders one card per passing record; both the original reject record and the checkpoint record pass, and GoalStateCard renders them as near-identical cards (both snapshots status 'active' / activity 'idle', same objective/turnCount/lastReason — the reason line is only shown for non-active or verifying states, and the only differing fields are evidenceCheckpoint/evidenceCursor/updatedAt, which the card doesn't render). So every rejection→checkpoint turn shows one card live but two duplicate cards after resume, growing by one per such turn.
Suggested fix: suppress checkpoint-bookkeeping records in resume replay — e.g. skip goal_state records whose snapshot delta vs the prior item is limited to evidenceCursor/evidenceCheckpoint/activeTimeMs/updatedAt — or dedupe consecutive near-identical goal cards.
中文说明
在此处隐藏 cause 'checkpoint' 并不能在会话恢复时抑制紧随 verifier 拒绝之后的 checkpoint 记录：runtime 会把该后续记录以 cause 'verifier_reject' 持久化（finishCheckpointCheck 与 recordCheckpoint 中均为 persistedCause = nextVerifierFeedback === undefined ? 'checkpoint' : 'verifier_reject'）——而该 cause 在本函数中返回 true。PR 自己的测试就钉住了 resulting 日志序列 ['create', 'turn_finished', 'verifier_reject', 'verifier_reject']。
失败场景：resume 时 resumeHistoryUtils 用该过滤器回放 goal_state 记录并逐条渲染卡片；原始拒绝记录与 checkpoint 记录都会通过，且 GoalStateCard 把它们渲染成几乎相同的卡片（两个快照均为 status 'active' / activity 'idle'，objective/turnCount/lastReason 相同——reason 行只在非 active 或 verifying 状态显示，唯一不同的字段 evidenceCheckpoint/evidenceCursor/updatedAt 卡片并不渲染）。因此每个「拒绝→checkpoint」轮在实时会话中显示一张卡片，resume 后却显示两张重复卡片，并随此类轮数逐次累加。
建议修复：在 resume 回放中抑制 checkpoint 簿记记录——例如跳过相对上一条快照差异仅限于 evidenceCursor/evidenceCheckpoint/activeTimeMs/updatedAt 的 goal_state 记录——或对连续几乎相同的 goal 卡片去重。
— qwen3.8-max via Qwen Code /review (v0.21.5)
|
@qwen-code /takeover |
qwen-code-dev-bot
added
the
autofix/takeover
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. This is a fork PR, so the first round comes from the next scheduled scan (usually within minutes). Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。本 PR 来自 fork，首轮处理将由下一次定时扫描执行（通常几分钟内）。移除 |
|
@qwen-code /review |
| _Qwen Code review request accepted. Review is queued in [workflow run](https://github.com/QwenLM/qwen-code/actions/runs/31243176905)._ |
|
🔀 Base updated: red check(s) [review-pr] pass on current main — merged current main via update-branch; CI will re-run. 中文说明🔀 已更新 base：红色检查 [review-pr] 在当前 main 上通过 —— 已通过 update-branch 合入当前 main，CI 将重新运行。 |
yiliang114
left a comment
yiliang114
left a comment
There was a problem hiding this comment.
LGTM, no blockers. State machine is correct (checkpointPending persisted atomically, stable recordUuid across crash-restore, sound race handling via enqueue serialization + isCurrentCheckpointAttempt re-checks); verifier output has good defense-in-depth; 323/323 goals tests pass at head. Two P2s worth addressing or documenting soon — both are fail-closed choices that can permanently usage_limited exactly the long-running Goals this protects: (1) checkpoints only fire at turn end, so one turn jumping threshold->hard-limit truncates and now blocks ALL terminal proposals (even abandoned); (2) trigger measures preview bytes but the verifier payload carries full evidenceContent capped 256KB, so a window of large tool outputs fails the Goal on a recoverable compression problem. Suggest bounding per-record content in the payload or falling back to previews on oversize.
…enLM#8465) Address the review findings on the Goal evidence checkpoint change: - Keep the verifier rejection feedback when a follow-up checkpoint fails, so the resumed continuation still learns why the proposal was rejected. - State the cumulative claim byte budget in the checkpoint verifier prompt so conforming output is steered inside the materialize gate. - Suppress checkpoint-bookkeeping goal_state records in ACP transcript replay and CLI resume rendering, matching the live TUI display path. - Pin the reviewed behaviors with mutant-killing tests and drop five unneeded runtime-options casts.
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/100 轮）。改动内容与我反驳保留之处如下： Review feedback round — all 22 findings resolved in codeCommit: Code fixes
New / extended tests (each mutation-probed where noted)
Already fixed in a prior commit (re-verified, no change needed)
The three @qqqys inline notes confirming those fixes are informational; no action was required beyond re-verification. VerificationCommands actually run (all on the final committed tree
中文说明评审反馈本轮处理 — 全部 22 条发现已在代码中解决提交： 代码修复
新增/扩展测试（关键项已做变异探针验证）
已在先前提交中修复（本轮重新验证，无需改动）
@qqqys 确认上述修复的三条行内说明属于信息性内容；除重新验证外无需处理。 验证实际执行的命令（除注明外均在最终提交树
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
|
📊 Takeover milestone — round 10/100, in the current window. Census: 7 pushed fix(es), 2 no-change review(s), 2 timeout(s), 0 rejected attempt(s), 1 other round(s) (crash / model error / gate error / infra), 3 base update(s). This many rounds deserves a human look. Options: keep going (fine — nothing changes), split or reduce the PR if rounds keep accumulating, or release takeover (remove the 中文说明📊 接管里程碑 —— 第 10/100 轮（当前窗口）。统计：推送修复 7 次、审阅无需改动 2 次、超时 2 次、验证拒绝 0 次、其他轮次（崩溃/模型错误/门错误/infra）1 次、base 更新 3 次。 轮次到这个量值得人工看一眼。可选：继续（无需操作）；若轮次持续累积，考虑拆分或缩减 PR；或释放接管（移除 |
Local runtime verification — PR #8465 @
|
base a8fb97d1 |
PR 529dde21 |
|
|---|---|---|
| outcome at turn 90 | Goal usage limited |
Goal complete |
| reason | exceeded the bounded evidence catalog… | Cited evidence supports the claim. |
| checkpoint verifier calls | 0 | 1 |
| terminal verifier calls | 0 — never reached | 1 |
The base arm never even runs terminal verification: the truncated catalog fails closed first. That is the #8430 behaviour this PR is meant to make unnecessary.
2. The catalog actually compacts (wire-observed get_goal payloads)
Every Goal turn calls get_goal, so the catalog the next turn sees is captured off the wire:
| model request # | base — entries / truncated | PR — entries / truncated / provenance |
|---|---|---|
| 24 | 23 / false |
23 / false — 23 assistant_output |
| 48 | 47 / false |
47 / false — 47 assistant_output |
| 64 | 63 / false |
63 / false — 63 assistant_output |
| 66 | 65 / false |
2 / false — 1 goal_checkpoint + 1 assistant_output |
| 80 | 78 / true |
16 / false — 1 goal_checkpoint + 15 raw |
| 82 | 78 / true (and stays there) |
18 / false — 1 goal_checkpoint + 17 raw |
The next turn sees exactly checkpoint claims + only newer raw evidence, as documented. Base saturates at 78 entries with truncated: true and stays there forever.
Over a longer run (329 Goal turns, unattended) the PR arm committed 7 compactions at turnCount 47 / 94 / 141 / 188 / 235 / 282 / 329, each advancing evidenceCursor.recordId to the new checkpointId, and never reached usage_limited.
3. Proof-kind preservation over mixed evidence
Ran a variant where each turn emits assistant text and a real run_shell_command tool result. At the checkpoint, 63 entries (47 delivered_output + 16 external_fact) collapsed to 2 claims — one per proof kind, and the mix stayed correct in every later catalog. The sourceRefs of the second checkpoint cite the first checkpoint's claim id (<uuid>:1), i.e. cumulative carry-forward works.
4. Continuation is genuinely gated on the checkpoint
Held the checkpoint verifier response open for 25 s. Zero model turns ran during the entire hold (wire log: last agent request at +1.8 s, checkpoint held +1.8 s → +26.8 s, next agent request at +26.8 s), and the TUI sits on /goal checking:
5. Crash recovery is exact and idempotent
SIGKILL while the checkpoint verification was in flight. Last durable record was turn_finished (turn 47) carrying checkpointPending.recordUuid = 2eed1470…. After qwen -c:
- exactly one replayed checkpoint request went out (same 47-record window,
previousClaims: []); - the committed record uuid ==
evidenceCheckpoint.checkpointId== the newevidenceCursor.recordId==2eed1470…— byte-identical to the pre-crash marker; - exactly one continuation followed (turn 48).
6. Fail-closed matrix — 6/6, no cursor advance
Injected each failure at the checkpoint verifier. Every one stopped at turn 47 with usage_limited, and on-disk evidenceCursor was unchanged and no evidenceCheckpoint was ever persisted.
| injected fault | reason surfaced |
|---|---|
| proof kind flipped | Goal checkpoint claim 1 changes the proof kind of source <uuid> |
| unknown source cited | Goal checkpoint claim 1 cites unknown source bogus-uuid-not-in-catalog |
| non-JSON body | Goal checkpoint verifier returned invalid JSON |
empty claims |
Goal checkpoint verifier returned invalid claims |
| provider HTTP 500 | Failed to generate text content (side-query:goal-checkpoint-verifier): 500 … |
| oversize claims | Goal checkpoint exceeds the 16000-byte claim limit |
7. Display suppression works (CLI + ACP projection)
The reject scenario persists two verifier_reject records per real rejection (the rejection, then the checkpoint follow-up write). Feeding that real transcript through the built buildResumedHistoryItems:
- persisted: 182
goal_staterecords —create1,turn_finished63,checkpoint9,verifier_reject108,pause1 - rendered: 56 cards —
create1,verifier_reject54,pause1
Exactly one card per real rejection; zero checkpoint cards. No duplicates.
9. Queued-user priority — Core yes, TUI no (and that is pre-existing)
The one Test Plan item I could not reproduce through the interactive TUI. Queuing a real user message while the checkpoint verifier was held open: the TUI accepted it into its own queue (↑ to edit queued messages), no turn ran during the hold, but the first request after the checkpoint released was still a synthetic continuation, not the user's text.
Control, with no checkpoint anywhere in the picture — set a Goal, queue a plain message during an ordinary active Goal, wait 25 s:
| arm | Goal turns run | user message delivered at |
|---|---|---|
base a8fb97d1 |
21 | never |
PR 529dde21 |
21 | never |
Identical on both arms, so this is the pre-existing TUI queue gate (plain input is withheld from the Goal runtime in every non-terminal Goal state), not anything this PR changed. The Core-side behaviour the PR actually implements is real and tested: deleting the promoteQueuedUserTurn() branch from recordCheckpoint fails 2 of the 348 src/goals tests. Flagging it only so the Test Plan line is not read as a TUI-observable guarantee.
8. Gates and mutation testing
Gates on 529dde21: build ✅ · bundle ✅ · typecheck ✅ · lint (eslint + integration-tests) ✅ · prettier --check ✅ · core src/goals + session-transcript-reader 408/408 ✅ · CLI touched suites 95/95 ✅ · acp-bridge 1074/1074 ✅ · full core 19338 passed / 2 failed — both failures (session-writer-lease, skill-curator) reproduce identically on the base arm; container/root artifacts, not this PR.
To confirm the new guards are load-bearing rather than decorative, I reverted each one against the 348 src/goals tests:
| mutation | tests failed |
|---|---|
shouldCheckpoint forced false |
19 |
checkpointPending never written on turn_finished |
19 |
| checkpoint entries excluded from the catalog | 4 |
| truncated-catalog gate reverted to the pre-PR complete-only form | 3 |
| cursor not advanced on checkpoint commit | 2 |
| proof-kind-preservation guard removed | 2 |
window.truncated fail-closed removed |
1 |
| checkpoint byte limit disabled | 1 |
| queued-user promotion dropped after a committed checkpoint | 2 |
| unknown-source guard removed | 0 † |
† Benign: with that guard gone an unknown source falls into the adjacent proof-kind check and still fails closed — only the message differs. The live run above shows the intended message is produced in production. Worth a one-line test if you want the message pinned.
Non-blocking observations
(a) Downgrade is silent, not loud — and it is a bigger drop than the PR body suggests. The PR notes downgrading is unsupported. Measured: feeding a PR-written transcript to the base parseGoalStateRecordPayloadV2/parseGoalRecord, 106 of 182 goal_state records (58%) parse to undefined and are skipped — 63 carry checkpointPending, 9 use cause: 'checkpoint', 50 carry evidenceCheckpoint on the goal record. The first drop is the second record written after goal creation. Rendering the same transcript on both builds gives 56 cards (PR) vs 38 cards (base) — the base build silently loses the trailing lifecycle, including the final pause card. Because recovery scans for the newest parseable payload, an old binary would resume from a pre-checkpoint snapshot rather than refusing. Not a blocker (downgrade is out of contract), but "not supported" reads as "errors out", whereas the observed behaviour is silent regression to stale state — worth a sentence in the release note.
(b) Every committed checkpoint costs two turns of repeated-blocker eligibility. A checkpoint advances evidenceCursor, which resets lineageTurnIds. Measured live at a checkpoint boundary: lineageTurnIds drops 16 → 1 and only returns to 3 two turns later. Probing the built validateGoalEvidenceReferences directly:
checkpoint=true rawTurnsSinceCursor=1 lineageTurnIds=1 -> repeated_blocker_turn_coverage
checkpoint=true rawTurnsSinceCursor=2 lineageTurnIds=2 -> repeated_blocker_turn_coverage
checkpoint=true rawTurnsSinceCursor=3 lineageTurnIds=3 -> ACCEPTED
The PR correctly skips checkpointing on the verifier-reject path for repeated blockers, so the three-turn lineage claim holds for the case it guards. But an ordinary turn_finished checkpoint still resets the lineage, so a repeated-blocker proposal made in the two turns right after any checkpoint is rejected with repeated_blocker_turn_coverage even when the blocker has genuinely recurred for many turns. The two-turn window itself is pre-existing (identical on base, and a checkpoint claim can never satisfy turn coverage since its turnId is checkpoint:<id> — by design). It is fail-safe: the proposal is rejected, the Goal keeps running, and the same proposal succeeds two turns later. What is new is that this window now recurs — roughly every 47 turns in my runs — rather than once per Goal revision.
(c) Per-turn cost, for awareness. finishTurn now flushes and re-reads the active transcript chain on every turn (previously only when a proposal needed verification), and writes a second goal_state record per turn even when nothing is compacted (335 turns → 670 goal_state records). Against a zero-latency mock this measures as 3.93 s → 6.07 s per 100 turns on the PR arm vs 3.18 s → 3.83 s on base, with the gap widening as the transcript grows (the read is O(transcript)). Against a real provider the latency is invisible, but transcript size for goal_state records doubles. Not a blocker; just sizing information.
Recommendation: LGTM to merge. The core claims — bounded-catalog compaction, cumulative carry-forward, proof-kind preservation, gated continuation, exact crash recovery, and fail-closed on every corrupted/failed checkpoint — all reproduce on real Linux runtime, and the guards are covered by non-vacuous tests. Observation (a) is a release-note nit; (b) and (c) are behavioural consequences worth knowing, not defects.
中文版
PR #8465 本地运行时验证 @ 529dde21
我为这个 PR 搭建了真实的端到端环境，而不是只依赖它自带的单元测试：生产代码中的 Goal runtime、证据目录、checkpoint window、materializer、持久化、恢复以及 terminal verifier 全部原样运行，只有模型的 token 流由一个 mock OpenAI 兼容服务伪造，因此 checkpoint verifier 的请求/响应是真实过网的。两个 arm 都是真实打包的 CLI，在 tmux 中驱动。
- PR arm —
529dde214f（worktree，npm ci+npm run build+npm run bundle） - Base arm —
a8fb97d16f（merge-base，即 fix(core): stop Goal retries after evidence catalog exhaustion #8430 之后的状态） - 两个 arm 使用相同 objective、相同 mock、相同场景、相同轮数。
结论：PR 实现了它声称的功能。 Reviewer Test Plan 中 7 项有 6 项在 Linux 上端到端复现；第 7 项（排队用户输入优先）只在 Core 层面得到验证，原因不在本 PR —— 见 §9。三条非阻塞观察见文末。
1. 核心 A/B —— 死路确实被修复了
一个 90 轮的 Goal，先撑大有界目录，然后在第 90 轮提出完成。两个 arm 输入完全一致。
base a8fb97d1 |
PR 529dde21 |
|
|---|---|---|
| 第 90 轮结果 | Goal usage limited |
Goal complete |
| 原因 | exceeded the bounded evidence catalog… | Cited evidence supports the claim. |
| checkpoint verifier 调用 | 0 | 1 |
| terminal verifier 调用 | 0 —— 根本没走到 | 1 |
base arm 连 terminal verification 都没跑：truncated 的目录先 fail closed。这正是本 PR 要消除的 #8430 行为。
2. 目录确实被压缩了（从 wire 上抓到的 get_goal payload）
每个 Goal turn 都调用 get_goal，因此下一轮看到的目录可以直接从网络请求中抓取：
| 模型请求序号 | base —— entries / truncated | PR —— entries / truncated / provenance |
|---|---|---|
| 24 | 23 / false |
23 / false —— 23 assistant_output |
| 48 | 47 / false |
47 / false —— 47 assistant_output |
| 64 | 63 / false |
63 / false —— 63 assistant_output |
| 66 | 65 / false |
2 / false —— 1 goal_checkpoint + 1 assistant_output |
| 80 | 78 / true |
16 / false —— 1 goal_checkpoint + 15 raw |
| 82 | 78 / true（此后不再变） |
18 / false —— 1 goal_checkpoint + 17 raw |
下一轮看到的正是 checkpoint claims + 仅更新的原始证据，与文档一致。base 在 78 条处饱和并永久保持 truncated: true。
在更长的一次运行中（329 个 Goal turn，无人值守），PR arm 在 turnCount 47 / 94 / 141 / 188 / 235 / 282 / 329 提交了 7 次压缩，每次都把 evidenceCursor.recordId 推进到新的 checkpointId，且从未进入 usage_limited。
3. 混合证据下的 proof kind 保持
另跑一个变体：每轮既产出 assistant 文本，又执行真实的 run_shell_command 工具调用。在 checkpoint 处，63 条（47 条 delivered_output + 16 条 external_fact）被压缩成 2 条 claim —— 每种 proof kind 一条，之后每次目录中的比例都保持正确。第二个 checkpoint 的 sourceRefs 引用了第一个 checkpoint 的 claim id（<uuid>:1），说明累计式 carry-forward 生效。
4. 续跑确实被 checkpoint 挡住
把 checkpoint verifier 的响应挂起 25 秒。整个挂起期间没有任何模型轮次运行（wire log：最后一次 agent 请求在 +1.8s，checkpoint 挂起 +1.8s → +26.8s，下一次 agent 请求在 +26.8s），TUI 停在 /goal checking。
5. 崩溃恢复精确且幂等
在 checkpoint 验证进行中 SIGKILL。最后一条持久记录是 turn_finished（第 47 轮），携带 checkpointPending.recordUuid = 2eed1470…。执行 qwen -c 之后：
- 只重放了一次 checkpoint 请求（同样的 47 条 window，
previousClaims: []）； - 提交的 record uuid ==
evidenceCheckpoint.checkpointId== 新的evidenceCursor.recordId==2eed1470…，与崩溃前的标记逐字节一致； - 之后只启动了一次 continuation（第 48 轮）。
6. Fail-closed 矩阵 —— 6/6，游标未推进
在 checkpoint verifier 处注入各类故障。每一种都在第 47 轮停下并转为 usage_limited，磁盘上的 evidenceCursor 未变，且从未持久化任何 evidenceCheckpoint。
| 注入的故障 | 暴露的原因 |
|---|---|
| 翻转 proof kind | Goal checkpoint claim 1 changes the proof kind of source <uuid> |
| 引用未知来源 | Goal checkpoint claim 1 cites unknown source bogus-uuid-not-in-catalog |
| 非 JSON 响应 | Goal checkpoint verifier returned invalid JSON |
空 claims |
Goal checkpoint verifier returned invalid claims |
| provider HTTP 500 | Failed to generate text content (side-query:goal-checkpoint-verifier): 500 … |
| 超长 claims | Goal checkpoint exceeds the 16000-byte claim limit |
7. 显示抑制生效（CLI + ACP projection）
reject 场景下，每次真实拒绝会持久化两条 verifier_reject 记录（拒绝本身，以及 checkpoint 的后续写入）。把这份真实 transcript 喂给构建产物中的 buildResumedHistoryItems：
- 持久化：182 条
goal_state——create1、turn_finished63、checkpoint9、verifier_reject108、pause1 - 渲染：56 张卡片 ——
create1、verifier_reject54、pause1
每次真实拒绝恰好一张卡片；没有 checkpoint 卡片，没有重复。
9. 排队用户输入优先 —— Core 层成立，TUI 层不成立（且是既有问题）
这是唯一一项我无法通过交互式 TUI 复现的 Test Plan 条目。在 checkpoint verifier 被挂起期间排入一条真实用户消息：TUI 接受了它并放进自己的队列（↑ to edit queued messages），挂起期间确实没有任何轮次运行，但 checkpoint 释放后的第一个请求仍然是合成 continuation，而不是用户输入。
对照实验（完全不涉及 checkpoint）—— 设置 Goal，在普通 active 状态下排入一条普通消息，等待 25 秒：
| arm | 运行的 Goal 轮数 | 用户消息何时送达 |
|---|---|---|
base a8fb97d1 |
21 | 从未 |
PR 529dde21 |
21 | 从未 |
两个 arm 完全一致，因此这是既有的 TUI 队列门控（在任何非终态 Goal 状态下，普通输入都不会交给 Goal runtime），与本 PR 无关。PR 真正实现的 Core 侧行为是有效且有测试的：把 recordCheckpoint 中的 promoteQueuedUserTurn() 分支删掉，348 个 src/goals 测试中有 2 个失败。此处指出，只是为了避免把这条 Test Plan 读成一个在 TUI 上可观察的保证。
8. 门禁与变异测试
529dde21 上的门禁：build ✅ · bundle ✅ · typecheck ✅ · lint（eslint + integration-tests）✅ · prettier --check ✅ · core src/goals + session-transcript-reader 408/408 ✅ · CLI 相关套件 95/95 ✅ · acp-bridge 1074/1074 ✅ · core 全量 19338 通过 / 2 失败 —— 两个失败（session-writer-lease、skill-curator）在 base arm 上完全相同地复现，属于容器/root 环境问题，与本 PR 无关。
为确认新增的守卫是真的在起作用而不是摆设，我逐个把它们改回去，对 348 个 src/goals 测试重跑：
| 变异 | 失败测试数 |
|---|---|
shouldCheckpoint 强制为 false |
19 |
turn_finished 不再写 checkpointPending |
19 |
| checkpoint 条目不进目录 | 4 |
| truncated 目录门禁改回 PR 之前的 complete-only 形式 | 3 |
| checkpoint 提交时不推进游标 | 2 |
| 移除 proof-kind 保持守卫 | 2 |
移除 window.truncated fail-closed |
1 |
| 关闭 checkpoint 字节上限 | 1 |
| checkpoint 提交后不再提升排队用户输入 | 2 |
| 移除未知来源守卫 | 0 † |
† 无害：移除该守卫后，未知来源会落到相邻的 proof-kind 检查上，仍然 fail closed，只是消息不同。上面的真实运行显示生产环境输出的正是预期消息。如果想把消息固定下来，可以补一行测试。
非阻塞观察
(a) 降级是静默的，而不是报错的 —— 且丢弃范围比 PR 描述暗示的更大。 PR 指出不支持降级。实测：把 PR 写出的 transcript 交给 base 的 parseGoalStateRecordPayloadV2/parseGoalRecord，182 条 goal_state 中有 106 条（58%）解析为 undefined 而被跳过 —— 63 条带 checkpointPending，9 条用 cause: 'checkpoint'，50 条在 goal record 上带 evidenceCheckpoint。第一条被丢弃的记录就是 goal 创建后的第二条。同一份 transcript 在两个构建上渲染分别得到 56 张（PR）和 38 张（base）卡片 —— base 静默丢掉了尾部生命周期，包括最后的 pause 卡片。由于恢复逻辑会寻找最新的可解析 payload，旧二进制会从 checkpoint 之前的快照恢复，而不是拒绝启动。这不是阻塞项（降级本就不在契约内），但 "not supported" 读起来像"会报错"，而实测行为是静默回退到陈旧状态 —— 值得在 release note 里加一句。
(b) 每次提交 checkpoint 都要花掉两轮的 repeated-blocker 资格。 checkpoint 会推进 evidenceCursor，从而重置 lineageTurnIds。在 checkpoint 边界处实测：lineageTurnIds 从 16 掉到 1，两轮后才回到 3。直接探测构建产物中的 validateGoalEvidenceReferences：
checkpoint=true rawTurnsSinceCursor=1 lineageTurnIds=1 -> repeated_blocker_turn_coverage
checkpoint=true rawTurnsSinceCursor=2 lineageTurnIds=2 -> repeated_blocker_turn_coverage
checkpoint=true rawTurnsSinceCursor=3 lineageTurnIds=3 -> ACCEPTED
PR 在 verifier-reject 路径上正确地跳过了 repeated blocker 的 checkpoint，所以三轮 lineage 的说法对它所保护的场景成立。但普通的 turn_finished checkpoint 仍会重置 lineage，因此在任何 checkpoint 之后的两轮内提出的 repeated-blocker 提议都会被 repeated_blocker_turn_coverage 拒绝，哪怕该 blocker 确实已经重复了很多轮。这个两轮窗口本身是既有行为（base 上完全相同；而且 checkpoint claim 的 turnId 是 checkpoint:<id>，按设计永远无法满足 turn coverage）。它是 fail-safe 的：提议被拒绝，Goal 继续运行，两轮后同样的提议就会通过。新增的部分是这个窗口现在会反复出现 —— 在我的运行中大约每 47 轮一次 —— 而不再是每个 Goal revision 只出现一次。
(c) 每轮开销，供参考。 finishTurn 现在在每一轮都会 flush 并重读活动 transcript chain（此前只在有提议需要验证时才读），并且即使没有压缩也会每轮多写一条 goal_state 记录（335 轮 → 670 条 goal_state）。在零延迟 mock 下实测：PR arm 每 100 轮 3.93s → 6.07s，base arm 3.18s → 3.83s，且随 transcript 增长差距扩大（该读取是 O(transcript)）。面对真实 provider 时这点延迟不可见，但 goal_state 记录的 transcript 体积会翻倍。不是阻塞项，仅作为容量参考。
建议：LGTM，可以合并。 核心主张 —— 有界目录压缩、累计 carry-forward、proof kind 保持、续跑门控、精确崩溃恢复，以及对任何损坏/失败 checkpoint 的 fail-closed —— 全部在真实 Linux 运行时复现，且守卫都有非空洞的测试覆盖。观察 (a) 是 release note 层面的小事；(b) 和 (c) 是值得知晓的行为后果，不是缺陷。
|
@qwen-code /triage |
|
Sandboxed verification: The verification run did not complete, so the phases below may be partial or missing entirely. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 585 passed · 0 failed · 585 total 中文 — 判定：
|
| cell / oracle | base 9e1b1eb49e |
head 529dde214f |
|---|---|---|
| causes after turn 1 (80 evidence records, no proposal) | create, turn_finished |
create, turn_finished, checkpoint |
| checkpoint verifier calls | 0 | 1 |
evidenceCheckpoint / cursor advanced |
absent / no | present / cursor == checkpointId |
turn 2 complete citing (claim + new) vs (old + new) evidence |
usage_limited — dead end, reason = catalog exhaustion |
complete |
| terminal verifier evidence | — | includes the claim with provenance: goal_checkpoint |
The base cell failing (dead-ending) is the predicted control outcome; both control assertions passed. Witness: evidence/01-ab-cells-head-vs-base.png.
S2 — truncated catalog + blocked/external proposal (109 records over two turns, citing one real-user constraint): base accepts the terminal blocked (the hole the PR closes — exhaustion reinterpreted as an external blocker, resumable forever); head fails closed to usage_limited with the exhaustion reason. Same capture.
S3 — crash recovery (crash-recovery.mjs, head): the turn_finished payload with checkpointPending is durable at crash time; a fresh runtime's restore() replays exactly one checkpoint reusing the pending recordUuid, writes exactly one checkpoint record, starts exactly one continuation, and makes zero terminal-verifier calls. S4 — downgrade: base's strict parser rejects the head-shaped payload; with an older parseable record present base silently rolls back to it (turnCount=0), with only the new-shaped record it throws GoalPersistenceUnavailableError; head fails closed when checkpoint dependencies are absent.
Secondary claim (display dedupe): isGoalCheckpointBookkeepingTransition/isGoalCheckpointBookkeepingCause verified at function level (gates G12) and by the acp-bridge page-boundary test; the CLI resume path's shape-only check analyzed unreachable for every displayable cause (see Findings).
Findings
No blocking findings. Two informational observations:
- Downgrade failure mode is silent rollback, not an error. Measured in S4: a pre-checkpoint binary restoring a session whose newest record carries
checkpointPendingfalls back to the last parseable record and silently loses the committed turn progress (goal rewinds toturnCount=0, still active). This is consistent with the PR's documented "downgrade not supported" tradeoff, but the silent variant is the quieter of the two failure modes — worth a sentence in the migration note if maintainers care. - CLI resume suppresses shape-equal transitions without the cause check ACP uses.
resumeHistoryUtils.tscallsisGoalCheckpointBookkeepingTransitionalone, whiletranscript-replay.tsalso requiresisGoalCheckpointBookkeepingCause. I walked every displayable cause (create/replace/edit/pause/resume/complete/blocked/usage_limited/clear/migrated/verifier_reject): all except duplicateverifier_rejectchange the compared shape fields (status/objective/lastReason/…), and duplicateverifier_rejectis suppressed by both paths. The asymmetry is therefore unreachable in practice; not a defect, recorded so a future cause addition re-checks it.
Reviewer Test Plan walk-through
| plan step | result |
|---|---|
| 1. Threshold turn without terminal proposal; next turn only after verified durable checkpoint; sees claims + newer evidence | verified — S1 A/B (both halves) |
| 2. Interrupt after pending marker; restore; verification completes before exactly one continuation | verified — S3 |
| 3. Terminal reject near threshold; checkpoint completes; next turn receives exact verifier feedback | covered by suite (promotes queued user input with exact verifier feedback after rejection among 348/348 green); not independently A/B'd |
| 4. Queued real user input admitted before automatic continuation during checkpoint verification | covered by suite and proven load-bearing: mutation M1 turns promotes queued user input before an automatic post-checkpoint turn red while the verification-path queued-user siblings stay green |
| 5. Malformed claims / changed proof semantics / unknown sources / truncation / source I/O / provider failures fail closed without cursor advance | verified — gates G1–G13 (29 assertions) + suite's moves to usage_limited when checkpoint %s fails |
6. External-blocker proposal after truncation → usage_limited, no continuation |
verified — S2 A/B |
| 7. Repeated blocker keeps three-turn raw lineage | covered by suite (lets a repeated blocker streak reach the verifier when the catalog truncates, green) |
Mutation matrix (vacuity)
| # | mutation | suite | result |
|---|---|---|---|
| M1 | remove checkpoint-attempt creation in finishTurn |
goal-runtime.test.ts |
killed — 19/100 red with behavioral failures (verifier never called, cause sequences broken), incl. the central lifecycle test and the post-checkpoint queued-user test |
| M2 | truncation gate reverted to base (complete-only) |
goal-evidence.test.ts |
killed — exactly the 2 gate-scoping tests red (scopes the truncated catalog gate…, fails closed when truncation evicts a repeated blocker turn) |
| M3 | byte budget counts serialization overhead instead of claim text | goal-checkpoint.test.ts |
killed — exactly enforces the byte limit on claim text, not serialization overhead red (InvalidGoalCheckpointError: exceeds the 16000-byte claim limit) |
Witness: evidence/03-mutation-m1-checkpoint-wiring-red.png. No survivors among the mutated load-bearing hunks; every kill was a behavioral expected-vs-actual failure, and the restored tree re-ran green (426/426 core, control).
Targeted gates
| suite | result |
|---|---|
packages/core src/goals/ |
348/348 |
packages/core session-transcript-reader.test.ts |
78/78 |
packages/cli resume/goal-runtime/clipboard tests |
79/79 |
packages/acp-bridge transcript-replay.test.ts |
22/22 |
Gate liveness proven by the mutation runs above (same harness goes red under M1–M3). Witness: evidence/04-gate-suites-green.png.
Not covered
- Per-commit attribution: the checkout is depth-2 (merge commit + base tip + PR head); the metadata lists 16 commits, only 3 reachable locally. Verified the aggregate
HEAD^1..HEADdiff only. - Checkpoint verifier wire path (
createGoalCheckpointVerifier→runSideQuery): needs a live provider; the validation surface it feeds (parse + materialize + persisted-parser) is what the 29 gate assertions exercise, and G13 checks parse/materialize limit alignment at the shared edges. - Base control build: compiled against the head tree's
node_modules(lockfile identical, so deps are not a confound); a freshtscof base shows pre-existing type errors in unrelated files (mime/lite,fdir,ignore,ajvtypings) that do not exist when compiling head — attributed to the base worktree's relativetsconfigpaths (../../node_modules) rather than to the PR; the goals closure emitted completely in both arms and is all the harness consumes. - Windows/macOS; live TUI frames of the suppressed cards; ACP replay through a real client (machine-level tests only); repo-wide lint/typecheck (CI-covered; head
tsc --buildonpackages/coreis clean, 0 errors).
Methodology
Environment: node:22-bookworm container, repo at refs/pull/8465/merge (depth 2), pre-built at head; base control via git worktree add tmp/base-tree HEAD^1 + workspace-scoped tsc build. All harnesses (ab-goal-checkpoint.mjs, crash-recovery.mjs, checkpoint-gates.mjs, score-ab.mjs) import the compiled dist/ of the arm under test and drive createGoalRuntime through its public ports with fake journal/evidence-source/verifier/host — no module interception. Assertions: 26 A/B + lifecycle + downgrade, 29 gates, 527 suite tests, 3 mutation kills = 585, all executed, 0 unexpected failures. Raw logs in logs/; captures in evidence/ (01 A/B cells, 02 gates, 03 mutation red, 04 gates green).
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
|
🤖 Addressed the latest review feedback (round 11/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 11/100 轮）。改动内容与我反驳保留之处如下： Address-review summary — PR #8465 (round on
|
| # | Point | Classification | Disposition |
|---|---|---|---|
| 1 | Mutation-testing footnote †: removing the unknown-source guard fails 0 tests because the adjacent proof-kind check still fails closed with a different message; "worth a one-line test if you want the message pinned" | Optional suggestion (in scope) | Implemented |
| 2 | Observation (a): downgrade is silent rather than loud; worth a sentence in the release note | Optional / release-note nit | Declined as a code change — flagged for the release drafter |
| 3 | Observation (b): a committed checkpoint resets lineageTurnIds, costing two turns of repeated-blocker eligibility after each compaction |
Behavioral consequence, not a defect | Declined |
| 4 | Observation (c): per-turn flush/re-read and second goal_state record double transcript size; ~2 s/100 turns overhead at zero latency |
Informational sizing note | Declined |
| 5 | Overall LGTM and seven-item Test Plan verification | Positive review | No action required |
Changes
Point 1 — pinned unknown-source rejection message (test-only).
materializeGoalEvidenceCheckpoint catches unknown sources before the
proof-kind check; the runtime surfaces error.message as the fail-closed
reason, so the message is the observable behavior distinguishing the two
guards. The existing test only asserted the error type, which the adjacent
proof-kind check also produces — hence the reviewer's 0-test mutation result.
The matcher is now .toThrow(/cites unknown source missing/), making the
guard load-bearing. Verified by mutation: with the guard temporarily removed,
the test fails (previously 0 tests failed); with the guard restored, the whole
src/goals + session-transcript-reader selection passes 408/408. Net diff:
1 insertion, 2 deletions (the second deletion is the now-unused
InvalidGoalCheckpointError import).
Point 2 — declined as a code change, flagged for the release drafter.
CHANGELOG.md is generated automatically from GitHub Releases ("Do not edit
it by hand — run npm run changelog to regenerate"), and there is no
per-PR release-note file in this repository, so there is no in-diff place to
put the sentence. The observation itself is valid and should ride into the
release note drafted at release time: downgrading to a pre-checkpoint binary
silently skips ~58% of goal_state records and resumes from the newest
parseable (pre-checkpoint) snapshot rather than refusing. Recording it here so
it is not dropped.
Point 3 — declined with evidence. The reviewer's own report establishes
this is fail-safe (proposal rejected, Goal keeps running, same proposal
succeeds two turns later), that the two-turn window is pre-existing and
identical on base, and that a checkpoint claim can never satisfy turn coverage
by design (turnId is checkpoint:<id>). The lineage reset is inherent to
advancing evidenceCursor, which is the compaction mechanism this PR exists
for; reworking turn-coverage accounting around checkpoints would be a
behavioral expansion beyond this PR's scope for a case the reviewer explicitly
labels "not a defect".
Point 4 — declined with evidence. Explicitly "Not a blocker; just sizing
information", and invisible against real provider latency. Optimizing the
per-turn flush/re-read would be a speculative refactor (AGENTS.md Simplicity
First) with no reported user-visible defect.
Conflict notes
None (--conflict false; no merge performed).
Verification
npx vitest run src/goals/goal-checkpoint.test.ts(packages/core) — 4 passed (4)- Mutation check: unknown-source guard removed → same suite 1 failed (3 passed); guard restored before commit
npx vitest run src/goals src/services/session-transcript-reader.test.ts(packages/core) — 16 files, 408 passed (408)npm run build— passed (exit 0)npm run typecheck— passed (exit 0)npm run lint(eslint + integration-tests) — passed, no findingsnpx prettier --check packages/core/src/goals/goal-checkpoint.test.ts— passed- Commit:
ecefc20405test(core): pin unknown-source checkpoint rejection message (#8465)— 1 file changed, 1 insertion(+), 2 deletions(-); working tree clean after commit
中文说明
处理评审摘要 — PR #8465（本轮 529dde21 → ecefc20405）
反馈分诊
本轮唯一可处理的反馈是 @wenshao 的 issue 级本地运行时验证报告（结论：LGTM，可合并）。当前处于仅处理 Critical 的模式，因此审计区域中列出的非 Critical 机器人评论未做处理。没有 inline 评论、没有失败检查、没有与 base 的冲突。
| # | 要点 | 分类 | 处理 |
|---|---|---|---|
| 1 | 变异测试脚注 †：移除未知来源守卫后 0 个测试失败，因为相邻的 proof-kind 检查仍会 fail closed，只是消息不同；"如果想把消息固定下来，可以补一行测试" | 可选建议（范围内） | 已实现 |
| 2 | 观察 (a)：降级是静默的而非报错；值得在 release note 里加一句 | 可选 / release note 层面的小事 | 拒绝作为代码改动 —— 已标记给 release 起草人 |
| 3 | 观察 (b)：提交 checkpoint 会重置 lineageTurnIds，使每次压缩后的两轮内失去 repeated-blocker 资格 |
行为后果，不是缺陷 | 拒绝 |
| 4 | 观察 (c)：每轮 flush/重读加第二条 goal_state 记录使 transcript 体积翻倍；零延迟下约 2 秒/100 轮开销 |
供参考的容量信息 | 拒绝 |
| 5 | 整体 LGTM 及七项 Test Plan 验证 | 正面评审 | 无需处理 |
改动
要点 1 —— 固定未知来源拒绝消息（仅测试改动）。
materializeGoalEvidenceCheckpoint 会先于 proof-kind 检查捕获未知来源；runtime 将 error.message 作为 fail-closed 原因暴露出去，因此该消息正是区分两个守卫的可观察行为。原测试只断言了错误类型，而相邻的 proof-kind 检查抛出的也是同一类型 —— 这正是评审者变异测试得到 0 个失败的原因。现在匹配器改为 .toThrow(/cites unknown source missing/)，使该守卫真正被测试承载。已通过变异验证：临时移除该守卫后同一套件 1 个失败（此前为 0 个失败）；恢复守卫后，src/goals + session-transcript-reader 所选范围 408/408 全部通过。净 diff：1 行新增、2 行删除（其中一行删除是因此不再使用的 InvalidGoalCheckpointError 导入）。
要点 2 —— 拒绝作为代码改动，已标记给 release 起草人。
CHANGELOG.md 由 GitHub Releases 自动生成（"请勿手动编辑 —— 运行 npm run changelog 重新生成"），且本仓库没有按 PR 维护的 release note 文件，因此 diff 中没有可以放置这句话的位置。该观察本身成立，应写入发布时起草的 release note：降级到 checkpoint 之前的二进制会静默跳过约 58% 的 goal_state 记录，并从最新的可解析（checkpoint 之前的）快照恢复，而不是拒绝启动。在此记录以免被遗漏。
要点 3 —— 带证据拒绝。 评审者自己的报告已证明这是 fail-safe 的（提议被拒绝，Goal 继续运行，同样的提议两轮后即通过）、两轮窗口是既有行为且在 base 上完全一致、并且 checkpoint claim 按设计（turnId 为 checkpoint:<id>）永远无法满足 turn coverage。lineage 重置是推进 evidenceCursor 的固有结果，而游标推进正是本 PR 存在的压缩机制；围绕 checkpoint 重构 turn-coverage 计算将超出本 PR 范围，且评审者已明确将其定性为"不是缺陷"。
要点 4 —— 带证据拒绝。 评论明确写明"不是阻塞项，仅作为容量参考"，且面对真实 provider 延迟时不可见。优化每轮 flush/重读属于投机性重构（AGENTS.md Simplicity First），且没有任何用户可见缺陷报告。
冲突说明
无（--conflict false；未执行合并）。
验证
npx vitest run src/goals/goal-checkpoint.test.ts（packages/core）—— 4 通过（共 4）- 变异检查：移除未知来源守卫 → 同一套件 1 失败（3 通过）；提交前已恢复守卫
npx vitest run src/goals src/services/session-transcript-reader.test.ts（packages/core）—— 16 个文件，408 通过（共 408）npm run build—— 通过（exit 0）npm run typecheck—— 通过（exit 0）npm run lint（eslint + integration-tests）—— 通过，无发现npx prettier --check packages/core/src/goals/goal-checkpoint.test.ts—— 通过- 提交：
ecefc20405test(core): pin unknown-source checkpoint rejection message (#8465)—— 1 个文件，1 行新增、2 行删除；提交后工作树干净
Deferred non-Critical feedback
Critical-only mode is active after 10 change-producing rounds. The workflow excluded the non-Critical feedback below from this round's actionable sections; the items remain open for human follow-up. Maintainer feedback is deferred only after its author has used 2 regular feedback batches in this window's Critical-only tail; authors at that budget, if any, are named below. (@qwen-code /retry starts a fresh counting window.)
中文说明
完成 10 个产生改动的轮次后进入仅处理 Critical 的模式。本轮可执行区域已排除下方非 Critical 反馈；这些条目保持开放，留待人工跟进。维护者反馈仅在其本人于本窗口 Critical-only 阶段已使用 2 批常规反馈预算后才会延后；达到预算的作者（如有）在下方点名。（评论 @qwen-code /retry 可开启新的计数窗口。）
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。
Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。
🧠 Handled by Qwen Code · model/模型 qwen3.8-max
Local runtime re-verification — PR #8465 @
|
| commit | mutation | src/goals result |
|---|---|---|
529dde21 |
delete the if (!sourceProofKind) throw … block |
348 passed — 0 failed |
ecefc204 |
same deletion | 347 passed — 1 failed |
FAIL src/goals/goal-checkpoint.test.ts > materializeGoalEvidenceCheckpoint > rejects unknown sources and proof-kind upgrades
AssertionError: expected [Function] to throw error matching /cites unknown source missing/
but got 'Goal checkpoint claim 1 changes the p…'
The guard is now pinned by message, not just by error class. And the message the test pins is the one production actually emits — §5 shows it coming out of a live run.
Because the production tree is byte-identical to 529dde21, everything else below is a re-confirmation, not a new result.
2. Headline A/B — the dead end is still fixed on this head
A 90-turn Goal that grows the bounded catalog, then proposes completion at turn 90. Identical objective, identical mock, identical turn counts on both arms.
base a8fb97d1 |
PR ecefc204 |
|
|---|---|---|
| outcome at turn 90 | Goal usage limited |
Goal complete |
| reason | exceeded the bounded evidence catalog… | Cited evidence supports the claim. |
| checkpoint verifier calls | 0 | 1 |
| terminal verifier calls | 0 — never reached | 1 |
The base arm still never reaches terminal verification: the truncated catalog fails closed first.
One extra data point from a run where the mock's terminal verifier rejects every proposal: the PR arm ran 166 turns with 36 rejections and 2 committed checkpoints and never reached usage_limited, while base dies at turn 90 regardless. Repeated terminal rejection no longer walks the Goal into the dead end.
3. The catalog still compacts (wire-observed get_goal payloads)
Every Goal turn calls get_goal, so the catalog the next turn sees is captured off the wire:
| model request # | base — entries / truncated | PR — entries / truncated / provenance |
|---|---|---|
| 24 | 23 / false |
23 / false — 23 assistant_output |
| 48 | 47 / false |
47 / false — 47 assistant_output |
| 60 | 59 / false |
59 / false — 59 assistant_output |
| 66 | 65 / false |
2 / false — 1 goal_checkpoint + 1 assistant_output |
| 80 | 78 / true |
16 / false — 1 goal_checkpoint + 15 raw |
| 82 | 78 / true |
18 / false — 1 goal_checkpoint + 17 raw |
| 85 | 78 / true (and stays there) |
— |
| 113 | — | 46 / false — 1 goal_checkpoint + 45 raw |
The next turn sees exactly checkpoint claims + only newer raw evidence. Base saturates at 78 entries with truncated: true and never recovers.
4. Continuation is still gated on the checkpoint
Held the checkpoint verifier response open for 25 s. Zero model turns ran during the entire hold — last agent request at +1.6 s (request #47), hold +1.6 s → +26.7 s, next agent request at +26.7 s — and the TUI sits on /goal checking:
5. Crash recovery is still exact and idempotent
SIGKILL while checkpoint verification was in flight. Last durable record was turn_finished (turn 47) carrying checkpointPending.recordUuid = 95fc205a…. After qwen -c:
- the first wire event after resume is the replayed checkpoint request (same 47-record window,
previousClaims: []) — 0 agent requests before it; - the committed record uuid ==
evidenceCheckpoint.checkpointId== the newevidenceCursor.recordId==95fc205a…, byte-identical to the pre-crash marker; - exactly one continuation followed,
+0.34 slater (turn 48), then checkpoints every 47 turns — 7 of them over 292 turns, neverusage_limited.
6. Fail-closed matrix — 6/6, no cursor advance
Injected each failure at the checkpoint verifier. Every one stopped at turn 47 with usage_limited.
| injected fault | reason surfaced |
|---|---|
| proof kind flipped | Goal checkpoint claim 1 changes the proof kind of source fa2eba9d-… |
| unknown source cited | Goal checkpoint claim 1 cites unknown source bogus-uuid-not-in-catalog |
| non-JSON body | Goal checkpoint verifier returned invalid JSON |
empty claims |
Goal checkpoint verifier returned invalid claims |
| provider HTTP 500 | Failed to generate text content (side-query:goal-checkpoint-verifier): 500 … |
| oversize claims | Goal checkpoint exceeds the 16000-byte claim limit |
Checked on disk for the unknown-source run — 95 persisted goal_state records (create 1, turn_finished 47, checkpoint 46, usage_limited 1):
- exactly one distinct
evidenceCursor.recordIdacross all 95 records — the cursor never moved; - 0 records carrying
evidenceCheckpoint— nothing was ever committed; - final status
usage_limitedatturnCount: 47.
Note the live message in row 2 is exactly what the new test pins.
7. Queued-user priority — unchanged, still pre-existing
Same as last round: the Core branch is real and test-pinned, but the TUI does not surface it. Control with no checkpoint in the picture — set a Goal, queue a plain message during an ordinary active Goal, wait 25 s:
| arm | Goal turns run | user message delivered at |
|---|---|---|
base a8fb97d1 |
21 | never |
PR ecefc204 |
21 | never |
Identical on both arms, so this remains the pre-existing TUI queue gate, not anything this PR changed.
8. Gates on ecefc204
build ✅ · bundle ✅ · typecheck ✅ · lint ✅ · prettier --check on this PR's changed files ✅ · core src/goals + session-transcript-reader 408/408 ✅ · CLI touched suites 95/95 ✅ · acp-bridge 1074/1074 ✅ · full core 19338 passed / 2 failed.
Two caveats, both pre-existing and both verified identical on the base arm:
- the 2 core failures (
skill-curator,session-writer-lease) reproduce ona8fb97d1— container/root artifacts, not this PR; - a repo-wide
prettier --checkreports 6 files, the same 6 on base, none of them touched by this PR.
Non-blocking observations (unchanged from the previous round)
The production tree is byte-identical to 529dde21, so all three still stand exactly as measured:
(a) Downgrade is silent rather than loud — 106 of 182 goal_state records (58%) parse to undefined on a pre-checkpoint binary, and recovery falls back to a stale snapshot instead of refusing. Out of contract, but worth a sentence in the release note.
(b) Each committed checkpoint costs two turns of repeated-blocker eligibility (lineageTurnIds resets with the cursor). Fail-safe and pre-existing in shape; what is new is that the window now recurs roughly every 47 turns instead of once per Goal revision.
(c) finishTurn now flushes and re-reads the active transcript chain every turn and writes a second goal_state record per turn even when nothing is compacted, so goal_state transcript volume roughly doubles.
One harness note for the record: long runs with the 1500-byte filler eventually pause on the model context limit (Context is too large to send safely…). That happens identically on both arms and is an artifact of the mock's filler text, not of this PR.
Recommendation: LGTM to merge. The follow-up commit does what it says — the unknown-source guard is now pinned by message and the mutation that previously killed nothing now kills a test — and every claim from the previous verification reproduces unchanged on the merge head.
中文版
PR #8465 本地运行时复验 @ ecefc204
这是我在 529dde21 上那轮验证的后续。我在当前 head 上重建了同一套真实端到端环境并重跑了所有场景，因此下面的证据锚定的是真正会被合入的那个 commit。
- PR arm ——
ecefc20405（worktree，npm run build+npm run bundle） - Base arm ——
a8fb97d16f（merge-base，即 fix(core): stop Goal retries after evidence catalog exhaustion #8430 之后的状态） - 生产代码中的 Goal runtime、证据目录、checkpoint window、materializer、持久化、恢复以及 terminal verifier 全部原样运行；只有模型 token 流由 mock OpenAI 兼容服务伪造，因此 checkpoint verifier 的请求/响应仍然真实过网。两个 arm 都是真实打包的 CLI，在
tmux中驱动。
结论：可以合并。 所有结论在新 head 上都复现，并且我上一轮留下的那个缺口现在补上了。
1. 相比上一轮改了什么，这个改动是否真的有用
529dde21..ecefc204 的差异是一个文件、+1/−2、零行生产代码：
packages/core/src/goals/goal-checkpoint.test.ts | 3 +--
它把 .toThrow(InvalidGoalCheckpointError) 换成了 .toThrow(/cites unknown source missing/)，正是我上轮建议的那一行。上一轮我报告过：删掉 unknown-source 守卫后失败测试数为 0，因为未知来源会落到相邻的 proof-kind 检查上，仍然 fail closed，只是消息不同。在新 head 上重跑同一个变异：
| commit | 变异 | src/goals 结果 |
|---|---|---|
529dde21 |
删除 if (!sourceProofKind) throw … 块 |
348 通过 —— 0 失败 |
ecefc204 |
同样的删除 | 347 通过 —— 1 失败 |
FAIL src/goals/goal-checkpoint.test.ts > materializeGoalEvidenceCheckpoint > rejects unknown sources and proof-kind upgrades
AssertionError: expected [Function] to throw error matching /cites unknown source missing/
but got 'Goal checkpoint claim 1 changes the p…'
守卫现在是按消息固定的，而不只是按错误类型。而且测试固定的这条消息正是生产环境实际输出的那条 —— §5 展示了它出现在一次真实运行里。
由于生产代码树与 529dde21 逐字节相同，下面其余内容属于复现确认，而非新结论。
2. 核心 A/B —— 死路在新 head 上依然被修复
一个 90 轮的 Goal，先撑大有界目录，然后在第 90 轮提出完成。两个 arm 的 objective、mock、轮数完全一致。
base a8fb97d1 |
PR ecefc204 |
|
|---|---|---|
| 第 90 轮结果 | Goal usage limited |
Goal complete |
| 原因 | exceeded the bounded evidence catalog… | Cited evidence supports the claim. |
| checkpoint verifier 调用 | 0 | 1 |
| terminal verifier 调用 | 0 —— 根本没走到 | 1 |
base arm 依然连 terminal verification 都到不了：truncated 的目录先 fail closed。
额外一个数据点：在 mock 的 terminal verifier 对每一次提议都拒绝的运行中，PR arm 跑了 166 轮、被拒 36 次、提交了 2 次 checkpoint，且从未进入 usage_limited；而 base 无论如何都在第 90 轮死掉。反复的终态拒绝不会再把 Goal 逼进死路。
3. 目录依然会被压缩（从 wire 上抓到的 get_goal payload）
每个 Goal turn 都调用 get_goal，因此下一轮看到的目录可以直接从网络请求中抓取：
| 模型请求序号 | base —— entries / truncated | PR —— entries / truncated / provenance |
|---|---|---|
| 24 | 23 / false |
23 / false —— 23 assistant_output |
| 48 | 47 / false |
47 / false —— 47 assistant_output |
| 60 | 59 / false |
59 / false —— 59 assistant_output |
| 66 | 65 / false |
2 / false —— 1 goal_checkpoint + 1 assistant_output |
| 80 | 78 / true |
16 / false —— 1 goal_checkpoint + 15 raw |
| 82 | 78 / true |
18 / false —— 1 goal_checkpoint + 17 raw |
| 85 | 78 / true（此后不再变） |
—— |
| 113 | —— | 46 / false —— 1 goal_checkpoint + 45 raw |
下一轮看到的正是 checkpoint claims + 仅更新的原始证据。base 在 78 条处饱和并永久保持 truncated: true。
4. 续跑依然被 checkpoint 挡住
把 checkpoint verifier 的响应挂起 25 秒。整个挂起期间没有任何模型轮次运行 —— 最后一次 agent 请求在 +1.6s（第 47 次请求），挂起 +1.6s → +26.7s，下一次 agent 请求在 +26.7s —— TUI 停在 /goal checking。
5. 崩溃恢复依然精确且幂等
在 checkpoint 验证进行中 SIGKILL。最后一条持久记录是 turn_finished（第 47 轮），携带 checkpointPending.recordUuid = 95fc205a…。执行 qwen -c 之后：
- 恢复后的第一个网络事件就是重放的 checkpoint 请求（同样的 47 条 window，
previousClaims: []）—— 在它之前没有任何 agent 请求； - 提交的 record uuid ==
evidenceCheckpoint.checkpointId== 新的evidenceCursor.recordId==95fc205a…，与崩溃前的标记逐字节一致； - 之后只启动了一次 continuation（
+0.34s，第 48 轮），随后每 47 轮一次 checkpoint —— 292 轮内共 7 次，且从未进入usage_limited。
6. Fail-closed 矩阵 —— 6/6，游标未推进
在 checkpoint verifier 处注入各类故障，每一种都在第 47 轮停下并转为 usage_limited。
| 注入的故障 | 暴露的原因 |
|---|---|
| 翻转 proof kind | Goal checkpoint claim 1 changes the proof kind of source fa2eba9d-… |
| 引用未知来源 | Goal checkpoint claim 1 cites unknown source bogus-uuid-not-in-catalog |
| 非 JSON 响应 | Goal checkpoint verifier returned invalid JSON |
空 claims |
Goal checkpoint verifier returned invalid claims |
| provider HTTP 500 | Failed to generate text content (side-query:goal-checkpoint-verifier): 500 … |
| 超长 claims | Goal checkpoint exceeds the 16000-byte claim limit |
对 unknown-source 那次运行检查磁盘 —— 95 条持久化 goal_state 记录（create 1、turn_finished 47、checkpoint 46、usage_limited 1）：
- 95 条记录中
evidenceCursor.recordId只有一个取值 —— 游标从未移动； - 0 条记录带
evidenceCheckpoint—— 从未提交过任何 checkpoint； - 最终状态为
usage_limited，turnCount: 47。
注意第 2 行的真实运行消息，正是新增测试所固定的那条。
7. 排队用户输入优先 —— 未变，仍是既有问题
与上一轮一致：Core 侧分支是真实且有测试固定的，但 TUI 不会体现它。对照实验（完全不涉及 checkpoint）—— 设置 Goal，在普通 active 状态下排入一条普通消息，等待 25 秒：
| arm | 运行的 Goal 轮数 | 用户消息何时送达 |
|---|---|---|
base a8fb97d1 |
21 | 从未 |
PR ecefc204 |
21 | 从未 |
两个 arm 完全一致，因此这仍是既有的 TUI 队列门控，与本 PR 无关。
8. ecefc204 上的门禁
build ✅ · bundle ✅ · typecheck ✅ · lint ✅ · 对本 PR 改动文件的 prettier --check ✅ · core src/goals + session-transcript-reader 408/408 ✅ · CLI 相关套件 95/95 ✅ · acp-bridge 1074/1074 ✅ · core 全量 19338 通过 / 2 失败。
两点说明，均为既有问题且已在 base arm 上验证完全一致：
- 2 个 core 失败（
skill-curator、session-writer-lease）在a8fb97d1上同样复现 —— 属于容器/root 环境问题，与本 PR 无关； - 仓库级
prettier --check报告 6 个文件，在 base 上是同样的 6 个，且没有一个被本 PR 改动。
非阻塞观察（与上一轮相同）
生产代码树与 529dde21 逐字节相同，因此三条观察全部按原测量结果继续成立：
(a) 降级是静默的而不是报错的 —— 182 条 goal_state 中有 106 条（58%）在 checkpoint 之前的二进制上解析为 undefined，恢复会回退到陈旧快照而不是拒绝启动。虽然降级本就不在契约内，但值得在 release note 里加一句。
(b) 每次提交 checkpoint 都要花掉两轮的 repeated-blocker 资格（lineageTurnIds 随游标重置）。形态上是 fail-safe 且既有的；新增之处在于该窗口现在大约每 47 轮反复出现一次，而不再是每个 Goal revision 一次。
(c) finishTurn 现在每一轮都会 flush 并重读活动 transcript chain，且即使没有压缩也会每轮多写一条 goal_state 记录，因此 goal_state 的 transcript 体积大致翻倍。
另记一条 harness 说明：使用 1500 字节填充文本的长时间运行最终会因模型上下文上限而暂停（Context is too large to send safely…）。这在两个 arm 上完全一致，是 mock 填充文本的产物，与本 PR 无关。
建议：LGTM，可以合并。 这个后续 commit 做到了它声称的事 —— unknown-source 守卫现在按消息固定，之前杀不掉任何测试的那个变异现在能杀掉一个测试 —— 并且上一轮验证的每一条结论都在合入 head 上原样复现。
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix round summary — no action taken (PR #8465)This round's feedback contains no actionable findings, so no code changes were made and the PR head remains What was reviewed
Non-blocking observations — deferred, not actioned The comment restates three explicitly non-blocking observations from the previous round: (a) pre-checkpoint binaries parse new 中文说明Autofix 轮次总结 —— 未执行任何操作（PR #8465）本轮反馈中没有可执行的发现，因此未修改任何代码，PR head 保持为 本轮审查内容
非阻塞观察 —— 已延后，未处理 该评论重申了上一轮的三条明确标注为非阻塞的观察：(a) checkpoint 之前的旧版本二进制会静默解析新的 Deferred non-Critical feedbackCritical-only mode is active after 10 change-producing rounds. The workflow excluded the non-Critical feedback below from this round's actionable sections; the items remain open for human follow-up. Maintainer feedback is deferred only after its author has used 2 regular feedback batches in this window's Critical-only tail; authors at that budget, if any, are named below. ( 中文说明完成 10 个产生改动的轮次后进入仅处理 Critical 的模式。本轮可执行区域已排除下方非 Critical 反馈；这些条目保持开放，留待人工跟进。维护者反馈仅在其本人于本窗口 Critical-only 阶段已使用 2 批常规反馈预算后才会延后；达到预算的作者（如有）在下方点名。（评论 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
|
Review: architecture is solid and well-tested (claims bounded to <=32/<=16KB with Core-assigned ids, proofKind can't change across a checkpoint — anti-forgery invariant enforced, cursor advances atomically with the checkpoint, crash recovery re-runs checkpointPending with a double-apply guard, replay/resume suppression consistent, legacy goals unchanged, protocol/config strictly additive). No duplication with main. Holding approval for one P1 and a few P2s worth addressing or explicitly accepting: P1 — unbounded window content can permanently abort a healthy goal. The checkpoint window sends the FULL content of every post-cursor raw record (previews are bounded, contents are not — renderToolResponse stringifies entire tool responses); only the aggregate 256KB verifier-request cap applies. Exceeding it -> usage_limited with GOAL_EVIDENCE_CATALOG_EXHAUSTED_REASON, even though the catalog itself was never exhausted — misleading, and resume is blocked until edit/replace. A handful of large tool outputs (large file reads/greps, common in coding) in a revision's first checkpoint can trip this. Suggest: cap per-record content in the window (truncate with an omission marker) or checkpoint incrementally; at minimum use a distinct, accurate reason string. P2 — single-shot verifier failure -> usage_limited. Any checkpoint-verifier failure (30s timeout, transport error, invalid JSON, hallucinated sourceRef, proof-kind mismatch) goes through recordCheckpointFailure -> usage_limited with maxAttempts:1 and no repair. Checkpoints run automatically every turn past threshold, so a long-running goal faces this exposure on every turn; one transient LLM hiccup kills the goal. Consider bounded repair retry for structural failures, or 'skip checkpoint, retry next turn' while rawEntries are far below catalog caps. P2 — user_input fidelity after a checkpoint. Post-checkpoint, user_input claims are LLM-compressed summaries; the raw user messages are pre-cursor and no longer citable, yet the terminal verifier policy accepts user_input content as proof of user authorization. A distorted claim that 'the user approved X' could become the sole record and later legitimate a terminal decision. Consider keeping raw real_user records citable across checkpoints (exempt from cursor advance), or requiring original user records for authorization claims. P2 — truncation fail-closed expanded to all terminal proposals (previously only complete); confirm intended, and consider still permitting authority blockers whose required user evidence is catalogued. P3s: share the one replay/resume suppression predicate (currently two that could drift); release-note the mixed-version forward-compat degradation (old builds drop checkpointPending/'checkpoint' records); and consider hysteresis / off-critical-path checkpointing since dense checkpoints (claim text near the 16KB budget) keep catalogBytes >= the trigger and re-checkpoint every subsequent turn. |
|
🤖 Addressed the latest review feedback (round 12/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 12/100 轮）。改动内容与我反驳保留之处如下： Round summary for PR #8465Addressed the actionable issue-level review from @yiliang114 (one P1, three P2s, three P3s). Critical-only mode is active; the deferred non-Critical section was left untouched. No inline comments were present this round. P1 — unbounded window content can permanently abort a healthy goal — FIXED
P2 — single-shot verifier failure → usage_limited — FIXED
P2 — user_input fidelity after a checkpoint — ESCALATED (maintainer decision)This turns on a security/scope tradeoff that is not this bot's to make. The concern: post-checkpoint, raw user messages are pre-cursor and no longer citable, while the terminal verifier accepts P2 — truncation fail-closed expanded to all terminal proposals — CONFIRMED INTENDED, refinement DECLINED
P3 — share one replay/resume suppression predicate — FIXED
P3 — release-note the mixed-version forward-compat degradation — NOTED for the release author
P3 — hysteresis / off-critical-path checkpointing — DECLINED this roundWhen claims sit near the 16KB budget, Verification
Commit: 中文说明PR #8465 本轮总结处理了 @yiliang114 在 issue 层面的可执行评审意见（1 个 P1、3 个 P2、3 个 P3）。当前处于仅处理 Critical 的模式，被延后的非 Critical 部分未做任何改动。本轮没有行内评论。 P1 — 无上限的窗口内容可能永久中止健康 Goal — 已修复
P2 — 单次验证器失败即 usage_limited — 已修复
P2 — 检查点之后的 user_input 保真度 — 上报（需维护者决策）这涉及一个安全/范围权衡，不应由本机器人擅自决定。顾虑是：检查点之后，原始用户消息位于 cursor 之前、不再可引用，而终端验证器接受 P2 — 截断失败关闭扩展到所有终端提案 — 确认为有意设计，拒绝细化建议
P3 — 共享单一的 replay/resume 抑制谓词 — 已修复
P3 — 将混合版本前向兼容降级写入发布说明 — 已提请发布负责人注意
P3 — 迟滞（hysteresis）/ 将检查点移出关键路径 — 本轮拒绝当 claims 接近 16KB 预算时， 验证
提交： Deferred non-Critical feedbackCritical-only mode is active after 10 change-producing rounds. The workflow excluded the non-Critical feedback below from this round's actionable sections; the items remain open for human follow-up. Maintainer feedback is deferred only after its author has used 2 regular feedback batches in this window's Critical-only tail; authors at that budget, if any, are named below. (
中文说明完成 10 个产生改动的轮次后进入仅处理 Critical 的模式。本轮可执行区域已排除下方非 Critical 反馈；这些条目保持开放，留待人工跟进。维护者反馈仅在其本人于本窗口 Critical-only 阶段已使用 2 批常规反馈预算后才会延后；达到预算的作者（如有）在下方点名。（评论 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 693 passed · 0 failed · 693 total 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：693 通过 · 0 失败 · 693 总计 Verification reportPR #8465 — feat(core): checkpoint long-running Goal evidenceVerdict: 中文摘要
Previous-finding status (follow-up round)
Carried-forward central A/B (S1/S2/S3) and gates were rebuilt and re-run from scratch at the new head — sections below. Central claim and A/B proofCentral claim: before the bounded evidence catalog reaches its hard cap (100 entries / 24 KB), the runtime pauses continuation, compresses cumulative evidence through a tool-free verifier into ≤32 bounded claims (≤16,000 bytes of claim text), validates every citation and proof kind, persists the checkpoint atomically, advances the evidence cursor, and resumes — converting the #8430 dead end into a completable long Goal. Harness S1 — long Goal crosses the threshold, then completes across windows (turn A: 80 tool results; turn B: 25 more + delivered output; head cites the checkpoint claim, base cites the newest raw record, mirroring what each catalog presents):
S2 — truncated catalog (120 entries) + Delta verification (new probes for commits
|
| oracle (per fault) | measured |
|---|---|
| goal status after the fault | active (not usage_limited) |
| activity | running — continuation turn started |
| evidenceCheckpoint | absent; cursor not advanced |
| journal tail | bookkeeping checkpoint record; zero usage_limited records |
| next turn with healthy verifier | checkpoint commits, cursor advances (retry works) |
Fail-closed remains for evidence-source flush/read failures and catalog truncation at checkpoint time (suite moves to usage_limited when checkpoint %s fails, green in gates). The pinned unknown-source message (ecefc20) verified at materialize level: Goal checkpoint claim 1 cites unknown source ghost-uuid. Witness: 02-softened-failure-matrix.png.
D2 — bounded checkpoint windows (03-delta-bounding.mjs, 19/19). Scenario: 77 small + 3×120 KB tool outputs among the 80 threshold records.
| build | request bytes | outcome |
|---|---|---|
| head (2 KB content cap on) | ≤ 256 000 (all contents ≤ 2000 B, exactly 3 carry \n…[truncated]) |
checkpoint commits, goal active |
M4 mutant (cap removed from a copy of head dist) |
> 256 000 | usage_limited + GOAL_CHECKPOINT_REQUEST_TOO_LARGE_REASON, no checkpoint |
Identical scenario, only the cap differs — the cap is what keeps one oversized tool output from exhausting a healthy Goal. The real createGoalCheckpointVerifier size gate corroborated without network: oversized input throws the genuine GoalCheckpointVerifierInputTooLargeError (byteLength > 256 000) before any provider path; capped input passes the gate and fails later, differently. Runtime wiring: InputTooLarge → usage_limited with the request-limit reason, and the reducer blocks resume of a request-limited goal (edit or replace). Witness: 03-bounded-window-uncap-flip.png.
Corrections
- The PR description's test-plan bullet 5 is stale relative to the final head. It states "Exercise malformed claims, changed proof semantics, unknown sources, catalog truncation, source I/O failures, and provider failures; each must fail closed." The final commit
d143f20("soften verifier failures") deliberately changed 4 of those 6 — malformed claims, changed proof semantics, unknown sources, provider failures — to settle as bookkeeping with the goal stayingactiveand retrying on a later turn (measured in D1; pinned by the suite'sskips a checkpoint that changes source proof semantics/keeps a goal active when the checkpoint verifier provider fails). Only catalog truncation and source-I/O failures still fail closed, plus the new structurally-oversized-request case. The "without advancing the evidence cursor" half still holds for all six. This is a description correction, not a request to change code — the code and its tests agree.
Findings
No blocking findings.
- (informational) Downgrade is a silent rewind, not an error — carried-forward F1, re-measured and stands. With an older parseable record present, a pre-checkpoint binary silently rolls the goal back to
turnCount 0(checkpoint and committed-turn progress lost, goal still active and serviceable); with only new-shaped records it throwsGoalPersistenceUnavailableError. The silent variant is the quieter failure mode; the PR documents downgrade as unsupported, so this matches the stated tradeoff. - (informational) Harness-only artifact, not a PR defect:
instanceof GoalCheckpointVerifierInputTooLargeErroris module-identity scoped. Driving a mutant copy ofdistwhile throwing the head module's error class made the runtime misclassify the oversized request as transient (settled instead of fail-closed). In production the runtime andcreateGoalCheckpointVerifiershare one module instance, so the guard holds; recorded so future refactors that duplicate the class across bundles keep this in mind.
Reviewer Test Plan walk-through
| plan step | result at head d143f20 |
|---|---|
| 1. Threshold turn without proposal; next turn only after verified durable checkpoint; sees claims + newer evidence | verified — S1 A/B |
| 2. Interrupt after pending marker; restore; verification completes before exactly one continuation | verified — S3 |
| 3. Terminal reject near threshold; checkpoint completes; next turn receives exact verifier feedback | covered by suite (checkpoints before continuing after the terminal verifier rejects, keeps rejection feedback for continuation when the checkpoint fails after rejection; green in gates) — not independently A/B'd |
| 4. Queued real user input admitted before automatic continuation | covered by suite (promotes queued user input before an automatic post-checkpoint turn, green) |
| 5. Malformed claims / changed proof semantics / unknown sources / truncation / source I/O / provider failures fail closed without cursor advance | partially superseded by the delta — see Corrections: 4 of 6 now settle-and-retry (verified D1, cursor never advanced), truncation + source I/O still fail closed (suite + S2) |
6. External-blocker proposal after truncation → usage_limited, no continuation |
verified — S2 head cell |
| 7. Repeated blocker keeps three-turn raw lineage | covered by suite (blocked-audit tests green in the 352) |
Mutation matrix (vacuity of the delta's guards)
| # | mutation (scratch edit, restored after) | suite | result |
|---|---|---|---|
| M5 | softening reverted: generic checkpoint-verifier error → recordCheckpointFailure (fail-closed) |
goal-runtime.test.ts |
killed — exactly skips a checkpoint that changes source proof semantics + keeps a goal active when the checkpoint verifier provider fails red, behavioral (usage_limited/idle vs expected active/running); sibling fail-closed tests stayed green |
| M6 | InputTooLarge branch → finishCheckpointCheck |
goal-runtime.test.ts |
killed — exactly records the request limit when the checkpoint request is structurally oversized red (active vs expected usage_limited) |
| M7 | capCheckpointContent pass-through (cap removed) |
goal-evidence.test.ts |
killed — exactly the two cap tests red (caps oversized window content with a truncation marker, caps window content on a code point boundary for multi-byte text), 27 others green |
Positive controls: three independent kills, each failing the intended behavioral assertion; restored tree re-ran the full goals suite green (352/352). Witnesses: 06-mutation-m5-softening-red.png, plus the runtime-level M4 flip in 03-bounded-window-uncap-flip.png. No survivors among the mutated delta hunks.
Targeted gates
| suite | result |
|---|---|
packages/core src/goals/ (15 files) |
352/352 |
packages/core session-transcript-reader.test.ts |
78/78 |
packages/acp-bridge transcript-replay.test.ts |
30/30 |
packages/cli resumeHistoryUtils + goal-runtime + clipboardUtils + history-replay-page |
100/100 |
Gate liveness proven by M5–M7 (the same harness goes red under mutation). Witness: 07-gates-green.png.
Not covered
- Per-commit attribution: depth-2 checkout (merge commit + base tip + PR head); the metadata lists 18 commits, only 3 reachable locally (
git rev-list HEAD^1..HEAD^2returns 1 at the shallow boundary). Verified the aggregateHEAD^1..HEADdiff only; the delta behavior was attributed tod143f20/ecefc20via metadata messages + code/test content, not per-commit execution. - Checkpoint verifier wire path (
createGoalCheckpointVerifier→runSideQuery→ provider): needs a live model; its size gate, schema validation, and parse/materialize limits were exercised without network (D3a + gates). - Base control build compiled against the head tree's
node_modulesvia symlinks (lockfile byte-identical, so deps are not a confound); the first attempt reproduced pre-existing type errors in unrelated files (mime/lite,fdir,ignore,ajv) caused by the worktree missing npm's nestedpackages/core/node_modulescopies — environmental, fixed by symlinking the identical nested tree; the clean rebuild then exited 0. - Windows/macOS; live TUI frames of the suppressed bookkeeping cards; ACP replay through a real client (machine-level tests only); repo-wide lint/typecheck (CI-covered); the
goalCausebackward-pagination seed through a real multi-page transcript (unit tests only).
Methodology
Environment: node:22-bookworm container (node v22.23.2), repo at refs/pull/8465/merge (depth 2), pre-built at head; base control via git worktree add tmp/base-tree HEAD^1 + the repo's own build_package.js with the nested node_modules symlinked in. Harnesses (lib.mjs, 01-ab.mjs, 02-delta-softening.mjs, 03-delta-bounding.mjs, 04-downgrade.mjs, 05-bookkeeping.mjs, kept in this artifact dir) import each arm's compiled dist/ and drive createGoalRuntime through its public ports — fake journal (whose goal_state records land in the transcript chain exactly as the real recording service does), fake evidence source, fake terminal/checkpoint verifiers, fake host; no module interception, no stubbing of the unit under test. The M4 uncap mutant is a patched copy of head dist (single constant changed). Mutations M5–M7 were scratch edits to source, run under vitest, then restored (git checkout) with full-suite green confirmation. Assertion counts: 128 harness checks + 560 targeted-gate tests + 5 mutation kills = 693, all executed, 0 unexpected failures. Raw logs in logs/; captures in evidence/ (01 A/B cells, 02 softening matrix, 03 bounding/uncap flip, 04 downgrade, 05 bookkeeping, 06 M5 red, 07 gates green).
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix review round: no actionNo new actionable feedback arrived this round, so no changes were made.
No code was modified, no commit was created, no review threads were resolved, and no base-branch merge was performed ( 中文说明Autofix 审查轮次：无操作本轮没有收到新的可执行反馈，因此未做任何改动。
未修改任何代码，未创建提交，未解决任何审查线程，也未进行 base 分支合并（ Deferred non-Critical feedbackCritical-only mode is active after 10 change-producing rounds. The workflow excluded the non-Critical feedback below from this round's actionable sections; the items remain open for human follow-up. Maintainer feedback is deferred only after its author has used 2 regular feedback batches in this window's Critical-only tail; authors at that budget, if any, are named below. (
中文说明完成 10 个产生改动的轮次后进入仅处理 Critical 的模式。本轮可执行区域已排除下方非 Critical 反馈；这些条目保持开放，留待人工跟进。维护者反馈仅在其本人于本窗口 Critical-only 阶段已使用 2 批常规反馈预算后才会延后；达到预算的作者（如有）在下方点名。（评论 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
|
@qwen-code /review |
yiliang114
left a comment
yiliang114
left a comment
There was a problem hiding this comment.
LGTM. All round-6 requested changes are addressed at this head — verified in the diff: eligibility now trims part.text and mirrors the functionResponse.response !== undefined check; the byte budget is measured on trimmed claim text only with the persisted-record parser aligned via TextEncoder; and the checkpoint payload now ships only {id, proofKind, claim} instead of uncitable source refs.
The lifecycle core checks out: the evidence cursor never advances before the journal write resolves, continuations are gated on checkpointAttempt with no double-continuation path, crash recovery replays with the persisted recordUuid, and verifier output is strictly validated (source existence, proof-kind invariance, byte/code-point caps) before materialization and re-validated on recovery. CI is green on this head.
Two non-blocking follow-ups:
- The 256 KB verifier request limit is still reachable despite the new 2 KB content cap:
capCheckpointContentcaps raw UTF-8 bytes, butverifierContentsserializes viaJSON.stringify, which escapes quotes/backslashes (2-6x). A near-cap window of quote-heavy tool output can exceed the request limit and fail closed tousage_limited, parking exactly the dense long-running Goals this feature protects. Might be worth budgeting against serialized size, or conservatively loweringCHECKPOINT_CONTENT_BYTE_LIMIT. - Minor: a tool response whose raw
JSON.stringifythrows (e.g. BigInt values) can be catalog-eligible viasummarizeJsonValuewhilerenderToolResponsereturns empty — fail-closed, but worth sharing one renderability guard between the preview and content paths.
|
Released in v0.21.8. |
5 participants
What this PR does
This adds a durable, Core-owned evidence checkpoint for long-running Goals. Before the bounded evidence catalog reaches its hard limit, the runtime pauses automatic continuation, asks an independent tool-free verifier to compress the cumulative evidence into strictly bounded claims, validates every cited source and proof kind, persists the checkpoint atomically, advances the evidence cursor, and then resumes the next Goal turn.
The checkpoint lifecycle is recoverable across process crashes, preserves terminal-verifier feedback, gives queued real user input priority over automatic continuation, keeps repeated-blocker audits on their required three raw turns, and fails closed to
usage_limitedwhen evidence, persistence inputs, or checkpoint verification cannot be trusted. A terminalblockedproposal cannot reinterpret catalog exhaustion as an external blocker: a truncated catalog always fails closed before terminal verification.Why it's needed
#8430 is the first-stage safety fix: it stops the retry loop after the evidence catalog is already truncated. It intentionally does not make a long Goal completable. This follow-up prevents ordinary long-running Goals from reaching that dead end by compacting verified evidence before the bound is exhausted, while retaining the existing fail-closed behavior for unrecoverable cases.
Reviewer Test Plan
How to verify
usage_limitedwithout another continuation.Evidence (Before & After)
Before: #8430 stops the infinite retry after catalog exhaustion by moving the Goal to
usage_limited, but that Goal revision can no longer complete. After: automated lifecycle assertions showturn_finished -> checkpoint verification -> durable checkpoint -> continuation, including crash recovery, verifier-reject recovery, queued-user priority, cumulative replacement, and fail-closed paths. This is runtime behavior with no visual UI change, so screenshots are N/A.Tested on
Environment (optional)
macOS, Node.js 22; 323 Goal tests, 3 CLI projection tests, Core and root TypeScript checks, repository build, Prettier, ESLint, and independent lifecycle verification all passed locally.
Risk & Scope
Linked Issues
Follow-up to #8430. Related to #8458.
中文说明
本 PR 做了什么
本 PR 为长时间运行的 Goal 增加了一个持久化、由 Core 拥有的证据 checkpoint。在有界证据目录达到硬上限之前，runtime 会暂停自动续跑，让独立且无工具权限的 verifier 将累计证据压缩为严格受限的 claims，校验每个引用来源和 proof kind，原子持久化 checkpoint，推进证据游标，然后再恢复下一轮 Goal。
checkpoint 生命周期支持进程崩溃恢复，会保留 terminal verifier 的反馈，让排队中的真实用户输入优先于自动续跑，确保 repeated-blocker 审计仍保留所需的三个原始 turn；当证据、持久化输入或 checkpoint 验证不可信时，会 fail closed 到
usage_limited。终态blockedproposal 不能把目录耗尽重新解释成外部阻塞：目录一旦 truncated，就会在 terminal verification 前直接 fail closed。为什么需要
#8430 是第一阶段的安全修复：它会在证据目录已经 truncated 后停止重试循环，但有意没有让长 Goal 重新具备可完成性。本次第二阶段修复会在达到上限之前压缩已验证证据，使普通长时间运行的 Goal 不会走到该死路，同时对不可恢复情况保留现有的 fail-closed 行为。
Reviewer 测试计划
如何验证
usage_limited，且不会再启动 continuation。证据（Before & After）
Before：#8430 会在目录耗尽后把 Goal 转为
usage_limited，停止无限重试，但该 Goal revision 无法继续完成。After：自动化生命周期断言覆盖了turn_finished -> checkpoint verification -> durable checkpoint -> continuation，包括崩溃恢复、verifier-reject 恢复、用户输入优先级、累计 checkpoint 替换以及 fail-closed 路径。本次是 runtime 行为变化，没有视觉 UI 变化，因此截图为 N/A。已测试平台
环境（可选）
macOS、Node.js 22；本地已通过 323 个 Goal 测试、3 个 CLI projection 测试、Core 和根目录 TypeScript 检查、仓库构建、Prettier、ESLint 以及独立生命周期验证。
风险与范围
关联 Issue
#8430 的后续阶段。关联 #8458。