feat(goal): stop a Goal whose checkpoints stall three times in a row - #9975
Conversation
QwenLM#9835 made a truncated evidence window compact instead of stopping the Goal. That removed the only terminator on the path where compaction runs and gives no relief: a Goal whose evidence rate outruns the catalog pays a checkpoint verifier call every turn, loses evidence every turn, and never converges on its own. A checkpoint is counted as stalled when it comes back holding the maximum number of claims while the window it compacted was already truncated. Compaction has two levers, folding evidence into claims and moving the cursor past what was folded; that combination means the first is exhausted (the next checkpoint can only merge) and the second is not keeping up (eligible evidence was left behind). A busy turn that truncates with room in the claims is not a stall, and a full claim list on a quiet Goal is not either. The streak is persisted as `GoalRecord.checkpointStalls` (absent means zero) so a restart or resume cannot launder it. Any check that finds room resets it, and so do edit and replace. After three consecutive stalls the Goal settles as `usage_limited` with the existing `limitKind: 'evidence_catalog'` and a reason naming what stalled, how many times, and what to do. No new limit kind, no branch in `goalLimitKindForReason`, nothing crosses the wire. Mutation probes (goal-runtime + goal-reducer + goal-checkpoint, 192 tests): no increment -> 3 fail; no reset on an effective checkpoint -> 1; no reset on a quiet check -> 1; threshold >= to > -> 1; parse drops the key -> 1; parse never restores it -> 1; predicate ignores truncation -> 2; predicate ignores the claim cap -> 1; edit stops resetting -> 1. Every other test green in every run.
|
@qwen-code /takeover |
qwen-code-dev-bot
added
the
autofix/takeover
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. This is a fork PR, so the first round comes from the next scheduled scan (usually within minutes). Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。本 PR 来自 fork，首轮处理将由下一次定时扫描执行（通常几分钟内）。移除 |
|
Re-run at the new head — the autofix rounds landed three things since the last pass: the resume reset (#9840 merged into the base), the error-class plumbing for unusable verifier results, and the orphaned-comment fix. Re-gating from scratch against Template: complete, including the bilingual section. ✓ Problem: real, and now demonstrated rather than argued. After #9835 a truncated window compacts instead of stopping, and claims pinned at the cap while the window keeps overflowing leaves no exit: every turn pays a checkpoint verifier model call and still loses evidence, forever. The maintainer's independent A/B harness in this thread reproduces it live on the merge-base build — 19 checkpoint calls and still Direction: aligned. This is a core Goal-runtime bound in the same family as the existing catalog-exhausted and request-too-large stops, reusing their settle path and their resume semantics. The reference runtime bounds the analogous thrash the same way. Size: all nine files are in Approach: still minimal. One pure predicate, one persisted record field, counter plumbing at the existing checkpoint settle points, reuse of the Risk: no elevated signals — none of the changed files matches the revert-correlated paths. Moving on to code review. 🔍 中文说明在新 head 上重跑——自上次分诊后，autofix 轮次落地了三件事：resume 重置（#9840 已合入基线）、不可用 verifier 结果的错误类型接线、孤立注释修复。在 **模板：**完整，含中文对照 ✓ **问题：**真实存在，且现在是被演示出来的而非仅靠推理。#9835 之后截断窗口会压缩而非停止；claim 顶到上限而窗口持续溢出时没有出口：每轮都付一次 checkpoint verifier 模型调用，同时还在丢证据，永不收敛。维护者在本帖中的独立 A/B 验证已在 merge-base 构建上实时复现——19 次 checkpoint 调用后仍然 **方向：**对齐。这是 Goal 运行时边界，与既有的 catalog 耗尽、请求过大两个停止同族，复用它们的落定路径与 resume 语义。参考运行时对同类抖动的约束方式一致。 **规模：**九个文件全部位于 **方案：**仍然最小化。一个纯谓词、一个持久化记录字段、在既有 checkpoint 落定点做计数接线、复用 **风险：**无升级信号——改动文件均未命中与 revert 相关的高风险路径。 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code review (re-run at
|
| Check | Conclusion |
|---|---|
Classify PR |
✅ success |
Dependency CVE audit |
✅ success |
Desktop Shell (ubuntu-22.04) |
✅ success |
Desktop Shell (windows-2022) |
✅ success |
Secret scan (TruffleHog) |
✅ success |
Test (ubuntu-latest, Node 22.x) |
✅ success |
web-shell E2E Smoke (ubuntu-latest, Node 22.x) |
✅ success |
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。
The central claim is behavioural, and it is substantiated beyond the suite: the maintainer's independent harness in this thread ran an A/B on the real compiled artifacts — the merge-base build pays 19+ checkpoint calls and stays active forever, this build settles usage_limited at the third stall (3 calls, 17 s); the streak survives a real process restart; a 31-claim control arm is never counted; and a mutant that degrades the error class to a plain Error brings the unbounded loop back, proving the class plumbing load-bearing. A sandboxed @qwen-code /verify run is already in flight on this head and will post its own report; the claim does not rest on the author's word.
Not verified here: this run executes no PR code (triage rules); live behaviour rests on the maintainer's in-thread harness and the pending /verify report. The author's mutation-probe numbers are their own claim, but the maintainer's independent 12-mutant sweep (all killed) covers the same ground.
中文说明
代码审查（在 f45ae6f3a1 上重跑）：我独立写下的方案（持久化连续停滞计数；停滞 = 窗口截断且 claim 顶到上限；任何发现有空间的检查加 edit/replace 重置；经由现有 usage_limited/evidence_catalog 路径落定）与 PR 一致。新 head 还落地了评审循环提出的两个 Critical，我逐一在代码中核实了修复本身，而非只看 diff 叙述：
1）连续计数不该重置的路径（第 1 轮）。 finishCheckpointCheck 现在带 outcome 参数——'room' 重置、'stalled' 递增、缺省 'inconclusive' 保留计数。我走查了所有调用点：'room' 只出现在无需 checkpoint 的检查；'stalled' 只出现在已截断窗口上的不可用结果；空轮次的簿记关闭保留计数（有测试钉住）。verifier 瞬时失败不再洗掉计数——它对窗口什么都没证明，所以什么都不计。
2）错误类型成为承重墙（第 3 轮）。 不可用结果按 instanceof InvalidGoalCheckpointError 计数，因此解析移出了 runSideQuery 的 validate 钩子——我在 sideQuery.ts 中确认该钩子会把失败重新包成普通 Error，悄悄丢掉这个类。现在所有解析层拒绝都抛 InvalidGoalCheckpointError；materializeGoalEvidenceCheckpoint 本就对所有不可用形状（空 claims、畸形 claim、未知引用源、字节上限）抛该类；新增测试在三种拒绝形状上钉住了类型。validateGoalCheckpointVerifierText 删除后已无任何引用（grep 核实）。
3）孤立 JSDoc 小瑕疵已修——takeTurnTokens 重新持有自己的注释。
上次评审以来的新增：#9840 的后续项已落地。证据受限的 resume 清除计数（与 edit 相同的重置，因为它重启窗口）；暂停态 resume 保留计数（同一窗口、同一事实）；两者都有 reducer 测试钉住。我直接核对了 reducer 结构：edit 与证据受限 resume 分支向 transitionGoal 传入 checkpointStalls: undefined，暂停态 resume 只展开 { status: 'active' }，字段恰在该保留处保留。
其余机制依旧成立：计数发生在 materialization 之后，可重试失败不会推进计数；触发停止的 checkpoint 被丢弃，落定时连 limitKind: 'evidence_catalog' 一起持久化——安全，因为 resume 门以 limitKind 为准（已在 reducer 与 web-shell 的 goalGate.ts 中核实）；持久化严格（只接受非负整数，-1/1.5 整体拒绝，0 恢复为无字段）；checkpointStalls 在 packages/core/src/goals/ 之外零出现，与 SDK 既有的更窄投影一致（本就省略 evidenceCheckpoint/tokensUsed）——没有任何东西过线。
测试证据：按分诊规则本次运行不构建或执行 PR 代码。被审提交上的 CI（经 API 拉取）全绿、无失败——含单测套件 Test (ubuntu-latest, Node 22.x)；macOS/Windows 的 Test 检查为仓库 CI profile 门控下的跳过，不是失败。套件在设计上钉住了改动：停滞测试用真实的 buildGoalEvidenceCheckpointWindow 驱动真实窗口（101 条溢出、60 条压缩、10 条安静），没有熔断则失败。
核心主张是行为性的，且已被套件之外的证据支撑：维护者在本帖中的独立验证环境跑在真实编译产物上做 A/B——merge-base 构建支付 19+ 次 checkpoint 调用且永远 active，本构建在第三次停滞落定为 usage_limited（3 次调用、17 秒）；计数扛过真实进程重启；31 条 claim 的对照臂从不计数；把错误类型变异为普通 Error 后无界循环回来了，证明错误类型接线是承重墙。沙箱 @qwen-code /verify 已在该 head 上运行中，将自行发布报告；该主张不依赖作者自述。此处未验证：本次运行不执行任何 PR 代码（分诊规则），实时行为依据维护者帖内验证与待发布的 /verify 报告；作者的变异探针数字是其自述，但维护者独立的 12 个变异体清扫（全部被杀）覆盖了同一范围。
— Qwen Code · qwen3.8-max
Reviewed at f45ae6f3a1247642d7a9ed8e02dc6f008b541b7b · re-run with @qwen-code /triage
|
Confidence: 5/5 — every stage clean on my own audit at the new head: both review-loop criticals fixed in the code (not just claimed), CI green, wire surface untouched, and the maintainer's independent runtime verification found no defect. Stepping back: the problem is no longer just argued — the maintainer's in-thread A/B shows the merge-base build paying checkpoint calls forever while this build stops at the third stall, and the streak survives a real restart. My independent proposal and the PR converged; looking for a simpler route turned up nothing — an in-memory counter would be laundered by a restart, and rate-limiting the verifier would cap the token spend but keep losing evidence. The two criticals the review loop raised are fixed in the code itself: the reset now fires only where a check actually proved room, and the unusable-result count keys on an error class that the parse deliberately keeps out of Residuals, carried from the maintainer's verification and confirmed by my read, none blocking: the The two standing Approving, pinned to the reviewed commit. ✅ 中文说明回顾：问题不再只是推理——维护者的帖内 A/B 显示 merge-base 构建永远支付 checkpoint 调用，而本构建在第三次停滞时停止，且计数扛过真实重启。我的独立方案与 PR 收敛；寻找更简路径无果——内存计数会被重启洗掉，给 verifier 限流能省调用却仍丢证据。评审循环提出的两个 Critical 都在代码里修好了（而非只是声称）：重置只在检查真正证明了有空间时触发；不可用结果的计数以一个刻意绕开 残余项（来自维护者验证并经我复核，均不阻塞）：「窗口有空间时不计数不可用结果」这个测试命名的状态，真实的只追加 transcript 在下一轮就会自行离开（该防护实际上只买一轮；终态可辩护）；严格的快照解析意味着旧代码读到新记录会跳过——这一族既有如此（#9165 付过同样代价），也是下一个 bot 现存的两个 批准，钉在被审提交上。✅ — Qwen Code · qwen3.8-max Reviewed at |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
…nLM#9975) finishCheckpointCheck reset checkpointStalls on all three call sites, but only the room branch proved the window had relief. A transient verifier failure or an empty turn now preserves the streak, so intermittent checkpoint-verifier errors cannot launder the count and keep the stall breaker from firing. Also moves withCheckpointStalls out from between takeTurnTokens and its JSDoc.
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/100 轮）。改动内容与我反驳保留之处如下： Round summary (PR #9975, review rv:5014777655)Two findings from the automated reviewer's CHANGES_REQUESTED review; both are resolved in code this round. No conflict resolution was requested ( Feedback points and decisions[rc:3849420322] Critical — streak reset fires on paths that never proved roomDecision: Resolved in code. The claim is checkable and was reproduced before any fix: Reproduction: the new test Fix (minimal, as the finding prescribes):
[rc:3849420327] Suggestion —
|
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
Deferred under the convergence posture (round 2, not a blocker) — recorded, not requested in this round:
packages/core/src/goals/goal-runtime.ts:989 — [probe] sub-32-claim checkpoints on truncated windows reset the streak — breaker disarmed
中文说明
已审查——无阻断问题。 建议见行内评论。
收敛姿态下延后（第 2 轮，非阻断）——已记录，本轮不要求修改：共 1 条（原文未翻译，列表见上方英文部分）。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 2/100 轮）。改动内容与我反驳保留之处如下： Autofix review round — PR #9975 (round 3)Addressed both inline findings; no failed checks and no base conflict to resolve ( Findings and dispositions1. Reproduced before changing anything: added a focused test driving three consecutive overflowing windows (101 records/turn) against a verifier that always answers Fix (first of the two options the finding proposed — counting, distinguishing "verifier returned an unusable result" from "check never ran"):
2. Replaced the 11-line hand-pasted fixture in Mutation probes (witness verification)
Both new guards are witnessed by tests committed in this round; the reproduction test also fails on the pre-round commit, as required for a defect fix. Changed files
Commit: Verification
中文说明Autofix 审查轮次 — PR #9975(第 3 轮)处理了两条行内发现;没有失败的检查,也没有需要解决的与 base 分支的冲突( 发现与处置1. 在改动任何代码之前先复现:新增一个聚焦测试,用持续溢出的窗口(每轮 101 条记录)驱动连续三轮检查,verifier 始终返回 修复方式(采用该发现提出的两个选项中的第一个——计数,并区分「verifier 返回了不可用结果」与「检查从未运行」):
2. 将 变异探针(见证验证)
本轮新增的两个守卫都由本轮提交的测试见证;复现测试在轮次前的提交上也会失败,符合缺陷修复的要求。 变更文件
提交:`740f540cf2 fix(goal): count unusable checkpoint results toward the stall limit (#9975)``(追加式,位于 PR head 分支)。 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Convergence: round 3 posted 2 inline comment(s), 2 of them reported for the first time; the previous round posted 2 (2 new). Findings keep coming back to the same files: packages/core/src/goals/goal-runtime.ts (findings in round 2; 2 more now). The rate of new findings is not falling. A cluster that keeps producing siblings usually means the fixes are treating instances of a shared root cause — triaging that cause before the next round, or splitting an independent cluster into its own pull request, tends to end the loop faster than fixing them one at a time. Batching the remaining fixes and verifying them before the next push, or dropping this PR's reviews to --severity-floor critical, keeps the loop from re-deriving the same set. (Observation only — nothing was withheld from this review because of this observation.)
中文说明
收敛情况：第 3 轮发布了 2 条行内评论，其中 2 条是首次提出；上一轮发布了 2 条（其中 2 条首次提出）。发现反复回到同一批文件：packages/core/src/goals/goal-runtime.ts（第 2 轮已出过发现，本轮又有 2 条）。新发现的产出速度没有下降。一个不断再生兄弟发现的簇，通常意味着逐条修复只在处理同一根因的实例——先定位并处理该根因，或把独立的簇拆成单独的 PR，通常比逐条修复更快结束循环。把剩余修复攒成一批、验证后再推送，或将本 PR 的评审降到 --severity-floor critical，可以避免循环反复推导同一组发现。（仅为观察——本轮评审未因此扣留任何内容。）
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
🤖 Addressed the latest review feedback (round 3/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 3/100 轮）。改动内容与我反驳保留之处如下： Round summaryTwo inline findings, both resolved in code with one commit; one review-body observation that required no code change. The Critical and the Suggestion shared a root-cause area (the stall-breaker machinery added by this PR), so both were batched into this round per the convergence observation's advice. Findings and dispositions[rc:3851765994] [Critical] Parse-level unusable verifier results never reach the stall breaker — RESOLVEDReproduced before changing anything: a new witness test driving Root cause confirmed by code reading: parse-level rejections ( Minimal root-cause fix (the finding's first suggested shape, classify at the source rather than broadening the runtime catch):
On the finding's note that the existing runtime test [rc:3851766006] [Suggestion] Stall-limit stop contract duplicated in two siblings — RESOLVEDImplemented the suggested extraction: [rv:5017501066] CHANGES_REQUESTED review body — observation only, no code actionThe review explicitly states it is an observation. Its advice (triage the shared root cause before fixing instances, batch the fixes) is what this round did: both findings trace to the stall-breaker machinery, and both landed in one verified batch. Mutation probes (both restored to green afterwards)
Conflict notes
Verification
中文说明本轮摘要两条行内发现,均已在代码中解决,合入一个提交;一条评审正文观察,无需代码改动。Critical 与 Suggestion 同属一个根因区域(本 PR 新增的停滞熔断机制),因此按照收敛观察的建议,在本轮中打包处理。 发现与处置[rc:3851765994] [Critical] 解析层面的不可用 verifier 结果永远到不了停滞熔断器 —— 已解决先复现再改动:新增的见证测试驱动 根因经代码阅读确认:解析层面的拒绝( 最小根因修复(采用该发现建议的第一种形态:在源头分类,而不是放宽运行时 catch):
关于该发现提到现有运行时测试 [rc:3851766006] [Suggestion] 停滞上限停止契约在两个兄弟分支中重复 —— 已解决按建议实施抽取: [rv:5017501066] CHANGES_REQUESTED 评审正文 —— 仅为观察,无代码动作该评审明确声明仅为观察。其建议(先定位共享根因再修实例、批量修复)正是本轮的做法:两条发现都追溯到停滞熔断机制,并在同一个经过验证的批次中落地。 变异探针(事后均已恢复为绿)
冲突说明
验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
… window QwenLM#9840 landed after this branch opened: an evidence-limited Goal now resumes by repointing the cursor and dropping the checkpoint, which is a different evidence window from the one the streak was counted against. Carrying the count across it spends the new window's allowance on the old window's failures -- a Goal resumed at two stalls would stop again after a single stalled checkpoint. A resume that does NOT restart the window (paused, blocked) keeps the streak: that Goal comes back to the same window, so what it learned about that window is still true. Mutation probe: removing the reset fails exactly the new resume test (75 others green).
|
不重启窗口的 resume(paused / blocked)则保留 streak——那个 Goal 回到的是同一个窗口,它对该窗口的判断依然成立。两种情形各有一个测试。 变异检验:去掉这次清零,恰好挂新增的 resume 测试(其余 75 绿)。全量 本轮 6 条线程已全部逐条回复并 resolve——其中 5 条由 autofix 环路的三个 commit 修复(含两条 Critical:catch 路径洗掉 streak、生产 verifier 抛的是普通 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
Deferred under the convergence posture (round 4, not a blocker) — recorded, not requested in this round:
packages/core/src/goals/goal-protocol.ts:131 — [probe] no runtime-level restore test for the checkpointStalls anti-laundering claimpackages/core/src/goals/goal-protocol.ts:25 — [review] stalled-reason prose hard-codes "three" instead of deriving it from the limit constant
中文说明
已审查——无阻断问题。 建议见行内评论。
收敛姿态下延后（第 4 轮，非阻断）——已记录，本轮不要求修改：共 2 条（原文未翻译，列表见上方英文部分）。
— qwen3.8-max via Qwen Code /review (v0.22.0)
| ); | ||
| }); | ||
|
|
||
| it('surfaces unusable model output as InvalidGoalCheckpointError', async () => { |
There was a problem hiding this comment.
[Suggestion] This new class-pin test covers three of the four rejection sites the diff converts to InvalidGoalCheckpointError, but not parseClaim's shape rejection (invalid proofKind, duplicate sourceRefs, wrong field types) — that site is asserted only by a message match in the pre-existing test, which passes for any Error subclass. A probe confirmed the gap: reverting just that throw site to a plain Error leaves the whole goals suite green (412/412), while a class-pinned probe for the duplicate-sourceRefs shape fails against that mutant and passes on restored code. In production the stall breaker dispatches on error instanceof InvalidGoalCheckpointError && window.truncated, so a verifier reply with duplicated sourceRefs arriving while the evidence window overflows would settle as inconclusive instead of stalled: the streak never advances for that class of unusable output, and the Goal keeps paying a checkpoint call per turn while shedding uncatalogued evidence — the loop this breaker exists to stop. Add one more reply to the list so every converted throw site is class-pinned:
JSON.stringify({
claims: [
{ proofKind: 'external_fact', claim: 'c', sourceRefs: ['r-1', 'r-1'] },
],
}),中文说明
这个新的错误类固定测试覆盖了本 diff 转换为 InvalidGoalCheckpointError 的四个抛出点中的三个,但没有覆盖 parseClaim 的形状拒绝(非法 proofKind、重复 sourceRefs、字段类型错误)——该抛出点只在既有测试中通过错误消息匹配来断言,而消息匹配对任何 Error 子类都通过。探针确认了这个缺口:仅把该抛出点还原为普通 Error,整个 goals 测试套件仍然全绿(412/412);而对重复 sourceRefs 形状做错误类固定的探针在该变异体上失败、在还原后的代码上通过。生产环境中熔断器以 error instanceof InvalidGoalCheckpointError && window.truncated 分派,因此当证据窗口溢出时,一份含重复 sourceRefs 的 verifier 回复会被归为 inconclusive 而非 stalled:这类不可用输出永远不会推进计数,Goal 会每轮继续支付一次 checkpoint 调用并持续丢失未编目的证据——正是本熔断器要终止的循环。向列表再加一条回复,使所有被转换的抛出点都固定错误类:
JSON.stringify({
claims: [
{ proofKind: 'external_fact', claim: 'c', sourceRefs: ['r-1', 'r-1'] },
],
}),— qwen3.8-max via Qwen Code /review (v0.22.0)
Independent runtime verification (maintainer review)I built a local harness for this PR and ran it in two layers against real compiled artifacts, not just the PR's vitest suite. Both new stop paths reproduce end-to-end, the false-positive guards hold, and the persistence claim survives a real process restart. No correctness defect found — I'm happy to merge this. Two low-severity observations are at the bottom. Arms. Harness (how the numbers below were produced)Layer 1 — compiled runtime, file-backed transcript. Layer 2 — the real CLI. Truncation was verified, not assumed. A temporary probe compiled into The probe was removed before every measured arm below. 1. The loop this PR bounds is real, and the breaker closes itSame objective, same mock, same CLI — only the core differs:
BEFORE paid a model call and lost ~⅔ of the turn's evidence every turn, forever. AFTER stops on the third stalled checkpoint with the new reason, and mints no further continuation.
The journal agrees, and shows the streak persisted on the record rather than held in memory: The stopping checkpoint is discarded exactly as described: the final record still carries the previous checkpoint's 32 claims and the previous cursor. 2. The streak survives a real process restartKilled the CLI mid-streak (
3. Unusable verifier output counts — over the wire, and the error class is load-bearingWith the mock returning Then I mutated the compiled verifier —
4. False-positive guards holdControl arm on the PR build: identical evidence pressure (window truncated every turn) but the verifier returns 31 claims instead of 32 → never counted, Goal still running after 10 checkpoints. One claim below the cap is the whole difference.
5. Full behaviour matrix (Layer 1, compiled runtime + real JSONL)
6. Tests, mutation, typecheck
Observations (neither blocks merge)O1 — The PR's helper takes the O2 — a downgrade silently rewinds the Goal. The pre-PR End to end: I ran the PR build to Note on the risk you already flagged: in my canonical arm the Goal died 17 seconds and 3 turns in. Models do tend to fill an allowed list to its maximum, so a verifier that habitually returns 32 claims will stop any high-volume Goal quickly. That is the documented tradeoff and the reason text does tell the user what to do — flagging the observed magnitude, not disputing the design. 中文版报告独立运行时验证（维护者复核）我为这个 PR 搭了一套本地验证环境，分两层跑在真实编译产物上，而不只是重跑 PR 自带的 vitest。两条新的停止路径都能端到端复现，误报防护成立，持久化的说法也扛过了真实的进程重启。没有发现正确性缺陷，我认为可以合入。 底部有两条低优先级观察。 对照臂。 验证环境。 第一层：两个 worktree 各自把 1. 这个 PR 要收敛的循环真实存在，熔断确实把它关上了。 同一目标、同一 mock、同一 CLI，只换 core：BEFORE 跑 3 轮后仍是 2. 连续计数扛过真实重启。 在 3. 不可用的 verifier 输出会计数，且错误类型是承重的。 mock 返回 4. 误报防护成立。 对照臂：证据压力完全相同（每轮窗口都截断），但 verifier 返回 31 条 claim 而非 32 → 从不计数，10 次 checkpoint 后 Goal 仍在跑。差别就在于比上限少一条。 5. 完整行为矩阵（第一层）： 截断窗口 + 32 claim ×3 → 停止；BEFORE 同场景 12 轮仍 active；截断 + 31 claim → 6 轮不计数；满 claim 但窗口有空间 → 计数归零；无需 checkpoint 的检查 → 归零；verifier 503 → 连续计数保持不变，之后 2 → 3 停止；不可用结果 + 截断窗口 → 3 次后停止；重启（两个进程、同一 JSONL）→ 恢复为 2，下一次停滞即停止；证据受限 Goal 的 6. 测试、变异与类型检查。 PR head 上 观察 1（不阻塞）——「窗口有空间时不计入不可用结果」这条测试断言的是真实会话到不了的状态。 不可用的 checkpoint 不会推进游标，因此在只增不减的 transcript 上窗口每轮都在长，很快自己就截断了。用真实 JSONL 驱动真实运行时（每轮 85 条证据、每次都返回 观察 2（不阻塞）——降级会静默回退 Goal。 PR 之前的 关于你已经标注的风险： 在我的标准场景里，Goal 在 17 秒、3 轮内就被停掉了。模型确实倾向于把允许的列表填满，因此一个习惯性返回 32 条 claim 的 verifier 会很快停掉任何高证据量的 Goal。这正是你写明的权衡，reason 文案也告诉了用户该怎么做——我只是把观察到的量级摆出来，不是反对这个设计。 |
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 91 passed · 0 failed · 91 total Flakiness gate: ✅ 4 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：91 通过 · 0 失败 · 91 总计 抖动门：✅ 4 changed test file(s) x 5 identical rounds, no divergence Verification reportPR 9975 — feat(goal): stop a Goal whose checkpoints stall three times in a rowVerdict: 中文摘要
Central claim and A/B proofCentral claim: a Goal whose checkpoints stall three times in a row (window truncated and claim list at the 32-cap, or a verifier result that could not be folded while the window overflowed) is settled Both arms drive the real compiled runtime (
Witnesses: Secondary cells (head, compiled reducer/parser)
Mutation matrix (vacuity proof of the new tests)Control (no mutation): 199/199 green on the PR's probe set (runtime + reducer + checkpoint). Every mutant killed; each failure quoted in
No survivors, so nothing to adjudicate. M1/M2 exceed the PR's counts only upward: commits 2–6 of the PR added tests pinning the same guards after the description's probe numbers were written (see Corrections). Corrections (description vs. code at the merged head)
Findings (non-blocking)
Not covered
MethodologyEnvironment: CI verify container ( Flakiness gate logEvidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention). — Qwen Code · sandboxed verification |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
|
Released in v0.22.2. |
4 participants
What this PR does
Adds a breaker for the one compaction loop #9835 left unbounded. A checkpoint is counted as stalled when it comes back holding the maximum number of claims while the evidence window it compacted was already truncated — compaction has two levers, folding evidence into claims and moving the cursor past what was folded, and that combination means the first lever is exhausted (the next checkpoint can only merge) and the second is not keeping up (eligible evidence was left behind uncatalogued). A busy turn that truncates while the claims still have room is not a stall, and a full claim list on a quiet Goal is not either; both levers have to be pinned at once.
The streak is persisted on the Goal record as
checkpointStalls(absent means zero), so a daemon restart or session resume cannot launder it. Any check that finds room resets it — a checkpoint on an un-truncated window, or a check that needs no checkpoint at all — and so do edit and replace. After three consecutive stalls the Goal is settled asusage_limitedwith the existinglimitKind: 'evidence_catalog'and a new reason that says what stalled, how many times, and what to do (narrow the objective via edit/replace before resuming). It goes through the same failure path as the other checkpoint bounds; no newGoalLimitKind, no branch ingoalLimitKindForReason(new records always persistlimitKind), so nothing crosses the wire and the pre-C3 Web Shell drift guard is untouched.Why it's needed
Before #9835 a truncated window stopped the Goal; after it, the window compacts. That is right, but it removed the only terminator on the "compaction runs but gives no relief" path: a Goal whose evidence rate outruns the catalog now pays a checkpoint verifier call — a model call — every single turn, loses evidence every turn, and never converges on its own. CC bounds the analogous thrash with per-query-chain counters (3 and 3) that reset on every user turn; ours has to survive restarts, hence the record field. The threshold of three matches: one stalled checkpoint is a busy turn, two is a pattern, three is the loop.
Reviewer Test Plan
How to verify
cd packages/core && npx vitest run src/goals/— 404 tests, 16 files (8 new). The runtime cases drive real windows through the checkpoint check: 101 records overflow the raw-entry budget (truncated), 60 compact without overflowing, 10 stay below threshold; the verifier fake answers with 32 claims.goal-runtime+goal-reducer+goal-checkpoint(192 tests): remove the increment → 3 fail; remove the reset on an effective checkpoint → 1; remove the reset on a quiet check → 1; threshold>=→>→ 1; drop the key fromparseGoalRecord→ 1; never restore it → 1; predicate ignores truncation → 2; predicate ignores the claim cap → 1; edit stops resetting → 1. Every other test green in every run.npx tsc --noEmitinpackages/core: identical error count with the diff stashed (35, all pre-existing dependency skew outsidesrc/goals/, 0 in goals). prettier + eslint clean on the seven changed files.Evidence (Before & After)
N/A (runtime bound; the stop renders through the existing usage-limited surfaces).
Tested on
Environment (optional)
N/A (unit tests only).
Risk & Scope
checkpointStalls, exactly as edit does here — a one-line follow-up. No UI, SDK, or CLI change.0restores as no field.Linked Issues
中文说明
这个 PR 做了什么
为 #9835 留下的那个无界压缩循环加一个熔断。当一次 checkpoint 返回时已持有最大数量的 claim,而它所压缩的证据窗口本身已经被截断时,记为一次停滞——压缩有两个杠杆:把证据折叠进 claim,以及把游标移过已折叠的部分;这种组合意味着第一个杠杆已耗尽(下一次 checkpoint 只能合并),第二个杠杆也跟不上(合格证据被遗留在目录之外)。忙碌轮次在 claim 仍有空间时截断不算停滞,安静 Goal 的满 claim 列表也不算;两个杠杆必须同时卡死。
连续次数以
checkpointStalls持久化在 Goal 记录上(缺省即零),daemon 重启或会话恢复都无法洗掉它。任何发现有空间的检查都会重置它——未截断窗口上的 checkpoint,或根本不需要 checkpoint 的检查——edit 与 replace 也会重置。连续三次停滞后,Goal 以现有的limitKind: 'evidence_catalog'落为usage_limited,并带上一条新的 reason,说明什么停滞了、几次、该做什么(resume 前通过 edit/replace 收窄目标)。它走与其他 checkpoint 边界相同的失败路径;不新增GoalLimitKind,不在goalLimitKindForReason加分支(新记录总是持久化limitKind),因此没有任何东西跨越线上传输,C3 之前的 Web Shell drift guard 也不受影响。为什么需要
#9835 之前,截断的窗口会停止 Goal;之后则会压缩。这是对的,但它移除了「压缩运行却无缓解」路径上唯一的终止器:证据产出速度超过目录容量的 Goal 现在每一轮都要付一次 checkpoint verifier 调用——一次模型调用——每轮丢证据,靠自己永远收敛不了。CC 用按查询链的计数器(3 和 3)约束类似的抖动,但那些计数器在每个用户轮次都会清零;我们的必须能撑过重启,因此放在记录字段上。阈值 3 与之一致:一次停滞是忙碌轮次,两次是模式,三次就是循环。
评审验证计划
如何验证
cd packages/core && npx vitest run src/goals/——404 个测试,16 个文件(新增 8 个)。runtime 用例驱动真实窗口经过 checkpoint 检查:101 条记录溢出原始条目预算(截断),60 条压缩但不溢出,10 条低于阈值;verifier 桩返回 32 条 claim。goal-runtime+goal-reducer+goal-checkpoint(192 个测试)上运行:去掉递增 → 3 个失败;去掉有效 checkpoint 上的重置 → 1;去掉安静检查上的重置 → 1;阈值>=改>→ 1;从parseGoalRecord去掉字段 → 1;永不恢复它 → 1;谓词忽略截断 → 2;谓词忽略 claim 上限 → 1;edit 不再重置 → 1。每次运行其余测试全绿。packages/core的npx tsc --noEmit:stash 掉 diff 后错误数相同(35,全是src/goals/之外的已有依赖偏差,goals 内为 0)。七个改动文件 prettier + eslint 干净。证据(前后对比)
N/A(运行时边界;停止状态通过现有的 usage-limited 界面渲染)。
已测试平台
Linux ✅;macOS / Windows⚠️ (CI 覆盖)。
环境(可选)
N/A(仅单元测试)。
风险与范围
checkpointStalls,与此处 edit 的做法一致——一行的后续改动。无 UI、SDK 或 CLI 变更。0恢复为无字段。关联 Issue