feat(goal): stop autonomous continuation at a token budget the user re-arms - #9891
Conversation
…e-arms A Goal run in this repository has no autonomous termination path: every stop so far is either the model completing, a specific enumerated bound, or a human typing /goal pause. The two runaway sessions that motivated this series both ended the third way -- one after 8.6M tokens in 34 minutes. Precision fixes remove the loop families we have found; this adds the bound that covers the families we have not. Every newly created Goal is armed with an autonomous spend window (GOAL_DEFAULT_TOKEN_BUDGET, 30M tokens on the `tokensUsed` metric the recorder already bills per turn). The gate sits in `queueContinuation`, the single point every autonomous continuation is minted through, so one check bounds turn cadence, verifier-rejection retries, checkpoint cycles, and loops not yet discovered. When the window is spent the runtime settles the Goal as `usage_limited` with the new `limitKind: 'token_budget'` instead of minting the continuation. User-driven turns never pass through the gate and are never blocked. The budget is an authorization quantum, not a fault: resuming a budget-stopped Goal moves the ceiling to `tokensUsed + grant` -- the meter itself is never reset -- and the same re-arm applies to an edit of a spent Goal, so both explicit user actions buy another window. An unattended runaway stops and stays stopped, because nobody is there to resume it. Goals persisted before budgets existed restore unbounded, and a host can opt out with a non-finite grant, which arms nothing rather than persisting a value the JSON journal cannot carry. The reducer's evidence-limited resume refusal now matches the two evidence kinds instead of any `limitKind`, so a budget-stopped Goal is not misread as evidence-limited. The SDK union and the webui mapper whitelist carry the new kind across the wire, and the unpermitted get_goal summary reports `tokenBudget` beside the `tokensUsed` it already exposed. Mutation probes: deleting the continuation gate fails exactly the budget-stop test (110 others green); forcing the re-arm helper to return nothing fails exactly the three re-arm tests (187 others green).
|
Gate re-run at the current head Template looks good ✓ Problem: observed, not theoretical. #9877 documents a Goal runaway that burned 8.6M tokens in 34 minutes before a human killed it; this is the budget half of that fix, promised in #9880. Unchanged since the prior pass — and the maintainer's local real-stack verification in the thread below confirms the stop fires in practice. Direction: aligned. The comparable systems converged on the same first line of defense — claude-code's CHANGELOG carries Size: core paths touched ( Approach: scope unchanged and still right — one gate in Risk: no elevated risk signals (no high-risk paths matched). Moving on to code review. 🔍 中文说明在当前 head 模板完整 ✓ 问题：已观测到，不是理论性的。#9877 记录了一次 34 分钟烧掉 860 万 token 的 Goal 失控；本 PR 是 #9880 承诺的"预算"一半。与上轮一致——且线程下方维护者的真实栈本地验证证实停止确实触发。 方向：对齐。同类系统收敛到同样的第一道防线——claude-code CHANGELOG 中有到达上限即停止后台 subagent 的 规模：触及核心路径（ 方案：范围不变且仍然合理——闸门设在 风险：无升级风险信号（未命中高风险路径）。 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code reviewThe head moved only on test and comment lines since the last reviewed pass (
No critical blockers. One standing nit, non-blocking: the one-line sequenceDiagram
participant P1 as User
participant P2 as Goal runtime
participant P3 as Journal
participant P4 as Web Shell resume gate
P2->>P2: finishTurn bills tokensUsed
P2->>P2: queueContinuation checks the budget
alt budget spent
P2->>P3: record usage_limited, limitKind token_budget
P2-->>P1: Goal stops and waits
P4-->>P4: Resume offered (status-only gate)
P1->>P2: resume or edit
P2->>P2: ceiling moves to tokensUsed plus grant
P2->>P2: continuation admitted again
else budget unspent
P2->>P2: mint the next turn as before
end
Files changed (12 of 12 shown)
CI test evidenceAll PR-CI runs on this commit are settled and green:
The green ubuntu unit run is the decisive one: it executes the 414-test goals suite that pins this change, including the new cursor-retention assertion. One note for anyone remembering the red from the round-7 window: at Sandboxed verification is already closing the behavioural gap — no new trigger needed: the maintainer-sponsored 中文说明代码审查自上次审查（
无关键阻塞。唯一遗留小点（非阻塞）： （时序图与上方英文一致；文件表见上。） CI 测试证据本提交的所有 PR-CI 运行均已结束且为绿（表格见英文部分）。决定性的绿色运行是 ubuntu 单测：它执行了锁定本改动的 414 个 goals 测试，包括新增的游标保留断言。给记得第 7 轮红条的人一个说明： 沙箱验证已在闭合行为缺口——无需新触发：维护者资助的 — Qwen Code · qwen3.8-max Reviewed at |
|
Confidence: 4/5 — clean across every stage at the current head; the only residuals are the one-line unrelated reflow and the in-flight Stepping back: this pass had less to decide than it looks. The head moved since the dismissed approval, but the movement is the review feedback being absorbed, not new risk: the follow-up test pins exactly the evidence-cursor axis the sponsored Going back to my independent proposal — gate at the single minting point, ceiling-forward re-arm, no meter reset, no retrofit, kind widening down the wire, absent-field opt-out — this PR still matches it point for point, and the re-verification at this head found no drift: the gate sits behind every re-entry guard, the user-turn path bypasses it by construction, the reducer's branch order keeps budget stops out of the evidence cursor reset, and the parser is strict about what a persisted budget may be. Every edit in the diff is in service of the bound except one reflow line that round 7 already recorded and deferred under convergence. If I had to maintain this in six months I'd thank the author — the constant carries its sizing rationale, the helpers carry their failure-mode reasoning, and the tests read like the design doc. What keeps it at 4 rather than 5: the 中文说明置信：4/5 —— 当前 head 上每个阶段都干净；仅剩的两点是一行无关重排，以及针对本提交的 退一步看：本轮要决定的其实比表面更少。自被驳回的批准以来 head 确实移动了，但移动是评审反馈被吸收，而非新增风险：后续测试恰好钉住了受资助 回到我的独立方案——闸门设在唯一铸造点、上限前移式重新武装、计量表不重置、不向旧 Goal 追加、类型沿传输链路扩展、以字段缺省表达退出——本 PR 依然逐点吻合，且本 head 上的重新核实未发现漂移：闸门位于所有重入守卫之后，用户 turn 路径在构造上绕过它，reducer 分支顺序使预算停止永不进入证据游标重置，解析器对持久化预算的取值严格。除第 7 轮已记录并按收敛姿态延后的一行重排外，diff 中每一处改动都服务于该边界。半年后维护这段代码我会感谢作者——常量带着量纲推导，helper 带着失败模式推理，测试读起来像设计文档。 停在 4 而非 5 的原因： — Qwen Code · qwen3.8-max Reviewed at |
|
✅ Defer withdrawn — the blocking condition no longer holds. The earlier defer stood because the reviewed commit had no green test run anywhere (main's base was broken at 中文说明✅ 推迟已撤回——阻塞条件已不存在。 此前的推迟是因为被审查的提交在任何地方都没有绿色的测试运行（main 基线在 — Qwen Code · qwen3.8-max |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Partially reviewed — gaps disclosed.
Not reviewed: build-and-test — no test suite could run: a pre-existing compile error at the merge base (packages/core contentGenerator.test.ts:776, 'countTokens' missing on ContentGenerator) halts the workspace build before the test phase; the break is byte-identical at the merge base and in files this diff does not touch, so it is not filed against the PR, but no suite verified this change locally.
中文说明
仅完成部分审查，审查缺口已披露。
未审查：build-and-test — no test suite could run: a pre-existing compile error at the merge base (packages/core contentGenerator.test.ts:776, 'countTokens' missing on ContentGenerator) halts the workspace build before the test phase; the break is byte-identical at the merge base and in files this diff does not touch, so it is not filed against the PR, but no suite verified this change locally。
— qwen3.8-max via Qwen Code /review (v0.22.0)
Superseded by subsequent commit bdf0b3e. Current head requires a fresh review.
🖼️ web-shell visual previewRendered against a mock daemon (no real backend): the PR base vs this PR head Screenshots · before / afterFull-resolution recordings (.webm) are attached to the workflow run. — Qwen Code · web-shell visuals |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
Not explored to full depth (tool budget reached): "agent 4": none — no check was cut short..
Deferred under the convergence posture (round 2, not a blocker) — recorded, not requested in this round:
packages/core/src/goals/goal-protocol.ts:30 — [review] 30M-token budget undocumented in docs/users/features/headless.md (deferred by the code-age rule)packages/core/src/goals/goal-runtime.ts:349 — [review] stopForSpentBudget post-journal staleness re-check has no test (deferred by the code-age rule)
中文说明
已审查。 建议见行内评论。
未探索到全部深度（达到工具调用预算）："agent 4"：none — no check was cut short.。
收敛姿态下延后（第 2 轮，非阻断）——已记录，本轮不要求修改：共 2 条（原文未翻译，列表见上方英文部分）。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
@qwen-code /takeover |
qwen-code-dev-bot
added
the
autofix/takeover
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. This is a fork PR, so the first round comes from the next scheduled scan (usually within minutes). Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。本 PR 来自 fork，首轮处理将由下一次定时扫描执行（通常几分钟内）。移除 |
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/100 轮）。改动内容与我反驳保留之处如下： Round 3 summary — PR #9891Commit: Feedback dispositions
Round-1 findings re-verified as already fixed by Not requested this round (reviewer's convergence deferrals, recorded only): the Not actionable locally: Mutation probes (each new behavior witnessed)
The settle extraction and predicate unification are behavior-preserving; their witnesses are the pre-existing Verification
中文说明第 3 轮处理总结 — PR #9891提交: 反馈处理
第一轮发现经复核确认已由 本轮不要求处理(审查方收敛姿态下的延后项,仅记录): 本地无法处理: 变异探针(每个新行为都有见证)
落定提取与谓词统一是行为保持的重构;其见证是既有的覆盖验证者/checkpoint/预算路径的 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
No issues found. LGTM! ✅
Not explored to full depth (tool budget reached): "agent 1c": (none — all planned checks completed; ~15 of ~61 tool calls used).
中文说明
未发现问题。LGTM！✅
未探索到全部深度（达到工具调用预算）："agent 1c"：(none — all planned checks completed; ~15 of ~61 tool calls used)。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix review round — no action takenOutcome: no change. This round's feedback contained nothing actionable, and the listed check failures are workflow-infrastructure cancellations that this PR's code cannot cause. The branch remains at What this round triagedThe prepared feedback ( The only entries under "Failed checks" were 13 identical Diagnosis of the cancelled checks (evidence-based)
The check data matches that coalescing behavior exactly:
Finally, that workflow is the autofix loop's own CI machinery, which address-review rounds are not permitted to modify — but no modification is needed or wanted; the observed behavior is its documented, intended behavior. Diff growth noteThe window report (source 13 / test 95 net lines vs 400/400 budget, 0 prior rounds over budget) is informational only — well within budget, and no VerificationNo code changed this round, so no build/typecheck/lint/test commands were run; there was no commit to verify. Working tree confirmed clean; HEAD unchanged at 中文说明Autofix 审查轮次 — 未采取任何操作结论：无改动。 本轮的反馈中没有任何可处理的内容，所列的检查失败属于工作流基础设施层面的取消（cancellation），本 PR 的代码不可能导致这些取消。分支保持在 本轮分类处理的内容预处理后的反馈（ "Failed checks" 下唯一的条目是 13 条相同的 对被取消检查的诊断（基于证据）
检查数据与上述合并行为完全吻合：
最后，该工作流属于 autofix 循环自身的 CI 机制，address-review 轮次本就无权修改——但实际上也完全不需要、不应该修改；观察到的行为正是其文档中记载的、预期内的行为。 关于 diff 增长窗口报告（源码净增 13 行 / 测试净增 95 行，预算为 400/400，此前 0 轮超出预算）仅为信息性提示——远低于预算，且没有 验证本轮没有改动任何代码，因此未运行 build/typecheck/lint/测试命令；没有需要验证的提交。已确认工作树干净，HEAD 保持为 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
|
@qwen-code /resolve |
|
Qwen Code attempted to resolve merge conflicts but the run did not complete successfully. Check the workflow run for full logs. |
|
Qwen Code review request accepted. Review is queued in workflow run. |
|
Qwen Code review did not complete successfully. The review pipeline failed before a review could be posted. A transient error is retried automatically; if you are seeing this, retry with |
|
🤖 Addressed the latest review feedback (round 4/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 4/100 轮）。改动内容与我反驳保留之处如下： Round summary — PR #9891 (address-review, conflict: false)One commit landed this round: Both maintainer verification reports recommend merge and raise no blocking findings. The round implemented the one actionable suggestion, attempted the second, and triaged the rest. No inline review comments and no review bodies existed this round; every point below comes from issue-level comments and is answered here. Feedback dispositions[ic:5410964915] Note 1 — the meter bills Goal-turn model calls only; worth one clause in the constant's doc comment → ACT. [ic:5410964915] Note 6 — the
Removing the hunk from this PR would require committing with Correction on the landed commit message: [ic:5410964915] Note 4 — [ic:5410348736] — Web Shell transcript card labels every [ic:5410348736] / [ic:5410964915] — overshoot is one turn wide (the crossing turn completes first) → DECLINE. [ic:5410348736] — the PR description's "Risk" paragraph about Web Shell withholding Resume until #9840 is stale → DECLINE (not actionable in code). [ic:5410348736] — headless budget stop ends with [ic:5410964915] Note 3 — pre-budget Goals stay unbounded forever (no retrofit on resume/edit) → DECLINE. [ic:5410964915] Note 5 — the mapper widening has no reader yet → DECLINE. [ic:5411506533] (qwen-code-ci-bot) — review pipeline did not complete; listed checks CANCELLED → NO ACTION. Mutation probesNot applicable this round: the landed commit adds no guard, branch, or behavior — one doc-comment sentence (the pre-commit hook's prettier pass confirmed it format-clean by leaving it untouched). The attempted test revert was itself probe-verified (prettier evidence above). ConflictNone — Verification
中文说明本轮摘要 — PR #9891（address-review，无冲突）本轮落地一个提交： 两位维护者的验证报告均建议合入，且没有阻塞性发现。本轮实现了一条可执行建议，尝试了第二条（未能落地，见下），其余均已分类处理。本轮没有行内评审意见、也没有评审主体（review body），以下所有条目均来自 issue 级评论，并在此逐一答复。 反馈处置[ic:5410964915] 备注 1 —— 计量表只统计 Goal turn 的模型调用；值得在常量文档注释里补一句 → 已处理。 [ic:5410964915] 备注 6 ——
要在本 PR 中去掉这个 hunk，只能 对已落地提交信息的更正： [ic:5410964915] 备注 4 —— 对耗尽的 Goal 执行 [ic:5410348736] —— Web Shell 转录卡片把所有 [ic:5410348736] / [ic:5410964915] —— 超调恰为一个 turn（越线的那一轮先跑完）→ 不采纳。 [ic:5410348736] —— PR 描述「风险」一段关于 Web Shell 在 #9840 合入前隐藏 Resume 的说法已过时 → 不采纳（代码层面无法处理）。 [ic:5410348736] —— headless 模式下预算停止以 [ic:5410964915] 备注 3 —— 预算出现之前的 Goal 永远不受限（resume/edit 都不追加上限）→ 不采纳。 [ic:5410964915] 备注 5 —— mapper 的放宽目前还没有读者 → 不采纳。 [ic:5411506533]（qwen-code-ci-bot）—— 评审流水线未完成；所列检查均为 CANCELLED → 无需处理。 变异检验本轮不适用：落地的提交没有新增任何守卫、分支或行为——只有一句文档注释（pre-commit 钩子的 prettier 检查未改动它，确认其格式干净）。尝试的测试还原本身已经过检验（见上方 prettier 证据）。 冲突无 —— 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Deferred under the convergence posture (round 7, not a blocker) — recorded, not requested in this round:
packages/core/src/telemetry/uiTelemetry.test.ts:1412 — [review] uiTelemetry.test.ts reflow churn still in diff despite HEAD commit message claiming a revert
中文说明
收敛姿态下延后（第 7 轮，非阻断）——已记录，本轮不要求修改：共 1 条（原文未翻译，列表见上方英文部分）。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： No code changes this round — the red Windows lane is a pre-existing failure on Feedback triage1. Review [rv:5022705540] — "Downgraded from Approve to Comment: CI failing: Test (windows-latest, Node 22.x)" → investigated, deferred to the follow-up queue ( Evidence that the failure is not this PR's:
Disposition: deferred to the follow-up queue rather than fixed here, because all three files are outside this PR's footprint; a dedicated Windows-lane repair PR is the right vehicle. 2. Deferred under convergence posture (round 7): 3. Issue-level comment [ic:5414320648] → stale fallback notice from an earlier review-pipeline run that failed before posting; it is superseded by the completed review in item 1. Nothing to address in code. Verification
中文说明本轮不做任何代码修改——红灯的 Windows 测试通道是 反馈分类处理1. 评审 [rv:5022705540] —— "已从批准降级为评论：CI 失败：Test (windows-latest, Node 22.x)" → 已调查，延后到后续跟进队列（ 证明该失败与本 PR 无关的证据：
处理方式：延后到跟进队列而不在本 PR 中修复，因为这三个文件都在本 PR 的改动范围之外；应由专门的 Windows 通道修复 PR 来处理。 2. 收敛姿态下延后（第 7 轮）： 3. Issue 级评论 [ic:5414320648] → 是早前一次评审流水线在发布前失败时留下的兜底提示；第 1 条中已完成的评审已经取代了它。代码层面无需处理。 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Partially reviewed — gaps disclosed.
Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
中文说明
仅完成部分审查，审查缺口已披露。
未审查：build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 76 passed · 0 failed · 76 total Flakiness gate: ✅ 6 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：76 通过 · 0 失败 · 76 总计 抖动门：✅ 6 changed test file(s) x 5 identical rounds, no divergence Verification reportPR 9891 Deep Verification (follow-up round) — feat(goal): stop autonomous continuation at a token budget the user re-armsVerdict: 中文摘要
Previous-finding status (follow-up round)Previous round verified head
The previous round's four corrections were also re-measured at the new head (they concern the description, which is unchanged) — see Corrections below. Central claim and A/B proofClaim under test: when Harness:
The flip is complete: base mints the continuation, head settles instead — re-established on the post-merge base, where main had independently evolved the goals subsystem (+1210/−113 lines across 12 goal files between the two base tips, measured by tree-diff of Control purity: the PR touches no Production wiring re-traced (static, new head): the single production call site ( Corrections (description vs measured — no code change requested)
Findings1. (Nit) Claimed uiTelemetry revert is not in the final merge resultCommit Mutation matrix (head worktree @
|
| row | mutation | result | classification |
|---|---|---|---|
| control | unmutated HEAD | 435 green | baseline |
| M1 | delete the queueContinuation budget gate |
3 red / 432 green (exactly the three budget-stop runtime tests) | load-bearing |
| M2 | rearmedTokenBudget() always returns {} |
8 red / 427 green (all re-arm/opt-out transitions + runtime stop-and-re-arm) | load-bearing |
| M3 | isEvidenceLimited reverted to presence-matching |
survived, 435 green | redundant defence (the earlier token_budget branch closes the same hazard — unchanged from the previous round) |
| M4 | M3 + token_budget resume branch dead |
1 red / 434 green — the retention pin, failing on r-100 vs r-200 |
killed this round (previous round: survived = coverage gap; closed by 353da2ee) |
| M5 | branch dead alone, narrowing kept | 2 red / 433 green (resume tests' lastReason/limitKind assertions) |
branch individually load-bearing |
| M6 | budget branch made to reset evidenceCursor (branch otherwise intact) |
1 red / 434 green — the same retention pin | positive control: the new pin catches its own axis |
| PC | createGoal never stamps the grant |
5 red / 430 green (stamping tests, default-arm test, all stop tests) | positive control for the reducer file |
Wire oracle and restart boundary
Wire oracle (wire-limit-kind.mts, run under tsx; producer = real head runtime settling a spent budget, wire = real JSON.stringify → JSON.parse, consumer = webui's real updateConnectionFromDaemonEvent via update._meta.goalState). Capture: 04-wire-oracle-limit-kind.png. 11/11 PASS: token_budget + status + lastReason survive producer → wire → mapper; both pre-existing kinds still cross; an unknown kind is dropped while the Goal still maps; Infinity demonstrably becomes null through JSON — why opt-out persists as an absent field. Live-gate control: deleting token_budget from the mapper whitelist turns exactly the new webui test red (1 failed / 37 passing).
Restart boundary (restart-boundary.mjs, head). Capture: 05-restart-boundary.png. 12/12 PASS: a journal holding an ACTIVE Goal whose spend already crossed the ceiling (the daemon-crash shape) re-settles to usage_limited/token_budget on restore() + bindHost() with zero turns minted and the settle journaled; a persisted budget stop stays stopped with zero turns and no redundant re-settle write; the persistence parser round-trips a budget stop deep-equal, rejects a negative budget, and restores a pre-budget Goal unbounded (no retrofitted default).
Targeted gates
| gate | result |
|---|---|
packages/core src/goals/ suite (HEAD worktree + main tree, clean) |
435/435 passed (16 files) |
packages/core src/telemetry/uiTelemetry.test.ts (the one other touched file) |
53/53 passed |
packages/webui src/daemon/session/mappers.test.ts |
38/38 passed (matches PR claim) |
packages/web-shell client/utils/goalGate.test.ts |
4/4 passed |
tsc --noEmit webui |
0 errors (PR claim held) |
npm run typecheck sdk-typescript |
exit 0 (PR claim held) |
prettier --check on the 12 changed files |
clean, live-gate proven (a planted violation in a scratch file was flagged, then removed — previous round's open item) |
| Cold-build parity core @ base vs @ head | identical single error (shellExecutionService.ts:13), zero in PR files (logs/build-parity.txt) |
Not covered
- Per-commit attribution. Shallow depth-2 checkout (
git rev-parse --is-shallow-repository= true);git rev-list HEAD^1..HEAD^2returns 1 at the boundary while the snapshot lists 12 commits → intermediate commits (353da2ee,b8af58bd, the four merges) unreachable. Their effects were verified in the aggregate state (the retention pin exists and kills M4; the doc comment matches the ledger code; the claimed uiTelemetry revert does not appear — Findings pre-release: fix ci #1), not via isolated per-commit diffs. The old-base→new-base comparison was possible as a tree-diff between the two base tip objects (ef18a73885vs0756be0ce7) even with grafted history. - Real model traffic. Journal/host/ledger are seams around real runtime code; the 30M default's sizing (vs the 8.6M runaway) is the documented rationale, not measured. Mechanism verified with scaled grants, including the exact-ceiling and below-ceiling gradient.
- The doc comment's unmetered-path claim was verified by tracing the permit gate (
accumulateGoalTurnTokensfires only on records withgoalContext), not by measuring side-query traffic. - User-driven turns on a live session — verified structurally (all five
beginTurncall sites are user-turn paths; the gate sits only inqueueContinuation). - Browser rendering of the stop — wire mapping proven at the function level; no webui render pass.
- sdk-typescript test suite — the SDK diff is type-level (union widening + doc); the typecheck gate substitutes.
- Flakiness gate — the workflow runs the changed-test-file repeat rounds outside the agent clock; this round's changed test files are the same six as last round.
Methodology
Environment: CI merge-ref checkout (HEAD = merge e4d88f9785, HEAD^1 = base tip 0756be0ce7, HEAD^2 = verified PR head 17fc1b86f5), node_modules preinstalled by the workflow. A/B: git worktree add tmp/base-tree HEAD^1, symlinked the lockfile-determined per-package node_modules (nested ignore@7.0.5/ajv copies the worktree would otherwise miss), rebuilt only packages/core with the repo's TypeScript, then drove both dists with ab-budget-gate.mjs (realpath + marker-census witnesses printed per arm). Mutation matrix: fresh HEAD worktree (tmp/mutant-tree), single-point source mutants applied/reverted per row with occurrence-count guards, npx vitest run src/goals/ per row (~7 s each), expectations adjudicated by adjudicate-matrix.mjs. Wire harness: npx tsx against real core dist + real webui source. Raw logs in logs/ (A/B arms, per-row matrix output, gate outputs, typechecks, build logs, parity check); harnesses live in this directory for rerun. All counts in assertions.json map to executed scripted checks; expected-failure cells (base arm, mutant kills) are encoded as passing assertions. Both scratch worktrees removed after capture; the main tree was never modified (verified clean with git status).
Flakiness gate log
rounds=5 files=6 skipped=0
file packages/core/src/goals/goal-reducer.test.ts: (cd packages/core) npx --no-install vitest run ./src/goals/goal-reducer.test.ts
file packages/core/src/goals/goal-runtime.test.ts: (cd packages/core) npx --no-install vitest run ./src/goals/goal-runtime.test.ts
file packages/core/src/goals/goal-tools.test.ts: (cd packages/core) npx --no-install vitest run ./src/goals/goal-tools.test.ts
file packages/core/src/telemetry/uiTelemetry.test.ts: (cd packages/core) npx --no-install vitest run ./src/telemetry/uiTelemetry.test.ts
file packages/web-shell/client/utils/goalGate.test.ts: (cd packages/web-shell) npx --no-install vitest run ./client/utils/goalGate.test.ts
file packages/webui/src/daemon/session/mappers.test.ts: (cd packages/webui) npx --no-install vitest run ./src/daemon/session/mappers.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/core/src/goals/goal-reducer.test.ts: PPPPP
packages/core/src/goals/goal-runtime.test.ts: PPPPP
packages/core/src/goals/goal-tools.test.ts: PPPPP
packages/core/src/telemetry/uiTelemetry.test.ts: PPPPP
packages/web-shell/client/utils/goalGate.test.ts: PPPPP
packages/webui/src/daemon/session/mappers.test.ts: PPPPP
verdict: pass
summary: 6 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/core/src/goals/goal-reducer.test.ts: P (exit 0)
round 1 · packages/core/src/goals/goal-runtime.test.ts: P (exit 0)
round 1 · packages/core/src/goals/goal-tools.test.ts: P (exit 0)
round 1 · packages/core/src/telemetry/uiTelemetry.test.ts: P (exit 0)
round 1 · packages/web-shell/client/utils/goalGate.test.ts: P (exit 0)
round 1 · packages/webui/src/daemon/session/mappers.test.ts: P (exit 0)
round 2 · packages/core/src/goals/goal-reducer.test.ts: P (exit 0)
round 2 · packages/core/src/goals/goal-runtime.test.ts: P (exit 0)
round 2 · packages/core/src/goals/goal-tools.test.ts: P (exit 0)
round 2 · packages/core/src/telemetry/uiTelemetry.test.ts: P (exit 0)
round 2 · packages/web-shell/client/utils/goalGate.test.ts: P (exit 0)
round 2 · packages/webui/src/daemon/session/mappers.test.ts: P (exit 0)
round 3 · packages/core/src/goals/goal-reducer.test.ts: P (exit 0)
round 3 · packages/core/src/goals/goal-runtime.test.ts: P (exit 0)
round 3 · packages/core/src/goals/goal-tools.test.ts: P (exit 0)
round 3 · packages/core/src/telemetry/uiTelemetry.test.ts: P (exit 0)
round 3 · packages/web-shell/client/utils/goalGate.test.ts: P (exit 0)
round 3 · packages/webui/src/daemon/session/mappers.test.ts: P (exit 0)
round 4 · packages/core/src/goals/goal-reducer.test.ts: P (exit 0)
round 4 · packages/core/src/goals/goal-runtime.test.ts: P (exit 0)
round 4 · packages/core/src/goals/goal-tools.test.ts: P (exit 0)
round 4 · packages/core/src/telemetry/uiTelemetry.test.ts: P (exit 0)
round 4 · packages/web-shell/client/utils/goalGate.test.ts: P (exit 0)
round 4 · packages/webui/src/daemon/session/mappers.test.ts: P (exit 0)
round 5 · packages/core/src/goals/goal-reducer.test.ts: P (exit 0)
round 5 · packages/core/src/goals/goal-runtime.test.ts: P (exit 0)
round 5 · packages/core/src/goals/goal-tools.test.ts: P (exit 0)
round 5 · packages/core/src/telemetry/uiTelemetry.test.ts: P (exit 0)
round 5 · packages/web-shell/client/utils/goalGate.test.ts: P (exit 0)
round 5 · packages/webui/src/daemon/session/mappers.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix review round — no code changesThe only new feedback was the automated reviewer's partial-review disclosure ( Why the lane was skipped in CIThe lane is merge-queue-only by design ( Local run of the exact laneAfter a fresh
The single failure is environmental, not caused by this PR
The merge-queue lane runs on clean runners with writable home directories, so this failure does not predict a CI failure there. Verification
No code changes and no commits; the working tree and PR head are unchanged. 中文说明Autofix 审查轮次 — 无代码变更本次唯一的新反馈是自动审查器披露的部分审查缺口（ 该检查为何在 CI 中被跳过该检查按设计仅在合并队列中运行（ 在本地运行完全相同的检查在从 HEAD 重新执行
唯一失败的测试属于环境问题，与本 PR 无关
合并队列中的该检查运行在家目录可写的干净运行器上，因此该失败不预示 CI 中会出现同样的失败。 验证
无代码变更、无提交；工作树与 PR head 均保持不变。 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： 🤖 Reviewed the latest feedback — no changes needed. Why, point by point: Autofix address-review round — no code changesEvery feedback item newer than the last evaluation (2026-08-26T08:18:19Z) was inspected, and none requests a code change:
Additionally:
No commit was made and no thread was resolved. The branch stays at 中文说明🤖 已审阅最新反馈——无需改动。逐点说明如下： Autofix address-review 轮次——无代码改动已检查所有晚于上次评估时间（2026-08-26T08:18:19Z）的反馈条目，均不要求代码改动：
此外：
未创建任何提交，也未将任何讨论线程标记为已解决。分支保持在 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
Post-merge verification of #9891 — real stack, on the tree that actually landedI re-ran a full local verification against the current head Verdict: the merged code does what it says. The budget stops autonomous continuation, resume/edit re-arms it, the meter is never reset, user turns are never blocked, pre-budget Goals stay unbounded, and the new Setup
What is new since my two earlier reports (Aug 25)Those ran against
Decision table over the built reducer9,603 enumerated transitions (status × limitKind × lastReason × budget shape × stalls × grant × action), run against each arm's compiled
Mutation probesAgainst the PR's own suite (source) and against the shipped bundle driven by the real CLI:
Suites and typecheck on head: ScreenshotsTUI — the Goal stops itself at the 30,000,000 ceiling and
TUI — after
TUI, base arm — same model, same objective: 63 turns / 189,000,000 tokens and still running until a human typed
Web Shell (real daemon) —
Web Shell — both windows in one view: stop at 30,000,000, Resume, stop again at 60,000,000:
Follow-up notes — none of them defects in this change
Also confirmed resolved from my earlier round: the meter's coverage ( 中文说明#9891 合入后验证 —— 真实环境，针对真正落地的那棵树我对当前 head 结论：合入的代码与描述一致。 预算会停止自主续跑，resume/edit 重新武装，计量表从不重置，用户驱动的 turn 永不被阻塞，预算出现之前的 Goal 保持无界，且新的 环境
相对 8 月 25 日两份报告的新增部分那两份跑的是
已构建 reducer 上的决策表枚举 9,603 条转换（status × limitKind × lastReason × 预算形态 × stalls × grant × action），分别跑在两条臂编译后的
变异检验同时打在 PR 自带套件（源码）与已发布 bundle（由真实 CLI 驱动）上：
head 上的套件与类型检查： 截图见上方英文部分：TUI 3000 万触顶停止并给出原因； 后续备注 —— 均非本次改动的缺陷
另外，上一轮的两点已确认收敛：计量口径（ Verified locally on macOS 26.6.2 / Node v24.18.1 — harness: scripted OpenAI-compatible server, production |
…wenLM#10125) A Goal whose objective cannot be satisfied as written -- it contradicts itself, names a target that verifiably does not exist, or needs an action no tool can perform -- had no sanctioned way to say so. Blocked proposals stop immediately only for user authority or an external change; everything else is treated as a repeated technical blocker and must recur on three consecutive turns before the verifier sees it. So an impossible objective burned turns until the token budget (QwenLM#9891) or a human stopped it. The session that motivated this series ran 34 minutes on an objective ("验证下版本") too under-specified to ever complete. CC's stop evaluator can answer `impossible` and end the loop; this is the counterpart. `blockerKind: 'infeasible'` joins authority, external and repeated. It is not a new status: the Goal settles as `blocked`, which every surface already renders and which resumes into `/goal edit` -- the only fix for an objective that cannot hold. Three rules keep it from becoming an "I think this can't be done" exit: - It bypasses the three-turn repetition rule. Waiting three turns to report an impossibility is the runaway this kind exists to end, and the evidence bar below is what earns the early exit. - The cited evidence must include an external_fact. User input can authorise a stop (that is `authority`) but cannot make an objective impossible, and assistant prose saying so is exactly what must not count. Like the other immediate blockers it must also cite every newer record, so a contradicting fact cannot be left out. - The verifier policy accepts it only for self-contradiction, a target that verifiably does not exist, or an action outside the tools, and rejects difficulty, uncertainty, obtainable information, or a preference to ask. An accepted infeasible stop appends a fixed next step to lastReason, so the stopped Goal tells the user what to do, not only what went wrong. Mutation probes (goal-evidence + goal-runtime + goal-tools, 194 tests), each killing exactly one test: infeasible routed through the repetition audit; policy sentence removed; external_fact requirement removed; next step suffix dropped; 'infeasible' removed from the tool schema enum.
…oal (QwenLM#10132) QwenLM#9891 stops a Goal the moment its autonomous token budget is spent: the continuation gate refuses the next turn and settles usage_limited. That bounds runaway spend, but it cuts the model off mid-thought -- whatever it had learned in the last window is stranded in the transcript, and the user who resumes gets no hand-off. A spent window now buys exactly one more continuation, flagged `windDown`, whose prompt says the budget is spent, forbids new work, and asks for a concise hand-off: what was accomplished with evidence refs, what remains, the one concrete next step. When that turn finishes, the gate stops the Goal as before. A hand-off turn that finds the objective already met and proposes completion still completes the Goal: the stop only ever refuses a continuation, never a verdict. Exactly one per window, and persisted: the wind-down turn's own turn_finished record stamps `GoalRecord.windDownTurnId`, so the gate can tell "hand-off delivered" from "hand-off owed" across a restart with no extra journal write and no new state cause. A hand-off the host dropped undelivered leaves no marker and is minted again; a restart that interrupted the hand-off turn grants it again for the same reason -- the user never got one. Re-arming the budget on resume or edit clears the marker, so each window owes its own. The flag rides the host boundary like verifierFeedback, through all three hosts, and the prompt block sits after the authoritative objective line and above verifier feedback; the ordinary prompt is byte-identical to before. Mutation probes (goal-runtime + goal-reducer, 218 tests): finishTurn never stamps the marker -> 3 fail; gate ignores the marker -> 4; gate never grants -> 6; re-arm keeps the old marker -> 3; parse never restores it -> 2. Each host hop deleted -> exactly one test fails in that host's suite (useMessageQueue, useGeminiStream, nonInteractiveCli, Session).
|
Released in v0.22.2. |
5 participants
What this PR does
Arms every newly created Goal with an autonomous spend window —
GOAL_DEFAULT_TOKEN_BUDGET, 30M tokens on thetokensUsedmetric the recorder already bills per Goal turn — and stops autonomous continuation when the window is spent. The gate sits inqueueContinuation, the single point every autonomous continuation is minted through, so one check bounds turn cadence, verifier-rejection retries, checkpoint cycles, and loop families not yet discovered. A spent budget settles the Goal asusage_limitedwith the newlimitKind: 'token_budget'instead of minting the continuation. User-driven turns never pass through that gate and are never blocked by the budget.The budget is an authorization quantum, not a fault. Resuming a budget-stopped Goal moves the ceiling to
tokensUsed + grant— the meter is never reset — and an edit of a spent Goal re-arms the same way, so both explicit user actions buy another window. An unattended runaway stops and stays stopped, because nobody is there to resume it. Goals persisted before budgets existed restore unbounded; a host can opt out with a non-finite grant, which arms nothing rather than persisting a value the JSON journal cannot carry (Infinitydoes not surviveJSON.stringify).Supporting changes: the reducer's evidence-limited resume refusal now matches the two evidence kinds instead of any
limitKind, so a budget-stopped Goal is not misread as evidence-limited; the SDKGoalLimitKindunion and the webui mapper whitelist carry the new kind across the wire; the unpermittedget_goalsummary reportstokenBudgetbeside thetokensUsedit already exposed.Why it's needed
A Goal run currently has no autonomous termination path: every stop is either the model completing, one specific enumerated bound, or a human typing
/goal pause. Both runaway sessions analyzed in this series ended the third way — one burned 8.6M tokens in 34 minutes before a human killed it (#9877 documents the loop that drove it). The merged precision fixes (#9165, #9835, #9880) each remove one discovered loop family; a budget is the blunt bound that covers the undiscovered ones. This is the same first line of defense the comparable systems ship — a hard stop-block cap in one, an arithmetic token budget in the other — and it deliberately does not try to be clever: it bounds autonomy, and hands the "keep going" decision back to the user at a fixed spend interval.The retry-bound half of #9877 lands here as promised in #9880: terminal-proposal retries burn tokens like every other continuation, so the budget bounds them without a second counter.
Reviewer Test Plan
How to verify
cd packages/core && npx vitest run src/goals/— 404 tests, 16 files. New coverage: the runtime stop-and-re-arm loop (stops autonomous continuation when the budget is spent, and resume re-arms it), the opt-out grant, and eight reducer transitions (create/replace stamping, resume/edit re-arm, unbounded goals never retrofitted, evidence kinds still refused, persistence round-trip incl. malformed budgets).cd packages/webui && npx vitest run src/daemon/session/mappers.test.ts— 38 tests;token_budgetcrosses the wire, unknown kinds still dropped.queueContinuationgate fails exactly the budget-stop test (110 others green); forcingrearmedTokenBudgetto return nothing fails exactly the three re-arm tests (187 others green).tsc --noEmitclean inpackages/webui;packages/corecarries 4 pre-existing errors on the merge base (verified identical with the diff stashed), none insrc/goals/; sdk builds clean.Evidence (Before & After)
N/A (runtime bound; no UI change — the existing usage-limited surfaces render the stop, and Resume is offered by the existing status-only gate once #9840 lands, see Risk).
Tested on
Environment (optional)
N/A (unit tests only).
Risk & Scope
/goal resumeinstead of running to completion unattended. That interruption is the feature: one explicit user action per spend window. The grant size is a named constant with the sizing rationale in its doc comment./goal set … --budget) and surfacing remaining budget to the model for a graceful wind-down — both deliberately deferred (D2/D3 in the series plan). Web Shell currently ships alimitKind-based Resume gate that would withhold Resume for budget-stopped Goals until fix(goal): resume an evidence-limited Goal from a fresh window #9840 (status-only gate) merges; sequencing either order is safe, the affordance is just conservative in the gap.Linked Issues
中文说明
这个 PR 做了什么
为每个新创建的 Goal 武装一个自主消费窗口——
GOAL_DEFAULT_TOKEN_BUDGET,即 recorder 已按 Goal turn 计费的tokensUsed口径下的 3000 万 token——窗口耗尽时停止自主续跑。闸门设在queueContinuation,即所有自主续跑被铸造的唯一通道,因此一个检查就能约束 turn 节奏、verifier 拒绝重试、checkpoint 循环,以及尚未被发现的循环家族。预算耗尽时,runtime 将 Goal 落为usage_limited并带上新的limitKind: 'token_budget',而不是铸造续跑。用户驱动的 turn 不经过该闸门,永远不会被预算阻塞。预算是一种授权额度,不是故障。恢复一个因预算停止的 Goal 会把上限移到
tokensUsed + grant——计量表永不重置——编辑一个预算耗尽的 Goal 也以同样方式重新武装,因此两种显式用户动作都购买了另一个窗口。无人值守的失控运行会停止并保持停止,因为没有人去恢复它。预算出现之前持久化的 Goal 恢复后不受限;宿主可以用非有限的 grant 选择退出,此时不武装任何预算,而不是持久化一个 JSON 日志无法承载的值(Infinity无法通过JSON.stringify)。配套变更:reducer 的证据受限恢复拒绝现在只匹配两种证据类 limitKind,而不是任意
limitKind,因此预算停止的 Goal 不会被误读为证据受限;SDK 的GoalLimitKind联合类型与 webui mapper 白名单让新类型跨越线上传输;无许可的get_goal摘要在已有的tokensUsed旁报告tokenBudget。为什么需要
Goal 运行目前没有任何自主终止路径:所有停止要么是模型完成、要么是某个具体的枚举边界、要么是人工输入
/goal pause。本系列分析的两个失控 session 都以第三种方式结束——其中一个在 34 分钟内烧掉 860 万 token 才被人工终止(#9877 记录了驱动它的循环)。已合入的精确修复(#9165、#9835、#9880)各自消灭一个已发现的循环家族;预算是覆盖未发现家族的钝性边界。这也是同类系统的第一道防线——一家是 stop-block 硬上限,另一家是算术 token 预算——它刻意不追求聪明:约束自主性,并以固定的消费间隔把「是否继续」交还给用户。#9877 的重试上界一半在此落地,兑现 #9880 的承诺:终局提案的重试和其他续跑一样燃烧 token,预算无需第二个计数器即可约束它们。
评审验证计划
如何验证
cd packages/core && npx vitest run src/goals/——404 个测试,16 个文件。新增覆盖:runtime 的停止-重新武装循环、退出 grant,以及八个 reducer 转换(create/replace 盖章、resume/edit 重新武装、无预算 Goal 永不追加、证据类仍被拒绝、持久化往返含畸形预算)。cd packages/webui && npx vitest run src/daemon/session/mappers.test.ts——38 个测试;token_budget跨线传输,未知类型仍被丢弃。queueContinuation闸门恰好挂预算停止测试(其余 110 绿);强制rearmedTokenBudget返回空恰好挂三个重新武装测试(其余 187 绿)。packages/webui的tsc --noEmit干净;packages/core在合并基线上有 4 个已存在错误(stash 验证与 diff 无关),无一在src/goals/;sdk 构建干净。证据(前后对比)
N/A(运行时边界;无 UI 变化——现有 usage-limited 界面渲染停止状态,#9840 合入后现有 status-only 闸门会提供 Resume,见风险)。
已测试平台
Linux ✅;macOS / Windows⚠️ (CI 覆盖)。
风险与范围
/goal resume,而不是无人值守地跑到完成。这个中断正是特性本身:每个消费窗口需要一次显式用户动作。grant 大小是带有量纲推导注释的命名常量。/goal set … --budget)与向模型暴露剩余预算以实现优雅收尾——均刻意推迟(系列计划中的 D2/D3)。Web Shell 当前的limitKindResume 闸门在 fix(goal): resume an evidence-limited Goal from a fresh window #9840(status-only 闸门)合入前会对预算停止的 Goal 隐藏 Resume;两种合入顺序都安全,间隙期内只是保守。关联 Issue