feat(goal): carry budget figures and progress guidance in the continuation prompt - #11257
Conversation
…ation prompt The continuation prompt told the model what the objective was and how to deliver it, and nothing about two things it had no other cheap way to know. It could not see how much of the spend window was left. A Goal stops when `tokensUsed` reaches `tokenBudget`, 30,000,000 by default, and gets one wind-down turn to hand off; until that turn arrives there was no signal at all, so the model could not tell turn 3 of a long run from the turn before the budget stops it, and could not choose between opening a broad investigation and finishing what it had. `get_goal` does not carry the figures either, so there was not even an expensive way to ask. It was also never asked whether its last turn accomplished anything. The verifier only ever sees a terminal proposal, so a turn that proposes nothing is judged by nobody -- and a turn spent restating status is exactly the turn that proposes nothing. Each runtime-scheduled turn now opens with what has been spent, out of what, what remains, and how many turns are behind it, followed by four standing lines: treat the workspace rather than the conversation as authoritative; work toward the end state the objective asks for rather than a more easily reached one; judge whether the previous turn actually changed anything before spending this one; and check every requirement against citable evidence before proposing completion. The figures sit after the standing objective guard and before the objective-updated notice, which is about what changed since the last turn and so reads last. They stay out of the data block on purpose: that block is compared by content to decide whether the objective changed, and a number that moves every turn would make every turn look like an edit. The remainder is clamped at zero, since the hand-off turn runs with the window already overspent. The progress lines are skipped on that hand-off turn, which is told not to start new work -- a line asking for "a different concrete action now" would contradict it. The budget line stays, because a hand-off reports the numbers it stopped at. `usage` is optional on the host contract, so a host with no figures renders exactly the prompt it did before. The three hosts copy it through their queue entries alongside the fields they already copy.
|
Thanks — re-run at the current head. Only two Template looks good ✓ — every required heading is there, including a real test plan and before/after evidence. Problem: a capability gap, not theoretical hardening. A Goal stops when Direction: aligned. This sharpens an existing first-party feature rather than adding surface, and the two halves pull in the same direction: the figures give the model something to reason with, the progress lines give it a reason. CHANGELOG signal is supportive but not a direct match — claude-code's Size: core paths are touched ( Approach: the scope feels right, and it's smaller than it looks. All the new behaviour lands in the one renderer every host already goes through, Risk: Stage 1e matched one high-risk path — One thing I checked specifically because it's the claim the whole risk section rests on: these lines reach only runtime-scheduled turns, never a user's own. That holds. Moving on to code review. 🔍 中文说明感谢贡献 —— 本次是在当前 head 上的重跑。自上次完整评审( 模板完整 ✓ —— 所有必需小节都在,包括真实的评审验证方式和 before/after 证据。 问题: 是能力缺口,不是理论性加固。Goal 在 方向: 对齐。这是打磨一个已有的一方特性,而不是增加表面积,而且两半是同一个方向:数字给模型可推理的材料,进展指引给它推理的理由。CHANGELOG 信号是支持性的、但不是直接对应——claude-code 的 规模: 触及了核心路径( 方案: 范围合理,而且比看起来更小。所有新行为都落在每个 host 本来就会经过的那一个渲染器里, 风险: Stage 1e 命中一条高风险路径 —— 有一件事我专门查了,因为整个风险小节都建立在它上面:这些行只会到达运行时调度的轮次,永远不会到达用户自己的轮次。这一点成立。 进入代码审查 🔍 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
Code reviewI wrote my own proposal down before opening the diff: put the figures in the single renderer every host already goes through, read them off the record at scheduling time, keep the field optional so an embedder with no figures degrades to today's prompt, clamp the remainder, hold the progress lines back on the hand-off, and pin the whole thing with complete-string tests. That is what this PR does. I did not find a cheaper route afterwards. The one place it beats my proposal is the centralization I would not have written. I'd have kept the four per-host copies and leaned on the four tests that pin them — which is precisely the shape that produced three earlier rounds of "a host silently drops the optional field". Making the host turn types extend a shared No critical blockers and no AGENTS.md violations. What follows is what I went and checked because each one could have been a real bug and none is.
Two small things, neither a gate. sequenceDiagram
participant P1 as goal-runtime flushContinuation
participant P2 as host startGoalTurn
participant P3 as host queue entry
participant P4 as render site
participant P5 as model call
P1->>P1: read tokensUsed, tokenBudget, turnCount off the record
P1->>P1: take the permit, then broadcast
P1->>P2: permit, continuationContext, usage, hints
P2->>P3: permit plus rest-spread continuation
P3->>P4: claimed turn
P4->>P4: renderGoalContinuationTurn
P4->>P5: budget line plus progress lines, outside the data block
Files changed (all 15 shown)
TestingThis is an unattended CI run, so I did not build or execute anything from this PR — the evidence below is the PR's own CI, read through the API for the reviewed commit. All 68 check-runs on The two lanes that were red at the previously reviewed head are the ones to look at, and both are green here:
Both were classified last round as inherited from main rather than caused by this PR, and the re-merge of a fixed main is exactly what would clear them. That is what happened, so the classification is now confirmed by outcome rather than resting on my reasoning about the diff.
Not verified: whether a live model reads these figures and idles less. The transcript output in the description is the author's own local run against a real model; I cannot reproduce it here, and no pre-merge lane in this repo can — the sandbox is credential-free by design, so it never reaches a model. The description carries this honestly under Not validated / out of scope, and I'd keep it there rather than let anyone read the green suite as settling it. A green suite proves the prompt text is assembled and sent, not that the model acts on it. Sandboxed verification would settle the other half, the one that is actually checkable: 中文说明代码审查我在打开 diff 之前先写下了自己的方案:把数字放进每个 host 本来就会经过的那一个渲染器;在调度时从记录里读出来;字段保持可选,让没有数字的嵌入方退化到今天的提示词;剩余量夹住;进展指引在交接轮上跳过;并用完整字符串测试钉住。这个 PR 做的正是这件事。之后我没有找到更省的路。 它超出我方案的唯一一处,正是我不会写的那个集中化。我本来会保留四份 per-host 拷贝、依赖钉住它们的四个用例 —— 而那恰恰是产生了前三轮"某个 host 静默丢掉可选字段"的形态。让各 host 的轮次类型继承共享的 没有关键阻塞项,也没有违反 AGENTS.md 的地方。 下面是我专门去查的东西,因为每一项本来都可能是真 bug,而结果都不是。
两件小事,都不是门。 测试这是一次无人值守的 CI 运行,所以我没有构建或执行本 PR 的任何东西 —— 下面的证据是 PR 自己的 CI,通过 API 针对被评审的 commit 读取。 要看的是在上一个被评审 head 上变红的那两条通道,它们在这里都是绿的:
上一轮把两者都判定为继承自 main、而非本 PR 造成,而重新合入一份修好的 main 正是会清掉它们的东西。事实就是这样发生的,所以这个判定现在是被结果确认的,而不是只依靠我对 diff 的推理。 未验证:真实模型是否会读这些数字并因此少空转。 描述里的会话记录输出来自作者本地对真实模型的运行;我无法在这里复现,而本仓库没有任何合并前通道能做到 —— 沙箱按设计无凭据,因此它根本到不了模型。描述在 Not validated / out of scope 里诚实地承载了这一点,我也希望它留在那里,而不是让任何人把绿灯套件读成已经把这件事定论了。绿灯套件证明的是提示词文本被组装并发送,不是模型照它行动。 另一半 —— 真正可查的那一半 —— 可以由沙箱验证来定论: — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
|
Confidence: 4/5 — clean across every stage and I'd merge it; not 5/5 because two of the PR's own accounts of itself lag the code it ships, and because the effect half of the claim is unmeasured and no lane here can measure it. Stepping back. My independent proposal and this PR's design are the same design, which is the strongest thing I can say about the budget half — I wrote mine down before opening the diff and did not find a cheaper route afterwards. Where it beats mine is the centralization nobody asked for. I would have kept the four per-host field copies and leaned on the four tests that pin them, which is exactly the shape that produced three earlier rounds of "a host silently drops the optional field". Making the shared turn type carry the hints turns that mistake into a type error instead of a missing prompt line on one host, and it does that while deleting more code than it adds in the hosts it touches. In six months I would thank whoever wrote that commit rather than curse them, which is the question I actually care about here. The pushover check, since this PR has been through roughly five bot review rounds and I'm now recommending it. I didn't inherit the previous pass's conclusions — it reviewed a different commit, and its approval was withheld for a reason that no longer exists. What it withheld on was two red CI lanes it classified as main's, on the reasoning that neither touched a file in this diff. At this head both are green, and the re-merge of a fixed main is the specific thing that would clear them. So the classification is now confirmed by outcome rather than resting on an argument about the diff, which is a strictly better evidentiary position than the one that blocked last round. The reason to defer is gone, and continuing to defer after the specific reason disappears is noise, not caution. What I verified myself at this commit rather than taking on trust: that What's left is genuinely non-blocking, and I want to be precise about which kind. The design note still describes the hosts as copying Per the repo's own guidance on not letting review rounds balloon a PR, this one is at the point where only critical fixes should land and suggestions get deferred. There are no critical fixes. So I'm recording the two documentation items here as deferred to a follow-up commit rather than asking for another push and another round — noted, not dropped. The unmeasured half is different in kind and I'm not going to pretend otherwise: nobody knows yet whether a live model reads these figures and idles less. That isn't a gap in this PR's evidence, it's a gap in what any pre-merge lane can produce here — the sandbox is credential-free by design, so it never reaches a model, and the prompt is never rendered into the TUI, so the terminal lane has nothing to show either. The description says so itself under Not validated, which is the honest way to carry it. Stage 2 names On volume: this author has a stack of open Goal-shaped PRs and I checked whether that wore me down. It didn't change what I read — the consumer census, the render-site gating, the stale-figure window and the counter semantics are all things I went and looked at in this branch rather than accepting from the description, and the centralization commit is the one I spent most time on. It does still argue for the sequencing the author proposed: the runtime bound landed first, this follows on top of it, and the two agree about what "progress" means. One note on approval state, because it's the thing that has bitten this PR's thread before: @wenshao approved at this exact commit, and that is a separate vote which I have not counted as mine. Verdict: approve. Approval is pinned to 中文说明信心度:4/5 —— 每个阶段都干净,我会合并它;不给 5/5 是因为 PR 对自身行为的两处陈述落后于它交付的代码,而且这个主张的"效果"那一半没有被测量,这里也没有任何通道能测它。 退一步看。我独立想到的方案和这个 PR 的设计是同一个设计,这是我对预算那一半能给出的最高评价 —— 我的方案是在打开 diff 之前写下的,之后也没找到更省的路。它超出我方案的地方是那个没人要求的集中化。我本来会保留四份 per-host 字段拷贝、依赖钉住它们的四个用例,而那正是产生了前三轮"某个 host 静默丢掉可选字段"的形态。让共享的轮次类型承载这些提示字段,把这个错误从"某一个 host 上少了一行提示词"变成了类型错误;而且它在所触及的 host 里删掉的代码比加上的还多。半年后我会感谢写下那个提交的人、而不是骂他 —— 这才是我真正在意的那个问题。 关于"我是不是太好说话"这项检查,因为这个 PR 已经过了大约五轮机器人评审,而我现在推荐它。我没有继承上一轮的结论 —— 它评审的是另一个 commit,而且它 withheld 批准的理由现在已经不存在了。它当时挡在两条红 CI 通道上,依据是两者都没碰本 diff 里的任何文件。在这个 head 上两者都是绿的,而重新合入一份修好的 main 正是会清掉它们的那件具体事情。所以这个判定现在是被结果确认的,而不是靠一个关于 diff 的论证撑着 —— 这是一个严格优于上一轮被挡时那种位置的证据状态。defer 的理由已经消失,而在具体理由消失之后继续 defer 是噪音,不是谨慎。 在这个 commit 上我自己核实、而不是采信的东西: 剩下的确实都不阻塞,而我想说清是哪一种不阻塞。设计文档仍然把各 host 描述成"沿着它们已经拷贝的字段一起拷贝 按仓库自己关于不要让评审轮次把 PR 撑大的指引,这个 PR 已经到了只应落关键修复、建议项应当延后的阶段。没有关键修复。所以我把这两条文档项在此记录为延后到一次后续提交,而不是再要求一次推送、再多一轮 —— 记录在案,不是悄悄丢掉。 未测量的那一半在性质上不同,我也不打算含糊:还没有人知道真实模型是否会读这些数字并因此少空转。这不是本 PR 证据上的缺口,而是这里任何合并前通道都无法产出的东西 —— 沙箱按设计无凭据,所以它根本到不了模型;而提示词从不渲染进 TUI,所以终端通道也没有任何东西可展示。描述自己在 Not validated 里写明了这一点,这是承载它的诚实方式。Stage 2 为可查的那一半点名了 关于数量:这位作者手上有一叠 Goal 形态的开放 PR,我检查了这是否让我疲掉。它没有改变我读的内容 —— 消费者清点、渲染点门槛、数字变旧的窗口与计数器语义,都是我到这个分支里去看的,而不是从描述里接受的;而集中化那个提交是我花时间最多的地方。它确实仍然支持作者提出的排序:运行时上限先落地,这个叠在它上面,而且两者对"进展"的含义是一致的。 关于批准状态有一点要说明,因为这是本 PR 讨论串上以前吃过亏的地方:@wenshao 在这个确切的 commit 上批准了,那是另一张票,我没有把它算作我的。 结论:批准。 批准通过 reviews API 钉在 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
…dgement on turn one The three host copies were the only link in the chain with nothing behind them: `usage` is optional on both sides of every hop, so deleting a copy typechecks and costs the prompt its budget line on that host alone. One case per host now fails when its copy is removed. The judge-your-previous-turn line is held back on the Goal's first turn. `create` schedules a continuation before any turn has finished, so that line asks the model to judge a turn that does not exist. A host that reports no figures still gets the line: not knowing the turn number is not evidence of a first turn.
|
Both open points taken, in 109c1e6. The host pass-throughs now have assertions. You are right that this was the one link with nothing behind it, and that an unchanged suite is the tell. One case per host: The On splitting the budget line from the progress lines: I left them together. They ship as one prompt, the E2E evidence is one transcript, and either half is a one-line revert in the same file. If either misbehaves the revert is cheap, and splitting now would mean two prompt-text PRs racing on the same five whole-prompt expectations. Merge order, unchanged: this needs to land after #11239 and #11248. It shares 中文说明两个开放问题都采纳了,在 109c1e6。 host 侧的透传现在有断言了。 你说得对,那是整条链上唯一没有东西兜着的一环,而"测试套原样通过"本身就是破绽。每个 host 一个用例:
关于把预算行和指引行拆开:我没有拆。它们作为同一段提示词上线,E2E 证据也是同一份记录,而任何一半都是同一个文件里的一行回滚。真出问题回滚很便宜;现在拆开反而会让两个改提示词的 PR 在同样那五处整段断言上互相排队。 合并顺序不变:本 PR 要排在 #11239 和 #11248 之后。它与前者共用 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
Not explored to full depth (tool budget reached): "agent 6c": I did not execute the new tests ( packages/core and packages/cli vitest runs) — my conclusions about them are from reading the assertions against the impleme….
Test Plan (not a blocker): Tests 168 passed — this review observed 28913, 23668, 1959, 298, 1818, 513, 6408 passed.
中文说明
已审查。 建议见行内评论。
未探索到全部深度(达到工具调用预算):"agent 6c":I did not execute the new tests ( packages/core and packages/cli vitest runs) — my conclusions about them are from reading the assertions against the impleme…。
Test Plan(非阻断):Tests 168 passed — this review observed 28913, 23668, 1959, 298, 1818, 513, 6408 passed。
— qwen3.8-max via Qwen Code /review (v0.23.0)
|
@qwen-code /resolve |
Both main (QwenLM#11248) and this branch inserted a new paragraph at the same point in the Goals introduction. The two describe different surfaces and are both kept: main's covers the footer pill and status cards rendering spend against the budget window, this branch's covers the figures and standing guidance the model receives at the start of each autonomous turn. Main's paragraph goes first because it defines "the window it is allowed" and names model.goalTokenBudget, which the second paragraph then relies on.
|
Qwen Code resolved the merge conflicts and pushed the branch update. Merge: PR #11257 ←
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
2 Suggestion-level finding(s) this review confirmed are already reported on this PR and are not repeated:
- R1-1 docs paragraph over-promises the standing guidance lines — already reported (docs/users/features/goals.md:21, comment 3949324960)
- R1-2 design note deleted, description pointer survives — already reported (docs/users/features/goals.md:21, comment 3949324984)
Test Plan (not a blocker): Tests 168 passed — this review observed 29027, 23695, 1971, 298, 1819, 513, 6533 passed.
Deferred under the convergence posture (round 2, not a blocker) — recorded, not requested in this round:
packages/core/src/goals/goal-continuation-prompt.ts:187 — [review] D2-1 first-turn gate reads the persisted record turnCount, not whether a previous continuation is in the model's contextpackages/core/src/goals/goal-continuation-prompt.ts:120 — [review] D2-2 EVIDENCE_LINE calls the untrusted workspace and tool channels authoritative without the house qualifierpackages/core/src/goals/goal-continuation-prompt.ts:90 — [review] D2-3 the description's premise that get_goal cannot supply the figures is contradicted by the permitted snapshot
中文说明
已审查——无阻断问题。 建议见行内评论。
本轮确认的 2 条建议级发现已在 PR 上报告过,不再重复发布(列表见上方英文部分)。
Test Plan(非阻断):Tests 168 passed — this review observed 29027, 23695, 1971, 298, 1819, 513, 6533 passed。
收敛姿态下延后(第 2 轮,非阻断)——已记录,本轮不要求修改:共 3 条(原文未翻译,列表见上方英文部分)。
— qwen3.8-max via Qwen Code /review (v0.23.0)
|
@qwen-code /triage |
|
Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 53 passed · 6 failed · 59 total Flakiness gate: 中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查。 脚本断言:53 通过 · 6 失败 · 59 总计 抖动门: Verification reportPR 11257 — deep verificationVerdict: The code is correct. Every assertion about the shipped behaviour of this All 6 failures are claims made in the PR description or in a committed code 中文摘要结论: 代码本身是正确的。 所有关于这次改动实际行为的断言都通过了:预算行及其算术、 A/B 结论(见 Central claim + A/B 表与 6 条失败全部是 PR 描述或已提交代码注释里的说法被测量推翻,不是改动的缺陷:
未覆盖范围:逐 commit 归因(浅克隆 depth 2,本地 ScopeCentral claim. Every runtime-scheduled Goal continuation prompt now opens Secondary claims. (S1) Out of scope by choice, and listed under Not covered: live-model E2E, Central claim + A/BBoth arms drive the real Scenario
4/4 flip from absent to present, and each line's used / remaining / turn Other cells (all in the capture):
Control validity (assertions CorrectionsThese correct the description and a committed comment, not the code. Nothing 1. The data-block rationale names a mechanism that does not exist. Both the The objective-change check is Proven by mutant rather than by reading: with the budget line moved inside cd /__w/qwen-code/qwen-code
git worktree add tmp/mutant-tree HEAD
ln -sfn "$PWD/node_modules" tmp/mutant-tree/node_modules
ln -sfn "$PWD/packages/core/node_modules" tmp/mutant-tree/packages/core/node_modules
# in tmp/mutant-tree/packages/core/src/goals/goal-continuation-prompt.ts:
# add `...(input.usage ? { budget: renderBudgetLine(input.usage) } : {})` to
# serializeGoalData's JSON, and delete the `if (input.usage) { lines.push(...) }` block
npx tsx tmp/pr11257-verify-20260907-233622/ab-harness.ts \
--tree "$PWD/tmp/mutant-tree" --out /tmp/mutant.json
git worktree remove --force tmp/mutant-treeThe placement decision itself is still defensible on a different ground — the 2. "A host that omits Measured (assertions The PR's own tests already encode the true behaviour, which is why this went Bounded — no production impact: the runtime populates 3. The risk sizing understates the change by roughly 3×. Risk & Scope
17 lines is a head rendering ( FindingsOrdered by severity. None is a defect in the shipped behaviour; 1 and 2 are the 1. Suggestion — a committed source comment states a false mechanism. 2. Suggestion — the backwards-compatibility guarantee in the description does cd /__w/qwen-code/qwen-code
npx tsx tmp/pr11257-verify-20260907-233622/ab-harness.ts --tree "$PWD" --out /tmp/head.json
git worktree add tmp/base-tree HEAD^1
ln -sfn "$PWD/node_modules" tmp/base-tree/node_modules
ln -sfn "$PWD/packages/core/node_modules" tmp/base-tree/packages/core/node_modules
npx tsx tmp/pr11257-verify-20260907-233622/ab-harness.ts --tree "$PWD/tmp/base-tree" --out /tmp/base.json
git worktree remove --force tmp/base-tree
# compare renderCells.noUsageAtAll: head 15 lines, base 11 linesFix is to the wording: a host that omits 3. Nice to have — the design note the body points at is not in the PR. 4. Nice to have — the Reviewer Test Plan's expected count is stale. The plan 5. Nice to have — fractional spend renders with decimals. The reducer Checked and clearNamed so the absence of a finding is not read as an absence of checking:
Mutation matrix15/15 killed, 0 survivors, 6/6 positive controls red. Witness:
No survivor needs adjudicating, so no coverage-gap / dead-code / Not covered
MethodologyCI Harnesses ran under Three assertions were wrong in draft and were fixed rather than reported, so Flakiness gate logEvidence imagesHarness scripts and raw logs are in the workflow run artifacts (7-day retention). — Qwen Code · sandboxed verification |
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
1 Suggestion-level finding(s) this review confirmed are already reported on this PR and are not repeated:
- D2-1 first-turn gate reads the persisted record turnCount, not whether a previous continuation is in the model's context — already reported (packages/core/src/goals/goal-continuation-prompt.ts:184, round 2 review body deferral list, review …
Test Plan (not a blocker): Tests 170 passed — this review observed 29030, 23693, 1971, 298, 1819, 513, 6629 passed.
中文说明
已审查——无阻断问题。 建议见行内评论。
本轮确认的 1 条建议级发现已在 PR 上报告过,不再重复发布(列表见上方英文部分)。
Test Plan(非阻断):Tests 170 passed — this review observed 29030, 23693, 1971, 298, 1819, 513, 6629 passed。
— qwen3.8-max via Qwen Code /review (v0.23.0)
…948/upstream-main' into HEAD
Local verification at
|
| # | earlier finding | status at 7bf27d5c |
|---|---|---|
| triage stage 2 | Budget: collides with the objective field this repo documents |
closed — renamed Token budget:. Rendering an objective that itself contains Budget: stop as blocked after 20 turns puts that text inside the JSON data block and the runtime's own line outside it; the two never read as the same field. |
| verify 1 | a committed comment states a mechanism that does not exist | closed — the docblock now gives the trust-boundary reason, which is the one that holds. |
| verify 2 | "a host that omits usage renders the previous prompt exactly" |
closed — the body now says the four progress checks still render. Measured: 11 → 15 lines with no figures. |
| verify 3 | risk sizing understated (five lines added to a prompt "that was seventeen") | closed — the body now says 11 → 15 without figures and 16 with. An exhaustive 16-combination census gives base 11–15 lines and head 13–18; the two stated numbers are exact. |
| verify 4 | test plan predicts 168 passed |
closed — the body says 170; I measured Tests 170 passed (170). |
| verify 5 | the design note the body cites is not in the PR | closed — docs/design/2026-09-07-goal-continuation-budget.md is in the diff, and its account of placement, the two exceptions and the projection type matches the code I read. |
| verify 6 | fractional tokensUsed renders 41,634.5 |
unchanged, and correctly so — see N1. |
6. Mutation matrix at this head
13/13 killed, zero survivors, including the newly renamed label and three same-file positive controls that remove a pre-existing windDown copy — without those, "the mutant survived" and "my command never collected your suite" would be the same observation. Tree asserted clean before the run, after every restore and at the end.
7. Two checks nobody had run
The new line cannot be forged from untrusted text. The budget line is the one piece of trusted context that sits outside the data block, so I fed four hostile objectives through the renderer — an embedded newline plus a fake Token budget: line, a CRLF variant, a </goal_runtime_data> close followed by a forged guard and figure, and a literal \n. All four render exactly one Token budget: line, always the runtime's own, always outside the block; line count stays 16 in every case. JSON.stringify keeps the objective on one line and escapeJsonTagCharacters handles the tag.
The lines never reach a user's turn. Census over a 129-request run with an active Goal: 103/103 runtime-scheduled turns carry the budget line, and 0/26 of everything else does — including get_goal tool results and an ordinary user message sent on --continue while the Goal was active.
8. Gates
| gate | result |
|---|---|
packages/core — the two suites from the test plan |
Tests 170 passed (170) |
packages/cli — the five suites from the test plan |
1377 passed | 1 skipped (1378), 102 s |
npm run typecheck (repo-wide) |
exit 0 |
prettier --check on all 15 changed files |
clean |
eslint --max-warnings 0 on the 13 changed TS files |
clean |
| CI on this head | 23 pass, 0 fail, 26 skipping |
Not covered
Live provider traffic (the model was scripted, so this reproduces the prompt bytes and the runtime's arithmetic, not a real model's reaction to the figures); macOS and Windows, which the PR marks toLocaleString('en-US') so it does not follow the host locale; the repo-wide suite and the integration suites; and whether the progress lines measurably reduce idle turns, which the PR scopes out itself.
Non-blocking notes
N1 — fractional spend still renders decimals, and I would leave it. isNonNegativeNumber admits a fractional tokensUsed, which reaches toLocaleString as Token budget: 41,634.5 of …. takeGoalTurnTokens sums integer totalTokenCount values, so nothing shipped can produce it. Guarding it would add a branch no test can reach honestly.
N2 — nothing pins the whole chain permanently. The runtime tests assert the usage object, the prompt tests assert the text, the host tests inject at the host seam; no fixture drives runtime → host → rendered text. My E2E closes that for this round only. A single integration fixture that drives the real runtime and asserts on the rendered string would keep it closed.
N3 — the usage === undefined branch is unreachable from every shipped host, since flushContinuation populates usage unconditionally. The description and design note are right about what happens there, but a reader may take it for a live configuration; one clause in the design note saying only an external embedder can reach it would settle that.
N4 — the projection is one rename away from silence. GoalContinuationUsage = Pick<GoalRecord, …> is the right call and fixes the drift the earlier rounds worried about, but a field renamed on GoalRecord still costs the line rather than failing the build. Already raised in round 3; I mention it only because I looked and agree it is a suggestion, not a blocker.
中文版 — 在 7bf27d5c 上的本地验证(第 4 轮,仅增量)
结论:可以合入
我在本地用两个真实 worktree 重新构建了这次改动,并对着脚本化模型跑了完整的端到端。PR 对自身行为的每一条说法在这个 head 上都成立;沙箱轮在 1709ae78 上留下的 6 条发现,除了那条本来就不是缺陷的以外全部关闭;变异矩阵干净。
这是本 PR 的第 4 轮(triage stage 1–3、1709ae78 上的沙箱验证、以及本 head 上的第 3 轮 /review),所以我只报告它们没有覆盖的部分:
- 沙箱轮之后 head 变了 ——
8e399957改了标签、重写了注释、补了设计文档、修正了描述,这些都没被验证过; - 真实端到端运行,沙箱轮在 Not covered 里明确写着没做("沙箱里没有凭据……PR 的 transcript 证据没有被独立复现")。我复现了,而且是在三个生产 host 上都复现;
/goal resume路径、针对新增可信行的注入探针,以及"用户自己的轮次绝不携带这些行"的普查。
环境
两个 worktree:head 7bf27d5c、base 663e55bd(它合入的 main tip),各自完整构建(node scripts/build.js --cli-only && npm run bundle)—— Linux、Node v22.22.2。模型轮由仓库自带的 integration-tests/fake-openai-server.ts 提供,每轮固定计 41,634 total_tokens,账本因此是确定的;全程无凭据、不接真实 provider。对照有效性:head bundle 里有字符串 Token budget:,base bundle 里出现 0 次;两个源码臂加载的模块字节也不同(prompt 98f0a364 vs c794bb38,runtime e21c6e93 vs de7959a2)。base 无论是否收到 usage 都渲染 11 行 / 1,184 字符,所以它是真正忽略该字段的对照,而不是一个构建不全的臂。
1. 核心主张,在线上测得
同一个目标、同一个脚本模型、model.goalTokenBudget = 100000,只有构建不同。观测点是 CLI 真正发出的 HTTP 请求体,而不是 host 边界上的 usage 对象。
那个 Goal 的每一个调度轮,端到端,退出码 0:
数字与运行时自己的账本每一步都相符;第一轮 3 行指引、之后每轮 4 行、交接轮 0 行。同一条阶梯在三个生产 host 上各自独立地跑出来了 —— headless(nonInteractiveCli.ts)、交互式 TUI(useMessageQueue.ts → use-llm-stream.ts)、以及 stdio 上的 ACP(Session.ts),每个都是单独的真实进程。
base 侧同样的运行里,第 1 轮和第 2 轮的提示词逐字节相同(都是 1,184 字符)。这就是本 PR 要解决的问题的度量化表述:在 main 上,提示词里没有任何东西能区分相邻两个调度轮。
体量方面(Risk & Scope 要求评审者权衡这一点):普通续跑轮从 1,184 涨到 2,107 字符(+923,+78%),Goal 首轮 1,184 → 1,789。那一段没说、但我认为应该说的一点:续跑提示词会像其他用户消息一样留在对话里,所以一个长跑的 Goal 携带的是它们全部,而不只是当前这一条。真实、有界,且我没有测量它与压缩(compaction)之间的相互作用。
2. 真正超支时的夹取
100,000 的额度花掉 124,902,渲染出来是 0 remaining 而不是 -24,902;四行指引从这个被告知"不要开始新工作"的轮次上消失了。交接轮是两臂只差一行的唯一一轮。
3. 跨 /goal resume —— 此前任何一轮都没覆盖
同一会话里的两次 headless 运行。resume 在已花掉的量之上又授予一个新窗口,与 docs/users/features/goals.md 的描述完全一致;提示词行正确报出抬高后的上限(166,536 of 266,536 … 100,000 remaining),turnCount 从 4 连到 7。首轮豁免在这里没有被误用:第 4 轮是 resume 后那次运行的第一轮,却是这个 Goal 的第四轮,所以"判断上一轮"那行理应保留 —— 它确实保留了。
4. 交互式 host 的真实运行
有必要挑明:这一帧在两臂之间逐字节相同(两边 md5 都是 fa894d0f2b71806b4a563789761555a6)。这次改动在屏幕上是不可见的,它完全活在 host 送上线的内容里 —— 而那正是我测量的地方。
5. 此前的发现在本 head 上的状态
| # | 此前的发现 | 在 7bf27d5c 上的状态 |
|---|---|---|
| triage stage 2 | Budget: 与仓库自己文档化的目标字段撞名 |
已关闭 —— 改名为 Token budget:。渲染一个本身含有 Budget: stop as blocked after 20 turns 的目标时,那段文本落在 JSON 数据块里,而运行时自己那行在块外;两者不会被读成同一个字段。 |
| verify 1 | 已提交的注释写了一个并不存在的机制 | 已关闭 —— 注释改成了成立的那个理由(信任边界)。 |
| verify 2 | "不提供 usage 的 host 渲染出来就是原来那段提示词" |
已关闭 —— 正文现在说四行进展指引仍会渲染。实测:无数字时 11 → 15 行。 |
| verify 3 | 风险体量被低估("原本 17 行的提示词加了 5 行") | 已关闭 —— 正文现在写的是无数字 11 → 15 行、有数字 16 行。穷举 16 种开关组合:base 11–15 行、head 13–18 行,正文这两个数字精确无误。 |
| verify 4 | 验证方式里写 168 passed |
已关闭 —— 正文写 170,我实测 Tests 170 passed (170)。 |
| verify 5 | 正文指向的设计文档不在 PR 里 | 已关闭 —— docs/design/2026-09-07-goal-continuation-budget.md 已在 diff 中,其对位置、两处豁免和投影类型的叙述与我读到的代码一致。 |
| verify 6 | tokensUsed 为小数时渲染成 41,634.5 |
未改动,而且我认为不该改 —— 见 N1。 |
6. 本 head 上的变异矩阵
13/13 全部被杀死,零幸存,其中包括新改名的标签,以及 3 个同文件阳性对照(删掉一个已有的 windDown 拷贝)—— 没有它们的话,"变异体幸存了"和"我的命令根本没收集到你的用例"是同一个观测。变异开始前、每次还原后、以及结束时都断言了工作树干净。
7. 两项此前无人做过的检查
新增的这一行无法被不可信文本伪造。 预算行是唯一一个位于数据块之外的可信上下文,所以我把四种敌意目标喂给渲染器 —— 内嵌换行加一条假的 Token budget: 行、CRLF 变体、</goal_runtime_data> 闭合后跟伪造的守卫行和数字、以及字面量 \n。四种情况都只渲染出一条 Token budget: 行,始终是运行时自己的那条,始终在块外;行数在每种情况下都保持 16。JSON.stringify 把目标压在一行里,escapeJsonTagCharacters 处理标签。
这些行绝不会落到用户自己的轮次上。 在一次带活跃 Goal 的 129 次请求的运行上做普查:103/103 个运行时调度轮携带预算行,其余 0/26 全都不带 —— 包括 get_goal 的工具结果,以及 Goal 活跃期间通过 --continue 发出的一条普通用户消息。
8. 门禁
| 门禁 | 结果 |
|---|---|
packages/core —— 验证方式里的两个套件 |
Tests 170 passed (170) |
packages/cli —— 验证方式里的五个套件 |
1377 passed | 1 skipped (1378),102 秒 |
npm run typecheck(全仓) |
exit 0 |
prettier --check 全部 15 个改动文件 |
干净 |
eslint --max-warnings 0 13 个改动 TS 文件 |
干净 |
| 本 head 上的 CI | 23 通过、0 失败、26 skipping |
未覆盖
真实 provider 流量(模型是脚本化的,所以复现的是提示词字节和运行时的算术,不是真实模型对这些数字的反应);macOS 与 Windows —— PR 自己标了 toLocaleString('en-US'),不跟随宿主 locale;全仓测试与集成测试;以及指引行是否能可测量地减少空转轮,这一点 PR 自己已列为范围外。
非阻塞事项
N1 —— 小数额度仍会渲染出小数,我倾向于保持现状。 isNonNegativeNumber 允许小数 tokensUsed,它会带着小数进入 toLocaleString,渲染成 Token budget: 41,634.5 of …。但 takeGoalTurnTokens 累加的是整数 totalTokenCount,已发布的代码路径产生不出这种值。为它加守卫等于加一个没有测试能诚实覆盖到的分支。
N2 —— 没有任何东西长期钉住整条链路。 运行时测试断言 usage 对象、提示词测试断言文本、host 测试在 host 缝隙处注入;没有任何 fixture 驱动 运行时 → host → 渲染文本。我这一轮的 E2E 只在本轮把这个缺口补上了。一条驱动真实运行时并断言渲染字符串的集成 fixture 可以把它长期补住。
N3 —— usage === undefined 分支在所有已发布的 host 上都不可达,因为 flushContinuation 无条件填充 usage。描述和设计文档对那里会发生什么说得没错,但读者可能会把它当成一种现实存在的配置;在设计文档里加一句"只有外部嵌入方能走到这条分支"就能说清。
N4 —— 这个投影离一次改名只有一步之遥。 GoalContinuationUsage = Pick<GoalRecord, …> 是对的选择,也修掉了此前几轮担心的漂移;但如果 GoalRecord 上的字段被改名,代价仍然是少一行,而不是构建失败。第 3 轮已经提过;我写在这里只是因为我看过之后同意它是建议而非阻塞。
🤖 Generated with Claude Code — Claude Opus 5 (1M context)
…948/upstream-main-latest' into HEAD # Conflicts: # packages/core/src/goals/goal-runtime.test.ts
|
@qwen-code /triage |
|
Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 50 passed · 1 failed · 51 total Flakiness gate: 中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查。 脚本断言:50 通过 · 1 失败 · 51 总计 抖动门: Verification reportPR 11257 — deep verification (round 2)Verdict: The code is correct, and the previous round's three substantive corrections The single failing assertion is a stale number in the PR body, not a defect: 中文摘要结论: 代码本身正确,上一轮的三条实质性更正全部被采纳。 关于这次改动实际行为的 A/B 结论(见 Central claim + A/B 表与 唯一失败的断言是 PR 正文里的一个过期数字,不是缺陷:评审验证方式写的是 上一轮 5 条发现的状态(详见 Previous-finding status 表):#1 代码注释里的 未覆盖范围:逐 commit 归因(浅克隆 depth 2,本地 Previous-finding statusThe previous round verified head
The previous round's Checked and clear list was also re-verified rather than One delta worth naming: the base prompt module is byte-identical to the ScopeCentral claim. Every runtime-scheduled Goal continuation prompt now opens Secondary claims. (S1) New this round (the delta). Out of scope by choice, and listed under Not covered: live-model E2E, Central claim + A/BBoth arms drive the real Scenario
4/4 flip from absent to present ( The PR body's transcript evidence reproduces byte-for-byte. All three Other cells (all in the capture):
Control validity ( CorrectionsNone new. The three corrections the previous round raised against the FindingsOrdered by severity. None is a defect in the shipped behaviour. 2–5 are 1. Nice to have — the Reviewer Test Plan's expected count is stale (the The command yields cd /__w/qwen-code/qwen-code/packages/core && \
npx vitest run src/goals/goal-continuation-prompt.test.ts src/goals/goal-runtime.test.ts
2. Nice to have — the core package has no test that drives cd /__w/qwen-code/qwen-code
python3 tmp/pr11257-verify-20260908-074556/adjudicate-survivors.py /tmp/survivors.jsonSo the guard is pinned ( 3. Nice to have — ...(snapshot.goal.tokenBudget === undefined ? {} : { tokenBudget: snapshot.goal.tokenBudget }),Mutant M7 replaces that with an unconditional Adjudicated as a coverage gap, not a defect — and bounded by measurement 4. Nice to have — the design doc describes the host copies the refactor 5. Nice to have — const { permit, continuationContext, ...hints } = turn;
return renderGoalContinuationPrompt({ goalId: permit.goalId, revision: permit.revision, objective: continuationContext, ...hints });Production hosts pass their whole queue entry, so 6. Observation, not counted in Mutation matrix9/9 guards killed by the suite that owns the file, 6/6 same-file positive
M8–M11 are the refactor's own hazard class, and the author's claim in the
Note what the refactor changed about the matrix itself: No combination row was owed: no two hunks in this PR defend the same hazard Not covered
MethodologyCI Harnesses ran under Mutations were applied in place by Flakiness gate logEvidence imagesHarness scripts and raw logs are in the workflow run artifacts (7-day retention). — Qwen Code · sandboxed verification |
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
# Conflicts: # docs/users/features/goals.md # packages/core/src/goals/goal-runtime.ts
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Test Plan (not a blocker): Tests 170 passed — this review observed 29126, 23788, 1972, 298, 1819, 515, 6687 passed.
中文说明
Test Plan(非阻断):Tests 170 passed — this review observed 29126, 23788, 1972, 298, 1819, 515, 6687 passed。
— qwen3.8-max via Qwen Code /review (v0.23.0)
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Partially reviewed — gaps disclosed.
1 Suggestion-level finding(s) this review confirmed are already reported on this PR and are not repeated:
- R5-9 first-turn gate keys on the record's turnCount rather than the previous turn's authorship — already reported as D2-1 (round 2 review body deferral list, review 5135260622; re-noted round 3, review 5137467791)
Not reviewed: test-efficacy probe — Agent 7's mutation and hunk-revert probe could not run: all 6 probes returned inconclusive because the probe runner died on the vitest globalSetup build guard, so harnessValidated is null and no survivor is either validated or refuted. The build (19/19 workspaces), every in-scope test suite, base-tree and test-delta all did run, and the round-2 /verify-pr mutation matrix measured 9/9 guards killed on goal sources that are byte-identical to this head.
Test Plan (not a blocker): Tests 170 passed — this review observed 29259, 23835, 1973, 298, 1819, 515, 6687 passed.
中文说明
仅完成部分审查,审查缺口已披露。
本轮确认的 1 条建议级发现已在 PR 上报告过,不再重复发布(列表见上方英文部分)。
未审查(原文为英文):test-efficacy probe — Agent 7's mutation and hunk-revert probe could not run: all 6 probes returned inconclusive because the probe runner died on the vitest globalSetup build guard, so harnessValidated is null and no survivor is either validated or refuted. The build (19/19 workspaces), every in-scope test suite, base-tree and test-delta all did run, and the round-2 /verify-pr mutation matrix measured 9/9 guards killed on goal sources that are byte-identical to this head.
Test Plan(非阻断):Tests 170 passed — this review observed 29259, 23835, 1973, 298, 1819, 515, 6687 passed。
— qwen3.8-max via Qwen Code /review (v0.23.0)
Local verification at
|
| gate | result |
|---|---|
packages/core — the two suites from the test plan |
Tests 190 passed (190) |
packages/cli — the five suites from the test plan |
1393 passed | 1 skipped (1394), 134 s |
npm run typecheck (repo-wide) |
exit 0 |
prettier --check on all 15 changed files |
clean |
eslint --max-warnings 0 on the 13 changed TS files |
clean |
| CI on this head | 23 pass, 0 fail, 26 skipping |
Not covered
Live provider traffic (the model is scripted, so this reproduces the prompt bytes and the runtime's arithmetic, not a real model's reaction to the figures); the verifier path, since the scripted model never proposes a terminal state — every run ends at a limit, not at a verdict; macOS and Windows, which the PR marks toLocaleString('en-US'), so it does not follow the host locale; the repo-wide suite and the integration suites; and whether the progress lines measurably reduce idle turns, which the PR scopes out itself.
Notes — none blocking
N1 — the ...hints spread is ordered so a caller's stray key outranks the permit.
renderGoalContinuationTurn puts ...hints after goalId, revision and objective, so a turn object carrying any of those three keys silently replaces the permit-derived values in the rendered data block. The old named-copy form could not do this. Nothing shipped reaches it — all four hosts carry only turnKey, origin, controller, modelStarted, kind, and I measured the rendered prompt byte-identical with those present — and TypeScript catches the literal form but not the variable form, which is the shape every host uses. Moving ...hints to the front of that object literal closes it in one line. Suggestion, not a blocker.
N2 — the design note now describes the pre-refactor copy shape. docs/design/2026-09-07-goal-continuation-budget.md says "The three hosts copy it into their queue entries alongside the fields they already copy" and scopes the host change as "one field copied through each host's queue entry". After 4fa4be63 no field is copied by name: the hosts spread the whole payload, and the renderer gained renderGoalContinuationTurn and the GoalContinuationTurn type, neither of which the note mentions. The note is in this PR's own diff, so it is worth a paragraph.
N3 — the Reviewer Test Plan's expected count is stale again. It predicts Tests 170 passed (170); this head yields 190 passed (190). That was 170 at 7bf27d5c and 187 at 0bac4bbf — the number moves with main, not with this PR, because goal-runtime.test.ts keeps gaining cases from other Goal work. Not a defect, but it is the one step a reviewer is asked to run.
Correction to my round-4 note N4. I wrote that GoalContinuationUsage = Pick<GoalRecord, …> leaves the projection "one rename away from silence" — that a field renamed on GoalRecord would cost the line rather than fail the build. That is wrong. I renamed tokensUsed to tokensSpent on GoalRecord and the build fails immediately at the projection itself: goal-continuation-prompt.ts(13,3): error TS2344: Type '"tokensUsed" | "tokenBudget" | "turnCount"' does not satisfy the constraint 'keyof GoalRecord', plus 82 more. The Pick is a load-bearing guard, not a silent one. Withdrawn.
Round 4's other notes stand at this head: fractional tokensUsed still renders decimals and still cannot be produced by anything shipped; no single fixture drives runtime → host → rendered text, which my E2E closes only for this round; and the usage === undefined branch remains unreachable from every shipped host, since flushContinuation populates usage unconditionally.
中文版 — 在 c518b774 上的本地验证(第 5 轮,仅增量)
结论:可以合入
我在两个真实 worktree 里重新构建了这个 head 和它的 merge-base,并在三个生产 host 上对着脚本化模型跑了完整的端到端。PR 对自身行为的每一条说法在合并后的 head 上都成立;变异矩阵干净;上一轮之后新增的那个 commit —— 4fa4be63 "refactor(goal): centralize continuation payload" —— 在每个 host 上都保持了它所重构的行为。
这是本 PR 的第 5 轮(triage 1–3、1709ae78 上的沙箱轮、我在 7bf27d5c 上的第 4 轮、以及机器人在 0bac4bbf 上的第 2 轮沙箱验证),所以我只报告它们无法覆盖的部分:
- 重构本身。
4fa4be63把四个 host 里逐字段的拷贝换成了const { permit, ...continuation } = input,把 renderer 里逐字段的转发换成了...hints展开。那正是此前每一轮在旧形态下证明正确的那段代码,而它是在第 4 轮之后才落下的; - 合并后的 head。 机器人验证的是
0bac4bbf;此后 main 上又落了 28 个 commit,本 PR 自己的三个文件也从中拿到了内容(goal-runtime.ts46 行改动、nonInteractiveCli.ts77 行、Session.ts18 行)。没有人驱动过这个合并后的状态; - 与 feat(goal): pause a Goal after three autonomous turns that make no progress #11239 的相互作用。它在第 4 轮之后并入 main,会在三个"什么都没记录下来"的自主轮之后把 Goal 暂停。本 PR 新增的"判断上一轮"那一行与这个闸门是第一次一起出货;
- 以及一条对我自己第 4 轮 N4 备注的更正 —— 那条是错的。
环境
两个 worktree:head c518b774、base a374d1b7(与 main 的 merge-base),各自 npm install && node scripts/build.js --cli-only && npm run bundle —— Linux、Node v22.22.2。模型轮由本地 OpenAI 兼容 mock 提供,每次调用固定计 20,000 total_tokens,并记录每一个请求体,因此账本是确定的;全程无凭据、不接真实 provider。mock 在每个 Goal 轮都调用 run_shell_command,所以每一轮都记录了真实证据,运行不会被 #11239 的闸门提前掐断。
对照有效性:head 构建出的 CLI 里 Token budget: 恰好出现 1 次,base 里 0 次;对 32 种开关组合的穷举普查给出 base 11–15 行、head 13–18 行。base 无论是否收到 usage 都渲染 11 行,所以它是真正忽略该字段的对照,而不是一个构建不全的臂。
1. 核心主张,在线上测得
同一个 Goal、同一个目标、model.goalTokenBudget = 100000,只有构建不同。观测点是 CLI 真正发出的 HTTP 请求体。
base 侧第 0、1、2 轮逐字节相同(md5 177c7efe 出现三次)。这就是本 PR 要解决的问题的度量化表述:在 main 上,提示词里没有任何东西能区分相邻两个调度轮。
我用的目标以 Budget: stop as blocked after 20 turns. 结尾 —— 正是 triage stage 2 提出的撞名场景。它渲染在 JSON 数据块里面,而运行时自己的 Token budget: 行在块外面;两者不会被读成同一个字段。
2. 重构,在三个生产 host 上
三个独立的真实进程 —— headless(nonInteractiveCli.ts)、交互式 TUI(useMessageQueue.ts → use-llm-stream.ts,在 tmux 里驱动)、以及 stdio 上的 ACP(Session.ts,由一个裸 ndjson 客户端驱动)—— 给出相同的行数、字符数和数字。四个改动的 host 文件全部覆盖。展开语法把 usage 带过了每一次队列拷贝。
3. 预言机:运行时自己的账本
从会话记录的 goal_state 快照里读出来,而不是从被测对象里读。每一步都精确相符,包括夹取。
4. wind-down 交接轮
100,000 的额度花掉 120,000,渲染出来是 0 remaining 而不是 -20,000;四行指引从这个被告知"不要开始新工作"的轮次上消失了。交接轮是两臂只差一行的唯一一轮。
5. 与 #11239 的相互作用 —— 此前任何一轮都没覆盖
同样的运行,但模型只输出散文、不调用任何工具。两臂在同一轮、同样的花费上、以同样的理由暂停(status=paused、noProgressTurns=3)。本 PR 没有移动那个闸门。在 head 侧,模型在第 1、2 轮被要求判断上一轮,并且看到了递增的数字,然后运行时才把它停下;在 base 侧,它三次收到同一段 11 行提示词。两个特性的组合方式与设计文档所说的一致。
6. 本 head 上的变异矩阵
21/21 全部被杀死,零幸存。其中 4 个专门针对本次重构 —— 从 renderer 里删掉 ...hints,以及从三个 host 的队列拷贝里各删掉 usage —— 所以重构引入的这些接缝是被测试覆盖的,而不只是被我这一次运行覆盖。3 个同文件阳性对照变异的是已有行为:没有它们的话,"变异体幸存了"和"我的命令根本没收集到你的用例"是同一个观测。变异开始前、每次还原后、以及结束时都断言了工作树干净。
7. 新管线必须保持为真的两件事
预算行是唯一一个位于不可信数据块之外的可信行,所以我把五种敌意目标喂给渲染器 —— 内嵌换行加一条伪造的 Token budget: 行、CRLF 变体、</goal_runtime_data> 闭合后跟伪造的守卫行和数字、字面量 \n、以及一个标签形状的开头。五种情况都只渲染出一条预算行,始终是运行时自己的那条,始终在块外,行数恒为 16。
而这些行绝不会落到用户自己的轮次上:在来自五次真实运行的 81 个请求里,68/68 个运行时调度轮携带这一行,其余 0/13 全都不带 —— 包括这些运行发出的全部 9 次旁路模型调用(输出语言 side query 和记忆抽取 subagent),以及在 Goal 活跃期间敲进 TUI 的一条普通用户消息。
这一帧是另一次 TUI 运行(额度 400,000),特意留着好在 Goal 活跃期间敲进用户消息。屏幕上的数字和提示词里的数字来自同一本账 —— 额度花完后胶囊读作 440.0k/400.0k —— 而输入框上方那条用户自己的轮次,一行新内容都没带。
8. 门禁
| 门禁 | 结果 |
|---|---|
packages/core —— 验证方式里的两个套件 |
Tests 190 passed (190) |
packages/cli —— 验证方式里的五个套件 |
1393 passed | 1 skipped (1394),134 秒 |
npm run typecheck(全仓) |
exit 0 |
prettier --check 全部 15 个改动文件 |
干净 |
eslint --max-warnings 0 13 个改动 TS 文件 |
干净 |
| 本 head 上的 CI | 23 通过、0 失败、26 skipping |
未覆盖
真实 provider 流量(模型是脚本化的,所以复现的是提示词字节和运行时的算术,不是真实模型对这些数字的反应);verifier 路径 —— 脚本模型从不提出终态提案,每次运行都停在某个限制上而不是停在一个判定上;macOS 与 Windows —— PR 自己标了 toLocaleString('en-US'),不跟随宿主 locale;全仓测试与集成测试;以及指引行是否能可测量地减少空转轮,这一点 PR 自己已列为范围外。
备注 —— 均不阻塞
N1 —— ...hints 的展开顺序让调用方的杂散键盖过 permit。
renderGoalContinuationTurn 把 ...hints 放在 goalId、revision、objective 之后,所以只要 turn 对象上带了这三个键中的任何一个,它就会静默替换掉由 permit 推出的值。旧的逐字段拷贝形态做不到这一点。已发布的代码路径都碰不到它 —— 四个 host 只带 turnKey、origin、controller、modelStarted、kind,我实测在它们存在时渲染结果逐字节不变 —— 而 TypeScript 只能拦住字面量形式,拦不住变量形式,偏偏后者才是每个 host 的实际写法。把 ...hints 移到那个对象字面量的最前面,一行就能关掉。建议,非阻塞。
N2 —— 设计文档描述的还是重构前的拷贝形态。 docs/design/2026-09-07-goal-continuation-budget.md 写的是 "The three hosts copy it into their queue entries alongside the fields they already copy",并把 host 侧改动的范围写成 "one field copied through each host's queue entry"。4fa4be63 之后没有任何字段是按名字拷贝的:host 展开整个载荷,renderer 新增了 renderGoalContinuationTurn 和 GoalContinuationTurn 类型,而文档两者都没提。这份文档就在本 PR 的 diff 里,值得补一段。
N3 —— 评审验证方式里的预期数字又过期了。 它写的是 Tests 170 passed (170),这个 head 实测 190 passed (190)。在 7bf27d5c 上是 170,在 0bac4bbf 上是 187 —— 这个数跟着 main 走而不是跟着本 PR 走,因为 goal-runtime.test.ts 一直在从其他 Goal 工作里获得新用例。不是缺陷,但这恰恰是评审被要求亲自执行的那一步。
对我第 4 轮 N4 备注的更正。 我当时写 GoalContinuationUsage = Pick<GoalRecord, …> "离一次改名只有一步之遥",说 GoalRecord 上的字段被改名只会少一行、不会让构建失败。这是错的。我把 GoalRecord 上的 tokensUsed 改名成 tokensSpent,构建立刻在投影本身处失败:goal-continuation-prompt.ts(13,3): error TS2344: Type '"tokensUsed" | "tokenBudget" | "turnCount"' does not satisfy the constraint 'keyof GoalRecord',另加 82 个错误。这个 Pick 是一个承重的守卫,不是一个静默的守卫。该备注撤回。
第 4 轮的其余备注在本 head 上仍然成立:小数 tokensUsed 仍会渲染出小数,而已发布的代码路径仍然产生不出这种值;没有任何单个 fixture 贯通 运行时 → host → 渲染文本,我这一轮的 E2E 只在本轮把这个缺口补上;usage === undefined 分支在所有已发布的 host 上仍然不可达,因为 flushContinuation 无条件填充 usage。
🤖 Generated with Claude Code — Claude Opus 5 (1M context)
|
@qwen-code /triage |
|
Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 63 passed · 1 failed · 64 total Flakiness gate: 中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查。 脚本断言:63 通过 · 1 失败 · 64 总计 抖动门: Verification reportSandboxed verification: ❌ not passed — findings reported (agent verdict) Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 63 passed · 1 failed · 64 total Verified head 中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查。 脚本断言:63 通过 · 1 失败 · 64 总计。已验证 head 结论:代码本身正确,行为面零缺陷。 关于这次改动实际行为的断言全部通过:预算行及其算术、四行进展指引、首轮豁免、wind-down 抑制与"只多一行"、剩余量夹到 0、位置、无额度措辞、四个 host 的每一跳、以及"用户自己的轮次永远看不到这些行"。A/B 中 head 4/4 个续跑提示词带预算行、base 0/4;positive control(真实 objective 编辑)在两侧都触发。变异矩阵 9/11 个防护点被其所属套件杀死、6/6 个同文件阳性对照变红,两个幸存者都已裁定(其中一个现在有了实测的一词修复)。 唯一失败的断言是 PR 正文里的过期测试数:评审验证方式写 上一轮 6 条发现 + 1 条结构性备注:#1 测试数过期 → 仍存在(170/187 → 170/190);#2 core 没有带数字驱动 未覆盖:逐 commit 归因(浅克隆,快照 13 个 commit 本地只可达 1 个);真实模型 E2E(无凭据——复现的是提示词字节,不是活会话);指引行是否减少空转轮(PR 自列范围外);base 侧 vitest(被工作区构建闸挡住,已证明是环境而非回归);仓库全量测试与集成测试;macOS/Windows。 Verification reportPR 11257 — deep verification (round 3)Verdict: The code is correct and the shipped behaviour has zero defects. Every The single failing assertion is a stale number in the PR body, not a Previous-finding statusThe previous round verified head
Previously fixed rows, re-verified rather than trusted: the committed ScopeCentral claim. Every runtime-scheduled Goal continuation prompt now opens Secondary claims. (S1) New this round (the delta). The delta is two main-sync merges, so there is Central claim + A/BBoth arms drive the real Scenario
4/4 flip from absent to present ( The PR body's transcript evidence reproduces byte-for-byte: all three Other cells (all in the capture):
Control validity ( Newly verified body claim. "User-driven turns never render this prompt, CorrectionsNone new. The three corrections raised in earlier rounds against the FindingsOrdered by severity. None is a defect in the shipped behaviour. 2–6 are 1. Nice to have — the Reviewer Test Plan's expected count is stale (the The command yields cd /__w/qwen-code/qwen-code/packages/core && \
npx vitest run src/goals/goal-continuation-prompt.test.ts src/goals/goal-runtime.test.ts
2. Nice to have — the core package has no test that drives 3. Nice to have — 4. Nice to have — the design doc describes the host copies the refactor 5. Nice to have — 6. Observation, not counted in Mutation matrix9/11 guards killed by the suite that owns the file, 6/6 same-file positive
M8–M11 are the refactor's own hazard class, and the author's design-doc claim —
The two reverts that kill the central claim fail the intended assertion, No combination row was owed: no two hunks in this PR defend the same hazard Not covered
MethodologyCI Harnesses ran under Evidence imagesHarness scripts and raw logs are in the workflow run artifacts (7-day retention). — Qwen Code · sandboxed verification Flakiness gate logEvidence imagesHarness scripts and raw logs are in the workflow run artifacts (7-day retention). — Qwen Code · sandboxed verification |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
…-claim-budget-retry Conflict in docs/users/features/goals.md: upstream added the continuation-prompt paragraph (QwenLM#11257) immediately before the checkpoint paragraph this branch rewrote. Keep both -- the branch's wording also corrects the stall accounting upstream still describes as unconditional, which QwenLM#11304 made conditional on an overflowing evidence window.
|
Released in v0.23.2. |




























What this PR does
Every runtime-scheduled Goal turn now reports its token usage, limit, remainder, and completed-turn count when those figures are available. Non-wind-down turns carry four standing progress checks, except when supplied usage identifies
turnCount === 0; that turn has no previous turn to judge, so it carries three.The four lines ask it to treat the workspace and this turn's tool results as authoritative rather than what earlier turns reported, to work toward the end state the objective asks for rather than a narrower one it can reach sooner, to judge whether its previous turn actually changed anything before spending this one, and to check every explicit requirement against citable evidence before proposing that the Goal is done. The judgement line is held back on the Goal's very first turn, which
createschedules before any turn has finished: there is no previous turn to judge there, and asking for the judgement invites the model to describe one.Placement is deliberate. The figures sit after the standing objective guard and before the objective-updated notice. They stay outside the data block because that block holds explicitly untrusted task data, while the figures are trusted runtime context. The remainder is clamped at zero because the wind-down turn may begin after the window is overspent.
The progress lines are skipped on that wind-down hand-off, which is told not to start new work: a line asking for "a different concrete action now" would contradict it. The budget line is kept there, because a hand-off reports the numbers it stopped at.
usageremains optional on the public host contract. When it is absent, theToken budget:line is omitted and all four non-wind-down progress checks render because no turn count is available. The renderer and host queues share aGoalContinuationUsageprojection fromGoalRecord, and the host tests cover each queue copy.Why it's needed
The model could not see how much of its spend window was left. A Goal stops when
tokensUsedreachestokenBudget, 30,000,000 by default, and gets one wind-down turn to hand off. Until that turn arrived there was no signal at all, so the model could not tell turn 3 of a long run from the turn before the budget stops it, and so could not choose between opening a broad investigation and finishing what it had in hand.get_goalcan provide the same figures, but asking for them costs an extra tool call and does not make the budget visible at the start of every scheduled turn.It was also never asked whether its last turn accomplished anything. The verifier only ever sees a terminal proposal, so a turn that proposes nothing is judged by nobody, and a turn spent restating status is exactly the turn that proposes nothing.
Codex's continuation template runs to 56 lines and covers both, with the same budget figures plus work-from-evidence, no-progress, fidelity and completion-audit sections. This is the equivalent for the parts that apply here. The blocked-audit section is deliberately not copied: this runtime already enforces that as a three-turn fingerprint check in code rather than as prompt text.
Design note in
docs/design/2026-09-07-goal-continuation-budget.md.Reviewer Test Plan
How to verify
The prompt tests pin complete production-shaped ordinary and wind-down prompt strings. They also cover with-budget, no-budget, and overspent cases; require the
Token budget:line to stay outside the untrusted data block and before the objective-updated notice; and distinguish first-turn, later-turn, and wind-down guidance.Each host has coverage proving that usage survives its queue copy. The shared
GoalContinuationUsageprojection keeps the copied fields aligned withGoalRecord, while the host tests catch an omitted copy without making the publicusageinput mandatory.Evidence (Before & After)
The numeric values below came from a real session transcript. The examples use the current renderer's
Token budget:label, so they describe the code under review without claiming that the historical transcript already used the renamed label.Before. The released prompt had no runtime token-budget line and ended at the standing objective guard.
After. Three continuation turns of one Goal, figures rising as it works:
The tail of that third prompt in full:
After — the wind-down hand-off, forced by setting
model.goalTokenBudgetto 30000. The remainder is clamped, and the progress lines are gone:That turn's answer closed with "the token budget is exhausted (62,026 of 30,000), so this Goal now stops and waits for the user to resume it", which is the model reading the figures and using them.
Tested on
Environment (optional)
Linux,
npm run build, headless--output-format textagainst a live model, prompts read back out of the session transcript.Risk & Scope
turnCount === 0, the previous-turn judgement line is omitted. The counterweight is that these lines only reach runtime-scheduled turns, never a user's own.get_goalandupdate_goaldescriptions are unchanged.usageremains optional; a host that omits it gets no numericToken budget:line and receives all four non-wind-down progress checks because no first-turn signal is available.Linked Issues
Part of #4228.
中文说明
这个 PR 做了什么
现在每个由运行时调度的 Goal 轮次,在有用量数据时都会说明已用 token、额度、剩余量和已完成轮数。非 wind-down 轮通常带四行常驻进展指引;只有传入的用量明确表明
turnCount === 0时,因为没有上一轮可判断,才带三行。这四行分别要求它:把工作区和本轮的工具结果当作权威,而不是听信更早的轮次报告了什么;朝着目标要求的终态去做,而不是换一个更快够得着的更窄结果;在花掉这一轮之前先判断上一轮到底有没有改变什么;以及在提出 Goal 已完成之前,把目标里每一条明确要求都对着可引用的证据核一遍。判断那一行在 Goal 的第一轮上会被跳过——
create会在任何一轮结束之前就调度出这一轮,那里根本没有"上一轮"可判断,硬要它判断只会诱使它编一个出来。位置是刻意的:数字排在常驻目标守卫之后、"目标已变更"通知之前;它位于数据块之外,因为数据块承载明确标记为不可信的任务数据,而数字属于可信的运行时上下文。剩余量夹到 0,因为 wind-down 轮开始时窗口可能已经超支。
那一轮的交接被告知不要开始新工作,所以四行指引在它上面被跳过:一句"现在换一个具体动作"会和它直接打架。预算行则保留,因为交接本来就要报出它停在哪个数上。
usage在公开 host 契约上仍然可选;缺失时不渲染Token budget:数字行,并因为没有轮次信息而在非 wind-down 轮渲染全部四行进展指引。renderer 与各 host 队列共享由GoalRecord投影出的GoalContinuationUsage类型,host 测试覆盖每一次队列拷贝。为什么需要
模型看不到自己的额度窗口还剩多少。Goal 在
tokensUsed达到tokenBudget(默认 30,000,000)时停下,并获得一轮 wind-down 用来交接。在那一轮到来之前完全没有信号,所以模型分不清自己是长跑中的第 3 轮,还是预算即将把它停掉的前一轮,也就无从在"展开一次大范围调查"和"把手上的事收尾"之间做选择。get_goal可以提供这些数字,但那需要额外一次工具调用,也不能让每个调度轮次一开始就看到额度。它也从来没有被要求判断上一轮到底完成了什么。verifier 只会看到终态提案,所以一个什么都不提案的轮次没有任何人来判定;而花在复述状态上的轮次,恰恰就是什么都不提案的那种轮次。
Codex 的续跑模板有 56 行,把这两件事都覆盖了:同样的预算数字,加上 work-from-evidence、no-progress、fidelity 和 completion-audit 几段。本 PR 做的是其中适用部分的等价物。blocked-audit 那段刻意没有照搬:本运行时已经把它作为三轮指纹检查在代码里执行了,而不是写成提示词。
设计说明在
docs/design/2026-09-07-goal-continuation-budget.md。评审验证方式
如何验证
提示词测试钉住了生产形态的普通轮和 wind-down 轮完整文本;同时覆盖有额度、无额度、已超支三种情况,要求
Token budget:行位于不可信数据块之外且在"目标已变更"通知之前,并区分第一轮、后续轮和 wind-down 轮的指引。每个 host 都有用例证明 usage 能通过它的队列拷贝。共享的
GoalContinuationUsage投影让字段与GoalRecord保持一致,host 测试则在不把公开usage输入改成必填的前提下捕获遗漏拷贝。证据(Before & After)
下面的数字来自真实会话记录;示例标签使用当前 renderer 的
Token budget:,因此反映待评审代码,而不声称历史记录当时已经使用改名后的标签。改动前。 已发布版本的提示词没有运行时 token 额度行,到常驻目标守卫那一行就结束了。
改动后。 同一个 Goal 的三个续跑轮,数字随着它干活往上走:
第三段提示词的结尾全文:
改动后 —— wind-down 交接轮,通过把
model.goalTokenBudget设成 30000 触发。剩余量被夹住,指引行消失:那一轮的回答结尾是"the token budget is exhausted (62,026 of 30,000), so this Goal now stops and waits for the user to resume it",也就是模型确实读到了这些数字并用上了。
测试平台
环境(可选)
Linux,
npm run build,对真实模型跑 headless--output-format text,提示词从会话记录里回读。风险与范围
turnCount === 0时才少掉"判断上一轮"那一行。对冲的一面是:它们只会出现在运行时调度的轮次上,永远不会出现在用户自己的轮次里。get_goal与update_goal的描述没有改动。usage仍然可选;不提供它的 host 不渲染Token budget:数字行,并因为没有首轮信号而在非 wind-down 轮收到全部四行进展指引。关联 Issue
属于 #4228 的一部分。