feat(goal): let a Goal stop early when its objective is infeasible - #10125
Conversation
A Goal whose objective cannot be satisfied as written -- it contradicts itself, names a target that verifiably does not exist, or needs an action no tool can perform -- had no sanctioned way to say so. Blocked proposals stop immediately only for user authority or an external change; everything else is treated as a repeated technical blocker and must recur on three consecutive turns before the verifier sees it. So an impossible objective burned turns until the token budget (QwenLM#9891) or a human stopped it. The session that motivated this series ran 34 minutes on an objective ("验证下版本") too under-specified to ever complete. CC's stop evaluator can answer `impossible` and end the loop; this is the counterpart. `blockerKind: 'infeasible'` joins authority, external and repeated. It is not a new status: the Goal settles as `blocked`, which every surface already renders and which resumes into `/goal edit` -- the only fix for an objective that cannot hold. Three rules keep it from becoming an "I think this can't be done" exit: - It bypasses the three-turn repetition rule. Waiting three turns to report an impossibility is the runaway this kind exists to end, and the evidence bar below is what earns the early exit. - The cited evidence must include an external_fact. User input can authorise a stop (that is `authority`) but cannot make an objective impossible, and assistant prose saying so is exactly what must not count. Like the other immediate blockers it must also cite every newer record, so a contradicting fact cannot be left out. - The verifier policy accepts it only for self-contradiction, a target that verifiably does not exist, or an action outside the tools, and rejects difficulty, uncertainty, obtainable information, or a preference to ask. An accepted infeasible stop appends a fixed next step to lastReason, so the stopped Goal tells the user what to do, not only what went wrong. Mutation probes (goal-evidence + goal-runtime + goal-tools, 194 tests), each killing exactly one test: infeasible routed through the repetition audit; policy sentence removed; external_fact requirement removed; next step suffix dropped; 'infeasible' removed from the tool schema enum.
|
@qwen-code /takeover |
qwen-code-dev-bot
added
the
autofix/takeover
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. This is a fork PR, so the first round comes from the next scheduled scan (usually within minutes). Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。本 PR 来自 fork，首轮处理将由下一次定时扫描执行（通常几分钟内）。移除 |
|
✅ Qwen Triage finished — CI landed green on ✅ Qwen Triage 已完成 —— |
|
Thanks for the PR! Template looks good ✓ Problem: observed, not theoretical. The PR documents a concrete runaway — a session that spent 34 minutes and 8.6M tokens on an objective too under-specified to ever complete — and the current rules genuinely have no exit for that case: only Direction: aligned. This completes the stop side of the Goal series (#9880 completion gate, #9891 token-budget backstop), mirroring Claude Code's stop-evaluator Size: core paths touched ( Approach: scope feels right. I worked through the problem independently before reading the diff and landed on essentially this design — a new Risk: no elevated risk signals — none of the changed files match the revert-correlated high-risk paths. Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题：已观测到，不是理论性的。PR 记录了一个具体的失控案例——一个 session 在含混到无法完成的 objective 上耗了 34 分钟、860 万 token——而现有规则确实没有为此提供出口：只有 方向：对齐。这补全了 Goal 系列的停止侧（#9880 完成闸门、#9891 token 预算兜底），对应 Claude Code 停止评估器的 规模：触及核心路径（ 方案：范围合理。我在读 diff 之前独立推演了同一问题，得到的设计基本一致——新增一个排除在三轮审计之外的 风险：无升级风险信号——改动文件均未命中与 revert 相关的高风险路径。 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code reviewI sketched my own approach from the title and motivation before reading the diff: a new
One non-blocking nit: the checkpoint-claim coverage test in One honest testing note: the new runtime test exercises the real runtime end to end, but the verifier's decision is mocked there — the evidence-validation, routing, and schema assertions run against real code, while the model judgment is faked. See the verification note below. CI evidenceAs of this fetch, the PR's own CI ( Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 The author reports 437 goal tests passing locally plus mutation probes killing each new rule — that is the author's claim, not independently re-run; the CI results above are the evidence that counts here. Sandboxed verification would settle the remaining gap: 中文说明代码审查：我在读 diff 前先独立推演了方案，PR 与之几乎一致，且路由上没找到正确性缺口：新成员被排除在 一个非阻塞小项： 诚实的测试说明：新的 runtime 测试端到端驱动了真实 runtime，但其中验证器的裁决是 mock 的——证据校验、路由、schema 断言跑在真实代码上，模型判断是假的。见下方验证说明。 CI 证据：截至抓取时，PR 自身 CI（ 沙箱验证可补齐剩余缺口： — Qwen Code · qwen3.8-max Reviewed at |
|
Confidence: 4/5 — clean, minimal change that closes a real gap; the open items are the still-running CI and one non-blocking test nit, not the code. This one earns its place. The runaway it fixes is documented — 34 minutes and 8.6M tokens on an objective that could never complete — and the V2 runtime genuinely had no sanctioned exit for it. The design pays for the early stop with a real evidence bar instead of trust: only a tool result can carry the claim, every newer record must be cited alongside it, and the independent verifier gets an explicit accept/reject policy that names the disqualifiers. The outcome is a resumable What keeps this at 4 rather than 5: the unit suite mocks the verifier's judgment, so how a real verifier behaves against the new policy sentence is argued, not demonstrated — that's what the Verdict: approve, but the PR's own CI ( 中文说明这个 PR 配得上合入。它修复的失控有记录在案——34 分钟、860 万 token 耗在一个永远无法完成的 objective 上——而 V2 runtime 此前确实没有为此提供被认可的退出方式。设计上用真实的证据门槛而非信任来换取提前停止：只有工具结果能承载该论断，所有更新的记录必须一并引用，独立验证器拿到的是点名了排除项的明确接受/拒绝策略。结果是可恢复的 没给到 5 分的原因：单元测试 mock 了验证器的裁决，真实验证器面对新策略句的行为只是被论证、尚未被演示——这正是审查评论中 结论：通过，但 PR 自身 CI（ — Qwen Code · qwen3.8-max Reviewed at |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
|
Released in v0.22.2. |
4 participants
What this PR does
Adds
blockerKind: 'infeasible'to Goal terminal proposals, so a Goal whose objective cannot be satisfied as written can stop on the turn that proves it instead of burning until the token budget does. It is not a new status: the Goal settles asblocked, which every surface already renders and which resumes into/goal edit— the only fix for an objective that cannot hold.Three rules keep it from becoming an "I think this can't be done" exit. It bypasses the three-turn repetition rule that ordinary technical blockers need — waiting three turns to report an impossibility is the runaway this kind exists to end, and the evidence bar is what earns the early exit. The cited evidence must include an
external_fact(a tool result): user input can authorise a stop, which is whatauthorityis for, but cannot make an objective impossible, and assistant prose saying so is exactly what must not count; like the other immediate blockers it must also cite every newer record so a contradicting fact cannot be omitted. And the verifier policy accepts it only for self-contradiction, a target that verifiably does not exist, or an action outside the tools, and rejects difficulty, uncertainty, obtainable information, or a preference to ask.An accepted infeasible stop appends a fixed next step to
lastReason("edit or replace the Goal with an objective the evidence allows before resuming it"), so the stopped Goal tells the user what to do, not only what went wrong. Theupdate_goaltool description and schema mirror the policy so the model learns when the kind applies and what it must cite.Why it's needed
Blocked proposals stop immediately only for user authority or an external change; everything else is treated as a repeated technical blocker and must recur with identical reason text on three consecutive turns before the verifier sees it. There was no sanctioned way for the model to say "this cannot be done as stated" that the verifier would honour early, so an impossible objective ran until #9891's budget or a human stopped it. The session that motivated this series spent 34 minutes and 8.6M tokens on an objective ("验证下版本") too under-specified to ever complete. CC's stop evaluator can answer
{ok: false, impossible: true}and end the loop; this is the counterpart, held to a stricter evidence bar because our verifier judges from cited transcript records, not from the model's own assessment.Reviewer Test Plan
How to verify
cd packages/core && npx vitest run src/goals/— 437 tests, 16 files. New: an evidence-validation case (assistant-only or user-only citations are refused withinfeasible_blocker_external_fact_required; a tool result is accepted withproofKind: 'external_fact'; leaving newer evidence uncited is refused like any immediate blocker), a runtime case (an infeasible proposal isreadyForVerificationon its first turn, the verifier receives the infeasible policy sentence, and the accepted Goal isblockedwith the next step appended tolastReason), and tool-schema/description assertions.external_factrequirement; drop thelastReasonsuffix; remove'infeasible'from the schema enum.npx tsc --noEmitinpackages/core: 0 errors insrc/goals/, total identical to the merge base. prettier + eslint clean on the seven changed files. No SDK, webui, web-shell or CLI code enumeratesblockerKind, so nothing crosses the wire.Evidence (Before & After)
N/A (runtime policy; the stop renders through the existing blocked-Goal surfaces).
lastReasonof an accepted infeasible stop, for reference:<verifier reason> The objective cannot be satisfied as written; edit or replace the Goal with an objective the evidence allows before resuming it.Tested on
Environment (optional)
N/A (unit tests only).
Risk & Scope
infeasibleto escape hard work. The three rules above are the mitigation — external-fact evidence, full newer-evidence citation, and a verifier policy that names the disqualifiers — and the outcome isblocked, which the user can resume or edit, never a silent completion./goal settime — that stays the model's call, judged by the verifier.blockerKindis optional and the new member is additive.Linked Issues
中文说明
这个 PR 做了什么
为 Goal 终局提案新增
blockerKind: 'infeasible',让一个「按现有表述无法达成」的 Goal 能在证明这一点的那一轮就停下,而不是烧到 token 预算耗尽。它不是新状态:Goal 落为blocked,所有界面已能渲染,且 resume 的出口就是/goal edit——对一个不可能成立的 objective,这是唯一的修法。三条规则防止它变成「我觉得做不到」的逃生口。它绕过普通技术性阻塞所需的三轮重复规则——等三轮再报告不可能,正是这个类型要终结的失控,而证据门槛才是换取提前退出的代价。引用的证据必须包含一条
external_fact(工具结果):用户输入可以授权停止(那是authority的用途),但不能让 objective 变得不可能,助手自己说不可能更是绝不能算数;和其他立即阻塞一样,它还必须引用所有更新的记录,以防遗漏相反事实。verifier 策略只在 objective 自相矛盾、指向可验证不存在的目标、或需要工具无法执行的动作时接受,并明确拒绝困难、不确定、可获取的信息、或想要询问这几种情形。被接受的 infeasible 停止会在
lastReason末尾附加固定的下一步(「编辑或替换 Goal,给出证据允许的 objective,再恢复」),让停止后的 Goal 告诉用户该做什么,而不只是哪里出了问题。update_goal的工具描述与 schema 同步了策略,使模型知道该类型何时适用、必须引用什么。为什么需要
阻塞提案目前只在用户授权或外部变化时立即停止;其余一律视为重复性技术阻塞,必须在连续三轮以完全相同的 reason 文本重现,verifier 才会看到。模型没有任何被认可的方式说出「按现在的表述做不到」并让 verifier 尽早采纳,于是一个不可能的 objective 会一直跑到 #9891 的预算或人工介入。触发本系列的那个 session 在「验证下版本」这种含混到无法完成的 objective 上耗了 34 分钟、860 万 token。CC 的停止评估器可以返回
{ok: false, impossible: true}结束循环;这是对应物,只是证据门槛更严,因为我们的 verifier 依据引用的转录记录判断,而不是模型的自我评估。评审验证计划
如何验证
cd packages/core && npx vitest run src/goals/——437 个测试,16 个文件。新增:证据校验用例(仅引用助手或用户记录以infeasible_blocker_external_fact_required拒绝;工具结果以proofKind: 'external_fact'接受;遗漏更新证据与其他立即阻塞一样被拒绝)、runtime 用例(infeasible 提案首轮即readyForVerification,verifier 收到 infeasible 策略句,接受后的 Goal 为blocked且lastReason附带下一步)、工具 schema/描述断言。external_fact要求;去掉lastReason后缀;从 schema 枚举移除'infeasible'。packages/core的npx tsc --noEmit:src/goals/内 0 错,总数与合并基线相同。七个改动文件 prettier + eslint 干净。SDK、webui、web-shell、CLI 均不枚举blockerKind,无线上变更。证据(前后对比)
N/A(运行时策略;停止状态通过现有的 blocked Goal 界面渲染)。被接受的 infeasible 停止的
lastReason形如:<verifier 理由> The objective cannot be satisfied as written; edit or replace the Goal with an objective the evidence allows before resuming it.已测试平台
Linux ✅;macOS / Windows⚠️ (CI 覆盖)。
环境(可选)
N/A(仅单元测试)。
风险与范围
infeasible逃避困难工作。上述三条规则即是缓解——外部事实证据、完整引用更新证据、点名排除项的 verifier 策略——且结果是blocked,用户可以 resume 或 edit,绝不会静默完成。/goal set时尝试检测不可行性——那仍由模型提出、verifier 判断。blockerKind为可选字段,新成员是增量。关联 Issue