Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions docs/design/2026-09-07-goal-continuation-budget.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Telling the model what its Goal has spent, and asking it to check its own progress

## Problem

The continuation prompt is four shared lines, a synthetic-turn guard, the data
block, and the standing objective guard. It tells the model what the objective
is and how to deliver it. It says nothing about two things the model has no
other cheap way to know.

**How much of the window is left.** A Goal stops when `tokensUsed` reaches
`tokenBudget`, 30,000,000 by default, and gets one wind-down turn to hand off.
Until that turn arrives the model has no signal at all: it cannot tell turn 3
of a long run from the turn before the budget stops it, so it cannot choose
between starting a broad investigation and finishing what it has. `get_goal`
can supply the figures, but asking on every turn would spend an extra tool call
to obtain context the runtime already holds when it schedules the turn.

**Whether the last turn accomplished anything.** The verifier only ever sees a
terminal proposal. A turn that proposes nothing is judged by nobody -- and a
turn spent restating status is exactly the turn that proposes nothing. Nothing
in the prompt asks the model to notice that its previous turn changed nothing
and to do something different.

Codex's continuation template runs to 56 lines and covers both: a budget block
with the same figures, plus work-from-evidence, no-progress, fidelity, and
completion-audit sections. Claude Code has no equivalent; its Stop hook feeds
back a refusal reason instead.

## Design

Two additions to the one place every host renders from.

**A token-budget line**, when the runtime supplies figures: what has been
spent, out of what, how much remains, and how many turns are behind it. Spelled
out with locale grouping rather than abbreviated, since a prompt is read once
by a model and not squeezed into a footer.

It sits after the standing objective guard and before the objective-updated
notice. The figures are context for the whole turn; the notice is about what
changed since the last one and reads last so it is acted on last.

It stays out of the data block deliberately. The line contains trusted runtime
figures, while the block is explicitly framed as untrusted task data. Keeping
the line outside preserves that trust boundary.

The remainder is clamped at zero. The wind-down turn runs with the window
already overspent, and a negative remainder would read as nonsense on the one
turn the figures matter most.

**Four progress lines**, on every turn except the hand-off. They ask the model
to treat the workspace rather than the conversation as authoritative, to work
toward the end state the objective asks for rather than a more easily reached
one, to judge whether its previous turn actually changed anything before
spending this one, and to check every explicit requirement against citable
evidence before proposing completion.

They are skipped on the wind-down turn, which is told not to start new work: a
line asking for "a different concrete action now" would contradict it. The
token-budget line is kept there, because a hand-off reports the numbers it
stopped at.

The judge-your-previous-turn line is also held back on the Goal's first turn.
`create` schedules a continuation before any Goal turn has finished, so on
that one turn there is no previous turn to judge, and asking for the judgement
invites the model to describe one. A host that reports no figures at all says
nothing about which turn this is, so the line stands there: silence is not
evidence of a first turn.

**Where the figures come from.** `GoalTurnHost.startGoalTurn` gains an optional
`usage`, and `flushContinuation` reads it off the record at scheduling time,
before the broadcast hands listeners a snapshot they may act on. The shared
`GoalContinuationUsage` type projects the three fields from `GoalRecord`, so
the runtime and renderer cannot drift into different shapes. The three hosts
copy it into their queue entries alongside the fields they already copy, and
pass it to the renderer. User-driven turns never render this prompt, so they
never carry figures.

`usage` remains optional on the public host and renderer contracts for
embedders that have no runtime figures. Its absence omits only the token-budget
line; the progress guidance still renders on non-wind-down turns.

## Scope

- `goal-continuation-prompt.ts`: the shared `usage` type, renderer input,
`renderBudgetLine`, the four progress lines, their placement and their two
exceptions, and the `buildGoalContinuationParts` pass-through.
- `goal-runtime.ts`: the `usage` field on the host contract, and reading it off
the record in `flushContinuation`.
- `useMessageQueue.ts` and `use-llm-stream.ts`, `Session.ts`,
`nonInteractiveCli.ts`: one field copied through each host's queue entry.
- `docs/users/features/goals.md`: what each continuation turn now tells the
model.

Not changed: the `get_goal` and `update_goal` tool descriptions; the blocked
audit, which qwen already runs as a three-turn fingerprint check in the
runtime rather than as prompt text; and the runtime's own bounds, which are
separate work.

## Verification

- `goal-continuation-prompt.test.ts`: complete-string expectations pin both
the ordinary production shape with usage and the wind-down shape. Focused
cases cover the with-budget, no-budget, and overspent renderings; ordering
checks keep the runtime figures outside the data block and above the
objective-updated notice; first-turn and no-usage cases pin the two guidance
exceptions.
- `useMessageQueue.test.ts`, `use-llm-stream.test.tsx`, `Session.test.ts`,
`nonInteractiveCli.test.ts`: one case per host, pinning that the figures
survive that host's copy. The field is optional on both sides of every hop,
so a dropped copy typechecks and would cost the token-budget line on that
host alone; each case fails when its copy is removed.
- `goal-runtime.test.ts`: the host receives the figures the record held when
the turn was scheduled, before and after a turn bills; a Goal with no ceiling
reports none; the wind-down hand-off carries them too.
- End to end against a real model: the rendered prompt read out of a session
transcript, in this change's pull request under Evidence.
2 changes: 2 additions & 0 deletions docs/users/features/goals.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Creating, editing, or resuming a Goal requires a trusted workspace (`/trust`). H

Once a Goal has billed a turn, the footer pill and every status card show what it has spent against the window it is allowed, as `1.2k/30.0m`. The figure counts the model calls the Goal makes in its own turns; subagents and the verifier's own checks are not included. The window is set by [`model.goalTokenBudget`](../configuration/settings.md); resuming a Goal that has spent its window grants another one on top of what it has already spent, so the figure reads `30.0m/60.0m` rather than starting over. A Goal with no budget shows only what it has spent. A Goal that has not billed a turn yet shows no figures at all.

Each turn the session takes on its own reports what the Goal has spent so far, how many turns are behind it, and — unless the Goal runs unbounded — the window it is allowed. Every such turn except the final wind-down hand-off also carries standing instructions to re-check the workspace rather than trust earlier turns' reports, to work toward the end state the objective asks for, to do something different when the previous turn changed nothing (from the second turn on, once there is a previous turn to judge), and to check every requirement against citable evidence before proposing that the Goal is done.

A long Goal periodically compresses the evidence it has recorded into checkpoint claims with a side model call, so later turns and the verifier still have it to cite. That call is bounded by [`model.goalCheckpointTimeoutSeconds`](../configuration/settings.md), 180 seconds by default; a checkpoint that does not finish in time is abandoned as an inconclusive check — the checkpoint stall streak is preserved rather than incremented — and a later turn retries it. The call is streamed, so the per-request transport timeout bounds only connect and first response, and the ceiling itself stops at the stream guards' 15-minute lifetime cap because past that the guard, not the setting, ends the call. That 15-minute limit on the setting is fixed, and raising the stream guard's own cap does not lift it.

## Interrupting a Goal
Expand Down
55 changes: 55 additions & 0 deletions packages/cli/src/acp-integration/session/Session.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24386,6 +24386,61 @@ describe('Session', () => {
);
});

it('carries the spend figures into the continuation prompt', async () => {
// `usage` is optional on both sides of the host hop, so a dropped
// copy typechecks and shows up only as a prompt that lost its budget
// line on this host.
const permit: core.GoalTurnPermit = {
goalId: 'goal-1',
revision: 1,
turnId: 'turn-usage',
};
mockGoalRuntime.getSnapshot.mockReturnValue({
v: 2,
activity: 'running',
goal: {
goalId: 'goal-1',
revision: 1,
objective: 'check weather',
status: 'active',
evidenceCursor: { recordId: 'cursor-1' },
turnCount: 4,
activeTimeMs: 0,
tokensUsed: 1_234,
createdAt: 1234,
updatedAt: 1234,
},
});
mockGoalRuntime.permitForTurn.mockImplementation((turnKey: string) =>
turnKey === 'goal-runtime:turn-usage' ? permit : undefined,
);
mockChat.sendMessageStream = vi
.fn()
.mockResolvedValue(createEmptyStream());

expect(boundGoalHost).toBeDefined();
await boundGoalHost!.startGoalTurn({
permit,
continuationContext: 'check weather',
usage: { tokensUsed: 1_234, tokenBudget: 30_000_000, turnCount: 4 },
});

await vi.waitFor(() => {
expect(mockChat.sendMessageStream).toHaveBeenCalled();
});
const request = (mockChat.sendMessageStream as ReturnType<typeof vi.fn>)
.mock.calls[0]?.[1] as { message: Array<Record<string, unknown>> };
expect(
request.message.some(
(part) =>
typeof part['text'] === 'string' &&
(part['text'] as string).includes(
'Token budget: 1,234 of 30,000,000 tokens used, 29,998,766 remaining; 4 Goal turns finished.',
),
),
).toBe(true);
});

it('settles a Goal turn whose prompt rejects before the turn body runs', async () => {
// `prompt()` rejects ahead of the try whose finally settles the turn
// when `assertCanStartTurn` throws — a session that began closing
Expand Down
21 changes: 6 additions & 15 deletions packages/cli/src/acp-integration/session/Session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import type {
GoalSnapshotV2,
GoalStateCause,
GoalTurnHost,
GoalContinuationTurn,
GoalTurnPermit,
ToolCallRequestInfo,
ToolCallResponseInfo,
Expand Down Expand Up @@ -615,15 +616,11 @@ type BeforeModelSendContext = {
compressionFailed: boolean;
};

interface AcpGoalTurn {
interface AcpGoalTurn extends GoalContinuationTurn {
permit: GoalTurnPermit;
turnKey: string;
controller: AbortController;
origin: 'runtime' | 'user';
continuationContext: string;
objectiveUpdated?: boolean;
windDown?: boolean;
verifierFeedback?: string;
modelStarted: boolean;
}

Expand Down Expand Up @@ -2305,19 +2302,13 @@ export class Session implements SessionContext {
) {
return;
}
const { permit, ...continuation } = input;
this.goalQueue.push({
permit: { ...input.permit },
turnKey: `goal-runtime:${input.permit.turnId}`,
permit: { ...permit },
turnKey: `goal-runtime:${permit.turnId}`,
controller: new AbortController(),
origin: 'runtime',
continuationContext: input.continuationContext,
...(input.objectiveUpdated
? { objectiveUpdated: input.objectiveUpdated }
: {}),
...(input.windDown ? { windDown: true } : {}),
...(input.verifierFeedback
? { verifierFeedback: input.verifierFeedback }
: {}),
...continuation,
modelStarted: false,
});
void this.#drainGoalQueue();
Expand Down
33 changes: 33 additions & 0 deletions packages/cli/src/nonInteractiveCli.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1295,6 +1295,39 @@ describe('runNonInteractive', () => {
);
});

it('carries the spend figures into a scheduled Goal continuation', async () => {
// The host copies `usage` onto its own turn record. The field is optional
// on both sides, so a dropped copy typechecks and costs the prompt its
// budget line on this host alone.
setupMetricsMock();
mockGetCommands.mockReturnValue([goalCommand]);
await prepareGoalState('paused');
mockFinishedGoalWorker();
vi.mocked(mockConfig.bindGoalTurnHost).mockImplementation((host) =>
goalRuntime.bindHost({
startGoalTurn: (input) =>
host.startGoalTurn({
...input,
usage: { tokensUsed: 1_234, tokenBudget: 30_000_000, turnCount: 4 },
}),
preemptGoalTurn: (reason) => host.preemptGoalTurn(reason),
}),
);

await runNonInteractive(
mockConfig,
mockSettings,
'/goal resume',
'goal-runtime-usage',
);

expect(mockLlmClient.sendMessageStream).toHaveBeenCalledOnce();
const [parts] = mockLlmClient.sendMessageStream.mock.calls[0]!;
expect(parts[0]?.text).toContain(
'Token budget: 1,234 of 30,000,000 tokens used, 29,998,766 remaining; 4 Goal turns finished.',
);
});

it('carries the objective-updated notice into a scheduled Goal continuation', async () => {
setupMetricsMock();
mockGetCommands.mockReturnValue([goalCommand]);
Expand Down
21 changes: 6 additions & 15 deletions packages/cli/src/nonInteractiveCli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import type {
GoalRuntime,
GoalSnapshotV2,
GoalTurnHost,
GoalContinuationTurn,
GoalTurnPermit,
ActiveGoal,
ToolCallRequestInfo,
Expand Down Expand Up @@ -223,15 +224,11 @@ function formatLoopDetectedMessage(loopType: LoopType | undefined): string {
return `Loop detection halted the run${detail}.${hint}`;
}

interface HeadlessGoalTurn {
interface HeadlessGoalTurn extends GoalContinuationTurn {
permit: GoalTurnPermit;
turnKey: string;
controller: AbortController;
origin: 'runtime' | 'user';
continuationContext: string;
objectiveUpdated?: boolean;
windDown?: boolean;
verifierFeedback?: string;
}

function sameGoalPermit(
Expand Down Expand Up @@ -658,19 +655,13 @@ export async function runNonInteractive(
) {
return;
}
const { permit, ...continuation } = input;
queuedGoalTurns.push({
permit: { ...input.permit },
turnKey: `goal-runtime:${input.permit.turnId}`,
permit: { ...permit },
turnKey: `goal-runtime:${permit.turnId}`,
controller: new AbortController(),
origin: 'runtime',
continuationContext: input.continuationContext,
...(input.objectiveUpdated
? { objectiveUpdated: input.objectiveUpdated }
: {}),
...(input.windDown ? { windDown: true } : {}),
...(input.verifierFeedback
? { verifierFeedback: input.verifierFeedback }
: {}),
...continuation,
});
},
preemptGoalTurn: (reason) => {
Expand Down
30 changes: 30 additions & 0 deletions packages/cli/src/ui/hooks/use-llm-stream.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -620,6 +620,36 @@ describe('useLlmStream', () => {
expect(syntheticPrompt).toContain('not evidence that the user supplied it');
});

it('renders the queued spend figures into the synthetic Goal turn', async () => {
// The render site reads `usage` off the queued turn. Dropping that read
// typechecks and only shows up as a prompt missing its budget line.
const goal: QueuedGoalTurn = {
kind: 'goal',
permit: {
goalId: 'goal-usage',
revision: 2,
turnId: 'turn-usage',
},
turnKey: 'goal-runtime:turn-usage',
continuationContext: 'report the figures',
usage: { tokensUsed: 1_234, tokenBudget: 30_000_000, turnCount: 4 },
};
const { result, mockSendMessageStream: streamMock } = renderTestHook([]);

await act(async () => {
await result.current.submitQuery(
goal.continuationContext,
SendMessageType.Goal,
'prompt-id-goal-usage',
{ goal },
);
});

expect(streamMock.mock.calls[0]?.[0] as string).toContain(
'Token budget: 1,234 of 30,000,000 tokens used, 29,998,766 remaining; 4 Goal turns finished.',
);
});

it('claims a Goal only after direct user input becomes model-facing', async () => {
const goal: QueuedGoalTurn = {
kind: 'goal',
Expand Down
11 changes: 2 additions & 9 deletions packages/cli/src/ui/hooks/use-llm-stream.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ import {
finalizeToolResponses,
endInteractionSpan,
getActiveInteractionSpan,
renderGoalContinuationPrompt,
renderGoalContinuationTurn,
} from '@qwen-code/qwen-code-core';
import { type Part, type PartListUnion, FinishReason } from '@google/genai';
import type {
Expand Down Expand Up @@ -3711,14 +3711,7 @@ export const useLlmStream = (
submitType === SendMessageType.Goal
? queuedGoal
? {
queryToSend: renderGoalContinuationPrompt({
goalId: queuedGoal.permit.goalId,
revision: queuedGoal.permit.revision,
objective: queuedGoal.continuationContext,
objectiveUpdated: queuedGoal.objectiveUpdated,
windDown: queuedGoal.windDown,
verifierFeedback: queuedGoal.verifierFeedback,
}),
queryToSend: renderGoalContinuationTurn(queuedGoal),
shouldProceed: true,
}
: { queryToSend: null, shouldProceed: false }
Expand Down
Loading
Loading