Repository navigation
feat(cli): /goal and /loop commands for long-horizon tasks #257
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+2,052
−64
Merged
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # Keep proto working toward a goal | ||
|
|
||
| Set a completion condition with `/goal` and proto keeps working across turns until the condition is met. After every turn a small fast model checks the transcript against your condition; if it isn't satisfied yet, proto starts another turn instead of returning control. The goal clears automatically once the condition is met. | ||
|
|
||
| Use a goal for substantial work with a verifiable end state: | ||
|
|
||
| - Migrating a module to a new API until every call site compiles and tests pass | ||
| - Implementing a design doc until all acceptance criteria hold | ||
| - Splitting a large file into focused modules until each is under a size budget | ||
| - Working through a labeled issue backlog until the queue is empty | ||
|
|
||
| ## Set a goal | ||
|
|
||
| Run `/goal` followed by the condition you want satisfied. | ||
|
|
||
| ``` | ||
| /goal all tests in test/auth pass and the lint step is clean | ||
| ``` | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| Setting a goal starts a turn immediately, with the condition itself as the directive — you do not need to send a separate prompt. While the goal is active, the evaluator's most recent reason is shown on `/goal` so you can see what proto is working toward. | ||
|
|
||
| > [!note] | ||
| > One goal can be active per session. Running `/goal <new condition>` replaces the previous one. | ||
|
|
||
| The condition can be up to 4,000 characters. To bound how long a goal runs, include a clause like `or stop after 20 turns` directly in the condition. | ||
|
|
||
| ## Write an effective condition | ||
|
|
||
| The evaluator only sees what proto has surfaced in the transcript — tool calls and the final assistant message. Write the condition so that proto's own output can demonstrate it. | ||
|
|
||
| A good condition usually has: | ||
|
|
||
| - **One measurable end state**: a test result, a build exit code, a file count, an empty queue. | ||
| - **A stated check**: how proto should prove it, such as `npm test exits 0` or `git status is clean`. | ||
| - **Constraints that matter**: anything that must not change on the way there, such as `no other test file is modified`. | ||
|
|
||
| "All tests in `test/auth` pass" works because proto runs the tests and the result lands in the transcript for the evaluator to read. "The code is good" does not, because nothing in the transcript can prove it. | ||
|
|
||
| ## Check status | ||
|
|
||
| Run `/goal` with no arguments to inspect the current state. | ||
|
|
||
| ``` | ||
| /goal | ||
| ``` | ||
|
|
||
| If a goal is active, the status shows the condition, how long it has been running, how many turns have been evaluated, the tokens spent on evaluation so far, and the evaluator's most recent reason. If no goal is active but one was achieved earlier in the session, the status shows the achieved condition along with how long it took. | ||
|
|
||
| ## Clear a goal | ||
|
|
||
| Run `/goal clear` to remove an active goal before its condition is met. Any of `stop`, `off`, `reset`, `none`, and `cancel` are accepted as aliases for `clear`. Starting a new conversation with `/clear` also removes any active goal. | ||
|
|
||
| ``` | ||
| /goal clear | ||
| ``` | ||
|
|
||
| ## How evaluation works | ||
|
|
||
| Each time the main agent finishes a turn, the condition and the conversation so far are sent to your configured content generator for a one-shot evaluator call. The evaluator returns a yes-or-no decision and a short reason. A "no" tells proto to keep working and includes the reason as guidance for the next turn; a "yes" clears the goal and records the achieved entry on `/goal`. | ||
|
|
||
| The evaluator does not call tools, so it can only judge what proto has already surfaced in the conversation. If the evaluator can't tell from the transcript, treat it as "no" and ask for the missing evidence. | ||
|
|
||
| ## How `/goal` differs from `/loop` | ||
|
|
||
| | Trigger for next turn | `/goal` | `/loop` | | ||
| | --------------------- | ------------------------------------------- | ------------------------------------------------ | | ||
| | When it fires | Previous turn ends | A time interval elapses | | ||
| | When it stops | The evaluator confirms the condition is met | You cancel it, or proto decides the work is done | | ||
| | Best for | Verifiable end states | Polling / babysitting on a cadence | | ||
|
|
||
| See [Schedule prompts](./scheduled-tasks.md) for `/loop`. | ||
|
|
||
| ## See also | ||
|
|
||
| - [Schedule prompts](./scheduled-tasks.md) — re-run a prompt on a time interval | ||
| - [Use hooks](./use-hooks.md) — write your own Stop hook when you need custom evaluation logic | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| /** | ||
| * @license | ||
| * Copyright 2026 protoCLI contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| import { describe, it, expect, beforeEach } from 'vitest'; | ||
| import { goalCommand, statusText } from './goalCommand.js'; | ||
| import { type CommandContext } from './types.js'; | ||
| import { createMockCommandContext } from '../../test-utils/mockCommandContext.js'; | ||
| import { GoalManager } from '@qwen-code/qwen-code-core'; | ||
|
|
||
| function makeContext(manager: GoalManager): CommandContext { | ||
| return createMockCommandContext({ | ||
| services: { | ||
| config: { | ||
| getGoalManager: () => manager, | ||
| }, | ||
| }, | ||
| } as unknown as CommandContext); | ||
| } | ||
|
|
||
| describe('goalCommand', () => { | ||
| let manager: GoalManager; | ||
| let ctx: CommandContext; | ||
|
|
||
| beforeEach(() => { | ||
| manager = new GoalManager(); | ||
| ctx = makeContext(manager); | ||
| }); | ||
|
|
||
| it('errors when config is not available', async () => { | ||
| const noConfigCtx = createMockCommandContext({ | ||
| services: { config: null }, | ||
| } as unknown as CommandContext); | ||
| const result = await goalCommand.action!(noConfigCtx, 'all tests pass'); | ||
| expect(result).toEqual({ | ||
| type: 'message', | ||
| messageType: 'error', | ||
| content: expect.stringMatching(/config/i), | ||
| }); | ||
| }); | ||
|
|
||
| describe('status mode (no args)', () => { | ||
| it('reports no active goal when none is set', async () => { | ||
| const result = await goalCommand.action!(ctx, ''); | ||
| expect(result).toMatchObject({ | ||
| type: 'message', | ||
| messageType: 'info', | ||
| }); | ||
| const content = (result as { content: string }).content; | ||
| expect(content).toMatch(/no active goal/i); | ||
| }); | ||
|
|
||
| it('reports the active goal', async () => { | ||
| manager.setGoal('all tests pass'); | ||
| manager.recordTurn(); | ||
| manager.recordEvaluation({ | ||
| met: false, | ||
| reason: 'tests still failing', | ||
| tokensUsed: 42, | ||
| }); | ||
| const result = await goalCommand.action!(ctx, ''); | ||
| const content = (result as { content: string }).content; | ||
| expect(content).toMatch(/active goal/i); | ||
| expect(content).toMatch(/all tests pass/); | ||
| expect(content).toMatch(/tests still failing/); | ||
| }); | ||
|
|
||
| it('reports the last achieved goal if no active goal', async () => { | ||
| manager.setGoal('cleanup'); | ||
| manager.markAchieved(); | ||
| const result = await goalCommand.action!(ctx, ''); | ||
| const content = (result as { content: string }).content; | ||
| expect(content).toMatch(/achieved/i); | ||
| expect(content).toMatch(/cleanup/); | ||
| }); | ||
| }); | ||
|
|
||
| describe('clear mode', () => { | ||
| it.each(['clear', 'stop', 'off', 'reset', 'none', 'cancel'])( | ||
| 'accepts "%s" as a clear alias', | ||
| async (alias) => { | ||
| manager.setGoal('working on something'); | ||
| await goalCommand.action!(ctx, alias); | ||
| expect(manager.hasActiveGoal()).toBe(false); | ||
| }, | ||
| ); | ||
|
|
||
| it('case-insensitive', async () => { | ||
| manager.setGoal('x'); | ||
| await goalCommand.action!(ctx, 'CLEAR'); | ||
| expect(manager.hasActiveGoal()).toBe(false); | ||
| }); | ||
|
|
||
| it('reports "no active goal" when nothing was set', async () => { | ||
| const result = await goalCommand.action!(ctx, 'clear'); | ||
| expect((result as { content: string }).content).toMatch(/no active/i); | ||
| }); | ||
| }); | ||
|
|
||
| describe('set mode', () => { | ||
| it('sets the goal and returns a submit_prompt for the first turn', async () => { | ||
| const result = await goalCommand.action!( | ||
| ctx, | ||
| 'all tests in test/auth pass', | ||
| ); | ||
| expect(result).toEqual({ | ||
| type: 'submit_prompt', | ||
| content: 'all tests in test/auth pass', | ||
| }); | ||
| expect(manager.hasActiveGoal()).toBe(true); | ||
| expect(manager.getActiveGoal()?.condition).toBe( | ||
| 'all tests in test/auth pass', | ||
| ); | ||
| }); | ||
|
|
||
| it('trims surrounding whitespace from the condition', async () => { | ||
| await goalCommand.action!(ctx, ' build is clean '); | ||
| expect(manager.getActiveGoal()?.condition).toBe('build is clean'); | ||
| }); | ||
|
|
||
| it('rejects conditions over 4000 characters', async () => { | ||
| const big = 'x'.repeat(4001); | ||
| const result = await goalCommand.action!(ctx, big); | ||
| expect(result).toMatchObject({ | ||
| type: 'message', | ||
| messageType: 'error', | ||
| }); | ||
| expect((result as { content: string }).content).toMatch(/4000/); | ||
| expect(manager.hasActiveGoal()).toBe(false); | ||
| }); | ||
|
|
||
| it('replaces an existing active goal', async () => { | ||
| manager.setGoal('first'); | ||
| await goalCommand.action!(ctx, 'second'); | ||
| expect(manager.getActiveGoal()?.condition).toBe('second'); | ||
| }); | ||
| }); | ||
| }); | ||
|
|
||
| describe('goalCommand statusText', () => { | ||
| it('renders the no-goal message', () => { | ||
| expect(statusText(new GoalManager())).toMatch(/no active goal/i); | ||
| }); | ||
|
|
||
| it('renders an active goal with recent reason', () => { | ||
| const m = new GoalManager(); | ||
| m.setGoal('build clean'); | ||
| m.recordTurn(); | ||
| m.recordEvaluation({ met: false, reason: 'lint failing', tokensUsed: 10 }); | ||
| const text = statusText(m); | ||
| expect(text).toMatch(/active goal/i); | ||
| expect(text).toMatch(/build clean/); | ||
| expect(text).toMatch(/lint failing/); | ||
| expect(text).toMatch(/1 turn/); | ||
| }); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.