Skip to content

fix(cli): make plan follow-up refinement deterministic - #11808

Merged
kirillk merged 3 commits into
mainfrom
keen-protocol
Jul 1, 2026
Merged

fix(cli): make plan follow-up refinement deterministic#11808
kirillk merged 3 commits into
mainfrom
keen-protocol

Conversation

@kirillk

@kirillk kirillk commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

What

  • Move the Plan-mode "finalize vs continue refining" decision out of model-written chat text and into the deterministic post-plan_exit follow-up panel.
  • Add a third follow-up option, Keep refining, which retargets the same session back to the plan agent instead of switching to implementation.
  • Add i18n strings, telemetry tracking, and regression coverage for the new option.

Why

PR #11170 (commit 38459184f2, feat(cli): align native plan with architect) changed the native Plan prompt to tell the model to ask the user to choose between "Finalize and save the plan" and "Continue refining". That choice became prompt/text-driven rather than tool-driven, so faster/weaker models can simply print numbered choices in chat instead of showing the interactive ask panel.

How it used to show

plan

The existing plan_exit follow-up is engine-driven and already works across CLI, VS Code, and JetBrains. Routing the refinement decision through that panel makes the UX reliable for models that can call plan_exit, instead of relying on the model to choose the structured question tool.

Validation

  • bun test ./test/kilocode/plan-followup.test.ts ./test/kilocode/plan-exit-detection.test.ts from packages/opencode
  • bun test ./tests/unit/plan-followup-locale-keys.test.ts ./tests/unit/question-dock-utils.test.ts ./tests/unit/followup-session.test.ts from packages/kilo-vscode
  • bun run typecheck from packages/opencode
  • bun run typecheck from packages/kilo-vscode
  • bun run lint from packages/kilo-vscode
  • bun run typecheck from packages/kilo-i18n
  • bun run typecheck from packages/kilo-telemetry
  • ./gradlew typecheck from packages/kilo-jetbrains
  • Push hook: bun turbo typecheck

Comment thread packages/opencode/src/kilocode/session/prompt.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Both previously flagged issues are resolved as of 82e58b264d:

  • The prompt.ts reminder already gated the finalize/refine wording by supportsPlanFollowup() (cli/vscode/jetbrains vs. ACP).
  • This commit updates native-plan-prompt.txt to defer to the dynamic "Plan File" reminder ("Follow the latest Plan File reminder for whether to confirm with the user before finalizing, and for what happens after plan_exit") instead of hardcoding the follow-up assumption, addressing the contradiction risk raised in review.
Files Reviewed (1 file changed in this update)
  • packages/opencode/src/kilocode/session/native-plan-prompt.txt
Previous Review Summaries (2 snapshots, latest commit febdc26)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit febdc26)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (2 files)
  • packages/opencode/src/kilocode/session/prompt.ts
  • packages/opencode/test/kilocode/plan-exit-detection.test.ts

Previous review (commit ce09eb3)

Status: 1 Issue Found | Recommendation: Address before merge

Fix these issues in Kilo Cloud

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/kilocode/session/prompt.ts 302 The new reminder assumes every client shows the post-plan_exit follow-up, but shouldAskPlanFollowup() only enables it for cli, vscode, and jetbrains, so ACP sessions lose the chance to keep refining.
Files Reviewed (28 files)
  • .changeset/plan-followup-refine.md - 0 issues
  • packages/kilo-i18n/src/ar.ts - 0 issues
  • packages/kilo-i18n/src/br.ts - 0 issues
  • packages/kilo-i18n/src/bs.ts - 0 issues
  • packages/kilo-i18n/src/da.ts - 0 issues
  • packages/kilo-i18n/src/de.ts - 0 issues
  • packages/kilo-i18n/src/en.ts - 0 issues
  • packages/kilo-i18n/src/es.ts - 0 issues
  • packages/kilo-i18n/src/fr.ts - 0 issues
  • packages/kilo-i18n/src/it.ts - 0 issues
  • packages/kilo-i18n/src/ja.ts - 0 issues
  • packages/kilo-i18n/src/ko.ts - 0 issues
  • packages/kilo-i18n/src/nl.ts - 0 issues
  • packages/kilo-i18n/src/no.ts - 0 issues
  • packages/kilo-i18n/src/pl.ts - 0 issues
  • packages/kilo-i18n/src/ru.ts - 0 issues
  • packages/kilo-i18n/src/th.ts - 0 issues
  • packages/kilo-i18n/src/tr.ts - 0 issues
  • packages/kilo-i18n/src/uk.ts - 0 issues
  • packages/kilo-i18n/src/zh.ts - 0 issues
  • packages/kilo-i18n/src/zht.ts - 0 issues
  • packages/kilo-telemetry/src/telemetry.ts - 0 issues
  • packages/kilo-vscode/tests/unit/plan-followup-locale-keys.test.ts - 0 issues
  • packages/opencode/src/kilocode/plan-followup.ts - 0 issues
  • packages/opencode/src/kilocode/session/native-plan-prompt.txt - 0 issues
  • packages/opencode/src/kilocode/session/prompt.ts - 1 issue
  • packages/opencode/test/kilocode/plan-exit-detection.test.ts - 0 issues
  • packages/opencode/test/kilocode/plan-followup.test.ts - 0 issues

Reviewed by claude-sonnet-5-20260630 · Input: 22 · Output: 4.3K · Cached: 450.5K

Review guidance: REVIEW.md from base branch main

@kirillk

kirillk commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the ACP review concern in febdc26.

The plan reminder now reuses the same client-support gate as shouldAskPlanFollowup(): cli, vscode, and jetbrains get the deterministic post-plan_exit follow-up wording, while unsupported clients such as ACP keep the previous in-chat "Finalize and save the plan" / "Continue refining" approval step.

Added a regression test covering both supported-client and ACP reminder text.

- If the user chooses "Finalize and save the plan", write the complete finalized Markdown plan to the chosen plan file, then call `plan_exit` as described above.
- If the user chooses "Continue refining", keep planning and do not write the final plan or call `plan_exit`.
- After `plan_exit`, rely on the client follow-up to ask whether the user wants to implement the saved plan in a new session.
- If the plan is implementation-ready, write the complete finalized Markdown plan to the chosen plan file, then call `plan_exit` as described above. Do not ask the user to choose between finalizing and refining in chat; the client follow-up after `plan_exit` asks whether to implement the saved plan or keep refining.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change. It is a great improvement. Just one comment on this part, I worry this part may cause a contradiction in regards to the branch created in code about supporting follow up vs not. We could potentially leave this part open and referencing the instructions that will be added conditionally?
e.g.

- If the plan is implementation-ready, write the complete finalized Markdown plan to the
  chosen plan file, then call `plan_exit` as described above.
- Call `plan_exit` only when the goal, constraints, ... are addressed or out of scope.
- Follow the latest Plan File reminder for whether to confirm with the user before
  finalizing, and for what happens after `plan_exit`.```

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, addressed in 82e58b2. The static native plan prompt no longer hardcodes the follow-up assumption — the finalize/refine and post-plan_exit behavior now defers to the dynamic "Plan File" reminder ("Follow the latest Plan File reminder for whether to confirm with the user before finalizing, and for what happens after plan_exit"), matching the existing "Follow the latest Plan File reminder for..." pattern in the same file.

This makes prompt.ts the single source of truth, so supported clients (cli/vscode/jetbrains) get the deterministic follow-up wording while ACP keeps the in-chat approval step, with no contradiction between the two reminders. Verified the dynamic reminder is always appended after the static prompt on the same user message, so the deferral target is always present.

@kirillk
kirillk enabled auto-merge July 1, 2026 13:08
@kirillk
kirillk merged commit 39d852d into main Jul 1, 2026
28 checks passed
@kirillk
kirillk deleted the keen-protocol branch July 1, 2026 13:26
t7tran pushed a commit to t7tran/kilocode that referenced this pull request Aug 14, 2026
fix(cli): make plan follow-up refinement deterministic
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants