Skip to content

feat(cli): set prompt cache breakpoints on stable prefix and user query before environment details - #13022

Merged
chrarnoldus merged 10 commits into
mainfrom
feat/openai-explicit-prompt-cache-breakpoints
Aug 10, 2026
Merged

feat(cli): set prompt cache breakpoints on stable prefix and user query before environment details#13022
chrarnoldus merged 10 commits into
mainfrom
feat/openai-explicit-prompt-cache-breakpoints

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Explicit prompt cache breakpoints for GPT-5.6+: Added support for promptCacheBreakpoint: { mode: 'explicit' } on stable prompt prefixes (system prompt / developer instructions) and user queries for OpenAI, Azure, and Kilo Gateway models on GPT-5.6+ in @kilocode/cli (ProviderTransform.message).
  • Improved user message cache breakpoint placement for all models with explicit caching: For all providers using content-level cache options (OpenAI, OpenRouter, Alibaba, etc.), cache breakpoints are now positioned on the last stable user prompt content part immediately preceding any trailing <environment_details> or synthetic context blocks. This prevents subsequent turns (which strip or regenerate ephemeral timestamps and open tabs) from invalidating the cached conversation history prefix across turns.
  • Model gating: Kept pre-5.6 OpenAI models on automatic implicit caching without emitting inline breakpoint markers.
  • Provider compatibility: Retained existing implicit mode behavior and existing provider breakpoints (Anthropic, Bedrock, etc.).

@chrarnoldus chrarnoldus self-assigned this Aug 8, 2026
Comment thread packages/llm/src/cache-policy.ts Outdated
Comment thread packages/llm/src/protocols/openai-responses.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/provider/transform.ts 496 For kilo-gateway models the explicit breakpoint never leaves the client — the SDK reads providerOptions.openrouter, not openai/azure

SUGGESTION

File Line Issue
.changeset/openai-explicit-prompt-cache-breakpoints.md 2 Feature shipped as patch — repo convention is minor for new features
Files Reviewed (3 files)
  • packages/opencode/src/provider/transform.ts - 1 issue
  • packages/opencode/test/provider/transform.test.ts - 0 issues
  • .changeset/openai-explicit-prompt-cache-breakpoints.md - 1 issue

Fix these issues in Kilo Cloud

Previous Review Summaries (4 snapshots, latest commit d3548d1)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit d3548d1)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/provider/transform.ts 489 For kilo-gateway models the explicit breakpoint never leaves the client — the SDK reads providerOptions.openrouter, not openai/azure

SUGGESTION

File Line Issue
.changeset/openai-explicit-prompt-cache-breakpoints.md 2 Feature shipped as patch — repo convention is minor for new features
Files Reviewed (2 files)
  • packages/opencode/src/provider/transform.ts - 1 issue
  • .changeset/openai-explicit-prompt-cache-breakpoints.md - 1 issue

Fix these issues in Kilo Cloud

Previous review (commit 7c7c11e)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 2
Issue Details (click to expand)

WARNING

File Line Issue
packages/opencode/src/provider/transform.ts 491 For kilo-gateway models the explicit breakpoint never leaves the client — the SDK reads providerOptions.openrouter, not openai/azure

SUGGESTION

File Line Issue
packages/opencode/src/provider/transform.ts 391 part.synthetic can never be true at this layer — and the as any[] cast is what hides it
.changeset/openai-explicit-prompt-cache-breakpoints.md 2 Feature shipped as patch — repo convention is minor for new features
Files Reviewed (3 files)
  • packages/opencode/src/provider/transform.ts - 2 issues
  • .changeset/openai-explicit-prompt-cache-breakpoints.md - 1 issue
  • packages/opencode/test/provider/transform.test.ts - no issues

Fix these issues in Kilo Cloud

Previous review (commit 435c029)

Status: No Issues Found | Recommendation: Merge

Both previous findings were addressed in the latest commits: openai-compatible-chat was removed from the inline-hint policy set, and the duplicated version gate was extracted into a shared supportsPromptCacheBreakpoint helper reused by all three call sites.

Files Reviewed (6 files)
  • packages/llm/src/cache-policy.ts
  • packages/llm/src/index.ts
  • packages/llm/src/protocols/openai-chat.ts
  • packages/llm/src/protocols/openai-responses.ts
  • packages/llm/src/protocols/utils/openai-options.ts
  • packages/opencode/src/provider/transform.ts

Previous review (commit 9dbf276)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
packages/llm/src/cache-policy.ts 46 openai-compatible-chat now receives inline cache hints, but the emission gate keys only on the model ID — compatible third-party endpoints with gpt-5.6+-named models may get prompt_cache_breakpoint fields and could 400 on the unknown field

SUGGESTION

File Line Issue
packages/llm/src/protocols/openai-responses.ts 33 supportsBreakpoint version gate duplicated verbatim in openai-chat.ts, openai-responses.ts, and transform.ts — consider a shared helper in the llm package
Files Reviewed (7 files)
  • .changeset/openai-explicit-prompt-cache-breakpoints.md
  • packages/llm/src/cache-policy.ts - 1 issue
  • packages/llm/src/protocols/openai-chat.ts
  • packages/llm/src/protocols/openai-responses.ts - 1 issue
  • packages/llm/test/cache-policy.test.ts
  • packages/opencode/src/provider/transform.ts
  • packages/opencode/test/provider/transform.test.ts

Fix these issues in Kilo Cloud


Reviewed by kimi-k3 · Input: 93.7K · Output: 12.5K · Cached: 407.8K

Review guidance: REVIEW.md from base branch main

chrarnoldus and others added 3 commits August 8, 2026 17:39
…ict inline hints to native openai routes

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
…etails

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
…el AI SDK path

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
@chrarnoldus chrarnoldus changed the title feat(llm): set explicit prompt cache breakpoints on stable prefix for GPT-5.6+ feat(cli): set explicit prompt cache breakpoints on stable prefix for GPT-5.6+ in AI SDK transform Aug 8, 2026
chrarnoldus and others added 3 commits August 8, 2026 18:25
…Kilo Gateway

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
…rkers in transform.ts

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
Comment thread packages/opencode/src/provider/transform.ts
Comment thread packages/opencode/src/provider/transform.ts Outdated
Comment thread .changeset/openai-explicit-prompt-cache-breakpoints.md
chrarnoldus and others added 2 commits August 8, 2026 18:52
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
…hing

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
@chrarnoldus chrarnoldus changed the title feat(cli): set explicit prompt cache breakpoints on stable prefix for GPT-5.6+ in AI SDK transform feat(cli): set prompt cache breakpoints on stable prefix and user query before environment details Aug 8, 2026
@chrarnoldus
chrarnoldus merged commit a4e5d34 into main Aug 10, 2026
30 checks passed
@chrarnoldus
chrarnoldus deleted the feat/openai-explicit-prompt-cache-breakpoints branch August 10, 2026 08:54
@passanger68

Copy link
Copy Markdown
2026-08-10_14h22_33 I', afraid it needs fix. BTW - Azure is FIXED.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants