Skip to content

fix(ai): advance Anthropic cache marker across tool results - #1927

Merged
sethkarten merged 4 commits into
mainfrom
fix/anthropic-tool-cache
Sep 1, 2026
Merged

sethkarten merged 4 commits into
mainfrom
fix/anthropic-tool-cache

Conversation

@sethkarten

@sethkarten sethkarten commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • include tool-result messages when placing Anthropic-style rolling cache markers
  • allow tool message text content to carry cache_control
  • add a regression test for sessions with tool-call-only assistant turns

Linear: ENG-5823

Why

Prime Agent's Python kernel loop often emits assistant messages with content: null after the first turn, while each kernel result is sent as a tool message. The cache-marker scan skipped tool messages, so it walked back to an earlier assistant text block and left the rolling breakpoint frozen there. Every later tool call and result was then resent as uncached input.

Test plan

  • npm test -w packages/ai
  • npx biome check packages/ai/src/providers/openai-completions.ts packages/ai/test/openai-completions-cache-control-format.test.ts
  • npx tsgo --noEmit

Note

Low Risk
Narrow change to where cache markers are attached on Anthropic-format completions; improves caching behavior without touching auth or data handling.

Overview
Fixes Anthropic-compatible rolling prompt cache placement so the ephemeral cache_control marker can land on the latest tool message, not only user or assistant turns.

In openai-completions.ts, the backward scan in addCacheControlToLastConversationMessage and the helpers addCacheControlToMessage / addCacheControlToTextContent now treat role: "tool" like other conversational messages. That matters when assistant turns are tool-call-only (e.g. empty text) and each kernel step ends with a tool result—the marker previously stuck on an earlier assistant block, so later tool traffic was treated as uncached input.

Adds a regression test that asserts the last outbound message is a tool payload with cache_control on its text part, plus a short changelog note.

Reviewed by Cursor Bugbot for commit eb64556. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Fix Anthropic cache marker to advance across tool result messages

Previously the rolling cache_control marker could only land on user or assistant messages, so it never advanced past a tool result. Expands addCacheControlToLastConversationMessage, addCacheControlToMessage, and addCacheControlToTextContent in openai-completions.ts to also consider role: 'tool' messages when placing the ephemeral cache marker. Adds a test confirming the marker lands on the final tool message in a conversation ending with a tool result.

Changes since #1927 opened

  • Updated model identifier in test case [eb64556]

Macroscope summarized be91ea9.

@sethkarten
sethkarten requested a review from samsja August 31, 2026 23:46
samsja
samsja previously approved these changes Aug 31, 2026
@sethkarten
sethkarten merged commit 9f71270 into main Sep 1, 2026
23 checks passed
@sethkarten
sethkarten deleted the fix/anthropic-tool-cache branch September 1, 2026 02:30
olety pushed a commit to oneiron-dev/prime-agent that referenced this pull request Sep 1, 2026
…ellect-ai#1927)

* fix(ai): cache Anthropic tool results

* docs(ai): add cache fix changelog fragment

* test(ai): trim Anthropic cache regression

* test(ai): use Haiku cache fixture
PR9000 pushed a commit to PR9000/prime-agent that referenced this pull request Sep 1, 2026
…ellect-ai#1927)

* fix(ai): cache Anthropic tool results

* docs(ai): add cache fix changelog fragment

* test(ai): trim Anthropic cache regression

* test(ai): use Haiku cache fixture
ketema added a commit to ketema/prime-agent that referenced this pull request Sep 1, 2026
- Direct session transport between TUI and worker (ENG-5817, PrimeIntellect-ai#1926)
- Event-driven supervisor agent roster with push subscriptions (PrimeIntellect-ai#1897, PrimeIntellect-ai#1900, PrimeIntellect-ai#1895)
- Hardened daemon startup, recovery ownership, and worker launch diagnostics (PrimeIntellect-ai#1929, PrimeIntellect-ai#1918)
- Python REPL runtime single-dump snapshots and bash preview tool (PrimeIntellect-ai#1945, PrimeIntellect-ai#1911)
- Non-blocking RLM subagent deletion and snapshot update suppression (PrimeIntellect-ai#1954, PrimeIntellect-ai#1944)
- Saved catalog loading on Agents View open (PrimeIntellect-ai#1960)
- Advanced Anthropic prompt caching marker across tool results (PrimeIntellect-ai#1927)
- TUI process replacement on update and empty draft eviction (PrimeIntellect-ai#1631, PrimeIntellect-ai#1946, PrimeIntellect-ai#1920)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants