Repository navigation
fix(sse): RTK must preserve cache_control-marked tool_result blocks (prompt-cache regression) - #4560
Merged
diegosouzapw merged 1 commit intoJun 21, 2026
Conversation
A tool_result block (or text sub-block) carrying cache_control is an explicit upstream prompt-cache breakpoint — the provider caches the prefix up to and including it. Since v3.8.32 RTK started compressing Anthropic tool_result blocks (processToolResultBlocks), rewriting the marked block's content and invalidating the cached prefix every turn → guaranteed cache miss (provider cache 'again broken'). Skip any cache_control-marked block/sub-block in processToolResultBlocks, preserving it byte-for-byte. Unmarked blocks still compress (no over-protection). Mirrors #3936: under caching, only ever preserve more of the prefix — never rewrite a declared breakpoint. TDD: rtk-cache-control-preserve.test.ts (red->green, 3 cases). Compression suite 777/777.
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This was referenced Jun 21, 2026
Merged
tkgo11
pushed a commit
to tkgo11/OmniRoute
that referenced
this pull request
Sep 23, 2026
…iegosouzapw#4560) Integrated into release/v3.8.33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Since v3.8.32 (PR #4468 added Anthropic
tool_resultsupport to RTK), enabling compression broke upstream prompt cache on caching providers — reported from the channels as "provider cache again broken in combo section" and "cache rate seems very low" right after upgrading.A
tool_resultcontent block (or text sub-block) carryingcache_controlis an explicit prompt-cache breakpoint: the provider caches the prefix up to and including that block.processToolResultBlocksrewrote the marked block's inner text, so the cached prefix no longer matched byte-for-byte → guaranteed cache miss on every turn.This re-broke the invariant #3936 established for #3890 (memory + system-prompt cache safety), which never covered tool_results.
Fix
processToolResultBlocksnow skips any block / text sub-block carryingcache_control, preserving it byte-for-byte. Blocks withoutcache_controlstill compress normally (no over-protection). Mirrors #3936's principle: under caching, only ever preserve more of the prefix — never rewrite a client-declared breakpoint.+21production lines, single file (open-sse/services/compression/engines/rtk/index.ts).Validation (TDD — Hard Rule #18)
tests/unit/compression/rtk-cache-control-preserve.test.ts— red→green, 3 cases:cache_control-marked tool_result (string content) preserved byte-for-bytecache_control-marked inner text sub-block (array content) preservedtypecheck:coreclean · lint 0 errors/0 warnings ·check:file-sizeOK ·check:any-budget:t11OKResearch
resolveCacheAwareConfig/preserveSystemPrompt, system-prompt only).processToolResultBlocksfirst shipped in tag v3.8.32 (absent in v3.8.31/.30).