Repository navigation
fix(claude): hoist tool_result.content cache_control and enforce 4-block cap - #15647
Merged
diegosouzapw merged 1 commit intoOct 7, 2026
Conversation
…ock cap Claude 5.x rejects nested cache_control inside tool_result.content[] and caps total cache_control blocks at 4; OmniRoute translations from OpenAI role:tool messages triggered 400s on ~80% of opus-5-5 traffic in an affected deployment. - hoistToolResultCacheControl: move inner cache_control markers onto the tool_result block itself, strip inner duplicates - enforceCacheControlBlockLimit: when more than 4 blocks carry cache_control, keep the last 4 (nearest the conversation head) - both run in prepareClaudeRequest for every Claude-format target, independent of preserveCacheControl (shape requirements, not policy) Verified: 7/7 new tests; hoist-disable mutation -> 3 red, cap no-op -> 1 red; 42 neighbouring suites referencing claudeHelper/openai-to-claude 442/442 green; lint, typecheck:core, prettier clean.
patrykkopycinski
marked this pull request as ready for review
October 6, 2026 15:12
diegosouzapw
merged commit Oct 7, 2026
8ee4929
into
cheaper-inference:release/v3.8.52
75 of 86 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Claude rejects two request shapes that the OpenAI-to-Claude translator currently produces when a client puts
cache_controlon the content parts ofrole: "tool"messages:Observed in production on a self-hosted deployment: roughly 80% of
claude-opus-5-5requests from such a client returned 400 before this change.prepareClaudeRequestnow:cache_controlmarker found insidetool_result.content[]onto the enclosingtool_resultblock and strips the inner markers. If the block already has its own marker, that one is kept.cache_controlblocks at 4 by keeping the last 4 (the most recent breakpoints, which give the best cache hits). The cap applies only to targets that support prompt caching. Other Claude-shape providers (e.g.kimi-coding) are left untouched.String
tool_resultcontent is not modified.Opus 5.5 model specs and thinking/effort handling are out of scope; #14546 covers them.
Related Issues
Validation
claudeHelper/openai-to-claude442/442npm run lint,npm run typecheck:core,prettier --check: cleanrelease/v3.8.52(branched from its tip, single commit)Mutation check: with the hoist call removed, 3 of the 7 tests fail. With
enforceCacheControlBlockLimitturned into a no-op, the cap test fails. Restoring either turns them green again.Tests Added Or Updated
tests/unit/claude-cache-control-placement-400s.test.ts(new)Coverage Notes
open-sse/translator/helpers/claudeHelper.ts: the three new helpers are covered by the new suite. It covers hoist in both preserve and non-preserve modes, an existing block-level marker, the cap above and at the limit, the non-caching provider exemption, and string content.Reviewer Notes