Preserve cache_control for Qwen/DeepSeek on OpenRouter + Anthropic adapter - #1
Merged
Merged
Conversation
…apter Three related fixes so cache_control survives end-to-end when routing Anthropic /v1/messages traffic to Qwen/DeepSeek upstreams via OpenRouter: 1. Add QWEN and DEEPSEEK to CacheControlSupportedModels so the OpenRouter chat handler stops stripping cache_control before the upstream call. 2. Widen is_anthropic_claude_model to recognize qwen/deepseek so the Anthropic->OpenAI adapter applies cache_control for those models. 3. Drop the x-anthropic-billing-header system block in the adapter path (mirrors _filter_billing_headers_from_system in the messages adapter). Its per-request hash otherwise invalidates the prefix cache every turn, pinning cache_read_input_tokens at 0. Adds unit tests for all three paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
When Claude Code is pointed at Qwen/DeepSeek through the LiteLLM proxy (Anthropic
/v1/messages-> OpenRouter),cache_controlwas being lost end-to-end, so every turn paid the full input rate (cache_read_input_tokensstayed at 0). These three fixes let the prefix cache actually land (measured ~0% -> ~95% hit rate on Alibaba/DeepSeek routes).Changes
openrouter/chat/transformation.py— addQWENandDEEPSEEKtoCacheControlSupportedModels. OpenRouter natively supportscache_controlcontent blocks for these upstreams (docs); without them the handler strips the marker before the upstream call.anthropic/.../adapters/transformation.py—is_anthropic_claude_model— recognizeqwen/deepseekso the Anthropic->OpenAI adapter appliescache_controlfor those models. (This method is also consulted for thinking translation: widening it means a raw Anthropicthinkingdict is passed through instead of being translated toreasoning_effortfor these models. In practice callers usedrop_params: trueand/or setreasoning_effortexplicitly, so this is benign, but it's the one behavioral tradeoff to be aware of.)anthropic/.../adapters/transformation.py—_add_system_message_to_messages— drop thex-anthropic-billing-header:system block. Claude Code injects this ahead of thecache_controlmarker with a per-request hash; left in, it invalidates the upstream prefix-cache key every turn. This mirrors_filter_billing_headers_from_systemin the sibling messages adapter, which already filters it.Tests
is_anthropic_claude_modelrecognizes qwen/deepseek (and still rejects gpt/gemini).cache_control.transform_requestpreservescache_controlfor qwen/deepseek models.All four touched files pass
black.