fix(relayconvert): Claude Messages 转 OpenAI 时不再向非 OpenRouter 上游透传 cache_control - #6691
fix(relayconvert): Claude Messages 转 OpenAI 时不再向非 OpenRouter 上游透传 cache_control#6691suqi8 wants to merge 1 commit into
Conversation
…eams Claude Messages -> OpenAI chat conversion forwarded the Anthropic-only `cache_control` marker to every upstream, not just OpenRouter. The system branch was already guarded, but the message-content branch was not, so a plain OpenAI-compatible upstream received a field it does not understand. Strict upstreams reject it outright with a 400, which is what QuantumNous#5982 reports. Permissive upstreams accept it and fail silently in a worse way: it destroys prompt caching. Clients such as Claude Code move the cache breakpoint forward as a conversation grows, so the marker lands on a different message index on each request. OpenAI's automatic caching keys on the longest common prefix, and the moved marker truncates that match at exactly the point it moved -- so a conversation that should hit cache on every turn keeps cold-starting instead. Also forward the client's `metadata.user_id` as `prompt_cache_key`, which this path never set. A stable cache key lets the upstream route a conversation to the same cache node. When the client sends no usable identifier the field is left unset rather than synthesised: a key that varies per request is worse than no key at all. Measured on a production deployment, comparing the same channel and model at the same input size, before and after this change: /v1/messages -> GPT cache-miss rate 74.5% -> 13.3% /v1/responses (native, unaffected) 14.4% The converted path now matches the native path, which is what it should have done all along. Regression tests cover both directions of the cache_control gate and the prompt_cache_key extraction, including malformed and non-string metadata. Introduced in QuantumNous#983. Refs QuantumNous#5982, QuantumNous#6538.
WalkthroughThe Claude-to-OpenAI converter now restricts ChangesClaude cache conversion
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR fixes Claude Messages → OpenAI Chat request conversion so Anthropic/OpenRouter-specific cache_control is no longer forwarded to non-OpenRouter upstreams, preventing strict upstream 400s and avoiding prompt-cache disruption. It also adds prompt_cache_key propagation from metadata.user_id to restore cache affinity for converted requests.
Changes:
- Introduce a single
keepCacheControlgate and apply it consistently to both system and message content conversion paths. - Populate
prompt_cache_keyfrom Claude requestmetadata.user_idwhen present and usable. - Add regression tests covering
cache_controlgating andprompt_cache_keyextraction/omission behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| relaykit/relayconvert/internal/claude_messages/to_oai_chat_req.go | Gates cache_control forwarding to OpenRouter+Anthropic models only, and forwards metadata.user_id into prompt_cache_key when available. |
| relaykit/relayconvert/claude_to_openai_cache_test.go | Adds tests ensuring cache_control is stripped/kept correctly and prompt_cache_key is extracted/omitted as intended. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
relaykit/relayconvert/claude_to_openai_cache_test.go (1)
59-74: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExpand the cache-control contract matrix.
The tests do not cover an OpenRouter request for a non-Anthropic model. They also do not cover structured system content.
Add regression cases where
OpenRouterDialectis true andUpstreamModelNameisopenai/gpt-4o, and where a system block containscache_control. Both cases must omitcache_control. This protects both terms ofkeepCacheControland the changed system branch.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@relaykit/relayconvert/claude_to_openai_cache_test.go` around lines 59 - 74, Expand the cache-control tests around ClaudeMessagesRequestToOpenAIChat with an OpenRouter non-Anthropic model (UpstreamModelName “openai/gpt-4o”) and a structured system-content block containing cache_control. Assert cache_control is omitted in both cases, while preserving the existing OpenRouter Anthropic test that expects it to be forwarded.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@relaykit/relayconvert/claude_to_openai_cache_test.go`:
- Around line 59-74: Expand the cache-control tests around
ClaudeMessagesRequestToOpenAIChat with an OpenRouter non-Anthropic model
(UpstreamModelName “openai/gpt-4o”) and a structured system-content block
containing cache_control. Assert cache_control is omitted in both cases, while
preserving the existing OpenRouter Anthropic test that expects it to be
forwarded.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 449d3095-5f62-4a64-9242-9ef30c65dd40
📒 Files selected for processing (2)
relaykit/relayconvert/claude_to_openai_cache_test.gorelaykit/relayconvert/internal/claude_messages/to_oai_chat_req.go
📝 变更描述 / Description
Claude Messages → OpenAI chat 的转换中，
cache_control这个 Anthropic/OpenRouter 专属字段被无条件透传给了所有上游。同一个函数里，system 部分有
isOpenRouterClaude判断保护（to_oai_chat_req.go:110），但消息内容部分漏了同样的判断（:156），导致普通 OpenAI 兼容上游也会收到这个它并不认识的字段。字段定义处的注释本身就标明了// OpenRouter Params。这个字段造成两种不同的故障：
严格上游直接返回 400，即 #5982 报告的现象。该 issue 评论中 @haowang02 的对照测试已独立验证，400 的成因是
cache_control本身，与 content 数组结构无关。宽松上游不报错，但会静默破坏 prompt 缓存，这一面此前没有被记录。Claude Code 这类客户端会随对话轮次增长把缓存断点向后移动，所以这个标记每次落在不同的消息下标上。OpenAI 的自动缓存按最长公共前缀匹配，标记一移动，前缀匹配就在移动点被截断——本该轮轮命中的对话变成反复冷启动。
本 PR 把该判断提升为函数级变量
keepCacheControl，两处统一使用，对 OpenRouter 的行为保持不变。同时补上了
prompt_cache_key：这条路径此前从未设置该字段（全仓库仅oai_responses/to_oai_chat_req.go:91的 Responses→Chat 路径传递过）。现从客户端的metadata.user_id转发，让上游能把同一会话路由到同一缓存节点。客户端未提供可用标识时保持不设置，而非自行生成——每次请求都变化的 key 比没有 key 更糟，会把本可命中的请求打散到不同节点。🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
cache_control两个问题，本 PR 只修复后者；数组结构部分留给维护者判断是否需要一并处理。prompt_cache_key，本 PR 顺带修复了 Chat 路径上的同一问题。✅ 提交前检查项 / Checklist
relaykit子模块与主模块./relay/...全量测试通过。📸 运行证明 / Proof of Work
生产环境实测，控制变量为同一渠道、同一模型
gpt-5.6-sol、输入规模相当：/v1/responses（原生，不受影响）/v1/messages（转换，修复前）/v1/messages（转换，修复后）修复后转换路径与原生路径持平。以下干扰因素已排除：
param_override为空，无 system_prompt 注入，强制缓存关闭测试结果：
主模块
./relay/...19 个包全部通过。新增 4 组回归测试，覆盖
cache_control门控的两个方向，以及prompt_cache_key的提取（含字段缺失、空值、非字符串、JSON 格式错误等边界）。已验证这些测试能真正捕获回归：临时还原旧逻辑后测试立即失败。Summary by CodeRabbit
Bug Fixes
Tests