fix(chatgpt): forward prompt cache key - #37630
Draft
steveonjava wants to merge 3 commits into
Draft
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
I removed context_management from the ChatGPT Responses allowlist after\nauthenticated GPT-5.6 probes did not establish normal endpoint support.\nprompt_cache_key remains forwarded, and the regression keeps unsupported\nChatGPT request fields filtered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
prompt_cache_keyHow it solves it:
prompt_cache_keythrough the ChatGPT provider allowlistcontext_managementUser Flow
Before: a developer sends a ChatGPT Responses request with a cache key, but the backend never receives it
https://litellm-domain/v1/responseswith modelchatgpt/gpt-5.6-soland"prompt_cache_key": "session_123"prompt_cache_keyAfter: the same request forwards the caller's cache key
https://litellm-domain/v1/responseswith"prompt_cache_key": "session_123"prompt_cache_keyunchangedRelevant issues
context_management; live subscription proof was unavailableLinear ticket
Pre-Submission checklist
Screenshots / Proof of Fix
Before (6d47468)
uv run pytest tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py::TestChatGPTResponsesAPITransformation::test_chatgpt_drops_unsupported_responses_params -q -o 'addopts='after assertingprompt_cache_keyis preserved.KeyError: 'prompt_cache_key'.After (025438c)
uv run pytest tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -q -o 'addopts='.uv run pytest tests/test_litellm/llms/openai/responses/test_openai_responses_transformation.py -q -o 'addopts='.Type
🐛 Bug Fix
Caveats (if any)
context_managementfiltered pending authenticated backend acceptance proof/responses/compactpath remains the likely compaction integrationFinal Attestation