Repository navigation
fix(caching): skip the cache past max_messages and keep tool_result text in semantic prompts #43878
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
f462fbe
fix(caching): keep tool calls and tool results in semantic cache prompts
kerry-berri 8b08e07
refactor(caching): keep semantic tool prompt helpers within lint budgets
kerry-berri 8bf4439
fix(caching): keep structured function_call_output text in semantic p…
kerry-berri 5165860
refactor(caching): split Responses text-field collection to stay with…
kerry-berri 19bc489
fix(caching): tag each tool result with the position of the call it a…
kerry-berri 6b4b375
fix(caching): encode tool result position and output together so tool…
kerry-berri 9379f65
test(caching): expect encoded tool result record in qdrant semantic p…
kerry-berri 1275062
test(caching): cover tool result arrangements, SDK clients, concurren…
kerry-berri 5aed3ab
feat(caching): embed every semantic cache prompt field except volatil…
kerry-berri 39efb5d
test(code-quality): allow the bounded semantic cache prompt walkers i…
kerry-berri 86c82b9
test(rust): expect structured JSON for unknown fields in redis and va…
kerry-berri 345bd97
Revert "test(rust): expect structured JSON for unknown fields in redi…
kerry-berri f8aad2f
Revert "test(code-quality): allow the bounded semantic cache prompt w…
kerry-berri ccca38f
Revert "feat(caching): embed every semantic cache prompt field except…
kerry-berri 9de92d7
Merge remote-tracking branch 'origin/main' into litellm_semantic_cach…
kerry-berri b9a7942
refactor(caching): rename get_str_from_messages_with_tools to get_sem…
kerry-berri b27722c
refactor(caching): split semantic cache prompt extraction by API format
kerry-berri 7c1390c
fix(caching): drop TypeIs guard and register Responses prompt walker …
kerry-berri 1c4a69b
fix(caching): pick the Responses text field without a Final inside a …
kerry-berri 4ac923b
refactor(caching): walk semantic cache prompts as plain dicts, dumpin…
kerry-berri ffd3a85
refactor(caching): write the semantic cache prompt builders as plain …
kerry-berri 394abff
fix(caching): skip the cache past max_messages and keep tool_result t…
kerry-berri 050e9c0
test(caching): drop formatting-only churn from the redis semantic cac…
kerry-berri 6231a68
ci(integration): drop the caching group wiring that main already carries
kerry-berri c8ae23c
refactor(caching): recurse into tool_result content in the semantic c…
kerry-berri 0dc9c91
test(integration): check the max_messages cap on the shared exact-cac…
kerry-berri f6e9be9
fix(caching): read list-form function_call_output text in semantic ca…
kerry-berri 0665296
refactor(caching): extract nested Responses input lookup to keep walk…
kerry-berri ba93ca5
Revert "refactor(caching): extract nested Responses input lookup to k…
kerry-berri 580a970
style(caching): suppress C901 on the Responses input walker instead o…
kerry-berri File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,6 +11,7 @@ | |
| "providers", | ||
| "streaming", | ||
| "messages_endpoint", | ||
| "caching", | ||
| "configuration", | ||
| "mcp", | ||
| "observability", | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.