feat: configurable prompt_caching.cache_ttl (5m default, 1h opt-in) — salvage #12659 - #15065
Merged
Conversation
- Load prompt_caching.cache_ttl in AIAgent (5m default, 1h opt-in) - Document DEFAULT_CONFIG and developer guide example - Add unit tests for default, 1h, and invalid TTL fallback Made-with: Cursor
Made-with: Cursor
This was referenced Apr 24, 2026
alt-glitch
added
type/feature
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Salvage of #12659 by @ericnicolaides onto current main. Chosen over parallel PRs #14812 (env-var only, violates '.env for secrets only') and #3082 (threads kwarg through gateway + session + state DB — unnecessarily heavy for a 2-value config).
Closes #14971.
What this PR does
Exposes Anthropic's 1h prompt-cache TTL tier as a config option. Default remains
5m— zero behavior change for existing users.Motivated by #14971 cost data: on a $246 Anthropic workload, 56.5% of spend was
input_cache_write_5m— the cache was rewriting more than reading because the 5m TTL kept expiring between turns. The 1h tier costs 2x on write (vs 1.25x for 5m) but amortizes across a full session. The 1h path has been supported insideapply_anthropic_cache_control()since it landed; this PR just exposes it.How
AIAgent.__init__readsprompt_caching.cache_ttlviahermes_cli.config.load_config(). Validates against{"5m", "1h"}; anything else falls back to"5m"without raising.DEFAULT_CONFIGgets aprompt_cachingsection so the merge resolves cleanly for upgraders. Same conflict resolution applied to match current main's_anthropic_prompt_cache_policy()helper refactor.Changes
cli-config.yaml.exampledocumentation blockagent@wildcat.local(Cursor agent commit email) → @ericnicolaidesValidation
_cache_ttlprompt_cachingsection)5m5m✓cache_ttl: "1h"1h1h✓cache_ttl: "30m"(invalid)5m(fallback)5m✓cache_ttl: "5m"(explicit)5m5m✓prompt_caching: {}(empty)5m5m✓tests/run_agent/test_run_agent.py -k cache_ttl— 3/3 pass (new tests)tests/agent/test_prompt_caching.py— 14/14 pass (regression guard)Co-authored-by: @ericnicolaides