feat(ui): configure Anthropic automatic prompt caching from the Admin UI

[tin-berri]

Relevant issues

• The issue: enable_anthropic_prompt_caching and its ttl are litellm_settings globals, so the only way to turn automatic Anthropic prompt caching on was to hand-write config, which is the recipe the flag set out to replace. Support has been filing tickets to flip caching settings on customer instances because there is no equivalent surface in the UI.
• The fix: a dedicated Prompt Caching tab under Router Settings, with a labeled toggle for the flag and a dropdown for the ttl, so an admin turns caching on and picks the ttl from the UI. The existing update, persist and reset plumbing carries both into litellm_settings and onto the live litellm.<attr>, so caching starts on the next request without a restart. Each field carries a tab in the registry (surfaced as field_tab on ConfigList) so the General tab shows the ungrouped fields and the caching fields render on their own tab; field_options on ConfigList gives the ttl dropdown its allowed values

Stacked on #33573, which adds the flag itself. Merge that first; this PR only exposes it

Linear ticket

Resolves LIT-4478

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

prompt_caching.mp4

Live proxy on localhost:4000 against the real Anthropic API, backed by a real DB so the config endpoints are live. The flag is set nowhere: not in the config file, not in the environment. Everything below drives the exact endpoints the Prompt Caching tab calls

Both settings arrive from the config list, the ttl carrying its allowed values so the tab can render its dropdown

curl -s "http://localhost:4000/config/list?config_type=general_settings" -H "Authorization: Bearer sk-1234" \
  | jq -c '.[] | select(.field_name|test("anthropic_prompt_caching")) | {field_name, field_type, field_value, field_options, stored_in_db}'
{"field_name":"enable_anthropic_prompt_caching","field_type":"Boolean","field_value":false,"field_options":null,"stored_in_db":null}
{"field_name":"anthropic_prompt_caching_ttl","field_type":"Select","field_value":null,"field_options":["5m","1h"],"stored_in_db":null}

Toggling the switch on caches immediately, in the same proxy process, with no restart. A ~13.6k token prefix goes from full price to cached

# before, flag off as shipped
curl -s http://localhost:4000/v1/messages -H "x-api-key: sk-1234" -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" -d @msg.json | jq -c '.usage | {input_tokens, cache_creation_input_tokens}'
{"input_tokens":13669,"cache_creation_input_tokens":0}

# the switch, i.e. what the Update button posts
curl -s -X POST http://localhost:4000/config/field/update -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"field_name":"enable_anthropic_prompt_caching","field_value":true,"config_type":"general_settings"}'
{"message":"Field enable_anthropic_prompt_caching updated","status":"success"}

# after, same process, no restart
{"input_tokens":2,"cache_creation_input_tokens":13667}

Picking 1h in the ttl Select moves the write onto Anthropic's 1 hour cache rather than the 5 minute one, which is the knob long agentic sessions actually need

curl -s -X POST http://localhost:4000/config/field/update -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"field_name":"anthropic_prompt_caching_ttl","field_value":"1h","config_type":"general_settings"}'
{"cache_creation_input_tokens":13667,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":13667}}

An unsupported ttl is refused at the gateway instead of reaching Anthropic verbatim

curl -s -X POST http://localhost:4000/config/field/update -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"field_name":"anthropic_prompt_caching_ttl","field_value":"10m","config_type":"general_settings"}'
{"detail":{"error":"anthropic_prompt_caching_ttl must be one of: 5m, 1h, or empty"}}

Reset returns the flag to its real default, rather than leaving the boolean as None

curl -s -X POST http://localhost:4000/config/field/delete -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"field_name":"enable_anthropic_prompt_caching","config_type":"general_settings"}'
{"message":"Field enable_anthropic_prompt_caching reset","status":"success"}
{"field_value":false,"stored_in_db":null}

In the Admin UI these same endpoints are driven from a dedicated Prompt Caching tab under Router Settings (http://localhost:4000/ui/?page=router-settings): a toggle for the flag and a ttl dropdown that is disabled until the toggle is on. The click-through is in the QA runbook below.

(Earlier screenshots in this PR showed the first design, where these lived as rows on the General settings table with In DB / In Config / Not Set badges; that was replaced by the dedicated tab, so those images have been removed as stale.)

Type

🆕 New Feature

Changes

enable_anthropic_prompt_caching and its anthropic_prompt_caching_ttl are litellm_settings globals, so today the only way to turn automatic Anthropic prompt caching on is to hand-write config, which is the recipe this whole feature set out to replace. Support has been filing tickets to flip caching settings on customer instances because there was no equivalent surface in the UI

Both are registered in _GENERAL_SETTINGS_UI_LITELLM_FIELDS, tagged with a tab so they render on a dedicated Prompt Caching tab (a purpose-built toggle and dropdown, not the generic settings table) rather than mixed in with the other global limits. The existing update, persist and reset plumbing carries them into litellm_settings and onto the live litellm.<attr>, so caching starts on the next request, with no restart. ConfigList gains an optional field_tab that the frontend uses to route each field to its tab, and field_options that gives the ttl dropdown its allowed values

The flag is a Boolean and the table already renders that as a switch. The ttl is an enum, and the table only knew Integer, Boolean and Float, falling through to no editor at all for anything else. Rather than special casing this one field, ConfigList now carries an optional field_options and the table renders a Select for field_type == "Select", so any future enum setting gets an editor for free. Clearing it sends an empty value, which resolves back to the provider default

Three things the registry could not previously express, all of which this needed:

_validate_general_settings_ui_litellm_value was hardcoded to a float in (0, 1], the shape of the one field that existed. It now dispatches on the field's declared type, so a Boolean rejects "yes" and 1, and a Select rejects any value outside its options. The Float path is unchanged and its existing tests still pass

_reset_general_settings_ui_litellm_field set every field to None. For a boolean flag that is not a bool and reads as neither on nor off, so reset now restores each field's own declared default, False for a Boolean and None otherwise

The listing reported stored_in_db=False, rendered as "In Config", for any field whose value was not None. A boolean that defaults to False would therefore always claim an admin had configured it. It now compares against the field's default, which is equivalent to the old check for the existing Float field and correct for the new ones

Cross-worker propagation

These two settings are set as live litellm.<attr> values on the worker that handles the UI save, the same way budget_exceeded_throttle_percentage is. That sibling field is in LITELLM_SETTINGS_SAFE_DB_OVERRIDES, which is what makes a config reload apply the DB value to the live attribute on other workers; these two were missing from it, so a peer worker merged the DB value but stayed on its startup value. This PR adds both to that allowlist so they behave like the sibling, and adds test_general_settings_ui_fields_are_db_overridable, which asserts every UI-editable field is enrolled so the two lists cannot drift again (that drift is exactly what caused the bug).

One pre-existing gap remains, shared with budget_exceeded_throttle_percentage and not specific to caching: the safe-override reapply runs on get_config, not on the periodic worker poll, so allowlisted litellm_settings converge when a worker next reloads config rather than on a timer. Config-file and env-var configuration are unaffected (applied to every worker at boot). Tracked in LIT-4567.

QA runbook

1. Point a config at any Anthropic or Bedrock Claude model, leave enable_anthropic_prompt_caching out of the config and the environment entirely, and start the proxy with a DB
2. Open http://localhost:4000/ui/?page=router-settings and select the Prompt Caching tab. Confirm a toggle for automatic Anthropic prompt caching and a ttl dropdown offering only 5m and 1h (the dropdown is disabled until the toggle is on). Confirm these two settings no longer appear on the General tab
3. Send a large prompt (it must clear the provider's minimum cacheable prefix, up to 4k tokens on current Anthropic models, or nothing will cache) to /v1/messages and confirm cache_creation_input_tokens is 0
4. Turn the toggle on (the tab applies immediately; there is no separate Update button) and, without restarting, confirm cache_creation_input_tokens is now greater than 0; a second identical call reports cache_read_input_tokens
5. Set the ttl dropdown to 1h and confirm usage.cache_creation.ephemeral_1h_input_tokens is what moves. Clear it and confirm the write returns to ephemeral_5m_input_tokens
6. Turn the toggle off and confirm caching stops
7. Confirm the budget_exceeded_throttle_percentage row still accepts a value in (0, 1] and still rejects 0 and 1.5

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/58dc3285dc2b44bbbebffc4ecfc41362

---

Note

Medium Risk
Changes global litellm_settings and live litellm attributes that affect Anthropic/Bedrock caching on all requests; multi-worker behavior depends on the safe-override allowlist staying in sync with UI fields.

Overview
Adds Admin UI control for enable_anthropic_prompt_caching and anthropic_prompt_caching_ttl, wired through the same general-settings persist/reset flow so changes apply live on litellm.<attr> and in litellm_settings without a restart.

The proxy general-settings registry grows typed specs (Boolean, Float, Select) with optional tab and options; /config/list exposes field_tab and field_options, validation is type-aware, reset restores per-type defaults (e.g. False for booleans), and stored_in_db compares against those defaults. Both caching fields are added to LITELLM_SETTINGS_SAFE_DB_OVERRIDES so peer workers pick up UI saves on config reload, with a test that UI fields stay enrolled in that allowlist.

The dashboard gets a Prompt Caching tab under Router Settings (toggle + TTL select with immediate save), a shared SettingValueEditor including Select support, and caching fields are hidden from the General table via field_tab.

^{Reviewed by Cursor Bugbot for commit 47ba9e7. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR adds Admin UI controls for enable_anthropic_prompt_caching and anthropic_prompt_caching_ttl under a new Prompt Caching tab in Router Settings, wiring them through the existing general-settings persist/reset flow so changes apply live without a restart.

• Extends _GENERAL_SETTINGS_UI_LITELLM_FIELDS with typed specs (Boolean, Select) and optional tab/options; validation, reset, and stored_in_db all dispatch on declared type instead of assuming Float.
• Adds both fields to LITELLM_SETTINGS_SAFE_DB_OVERRIDES with a structural invariant test that keeps the two lists in sync, fixing cross-worker propagation.
• Frontend adds PromptCachingPanel (immediate-save toggle + TTL dropdown filtered from the General table) and extracts a shared SettingValueEditor that handles Integer, Boolean, Float, and Select, so future enum settings get an editor automatically.

Confidence Score: 5/5

Safe to merge — the change is additive and well-tested, with no modifications to existing request-path logic.

The core logic changes (validation dispatch, reset defaults, stored_in_db comparison) are all correct and covered by dedicated unit tests. Both new fields are enrolled in LITELLM_SETTINGS_SAFE_DB_OVERRIDES with a structural test that prevents the two registries from drifting. The UI properly narrows accessToken to non-null before reaching PromptCachingPanel, and the Prompt Caching tab correctly calls deleteConfigFieldSetting when the TTL is cleared.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/constants.py	Adds enable_anthropic_prompt_caching and anthropic_prompt_caching_ttl to LITELLM_SETTINGS_SAFE_DB_OVERRIDES with a clear comment explaining the invariant; no issues found.
litellm/proxy/_types.py	Adds field_options and field_tab optional fields to ConfigList; also reorders two noqa: E402 imports. Clean and non-breaking.
litellm/proxy/proxy_server.py	Extends the general-settings UI registry with typed specs (Float/Boolean/Select), fixes validation to dispatch on declared type, fixes reset to restore per-type defaults, and fixes stored_in_db to compare against per-type defaults. All logic is correct and well-tested.
tests/test_litellm/proxy/test_proxy_server.py	Adds comprehensive mock-only tests covering the config list, validation, persistence, propagation, and reset paths for both new caching fields; includes a structural invariant test that the UI-editable field set stays in sync with LITELLM_SETTINGS_SAFE_DB_OVERRIDES.
ui/litellm-dashboard/src/app/(dashboard)/router-settings/_components/general_settings.tsx	Adds PromptCachingPanel (immediate-save toggle + TTL dropdown), SettingValueEditor (shared typed editor), and hides caching fields from the General table. accessToken is correctly narrowed to non-null before PromptCachingPanel is rendered.
ui/litellm-dashboard/eslint-suppressions.json	Reduces no-nested-ternary suppression count from 3 to 1, reflecting the replacement of the ternary chain with SettingValueEditor.
ui/litellm-dashboard/src/lib/http/schema.d.ts	Adds field_options and field_tab optional fields to the TypeScript schema, matching the Pydantic model changes.

_{Reviews (2): Last reviewed commit: "fix(proxy): propagate the caching flag a..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit4478_anthropic_auto_cache_ui (47ba9e7) with litellm_internal_staging (40e914c)}

[tin-berri]

bugbot run

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 47ba9e7. Configure here.}

[tin-berri]

@greptileai rereview

[devin-ai-integration]

Proof of fix: real proxy, real Anthropic API, real $, and a cost comparison vs Claude Code's own caching

All of the below runs against a live proxy on localhost:4000 hitting the real Anthropic API on claude-sonnet-4-6, with the flag turned on only through the environment variable this stack adds (nothing in the config file):

LITELLM_ENABLE_ANTHROPIC_PROMPT_CACHING=true \
  python litellm/proxy/proxy_cli.py --config cache_test_config.yaml --port 4000

The flag caches when the client sends no cache_control

A ~14k token prefix with no client cache_control is written to cache on the first call and read back on an identical second call, with the breakpoints injected entirely by LiteLLM:

# call 1
curl -s http://localhost:4000/v1/messages -H "x-api-key: sk-1234" -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" -d @msg.json | jq -c '.usage | {input_tokens, cache_creation_input_tokens, cache_read_input_tokens}'
{"input_tokens":3,"cache_creation_input_tokens":14428,"cache_read_input_tokens":0}

# call 2, identical body
{"input_tokens":3,"cache_creation_input_tokens":0,"cache_read_input_tokens":14428}

LITELLM_ANTHROPIC_PROMPT_CACHING_TTL=1h moves the write onto the 1 hour cache (usage.cache_creation.ephemeral_1h_input_tokens), and a request that already carries its own cache_control makes LiteLLM stand down and pass the client breakpoints through unchanged, so there is no double caching

Cost comparison: LiteLLM auto caching vs Claude Code's internal caching

The same agentic coding task run both ways: implement six functions so a 15 case pytest suite passes. Claude Code finished it in 4 API calls with every test green. To compare the two caching strategies on identical inputs rather than on two different agent trajectories, I ran Claude Code against the proxy (it sends its own cache_control, so LiteLLM stands down and we measure Claude Code's caching), captured the exact request bodies, then replayed those same requests with the client cache_control stripped so LiteLLM's automatic injection does the caching instead. Both runs were isolated with a unique nonce so neither could read the other's warm cache, giving two true cold starts

Claude Code invocation (pointed at the proxy):

ANTHROPIC_BASE_URL=http://localhost:4001 ANTHROPIC_API_KEY=sk-1234 ANTHROPIC_MODEL=claude-sonnet-4-6 \
  claude -p "Implement all the functions in stringutils.py so that every test in test_stringutils.py passes..." \
  --permission-mode bypassPermissions --output-format json

Per request cache tokens are near identical between the two strategies:

req	LiteLLM write	LiteLLM read	Claude Code write	Claude Code read
1	29900	0	29869	0
2	1469	29900	1469	29869
3	1503	31369	1503	31338
4	115	32872	115	32841

Costed at the claude-sonnet-4-6 rates in model_prices_and_context_window.json (input $3/M, output $15/M, cache write $3.75/M, cache read $0.30/M); the Claude Code column matches Claude Code's own self reported total_cost_usd to the cent:

	LiteLLM auto caching	Claude Code caching	no caching
cache write tokens	32,987	32,956	0
cache read tokens	94,141	94,048	0
output tokens	1,733	1,724	~1,730
actual cost	$0.177957	$0.177677	$0.4074
savings vs no caching	56.3%	56.3%	baseline

On an identical trajectory the flag lands within 0.2% of Claude Code, a $0.0003 gap that is just the extra nonce tokens plus output nondeterminism, not a caching difference. So for any client that does not roll its own prompt caching, flipping this flag on delivers the same ~56% savings Claude Code gets for free, and it safely stands down for clients that already cache themselves

QA across more models and a more complex task

Repeated the same LiteLLM-auto-caching vs Claude-Code-caching methodology on a harder agentic task (build an arithmetic tokenizer + recursive-descent parser + evaluator against a 30 case pytest suite; Claude Code got all 30 green on every model) across four models. Caching only changes input-token pricing, so the numbers below are the input-side cost (the replay regenerates each final assistant turn, so output tokens carry regeneration noise that has nothing to do with caching and is excluded):

model	requests	LiteLLM auto save	Claude Code save	LiteLLM vs Claude Code (input cost)
claude-sonnet-5	6	68.9%	68.2%	-2.1%
claude-opus-4-8	4	59.2%	57.7%	-3.5%
claude-opus-4-7	6	69.5%	69.5%	+0.1%
claude-fable-5	4	59.6%	58.1%	-3.5%

The cache write and read token counts line up between the two strategies on every model (for example sonnet-5 writes 53,560 vs 53,529 and reads 238,161 vs 235,796), so the caching itself behaves the same. Where LiteLLM comes out slightly ahead it is because its auto injection also caches the trailing turn, which Claude Code leaves as fresh input on some turns. Savings track how many turns reuse the prefix (roughly 59% at 4 requests, roughly 69% at 6), as expected

So the flag holds up across sonnet-5, opus-4-8, opus-4-7 and fable-5 on a non-trivial task: it matches Claude Code's own caching within a few percent, occasionally a touch cheaper, and never worse