feat(ui): show prompt caching requests and net savings

[tin-berri]

TLDR

Problem this solves:

• Prompt caching lacks request-level injection and savings visibility

How it solves it:

• Adds filterable requests with injection, usage, cost, and savings

User Flow

Before: an admin cannot inspect individual caching requests from Cost Optimization

1. Open /ui/cost-optimization/ and select Prompt Caching
2. See configuration and cache leakage, without a request table

After: an admin can inspect caching requests and open their logs

1. Open /ui/cost-optimization/ and select Prompt Caching
2. Select a date range and All caching, LiteLLM injected, or Cache hits
3. See recorded injection, cache reads/writes, actual cost, and estimated net savings; click a request to open its log

Relevant issues

• Show injected requests even when caching has no benefit
• Show provider cache hits separately from LiteLLM injection
• Preserve cache-write premiums and unavailable savings estimates
• Keep pagination stable while new requests arrive
• Bind cursor timestamps correctly through the Prisma query engine

Affected release

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible
• I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Local Postgres and proxy on port 16540, dashboard development server on 16541, automatic Anthropic caching enabled. Real Anthropic requests used the gateway account through a recording relay; no provider responses were mocked. Reproduction payloads are linked with the screenshots. The 55 clearly labeled pagination-qa-* rows are local fixtures used to exercise the 50-row dashboard limit

Before (f49fd22)

1. Open http://localhost:16540/ui/cost-optimization/, select Prompt Caching, and scroll below configuration. Only cache leakage is available

   

2. Request the caching table API:

   curl -sS -H 'Authorization: Bearer sk-cache-table-local' \
     'http://localhost:16540/cost_optimization/prompt_caching/requests?start_date=2026-09-19T00:00:00Z&end_date=2026-09-20T23:59:59Z'

   HTTP 404, {"detail":"Not Found"}

After (94b2fd8)

1. Open http://localhost:16541/cost-optimization/, select Prompt Caching, and scroll below configuration. Select Cache hits to see recorded injection alongside provider hits and estimated savings

   

2. Send the three payloads through the local proxy, then repeat the API request above:

   for payload in write read client; do
     curl -sS http://localhost:16540/v1/chat/completions \
       -H 'Authorization: Bearer sk-cache-table-local' \
       -H 'Content-Type: application/json' --data-binary @"$payload.json"
   done

   HTTP 200; recorded live results:

   Request	LiteLLM injection	Cache reads	Cache writes	Actual cost	Estimated net savings
   Initial write	Recorded	0	4,917	$0.0123365	-$0.0024585
   Subsequent read	Recorded	4,902	15	$0.0010619	$0.0088161
   Client caching	Not recorded	4,902	0	$0.0010624	$0.0088236

3. Select LiteLLM injected: the client-caching row disappears and the write-only row stays. Select Cache hits: both hit rows remain. Click the injected hit request: its log shows 4,902 cache reads and 15 writes

4. Select All caching, then click Next and Previous: page 2 loads 16 rows and page 1 restores 50. All 66 requests appear exactly once when following API cursors; injected and cache-hit filters also traverse completely. The SQL binds timestamps as text before conversion, fixing the page-2 Prisma binary-bind error

   

   Cursor regression and traversal evidence

make check and 145 focused tests pass. Pagination tests use real Prisma with one connection and cover null-to-cursor-to-null reuse, microsecond precision, timezone offsets, and concurrent inserts/deletions. Restoring the old SQL fails with the reported bind error. All 94 checks pass; Greptile is 5/5 and Bugbot/Veria report no issues on this commit

Type

New Feature

Caveats (if any)

Low

• Savings use logged usage and current configured pricing
• Older logs may lack recorded injection attribution

Final Attestation

• Tests cover filtering, attribution, pricing, pagination, authorization, and scope changes

---

Note

Medium Risk
New admin spend-log query endpoint and shared savings pricing path affect cost metrics visibility; scope is read-only with role checks but touches spend tracking SQL and pagination.

Overview
Adds request-level prompt caching visibility on Cost Optimization via a new admin API and dashboard table.

The proxy exposes GET /cost_optimization/prompt_caching/requests, which reads LiteLLM_SpendLogs for a date range with filters (all caching, LiteLLM injected, cache hits), cursor-based pagination, and per-row fields for injection attribution, cache read/write tokens, spend, and estimated net savings (shared with daily rollups through refactored prompt_caching_savings_for_request). Access is proxy-admin only; /cost_optimization/ is allowlisted for the UI backend.

On Prompt Caching, a shared date picker drives a new Prompt caching requests table (tabs, prev/next cursors, links to logs) alongside cache leakage; the leakage card no longer has its own date control.

^{Reviewed by Cursor Bugbot for commit 94b2fd8. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_prompt_caching_request_table (94b2fd8) with main (93e39d5)}

[greptile-apps]

Confidence Score: 5/5

The current PR appears safe to merge; no outstanding or newly introduced actionable issues remain.

Summary

Adds an admin-only prompt-caching request explorer to Cost Optimization.

• Introduces filtered, cursor-based spend-log queries for injected caching requests and provider cache hits.
• Calculates request-level net savings while preserving cache-write premiums and unavailable estimates.
• Adds the dashboard request table, log links, shared date-range behavior, pagination, and focused backend/UI regression coverage.

_{Reviews (4) · Last reviewed commit: "feat(ui): show prompt caching requests a..."}

[codecov]

Codecov Report

❌ Patch coverage is 98.07692% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...xy/management_endpoints/prompt_caching_requests.py	96.61%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[tin-berri]

@greptileai Please review the updated cursor pagination; live traffic and regression tests now cover inserts, deletions, and timestamp ties

[tin-berri]

bugbot run

Please review the current commit, including request attribution, pricing, cursor pagination, and date-scoped dashboard navigation

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai Please review the current commit, including explicit timestamp text binding and regression coverage through the real Prisma query engine

[tin-berri]

bugbot run

Please review the current commit for cursor parameter binding and pagination regressions after replacing the alternate test driver

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai Please refresh the current commit review after withdrawing the analytics finding; live cursor traversal and real Prisma regression tests pass

[ryan-crabbe-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 94b2fd8. Configure here.}