fix(batches): account for Anthropic cache TTL and search fees

[mavericksea-ai]

TLDR

Problem this solves:

• Anthropic batches undercount one-hour cache writes
• Batch cost records omit paid web searches

How it solves it:

• Price one-hour writes using their discounted TTL rate
• Carry search fees separately into both batch logging paths

I maintain DriftproofHQ and found this during usage-to-cost reconciliation work. This fixes the reproduced first-party Anthropic batch cases without claiming a production incident or Sev1

User Flow

Before, a developer reconciling batch usage gets an understated total

1. Save the offline reproduction from [Bug]: Anthropic batch accounting undercounts 1h cache writes and omits web-search fees #45830 as repro.py
2. Run it against the unmodified checkout with bundled local prices
3. The combined example reports $0.28 despite $0.53 of priced usage

After, the same usage produces the full cost

1. Save the offline reproduction from [Bug]: Anthropic batch accounting undercounts 1h cache writes and omits web-search fees #45830 as repro.py
2. Run it against this checkout with bundled local prices
3. The combined example reports $0.53, including $0.10 in searches

Relevant issues

Fixes #45830

Related prior work includes #39626, which acknowledges dropped special batch costs, #43602 and #43585 on Vertex batch cache prices, and #36762 on Bedrock cache TTL translation

Changes

Batch cache-write pricing reads the reported one-hour token count and uses the standard one-hour rate with the batch discount. The existing rate resolver handles standard token thresholds and explicit zero prices. Missing one-hour prices retain the configured batch write rate, including when the selected tier omits that price or a higher threshold does not apply. Remaining cache-write tokens retain their existing batch rate

Anthropic batch rows use the existing provider search-price calculator. Aggregation retains the search count and a separate tool_cost, adds it to the total, and forwards it through both retrieval-driven logging and the background poller. Unset deployment search prices inherit the published rate, while explicit zero and custom search prices remain overrides. Models without search usage keep their existing usage shape

Pre-Submission checklist

• I have added meaningful tests
• Focused affected test suites pass locally
• My PR passes all required CI/CD checks
• My PR's scope is isolated to batch cost accounting
• Greptile confidence is at least 4/5 before maintainer review

Screenshots / Proof of Fix

The requester required offline testing with no model calls. These are synthetic provider-result fixtures, not a live paid API or proxy test. Prices come from the bundled map. Save the Python reproduction in #45830 as repro.py, use the same installed dependencies on both sides, and run LITELLM_LOCAL_MODEL_COST_MAP=True PYTHONPATH=. python repro.py

Before (38d0333)

1. Run the command above on the base checkout
2. Observe these expected/actual USD pairs

plain       0.03 / 0.03
5m          0.28 / 0.28
1h          0.43 / 0.28
searches    0.13 / 0.03
combined    0.53 / 0.28

After (e880377)

1. Run the same command on this commit
2. Observe these expected/actual USD pairs

plain       0.03 / 0.03
5m          0.28 / 0.28
1h          0.43 / 0.43
searches    0.13 / 0.13
combined    0.53 / 0.53

The actual standard logging payload was also captured through both batch entry paths. The combined case now records response_cost=0.53 and tool_usage_cost=0.10. Seven fixtures per entry path check token-only, five-minute, one-hour, mixed-TTL, one-search, ten-search and combined usage

The offline SQLite-adapter budget test from the issue now stores $0.43 against a $0.40 limit and the real budget function blocks, whereas the old $0.28 record allowed it. This remains a component test with synthetic usage, not evidence of real customer losses

Validation

make check passed, including formatting, Ruff, import checks and the type/test-quality delta gates. The complete batch and calculator files passed 481 tests, batch-specific logging coverage passed 18 tests, and shared pricing helper suites passed 245 tests. The change adds 22 regression cases, including missing tier prices, uncrossed thresholds and deployment search overrides. Fourteen fixture-to-logging runs and twelve SQLite-adapter budget cases also passed offline

The broader three-file run encounters test_logging_result_for_bridge_calls. It also fails in the same combined-suite order on the unchanged de6a071 baseline, while the isolated test passes on both baseline and patch. This limitation is recorded rather than treating the broad run as green

Type

Bug Fix

Caveats

Low

• No live provider or PostgreSQL deployment validation
• Search-fee changes cover first-party Anthropic batches
• Existing recorded spend is not retroactively repaired
• CI and automated reviews remain pending

Final Attestation

• Regression tests cover the reproduced usage and logging cases
• Live end-to-end validation completed

[CLAassistant]

All committers have signed the CLA.

[greptile-apps]

Confidence Score: 3/5

[High impact] Changes how batch costs are calculated for cache and search fees.

Fix custom-deployment search prices and hour-price fallback selection before merging.

Findings

1.  Custom prices erase search fees ▶
2.  Missing hour prices change charges ▶

Summary

This PR prices Anthropic batch one-hour cache writes separately and includes paid searches in batch totals.

• Batch cache writes use the one-hour rate for one-hour tokens.
• Anthropic batch totals include web-search fees as tool costs.

_{Reviews (1) · Last reviewed commit: "fix(batches): account for Anthropic cach..." · Reviewed by Greptile}

[greptile-apps]

Custom prices erase search fees

When an Anthropic deployment sets custom token prices but no search price, deployment_pricing_model_info returns search_context_cost_per_query=None. This merge overwrites the published search price with that None, so paid searches still cost zero through retrieval logging and the background poller.

Ignore unset deployment values when merging, while preserving explicit zero prices.

[greptile-apps]

Missing hour prices change charges

has_hour_price treats any selected tiered_pricing entry as an explicit one-hour price, even when it has none. It also counts hour-price keys for thresholds the request has not crossed. In either case, standard_rates[3] can be an ordinary cache-write fallback that incorrectly replaces the supplied batch rate.

For example, a selected tier with cache writes priced at 6 and an explicit batch write rate of 4, but no hour price, charges one-hour tokens at 3 instead of retaining 4. Check whether the selected rate actually declares an hour price before replacing cache_creation_rate.

[codspeed]

Merging this PR will not alter performance

⚠️ 1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• test_completion_streaming

✅ 31 untouched benchmarks

---

_{Comparing mavericksea-ai:fix/anthropic-batch-cost-components (e880377) with main (09558ba)}

[codecov]

Codecov Report

❌ Patch coverage is 93.75000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/batches/batch_utils.py	89.47%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mavericksea-ai]

@greptileai please review commit e880377. Both prior findings are addressed with regression tests for missing pricing fallbacks and explicit zero overrides