chore(release): backport #33810, #33733 to rc/1.94.0 and bump litellm-proxy-extras to 0.4.79.post1

[yuneng-berri]

Relevant issues

Backports two staging features onto rc/1.94.0 and bumps litellm-proxy-extras to 0.4.79.post1 so the new migrations actually ship.

#33810 threads prompt-compression and prompt-caching savings through the daily spend aggregates: a normalizer sums compression-interception savings with Headroom guardrail tokens_saved, the value is written into the SpendLog metadata JSON under a typed compression_savings key, and compression_saved_tokens plus the two dollar-savings columns are threaded through BaseDailySpendTransaction, the queue aggregation, the daily upsert blocks, six new BigInt columns, and the daily-activity read path. It also adds the Cost Optimization dashboard page.

#33733 forwards bedrock_tags from config or optional params onto the Bedrock CreateModelInvocationJob request, validated through a Pydantic TypeAdapter, and adds bedrock_tags to the banned request-body params so the value stays admin-controlled.

The proxy-extras bump is the reason this is not a code-only backport. 0.4.79 is already published and pins the pre-migration package, and 0.4.80 is taken by the nightly on the 1.95.0 line, so 0.4.79.post1 is the only version that both carries the new migrations and stays out of that line's way. cz bump rewrote the subpackage version, the root pin, and the lock in one commit, matching how #33855 did the previous bump on this branch.

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

What is included

• feat(spend): track prompt compression saved tokens in daily spend aggregates #33810 feat(spend): track prompt compression saved tokens in daily spend aggregates, cherry-picked from 3f3295b3
• feat(bedrock): forward bedrock_tags to CreateModelInvocationJob for batch jobs #33733 feat(bedrock): forward bedrock_tags to CreateModelInvocationJob for batch jobs, cherry-picked from 3819ee5d with -m 1
• bump: version 0.4.79 -> 0.4.79.post1 covering litellm-proxy-extras/pyproject.toml, the root pyproject.toml pin, and uv.lock

Both picks are patch-id identical to their staging sources, so there are no adaptation notes. The root litellm version stays at 1.94.0; this branch keeps pyproject pinned across dev.N and rc.N, and the rc tag carries the release identity.

UI rebuild still outstanding

Three production .tsx files are in this branch and the Next.js bundle has NOT been rebuilt here:

• ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/CostOptimizationView.tsx
• ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/page.tsx
• ui/litellm-dashboard/src/components/leftnav.tsx

The proxy serves the prebuilt bundle from litellm/proxy/_experimental/out/, and --reload never runs next build, so without a rebuild the Cost Optimization page and its left-nav entry will not appear on a deployed image. The rebuild is being handled separately and must land on this branch before it ships.

Known noise on this line

The full tests/test_litellm suite on the rc/1.94.0 tip already fails 79 tests with 3 collection errors before any of these picks, largely live-API tests and duplicate-basename collisions. Judge CI as a delta against that, not in absolute terms.

Screenshots / Proof of Fix

All verification is complete. Every signal is judged as a delta against a baseline captured on the line tip at 6c6a07c before picking.

Targeted suite:

baseline (6c6a07c824, pre-pick):  738 passed, 1 skipped
post-pick (e1e47adeb7):           802 passed, 1 skipped

All 64 new tests from both PRs pass on this line, and no test that passed at baseline regressed.

Full tests/test_litellm suite, same delta treatment:

baseline   (6c6a07c824):  82 unique failures, 29994 passed
post-pick  (e1e47adeb7):  79 unique failures, 30065 passed

Four failures appear only in the post-pick run. Three of them pass when re-run in isolation, and the fourth fails identically on both trees, so the delta is clean. The line carries substantial pre-existing failure noise from live-API tests and duplicate-basename collection errors, which is why the counts are read as a set difference rather than in absolute terms.

Live proxy against a Postgres migrated in two stages, first to the current rc/1.94.0 tip and then to this branch, to confirm the new migrations apply on top of a database that already ran v1.94.0-rc.2:

$ uv run prisma migrate deploy --schema litellm-proxy-extras/litellm_proxy_extras/schema.prisma
Applying migration `20260717000000_add_compression_saved_tokens`
Applying migration `20260718000000_add_savings_spend`
All migrations have been successfully applied.

20260717000000_add_compression_saved_tokens sorts lexicographically behind 20260717000000_add_mcp_server_oauth_client_table, which is already applied on any deployment running rc.2. It applies without complaint, and all 18 columns land across the six daily spend tables.

A real completion, then the daily activity read path:

$ curl -s -X POST http://localhost:4007/v1/chat/completions -H "Authorization: Bearer $MK" \
    -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Reply with exactly: backport ok"}],"max_tokens":16}'
content: backport ok
usage: {'completion_tokens': 3, 'prompt_tokens': 14, 'total_tokens': 17}

$ curl -s -H "Authorization: Bearer $MK" "http://localhost:4007/user/daily/activity?start_date=2026-07-01&end_date=2026-07-22"
total_spend: 0.000135  total_tokens: 30  requests: 3
total_compression_saved_tokens=0  total_compression_savings_spend=0.0  total_prompt_caching_savings_spend=0.0

Real spend flows through the widened daily spend write path and the three new fields read back, so the added BaseDailySpendTransaction keys do not break the pre-existing writer.

The bedrock_tags path from #33733, with a control request to show it is not blanket-rejecting:

$ curl ... -d '{"model":"gpt-4o", ..., "bedrock_tags":[{"key":"team","value":"attacker"}]}'
{"error":{"message":"Rejected Request: bedrock_tags is not allowed in request body. ...","code":"401"}}

$ curl ... -d '{"model":"gpt-4o", ...}'          # same request without bedrock_tags
HTTP 200

A subagent gauntlet run over the two picks cleared identifier resolution and the BaseDailySpendTransaction question, and separately flagged that _saved_tokens_or_zero in compression_savings.py screens negatives but not NaN or Infinity, so a non-finite tokens_saved would raise inside int() and be swallowed by the broad handler in _batch_database_updates. That code is byte-identical to the staging source, and reading the upstream compression service confirms it builds the field as an integer subtraction of tokenizer counts, hardcodes zero on every error path, and guards the only division in the response, so it cannot emit a non-finite value. The two shapes it can produce, zero and negative, are both already handled. Tracking the missing guard as a separate hardening change on staging rather than diverging this branch from its source.

Type

🆕 New Feature

Changes

Two cherry-picks plus a proxy-extras version bump. No hand-written code; every commit is either a -x cherry-pick traceable to litellm_internal_staging or the cz bump output.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR adds cost-optimization reporting and Bedrock batch tags to the release branch. The main changes are:

• Tracks prompt-compression and prompt-caching savings in daily spend aggregates
• Adds database migrations and API fields for the new savings metrics
• Adds the Cost Optimization dashboard source and navigation entry
• Validates and forwards administrator-controlled Bedrock batch tags
• Bumps litellm-proxy-extras to 0.4.79.post1

Confidence Score: 5/5

No additional blocking issue was found in this follow-up.

• The new daily fields are carried through aggregation, persistence, and read responses.
• Bedrock tags are validated before forwarding and blocked from request-body overrides.
• No separate production failure remained after applying the required follow-up selection rules.

Important Files Changed

Filename	Overview
litellm/proxy/spend_tracking/savings.py	Adds per-request pricing for compression and prompt-cache savings.
litellm/proxy/db/db_spend_update_writer.py	Adds saved-token and savings-spend values to daily transactions and upserts.
litellm/proxy/db/db_transaction_queue/daily_spend_update_queue.py	Aggregates the three new savings fields across queued daily transactions.
litellm/llms/bedrock/batches/transformation.py	Validates configured Bedrock tags and forwards them to batch job creation.
litellm/proxy/auth/auth_utils.py	Prevents request bodies from overriding administrator-controlled Bedrock tags.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/CostOptimizationView.tsx	Adds charts and totals for compression and prompt-caching savings.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/page.tsx	Adds the new dashboard route and passes authenticated user context to the view.
litellm-proxy-extras/litellm_proxy_extras/migrations/20260717000000_add_compression_saved_tokens/migration.sql	Adds saved-token columns to all six daily spend tables.
litellm-proxy-extras/litellm_proxy_extras/migrations/20260718000000_add_savings_spend/migration.sql	Adds compression and prompt-caching savings columns to all six daily spend tables.

_{Reviews (2): Last reviewed commit: "bump: version 0.4.79 → 0.4.79.post1" | Re-trigger Greptile}

[greptile-apps]

Production Bundle Omits New Page

Proxy deployments serve the checked-in export under litellm/proxy/_experimental/out/, but this change only adds the source route and navigation entry. Images and packages built from this commit will keep serving the old dashboard, so users cannot open the new Cost Optimization page until the static export is rebuilt and included.

Rule Used: What: Ensure that any PR claiming to fix an issue ... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Model Aliases Lose Savings

When a spend log contains a deployment alias or custom model name that get_model_info cannot resolve, this branch silently prices both compression and prompt-cache savings at zero. Those requests still record saved tokens, but the new dashboard reports no dollar savings even when the deployment has a priced underlying model.

[yuneng-berri]

Moving this to draft pending a decision on one finding; not a problem with the picks themselves.

Everything the backport is responsible for checks out. Both picks are patch-id identical to their staging sources, the targeted suite goes 738 passed at baseline to 802 passed post-pick with zero new failures, the full tests/test_litellm delta is clean once the four apparent new failures are re-run in isolation (three pass, one fails identically on both trees), and a live proxy against a freshly migrated database writes real spend through the widened daily-spend path and reads the new fields back.

The finding is in litellm/proxy/spend_tracking/compression_savings.py, carried in verbatim from #33810. _saved_tokens_or_zero screens bools, non-numerics, and negatives, but value < 0 is False for NaN and +Infinity, so both reach int(value) and raise. The call now sits on the hot path of the pre-existing daily-user-spend write, and _batch_database_updates wraps that call in a broad except Exception that logs at debug level, so the raise silently drops the entire daily spend row for that request rather than just the compression fields. tokens_saved is copied straight out of the external compression service's response body, so a non-finite value from that service is a real path, not just a theoretical one.

This is byte-identical to staging, so it is a latent defect in the feature rather than something the backport introduced, and it is present on staging today. It is still a behavior change against the current rc/1.94.0, where no such call existed and the write would have succeeded.

Holding here rather than patching it inline, since a fix written directly on the rc line would be code that never went through staging review. The staging-first sequence would be to guard the coercion on staging, then bring that fix along with these picks.

[yuneng-berri]

Taking this out of draft; the finding above resolved on inspection of the upstream service.

I read the compression service's /v1/compress handler at headroomlabs-ai/headroom@5a0a5a7. It builds the field as result.tokens_before - result.tokens_after, where both operands originate from tokenizer counting functions returning int, so the result is an integer subtraction and cannot be non-finite. Every error and early-return path hardcodes "tokens_saved": 0, and compression_ratio, the only division in the response, is explicitly guarded against a zero denominator. The Rust crates do not serve that route and use integral token types regardless.

The two shapes the service can actually produce are zero and negative; the response is unclamped where the service's own telemetry applies max(0, ...), and its test suite covers compressed-exceeding-original. Both are already handled by the value < 0 branch.

So the missing non-finite guard is input validation against a misbehaving dependency rather than a defect on a live path. It is byte-identical to the staging source, so it is being tracked as a separate hardening change on staging instead of diverging this branch from what it was picked from.

The body now carries the full verification evidence: targeted suite 738 to 802 passed with no regressions, a clean full-suite set delta, the two-stage migration replay against a database already at rc.2, and live proxy output for both the spend write path and the bedrock_tags rejection.

Still outstanding and not part of this branch: the Next.js bundle rebuild for the three production .tsx files, which is being handled separately and needs to land before this ships.