chore(release): backport #30787, #30788, #31035, #31122, #31133 to stable/1.84.x and cut 1.84.10

[yuneng-berri]

Relevant issues

Backports five already-merged litellm_internal_staging changes onto stable/1.84.x and cuts 1.84.10. Three are the interrupted/agentic Anthropic stream cost-recovery family (#30787, #30788, #31035): a stream that breaks mid-flight, or an agentic tool-use stream that stream_chunk_builder cannot reassemble, was logged with undercounted or zero spend even though the provider billed every token; these recover realistic usage on the failure and pass-through paths and name the key in the virtual-key budget error. The other two are maintenance: a dependency relock that moves the runtime-facing packages off versions flagged by the upstream scanner (#31122) and a wolfi-base image digest refresh (#31133).

1.84.9 already shipped as a container image (ghcr.io/berriai/litellm:v1.84.9, built from the stable/1.84.x tip b860b2f64ff), so this PR bumps to 1.84.10 rather than re-cutting 1.84.9.

What is included

In staging merge order:

• fix(passthrough): recover output tokens for interrupted anthropic streams #30787 fix(passthrough): recover output tokens for interrupted anthropic streams
• fix(proxy): record partial spend on the failure row for interrupted streams #30788 fix(proxy): record partial spend on the failure row for interrupted streams
• fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams #31035 fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams
• fix(deps): bump osv-flagged dependencies to clear known CVEs #31122 fix(deps): bump osv-flagged dependencies to clear known CVEs
• fix(docker): bump wolfi-base digest to patch openssl CVE-2026-34182 #31133 fix(docker): bump wolfi-base digest to patch openssl CVE-2026-34182
• bump: version 1.84.9 -> 1.84.10
• chore: refresh uv.lock for 1.84.10

Every functional commit is a git cherry-pick -x of its staging squash; the bump and the lock refresh are the only generated commits.

Adaptation notes

stable/1.84.x predates several intermediate staging changes, so three picks were adapted on the line; each adaptation keeps the pick's claimed behavior and was content-verified against staging.

• fix(proxy): record partial spend on the failure row for interrupted streams #30788: on staging this change sits inside a failure-path block whose first_api_call_start_time lift came from an unrelated OTEL PR (fix(otel): cap metric attribute cardinality with include/exclude lists #30257) that is not on this line. Only fix(proxy): record partial spend on the failure row for interrupted streams #30788's own recovered-usage lift (combined_usage_object + response_cost) was taken; the OTEL lift was left out.
• fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams #31035: its usage-only fallback builds a server_tool_use dict and relies on Usage.__init__ coercing it to a ServerToolUse. That coercion is not on this line (its staging origin is a 7000-file rebase squash, not cleanly pickable), so the three-line coercion was folded into the pick, byte-identical to staging's Usage.__init__. Without it two of fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams #31035's own tests fail. The budget-error change was applied on the line's existing if spend >= valid_token.max_budget: guard; the staging context also carried a separate NaN-defense from a prior PR which was left out as out of scope.
• fix(deps): bump osv-flagged dependencies to clear known CVEs #31122: the line has no osv-scanner.toml or OSV workflow, so those parts are no-ops here and were dropped. The dependency bumps were applied to the line's pyproject (cryptography 48.0.1, aiohttp 3.14.1, vcrpy 8.2.1, langchain 1.3.9, langgraph 1.2.x, js-yaml 4.2.0, ws 8.21.0) and both lockfiles were regenerated on the line. Two looser pins were needed for the relock to resolve and match staging: mlflow to >=3.11.1,<4.0 (the exact ==3.11.1 pin capped cryptography below 47) and semantic-router to >=0.1.15,<1.0 (the pinned 0.1.12 is a yanked release). Reaching staging's starlette / python-multipart / pydantic-settings / pypdf would require a fastapi major bump and was left out for scope; nothing regressed.
• fix(docker): bump wolfi-base digest to patch openssl CVE-2026-34182 #31133: only Dockerfile, docker/Dockerfile.database and docker/Dockerfile.non_root exist on this line, so the digest was bumped in those; the line's older base digest and staging's both resolved to the same patched target. The backend / gateway / migrations Dockerfiles are not on this line.

Known noise on this line

stable/1.84.x carries five pre-existing targeted-test failures unrelated to these picks. They fail on the line before any backport and are not regressions:

• test_streaming_handler.py::test_gemini_legacy_vertex_stop_finish_reason_normalised
• test_streaming_handler.py::test_gemini_legacy_vertex_tool_calls_finish_reason_with_stop_enum
• test_proxy_utils.py::test_get_custom_url
• test_auth_checks.py::test_get_key_object_should_raise_if_reconnect_fails_on_db_connection_error
• test_auth_checks.py::test_get_key_object_should_reconnect_once_on_db_connection_error

Targeted suite before the picks: 431 passed, 5 failed. After all picks and the relock: 463 passed, 5 failed. The added 32 passing tests are the picks' own regression tests; the failing set is identical to the baseline, so the picks add zero new failures.

Screenshots / Proof of Fix

Live proxy on stable/1.84.x with the picks applied, against real provider APIs.

Budget error now names the key (#31035), same key exhausted before and after:

# before (stable/1.84.x baseline):
Budget has been exceeded! Current cost: 7e-05, Max budget: 1e-07

# after (this branch):
Budget has been exceeded! Key=bp-budget-canary-post (sk-...OK7g) Current cost: 7e-05, Max budget: 1e-07

Interrupted Anthropic pass-through stream reproduced (the #30787 path): a max_tokens: 4000 stream cut by client disconnect at ~2.5s carries only the message_start output_tokens placeholder (3) with no terminal message_delta; the proxy returns 200 and the silent re-tokenization recovery is covered by TestInterruptedStreamOutputTokenRecovery.

curl -sN --no-buffer --max-time 2.5 http://localhost:4010/anthropic/v1/messages \
  -H "anthropic-version: 2023-06-01" \
  -d '{"model":"claude-haiku-4-5","max_tokens":4000,"stream":true,"messages":[{"role":"user","content":"Write a 3000-word essay on the history of computing."}]}'
# -> message_start output_tokens placeholder = 3, no terminal message_delta, HTTP 200

Sanity (unchanged from baseline): /health/liveliness alive, a real gpt-4o completion succeeds, and a normal Anthropic pass-through stream logs its terminal message_delta usage.

Relocked environment imports cleanly: cryptography 48.0.1, aiohttp 3.14.1, litellm + proxy import OK.

Type

🐛 Bug Fix

Changes

Cost-recovery on interrupted and agentic Anthropic streams (#30787, #30788, #31035), a runtime dependency relock and a base-image digest refresh (#31122, #31133), and the 1.84.10 version bump with regenerated lockfiles

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 3 committers have signed the CLA.

✅ yuneng-berri
❌ yucheng-berri
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This backport cherry-picks five already-merged staging changes onto stable/1.84.x and cuts 1.84.10. The core fixes close a spend-tracking gap where streams interrupted by client disconnect (or that stream_chunk_builder could not reassemble) were logged with zero cost while Anthropic still billed every delivered token.

• Interrupted-stream cost recovery (#30787, #30788, #31035): A multi-layer fix covering the CustomStreamWrapper failure handlers (stash partial usage before the failure row is written), the AnthropicPassthroughLoggingHandler pass-through path (retokenize buffered output to correct the undercounted message_start placeholder, and fall back to usage-only recovery when stream_chunk_builder raises on agentic/tool-use streams), and the shared failure-path plumbing in proxy/utils.py, litellm_logging.py, spend_tracking_utils.py, and proxy_track_cost_callback.py that propagates the recovered usage through to the DB write. The budget-exceeded error is also updated to name the triggering key so operators don't have to reverse-map spend.
• Maintenance (#31122, #31133): Runtime dependencies with known CVEs are bumped (cryptography, aiohttp, vcrpy, langchain, js-yaml, ws) and the wolfi-base Docker image digest is refreshed to patch OpenSSL CVE-2026-34182; lockfiles are regenerated accordingly.

Confidence Score: 4/5

Safe to merge; all changes are targeted bug fixes with no observable regressions in the test suite and no modifications to auth or permission logic other than an informational improvement to the budget error message.

The cost-recovery chain threads through six files and coordinates state via combined_usage_object / response_cost keys written in the streaming handler and read in the failure callbacks. Each hand-off is tested with mocks. The retokenization path in _recover_interrupted_stream_output_tokens correctly guards against both empty output text and a re-tokenized count that is not higher than the existing count, so it cannot silently deflate a legitimate token count. The _stream_was_interrupted heuristic (tail-scan for message_delta) is sound for Anthropic's documented SSE ordering. The dependency bumps loosen two version pins (mlflow, semantic-router) relative to the baseline, which the PR describes and justifies. No new network calls appear in the mock test suite.

The largest single file change is anthropic_passthrough_logging_handler.py (+275 lines), which deserves the closest read; streaming_handler.py (+83 lines) is the second most complex. Both have dedicated test classes covering the new paths.

Important Files Changed

Filename	Overview
litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py	Adds three cooperative features: a tail-scan interrupted-stream detector (_stream_was_interrupted), a retokenizer that corrects the undercounted output_tokens placeholder on interrupted Anthropic streams, and a usage-only fallback (_build_usage_only_response_from_chunks) that recovers cost from raw SSE events when stream_chunk_builder fails on large agentic/tool-use streams. Also sets model_call_details["response_cost"] from the success path so the pass-through spend is recorded correctly.
litellm/litellm_core_utils/streaming_handler.py	Wraps the end-of-stream stream_chunk_builder call in a try/except in both sync and async iterators, recovering best-effort usage from raw chunks when assembly fails. Adds _record_partial_usage_for_failure called from the timeout/exception handlers to stash partial usage and cost on the logging object before the failure handler fires.
litellm/litellm_core_utils/litellm_logging.py	Failure-path response_cost initialization now skips zeroing when combined_usage_object is already set, preserving the recovered partial spend from an interrupted stream.
litellm/proxy/utils.py	In post_call_failure_hook, lifts combined_usage_object and response_cost from the logging object onto request_data before popping litellm_logging_obj, so downstream failure-path spend callbacks can record the real partial spend.
litellm/proxy/hooks/proxy_track_cost_callback.py	Failure hook now checks for combined_usage_object on request_data and passes through the recovered response_cost (floored at 0) to update_database instead of the hardcoded zero.
litellm/proxy/auth/auth_checks.py	Budget-exceeded error now names the key (key_alias + masked key_name) so operators can identify the exhausted key without reverse-mapping spend to a key.
litellm/proxy/spend_tracking/spend_tracking_utils.py	When normal usage extraction yields nothing (failed stream), falls back to combined_usage_object if it is a litellm.Usage instance so the failure row records the recovered partial token counts.
litellm/types/utils.py	Widens server_tool_use parameter in Usage.__init__ to accept dict and coerces it to ServerToolUse, enabling the usage-only fallback to pass a dict directly from the raw SSE event.
litellm/proxy/pass_through_endpoints/streaming_handler.py	Decodes accumulated bytes with errors="replace" so a stream cut mid-multibyte-sequence still produces decodable lines (including already-received usage events) rather than raising UnicodeDecodeError.
litellm/proxy/pass_through_endpoints/llm_provider_handlers/base_passthrough_logging_handler.py	Mirrors the model_call_details["response_cost"] assignment added to the Anthropic handler so the base pass-through success path also records spend in the correct dict key.
Dockerfile	Updates wolfi-base image digest to patch the OpenSSL CVE-2026-34182 in both build and runtime stages.
pyproject.toml	Bumps security-flagged runtime dependencies (cryptography 48.0.1, aiohttp 3.14.1, vcrpy 8.2.1, langchain 1.3.9) and loosens two pins (mlflow >=3.11.1, semantic-router >=0.1.15) to allow the lock to resolve; JS deps also bumped (js-yaml 4.2.0, ws 8.21.0).

_{Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.84.10" | Re-trigger Greptile}