chore(release): backport #27319, #27756, #27921, #28395, #29983, #29984, #29986, #30160, #30327 to stable/1.84.x and cut 1.84.8

[yuneng-berri]

Relevant issues

Backport of seven staging fixes plus one prerequisite onto stable/1.84.x, cutting 1.84.8. Covers streaming Anthropic reasoning token usage, grace-period key rotation, native Azure container id routing, cached-plan recovery, DB-infra auth error semantics, and Anthropic pass-through cost resolution.

Grace-period rotation needed two changes: #27756 fixes the deprecated-key lookup itself, and #30327 (found while verifying this backport, fixed on staging first) completes the end-to-end flow. Both are included, so 1.84.8 carries the complete rotation fix.

What is included

Picks, in staging merge order, each with a (cherry picked from commit ...) footer:

• Fix Anthropic streaming reasoning token usage #27319 Fix Anthropic streaming reasoning token usage (verbatim)
• Litellm key rotation bug #27756 grace-period key rotation lookup fix (verbatim)
• fix(router): use forwarded model_id for native Azure container IDs #27921 fix(router): use forwarded model_id for native Azure container IDs (adapted)
• fix(proxy): recover from cached-plan errors by reconnecting the Prisma client #29983 fix(proxy): recover from cached-plan errors by reconnecting the Prisma client (adapted)
• fix(proxy): expose Prisma idle/connect timeout + extra DB URL params #28395 fix(proxy): expose Prisma idle/connect timeout + extra DB URL params (adapted; prerequisite for feat(proxy): add option to disable server-side prepared statements for DB lookups #29984, shipped in v1.87.0)
• feat(proxy): add option to disable server-side prepared statements for DB lookups #29984 feat(proxy): add option to disable server-side prepared statements for DB lookups (adapted)
• fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures #29986 fix(proxy): return 5xx on DB infra errors during auth (adapted)
• fix(anthropic_passthrough): resolve costing model from message_start chunk, litellm_params and model_group instead of 'unknown' #30160 fix(passthrough): resolve costing model when body model is unknown (adapted)
• targeted subset of feat: litellm oss 110626 #30202: the anthropic passthrough SSE hardening hunks ([DONE] sentinels, non-JSON frames) and their four tests; the rest of that staging batch is excluded
• fix(proxy): grace-period key rotation 401s; return deprecated-key lookup result directly #30327 fix(proxy): return deprecated-key lookup result directly in get_data combined view (adapted; completes grace-period rotation)

Plus two test-only adaptation commits, the version bump to 1.84.8, and the uv.lock refresh.

Adaptation notes

Every production source file in every pick is patch-id identical to its staging counterpart, with one verified exception class:

• fix(router): use forwarded model_id for native Azure container IDs #27921: dropped the generated litellm/proxy/_experimental/out/...js build artifact
• fix(proxy): recover from cached-plan errors by reconnecting the Prisma client #29983: its test additions live in tests/test_litellm/proxy/utils/prisma_and_spend/, a directory this line does not have; the PR's six test functions plus the fixture subset they need were grafted verbatim from the staging files
• fix(proxy): expose Prisma idle/connect timeout + extra DB URL params #28395: two context-drift regions in proxy_cli.py resolved to the staging post-image; the end state of both regions is byte-identical to staging after fix(proxy): expose Prisma idle/connect timeout + extra DB URL params #28395
• feat(proxy): add option to disable server-side prepared statements for DB lookups #29984: dropped ui/litellm-dashboard/src/lib/http/schema.d.ts, which does not exist on this line
• fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures #29986 and fix(anthropic_passthrough): resolve costing model from message_start chunk, litellm_params and model_group instead of 'unknown' #30160: staging-only neighbor tests carried by conflict regions were dropped; only each PR's own test functions were taken
• fix(proxy): grace-period key rotation 401s; return deprecated-key lookup result directly #30327: utils.py diff patch-id identical; its regression test grafted into this branch's subset copy of the prisma_and_spend test file, dropping staging-only neighbors the conflict region carried
• test-only follow-up commits restore upstream imports (AsyncMock, fastapi.status) and remove patch() entries that target seed_request_identity, a helper that does not exist on this line

Known noise on this line

Two pre-existing failures on the stable/1.84.x tip before any pick, unchanged after the picks: tests/test_litellm/proxy/test_proxy_utils.py::test_get_custom_url and tests/test_litellm/containers/test_azure_container_transformation.py::TestAzureContainerConfig::test_validate_environment_uses_azure_env_var (reads a real env var over the monkeypatched one in this environment).

Screenshots / Proof of Fix

Targeted test delta on this line: baseline 419 passed / 2 failed; after picks 520 passed / 2 failed; zero new failures, both failures are the pre-existing noise above.

Live proxy (this branch, port 4001), streaming Anthropic with reasoning_effort and include_usage:

# before picks
"completion_tokens_details":{"reasoning_tokens":0,"text_tokens":168}
# after picks, same request
"completion_tokens_details":{"reasoning_tokens":93,"text_tokens":84}

Anthropic pass-through on the generic route still costs correctly after the picks (no regression on the working path):

curl -s -X POST "http://localhost:4001/anthropic/v1/messages" -H "Authorization: Bearer <key>" \
  -H "anthropic-version: 2023-06-01" -d '{"model":"claude-haiku-4-5-20251001","max_tokens":64,"stream":true,...}'
# /key/info after spend flush
spend: 6e-05

The unknown-model costing path that produced $0 spend logs needs a router-passthrough deployment shape that does not register on this line's dev setup; behavior there is pinned by the six tests #30160 brings, which pass on this branch.

Grace-period rotation, end-to-end on a live proxy running this branch (key generated, regenerated with grace_period: "1h", then the OLD key used):

# before picks: every old-key call returned 401 (token_not_found_in_db)
# after picks (#27756 + #30327):
-- OLD key, call 1: HTTP 200
-- OLD key, call 2 (cache hit): HTTP 200
-- OLD key, call 3: HTTP 200
-- OLD key, real completion: "OK"
-- NEW key still works: HTTP 200

An adversarial multi-agent audit of the pick set (diff fidelity vs staging, symbol resolution, pick-test execution, and drifted-caller analysis) returned SURVIVED on all four sub-claims with zero verified refutations, fingerprinted at this branch's HEAD.

Type

🐛 Bug Fix

Changes

Ten cherry-picks onto stable/1.84.x as listed above, two test-only adaptation commits, version bump 1.84.7 -> 1.84.8, uv.lock refresh

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
3 out of 5 committers have signed the CLA.

✅ harish-berri
✅ Sameerlite
✅ yuneng-berri
❌ ishaan-berri
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[yuneng-berri]

@greptileai

[greptile-apps]

Greptile Summary

Nine cherry-picks from staging onto stable/1.84.x, cutting 1.84.8. The changes address Anthropic streaming reasoning-token counting, grace-period key rotation cache tuple bug, native Azure container routing, Prisma cached-plan recovery via reconnect, DB connection URL configuration knobs, 503 surfacing for DB infrastructure failures during auth, and Anthropic passthrough cost resolution for unknown-model paths.

• Auth/DB reliability: _lookup_deprecated_key now writes and reads a 3-tuple (was a 2-tuple causing ValueError on every cached cache hit); _query_first_with_cached_plan_fallback reconnects the Prisma client instead of injecting a comment into the query; infrastructure DB failures during auth now surface as 503 instead of 401.
• Streaming & costing: Reasoning chunks are accumulated across streaming events so final usage correctly splits reasoning vs text tokens; passthrough handlers fall back through litellm_params and SSE chunk content to resolve the costing model when the logging object reports "unknown".
• Container routing (Azure): api_base endpoint-path stripping and api-version extraction ensure the containers URL is built from the resource root; empty query_params dicts are converted to None so httpx does not silently strip the existing query string.

Confidence Score: 4/5

The backport is safe to merge; the most sensitive paths (auth error classification, cached-plan recovery, grace-period key lookup) are all covered by dedicated mock tests and the logic checks out.

The set of changes is well-scoped and each fix is independently testable. The only nit is a cosmetic double .strip() call in the router. All auth-path changes behave correctly under the exception-type hierarchy, and the new Prisma reconnect path is gated by the existing singleflight attempt_db_reconnect to prevent concurrent client teardowns.

No files require special attention beyond the minor style note in litellm/router.py.

Important Files Changed

Filename	Overview
litellm/proxy/utils.py	Fixes the 2-tuple/3-tuple cache bug in _lookup_deprecated_key (revoke_at now stored and checked), removes duplicate cache read, and replaces the unsafe DEALLOCATE-then-retry cached-plan recovery with a clean reconnect-then-retry approach.
litellm/proxy/auth/auth_exception_handler.py	Adds a 503 path for infrastructure-level DB failures during auth so valid keys are never rejected as 401 during a DB outage; control flow is correct.
litellm/proxy/db/exception_handler.py	Adds is_database_service_unavailable_error and is_prisma_engine_internal_error to classify DB infrastructure failures as 503-worthy rather than 401 auth failures.
litellm/proxy/proxy_cli.py	Introduces _build_db_connection_url_params; adds four new DB URL config knobs with backward-compatible fallback.
litellm/router.py	Falls back to proxy-forwarded model_id for native upstream container IDs; minor double .strip() call.
litellm/proxy/container_endpoints/ownership.py	Adds _CONTAINER_STORED_ID_CACHE and _get_stored_container_id for model_id recovery without extra cold-path DB hits.
litellm/llms/anthropic/chat/handler.py	Accumulates streamed thinking blocks so _handle_usage can correctly split reasoning vs text tokens.
litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py	Adds model fallback resolution and SSE pipeline hardening ([DONE] sentinels, non-JSON frames).
litellm/llms/azure/containers/transformation.py	Adds _normalize_api_base and _extract_api_version helpers for correct containers URL construction.
litellm/llms/custom_httpx/container_handler.py	Converts empty query_params dict to None to prevent httpx stripping the existing query string.

_{Reviews (1): Last reviewed commit: "test: align adapted auth builder tests w..." | Re-trigger Greptile}

[greptile-apps]

.strip() is called twice on _forwarded_model_id — once in the guard condition and again in the value expression. Assigning the stripped value to a local variable eliminates the redundant call.

Suggested change

	model_id = decoded.get("model_id") or (
	_forwarded_model_id.strip()
	if isinstance(_forwarded_model_id, str) and _forwarded_model_id.strip()
	else None
	)
	_stripped_model_id = (
	_forwarded_model_id.strip()
	if isinstance(_forwarded_model_id, str)
	else ""
	)
	model_id = decoded.get("model_id") or (_stripped_model_id or None)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Greptile Summary

Nine cherry-picks from staging onto stable/1.84.x cutting 1.84.8, accompanied by two test-only adaptation commits, a version bump, and a uv.lock refresh. Every production file is patch-id identical to its staging counterpart with the documented exceptions (dropped build artifacts, grafted test fixtures, and two context-drift resolutions in proxy_cli.py).

• Auth/DB correctness (core fixes): Fixes the grace-period key rotation bug (2-tuple→3-tuple cache mismatch causing 401 on every request after the first rotation lookup); returns 503 on DB-infra failures during auth instead of 401; adds the Prisma-engine AttributeError edge case to the infra-error classifier; recovers from PostgreSQL "cached plan" errors by reconnecting rather than injecting a unique SQL comment.
• Anthropic streaming & passthrough: Accumulates streaming thinking blocks so reasoning tokens are correctly split from text tokens at stream end; caps estimated_reasoning_tokens at actual completion_tokens; resolves the costing model for passthrough calls when the body model is \"unknown\"; hardens the SSE pipeline against [DONE] sentinels and non-JSON frames.
• Azure native container routing: Strips endpoint-specific path suffixes from api_base, prefers the API version embedded in the responses URL, and passes params=None (not {}) to httpx so existing ?api-version=… query strings are not silently stripped.

Confidence Score: 4/5

Safe to merge; all production changes are correct adaptations of staging fixes with comprehensive test coverage and zero new failures on this line's test suite.

The auth DB-infra error classification relies on traceback frame inspection to catch a specific prisma-client-py AttributeError, which is a narrow detection mechanism that would silently regress if the upstream library renames its engine subpackage. Additionally, _get_stored_container_id introduces a direct Prisma table query in the container request path rather than going through a helper function, widening the surface of direct-query call sites. Both are quality/maintainability concerns rather than immediate correctness failures; all other changes are straightforward and well-tested.

litellm/proxy/db/exception_handler.py (traceback-based prisma engine detection) and litellm/proxy/container_endpoints/ownership.py (direct DB query in request path).

Important Files Changed

Filename	Overview
litellm/proxy/utils.py	Grace-period key rotation: removes duplicate cache read, fixes 2-tuple→3-tuple cache mismatch that caused ValueError on second auth with a rotated key, drops the select clause so revoke_at is populated.
litellm/proxy/db/exception_handler.py	Adds is_database_service_unavailable_error and is_prisma_engine_internal_error; the traceback-walk approach in is_prisma_engine_internal_error is brittle to prisma module renames but solves a real edge case with good test coverage.
litellm/proxy/auth/auth_exception_handler.py	Adds 503 response for DB-infra failures during auth (instead of 401), correctly placed after the ProxyException re-raise so only raw non-Proxy exceptions reach the new branch.
litellm/llms/anthropic/chat/handler.py	Accumulates streaming thinking-block deltas in reasoning_content_chunks so _handle_usage can split reasoning vs text tokens correctly at stream end.
litellm/llms/anthropic/chat/transformation.py	Caps estimated_reasoning_tokens at actual completion_tokens with min(), preventing text_tokens from going negative when the token-counter estimate overshoots.
litellm/proxy/proxy_cli.py	Refactors DB URL param construction into _build_db_connection_url_params; adds connect_timeout, socket_timeout, pgbouncer, and extra_params knobs; backward compatible (all new fields are opt-in).
litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py	Adds _resolve_costing_model to fall back to deployment/model_group when model is 'unknown', adds _extract_model_from_anthropic_chunks to recover model from SSE stream, and hardens the SSE pipeline against [DONE] sentinels and non-JSON frames.
litellm/proxy/container_endpoints/ownership.py	Adds _CONTAINER_STORED_ID_CACHE to recover model_id from the encoded ID stored at create time for native upstream container IDs; get_container_forwarding_params becomes async to support the DB lookup; makes a direct Prisma query bypassing helper functions.
litellm/router.py	Falls back to the proxy-forwarded model_id when a container_id carries no LiteLLM routing payload (native Azure hex IDs), so deployment credentials are applied correctly.
litellm/proxy/_types.py	Adds four new optional ConfigGeneralSettings fields: database_connect_timeout, database_socket_timeout, database_extra_connection_params, database_disable_prepared_statements — all None-defaulted, backward compatible.

_{Reviews (2): Last reviewed commit: "test: align adapted auth builder tests w..." | Re-trigger Greptile}

[greptile-apps]

Traceback inspection is brittle to prisma module renames

is_prisma_engine_internal_error classifies an AttributeError as a DB-infra failure by walking e.__traceback__ and checking whether any frame's __name__ starts with "prisma.engine". This works today, but if prisma-client-py ever renames its engine subpackage (e.g. to prisma._engine, prisma_client.engine, or anything that doesn't share this prefix) the check silently returns False, and the same AttributeError that caused the auth-outage regression would start returning 401 again.

Consider recording a recognizable sentinel on the exception at the raise site inside the prisma shim, or pinning a prisma version range in pyproject.toml alongside a comment that this detection depends on the current naming convention.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Direct Prisma query in request path bypasses helper functions

_get_stored_container_id issues a litellm_managedobjecttable.find_first query directly against prisma_client.db, mirroring the existing pattern in _get_container_owner. The codebase rule requires all DB queries in the request path to go through the get_team/get_user/get_key object helper functions. The container path isn't the auth hot-path, and the call is cache-guarded, so the practical impact is low — but it widens the set of direct-query call sites that need to be updated if the DB access pattern changes (e.g., read-replica splitting, query logging).

Rule Used: What: In critical path of request, there should be... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[mateo-berri]

LGTM; thanks!