chore(deploy): build image off 1.89.1 backport (#30502)

[tin-berri]

Relevant issues

This PR exists to run the build/deploy workflow off the exact file state of the 1.89.1 backport merge (#30502, commit 3983231) so the resulting image can be deployed to a vcluster for testing. It is not intended as new feature work; the head branch points directly at that commit

Pre-Submission checklist

• not applicable; this is a deploy/CI build branch carrying an already-reviewed backport set

Type

🚄 Infrastructure

Changes

The head branch is the 1.89.1 backport (#30502); base is litellm_internal_staging, so the diff reflects the 1.89.x patch set (MCP, model-info, passthrough, DB fixes plus the 1.89.0 -> 1.89.1 version bump). No changes were authored here beyond pointing a branch at that commit

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
8 out of 11 committers have signed the CLA.

✅ yuneng-berri
✅ mateo-berri
✅ Sameerlite
✅ aanchal22
✅ kingdoooo
✅ Ar-maan05
✅ ryan-crabbe-berri
✅ tin-berri
❌ shin-berri
❌ yassin-berriai
❌ ishaan-berri
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This is the 1.89.1 backport deployed against litellm_internal_staging for vcluster testing. The patch set bundles several independent fixes across MCP auth, model-info endpoints, DB resilience, Anthropic sampling-param gating, and team BYOK concurrency.

• MCP auth (user_api_key_auth_mcp.py): explicit litellm key is now validated before the delegate_auth_to_upstream bypass (security improvement), _target_servers_use_oauth2 is removed, and the anonymous cold-start fallback is narrowed to auth_type=none passthrough servers only.
• DB resilience (exception_handler.py, auth_exception_handler.py, utils.py): DB outages during auth now surface as 503 instead of 401; the PostgreSQL cached-plan error recovery no longer injects a unique SQL comment but instead calls attempt_db_reconnect; a silent bug where deprecated-key grace-period lookups fetched a result but never returned it is fixed.
• model_info_v1 (proxy_server.py): now reads from llm_router.model_list directly (aligning with v2), adds include_team_models/teamId query params, and applies team BYOK scoping to prevent cross-team metadata leakage.
• Anthropic sampling params (common_utils.py, transformation.py): Claude 4.7+ models that removed temperature/top_p/top_k now get a clean 400 or silent drop rather than a passthrough that the API rejects.
• Team BYOK concurrency (team_endpoints.py): team_model_add now uses an atomic SQL ARRAY append with DISTINCT unnest to eliminate the race condition when concurrent BYOK model creates overwrite each other's team.models.

Confidence Score: 4/5

The backport is largely additive and defensive, but two changes warrant a closer look before promoting to production: the removal of the model-ID-to-proxy-name mapping in the batch rate limiter auth loop may reject legitimate non-managed batch job submissions, and the _supports_sampling_params fallback embeds hardcoded Claude model-name patterns that bypass the JSON-driven capability system.

The deprecated-key grace-period fix, DB 503 promotion, and cached-plan reconnect are clear improvements. The MCP auth reorder correctly tightens the security boundary. The batch rate limiter change removes a mapping that the code comment explicitly noted was needed for post-replacement JSONL files; whether that path is still exercised for legacy (non-managed) files needs confirmation.

litellm/proxy/hooks/batch_rate_limiter.py (resolve_model_name_from_model_id removal), litellm/llms/anthropic/common_utils.py (_supports_sampling_params fallback names)

Important Files Changed

Filename	Overview
litellm/proxy/hooks/batch_rate_limiter.py	Introduces target_model_names from unified file ID for managed files, and removes the resolve_model_name_from_model_id mapping that previously normalized provider IDs to proxy names for all batch files in the auth loop.
litellm/llms/anthropic/common_utils.py	Adds _supports_sampling_params to gate top_p/top_k/temperature on Claude 4.7+ models. Primary lookup is model-map driven, but retains a hardcoded name-pattern fallback list that violates the no-hardcode-model-flags rule.
litellm/proxy/management_endpoints/team_endpoints.py	team_model_add replaces Python-side merge with an atomic SQL ARRAY append using DISTINCT unnest to fix race conditions in concurrent BYOK model creates; updates updated_at only for the follow-up find.
litellm/proxy/_experimental/mcp_server/auth/user_api_key_auth_mcp.py	MCP auth refactor: reorders condition tree so explicit litellm key is checked before delegate_auth_to_upstream bypass, removes _target_servers_use_oauth2 fallback, narrows anonymous cold-start fallback to auth_type=none passthrough servers only.
litellm/proxy/proxy_server.py	model_info_v1 now reads directly from llm_router.model_list (like v2), adds include_team_models/teamId params, applies team BYOK scoping; v2/model/info promoted out of include_in_schema=False; helper functions extracted for testability.
litellm/proxy/utils.py	Cached-plan error recovery now calls attempt_db_reconnect (singleflight) instead of injecting a unique comment; also fixes a silent bug where deprecated_response was fetched but never returned during grace-period key lookups.
litellm/proxy/db/exception_handler.py	Adds is_prisma_engine_internal_error and is_database_service_unavailable_error to distinguish DB outage from auth failure, enabling 503 responses during connectivity issues instead of 401.
litellm/proxy/auth/auth_exception_handler.py	Adds a 503 fast-path when the authentication database is unreachable, preventing DB outages from masquerading as 401 auth failures.
litellm/proxy/auth/user_api_key_auth.py	Adds disable_budget_reservation flag to general_settings; passes general_settings into _reserve_budget_after_common_checks for the new skip path.
litellm/proxy/_experimental/mcp_server/rest_endpoints.py	Adds include_disabled_tools query param (admin-only) that bypasses allowed_tools filtering for MCP configuration UI; gated on PROXY_ADMIN role so non-admins always see filtered tools.
litellm/proxy/_experimental/mcp_server/db.py	MCP server delete now cleans up both litellm_mcpusercredentials and litellm_mcpuserenvvars independently so a failure on one table still attempts the other.
litellm/proxy/_experimental/mcp_server/discoverable_endpoints.py	exchange_token_with_server: expires_in is now omitted instead of defaulted to 3600 when absent, matching RFC 6749 §4.2.2 which makes expires_in optional.
litellm/llms/anthropic/chat/transformation.py	Routes temperature, top_p, and top_k through _apply_sampling_param to enforce model capability checks; no logic changes for supported models.

Comments Outside Diff (1)

1. litellm/proxy/hooks/batch_rate_limiter.py, line 657-663 (link)

   resolve_model_name_from_model_id mapping removed from auth loop

   The deleted code translated provider-level model IDs back to proxy model names before running access checks. The comment it carried ("body.model may be the provider id after replace_model_in_jsonl") described a real case: when a non-managed JSONL file is pre-processed by replace_model_in_jsonl, the body.model fields become provider IDs (e.g. openai/gpt-4.1), not the proxy aliases the caller has access to. After this change, model_to_check equals the raw JSONL value for the non-managed path, so the access check compares a provider ID against a proxy-model allowlist and spuriously rejects legitimate callers.

   For managed files this is fine — target_model_names already carries the proxy alias and bypasses the JSONL parse. The regression risk is on the legacy non-managed path if replace_model_in_jsonl has already transformed the content before this hook runs.

_{Reviews (1): Last reviewed commit: "Merge pull request #30502 from BerriAI/l..." | Re-trigger Greptile}

[greptile-apps]

Hardcoded model-name fallback violates no-hardcode-flags rule

The fallback name list ("fable", "opus-4-7", "opus-4-8" variants) is evaluated when _get_model_capability returns None — i.e. when the model map has no supports_sampling_params entry. The policy here is to express model capabilities solely through model_prices_and_context_window.json so users benefit from updates without code changes. The "fable" substring in particular is very broad and could silently match unintended provider-routed IDs.

Since the verified model IDs (claude-opus-4-7, claude-fable-5, etc.) already carry "supports_sampling_params": false in the JSON, the fallback names add risk rather than safety. The preferred pattern is to gate solely on _get_model_capability and, if that returns None, treat the model as supported (fail-open for unknown models) rather than maintaining a parallel name list in code.

Rule Used: What: Do not hardcode model-specific flags in the ... (source)

[greptile-apps]

Silent success when team_id does not match any row

execute_raw returns None regardless of whether the WHERE team_id = $2 clause matched any rows. If data.team_id refers to an already-deleted or nonexistent team, the SQL succeeds silently, the subsequent update also finds no row and returns None, and the caller receives no error. The old add_new_models_to_team + update(data={"models": ...}) path would raise a Prisma RecordNotFound from the update. Consider checking updated_team is None after the update and raising a 404 if the team row was not found.

[codecov]

Codecov Report

❌ Patch coverage is 38.33333% with 185 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	11.11%	56 Missing ⚠️
..._handlers/anthropic_passthrough_logging_handler.py	22.50%	31 Missing ⚠️
litellm/proxy/db/exception_handler.py	13.79%	25 Missing ⚠️
...ardrail_hooks/crowdstrike_aidr/crowdstrike_aidr.py	4.76%	20 Missing ⚠️
litellm/proxy/proxy_cli.py	0.00%	8 Missing ⚠️
litellm/llms/anthropic/common_utils.py	78.12%	7 Missing ⚠️
litellm/proxy/utils.py	0.00%	7 Missing ⚠️
litellm/llms/custom_httpx/llm_http_handler.py	62.50%	6 Missing ⚠️
litellm/proxy/_experimental/mcp_server/db.py	0.00%	5 Missing ⚠️
litellm/litellm_core_utils/litellm_logging.py	72.72%	3 Missing ⚠️
... and 9 more

📢 Thoughts on this report? Let us know!

[veria-ai]

High: SigV4 signing to caller-controlled hosts

api_base can come from the request path through the proxy, and the new SigV4 fallback signs whatever URL is returned here when no bearer token is configured. An authenticated caller can set api_base to an attacker-controlled host for a Bedrock Mantle /v1/responses request and receive the proxy's AWS-signed headers and prompt body; restrict IAM signing to the standard Mantle host or an operator-configured allowlist, and reject request-supplied custom hosts before calling _sign_request.

[veria-ai]

PR overview

This PR backports the deployment image build to the 1.89.1 line and includes changes affecting the Bedrock Mantle /v1/responses request transformation path. The touched code handles how Bedrock Mantle response requests are prepared and signed when routed through the proxy.

There is one open security issue: an authenticated caller can influence the Bedrock Mantle api_base and cause the proxy to SigV4-sign requests to a caller-controlled host. That could expose AWS-signed headers and prompt contents to the caller’s infrastructure unless signing is limited to trusted Mantle hosts or an operator allowlist. No issues have been addressed yet, so the PR still carries a clear attacker-driven data exposure risk.

Open issues (1)

• High: SigV4 signing to caller-controlled hosts — litellm/llms/bedrock_mantle/responses/transformation.py:83

Fixed/addressed: 0 · PR risk: 7/10