chore(release): backport 1.84.8 patch set + MCP/model-info/DB fixes to stable/1.89.x and cut 1.89.1 - #30502
Merged
yuneng-berri merged 26 commits intoJun 16, 2026
Conversation
* feat(proxy): publish /v2/model/info in Swagger OpenAPI spec Expose the v2 model info endpoint in /docs by removing include_in_schema=False and documenting query parameters used by the admin UI and proxy CLI consumers. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(ui): regenerate schema.d.ts for /v2/model/info OpenAPI docs Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> (cherry picked from commit f5b11b7)
…#29665) (#29788) * feat(responses): add default no-op sign_request to BaseResponsesAPIConfig * feat(responses): call sign_request after body is final, send signed bytes when signed * feat(bedrock_mantle): add SigV4 sign_request via composed BaseAWSLLM (bearer path) * test(bedrock_mantle): cover SigV4 access-key, AssumeRole, body bytes, region/auth consistency * feat(bedrock_mantle): defer auth to sign_request; validate_environment no longer requires bearer * docs(bedrock_mantle): document SigV4 + Bearer auth on Responses route * test(responses): cover fake-stream signing order and mantle bearer arg/env precedence * fix(bedrock_mantle): wrap all botocore credential errors with both-paths guidance * fix(bedrock_mantle): catch specific credential errors, not all BotoCoreError, so STS transport failures are not masked * fix(bedrock_mantle): sign the compact Responses route too, not just create (cherry picked from commit 2c95d0b)
…gs (#29991) Capture user_id and extra_info from metadata or litellm_metadata. The single-bag read dropped identity whenever a request carried a present litellm_metadata field (null or a user-supplied dict), since /chat/completions routes the authenticated identity into metadata while the guardrail read litellm_metadata first (cherry picked from commit 1bbaf1c)
…29493) * feat(proxy): add disable_budget_reservation general setting (#27639) * feat(proxy): register disable_budget_reservation in ConfigGeneralSettings (#27639) * docs(proxy): document disable_budget_reservation concurrency tradeoff (#27639) * ci: re-trigger flaky docker build (prisma generate ECONNRESET) * fix(proxy): warn and document budget enforcement tradeoff when disable_budget_reservation is set (#27639) (cherry picked from commit 1032dd7)
(cherry picked from commit 92817cb)
…ware GET path (#29960) * fix(ui): load MCP tool configuration tools via the OBO/passthrough-aware GET path * fix(mcp): admin-only include_disabled_tools so the settings UI shows toggled-off tools * fix(ui): repopulate MCP server edit form when server data loads after mount (OAuth return) * fix(ui): persist MCP OAuth token on save and return to the Settings tab after authorize * fix(ui): scope MCP OAuth callback to the initiating form so create and edit flows don't cross-talk * fix(ui): derive OAuth-return Settings tab via lazy state init instead of setState-in-effect * Fix MCP OAuth edit token handling --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> (cherry picked from commit 51ba6e3)
…r server's token no longer leaks into the next add-server session (#30000) * fix(ui/mcp): reset OAuth hook state on modal close so a prior server's token no longer leaks into the next add-server session * fix(ui/mcp): clear in-flight OAuth guard on reset and reset form/tools on modal close so nothing leaks on a parent-driven dismiss (cherry picked from commit d8fe091)
…T-3593) (#30009) * fix(proxy): authorize batch files using upload target_model_names (LIT-3593) After replace_model_in_jsonl, body.model is a stripped provider id. Reverse-mapping it via resolve_model_name_from_model_id is first-match on model_list and caused false 403s when multiple deployments share the same stripped name. Use target_model_names from the unified file id instead. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(proxy): restore resolve_model_name_from_model_id for JSONL fallback path (LIT-3593) Restores the reverse-lookup for the JSONL body.model fallback path so that legacy/pre-target_model_names managed files still map stripped provider IDs back to proxy aliases before auth. Also cleans up redundant `or None`. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert "fix(proxy): restore resolve_model_name_from_model_id for JSONL fallback path (LIT-3593)" This reverts commit 30d2e96. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> (cherry picked from commit 2cd7e87)
* fix(proxy): align /v1/model/info with router deployments Return router model_list entries (including team-scoped models) with team access metadata instead of wildcard-expanded names from get_complete_model_list. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(proxy): gate v1 team filter and honor key allowlists Only apply get_all_team_and_direct_access_models for admin or user-bound keys, then intersect with key/team model restrictions to avoid empty lists for service tokens and metadata leaks for restricted keys. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(proxy): skip v1 team filter when user row is missing Require a DB-backed user before applying team-access filtering on /v1/model/info, and skip the trailing filter in get_all_team_and_direct_access_models when user context cannot be resolved. Co-authored-by: Cursor <cursoragent@cursor.com> * Revert "fix(proxy): skip v1 team filter when user row is missing" This reverts commit 74e1fbd. * fix(proxy): restore legacy v1 model access filtering Keep /v1/model/info on key/team allowlists instead of DB team-membership filtering, while still listing router deployments for team-scoped models. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(proxy): drop A2A agent entries from public /v1/model/info list * fix(proxy): scope team BYOK rows on /v1/model/info to caller's teams Listing the full router model_list let any authenticated key without explicit model restrictions enumerate other teams' BYOK deployments (public name, team_id, api_base) via /v1/model/info. Reuse the existing _get_caller_byok_team_scope check so non-admin callers only see global deployments plus their own team's BYOK rows; admins keep the full view. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> (cherry picked from commit 6068bb7)
Enable teams to configure their own Datadog credentials via
POST /team/{team_id}/callback, following the same pattern as Langfuse.
(cherry picked from commit f5e6012)
… responses (#26153) (#27346) * fix: coerce server_tool_use dict to ServerToolUse in Usage.__init__ (#26153) * fix: coerce server_tool_use to ServerToolUse in stream_chunk_builder (#26153) * fix: dict/pydantic-tolerant access in tool_call_cost_tracking (#26153) * fix: dict/pydantic-tolerant access in anthropic cost_calculation (#26153) * test: assert ServerToolUse type in existing stream_chunk_builder anthropic web search test * test: regression test for #26153 (stream_chunk_builder server_tool_use type) * test: dict/pydantic safety for tool_call_cost_tracking helper * test: dict/pydantic safety for anthropic web_search cost * refactor: consolidate _get_web_search_requests into shared cost-calc utils * test(realtime): use gpt-realtime; openai retired gpt-4o-realtime-preview OpenAI shut down the gpt-4o-realtime-preview family (incl. the undated alias) on 2026-05-07, causing the live realtime test to fail with a 4000 invalid_request_error.invalid_model close. gpt-realtime is the GA successor; switch the live-call tests to it, matching the base branch. * refactor(types): drop redundant server_tool_use coercion in Usage.__init__ --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> (cherry picked from commit 4a3860d)
fix(proxy): atomic merge for team model aliases and team.models on BYOK create (cherry picked from commit ac7c2dc)
* fix(proxy): populate access_via_team_ids on /v1/model/info Team metadata enrichment previously only ran on /v2/model/info with include_team_models=true, leaving /v1/model/info without access_via_team_ids for project model-picker flows. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(dashboard): sync OpenAPI schema for /v1/model/info query params Add include_team_models and teamId to the generated schema for /model/info and /v1/model/info after the proxy endpoint gained team-access filtering. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(proxy): always return direct_access on /v1/model/info Set direct_access to true or false on every enriched model so clients can filter without treating a missing field as ambiguous. Co-authored-by: Cursor <cursoragent@cursor.com> * perf(proxy): fail fast when teamId is set without a connected DB on /v1/model/info Raise the db_not_connected error before building, enriching, and translating the model list instead of after, so a teamId query against a proxy with no database no longer wastes the full enrichment pipeline. * fix(proxy): fail fast when include_team_models is set without a database include_team_models=True relies on _populate_team_access_on_models to set direct_access/access_via_team_ids, which only runs when a database is connected. Without one, _filter_models_to_user_accessible discarded every model and the endpoint returned an empty list with HTTP 200. Mirror the teamId guard so the request fails fast with a clear db_not_connected error before any model-list work. * fix(proxy): populate direct_access on single-model /model/info lookup The /v1/model/info list path populates model_info.direct_access (and access_via_team_ids) when a database is connected, but the litellm_model_id single-model lookup returned early without it. This made the two endpoints disagree, breaking the parity assertion in test_get_specific_model. Run the same population on the single-model path so both responses match. * fix(proxy): apply no-DB fast-fail before litellm_model_id branch The teamId/include_team_models no-DB guard sat after the litellm_model_id early return, so ?litellm_model_id=X&teamId=Y with no DB returned 200 with unpopulated access fields instead of the 500 raised on every other path. Move the guard ahead of the branch so the fast-fail is uniform. * fix(proxy): apply teamId/include_team_models filters on single-model lookup The litellm_model_id early-return branch in model_info_v1 populated the team access fields but returned before the teamId and include_team_models filters ran, so a single-model lookup surfaced the deployment regardless of team access when the DB was connected. Run both filters on the single-model list before returning so the documented query params behave the same with and without litellm_model_id. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> (cherry picked from commit 7d1f68e)
…thropic streaming logging Targeted subset of staging commit cfcdf87 (#30202): only the anthropic_passthrough_logging_handler.py hardening hunks and their four tests are taken; the rest of that staging batch is intentionally excluded. (cherry picked from commit cfcdf87) (cherry picked from commit 973c7eb)
…combined view (#30327) The grace-period branch assigned the recursive get_data result (a finished LiteLLM_VerificationTokenView) back into the variable that the combined-view dict normalization then subscripts, raising TypeError on every request made with a rotated key inside its grace window; auth surfaced that as a 401. Return the recursive result directly instead. Regression test drives the full get_data flow: old hash misses the view, deprecated table resolves to the active token, and the call must return the view object (cherry picked from commit 5047eaf)
The OAuth2 passthrough ran user_api_key_auth on the client's upstream bearer first and only recovered after the failed validation had already logged a 401 auth event to the tracer, so successful tool calls to a delegated server each carried a phantom 401 span. Check delegate_auth_to_upstream before validating: a delegated server skips the doomed call entirely so nothing is logged, and a non-delegated server validates normally and surfaces a real 401 rather than being exchanged for an anonymous upstream-passthrough session. (cherry picked from commit 039a2d8)
(cherry picked from commit d96ab46) Backport adaptation: applied only the pyproject.toml and ui/litellm-dashboard/package.json manifest hunks; uv.lock and package-lock.json are regenerated on this line in a separate commit rather than cherry-picked, and the package.json devDependencies are unioned with this line's @types/uuid and vite entries
Prerequisite for #27346 (completion_cost AttributeError on streaming Anthropic web_search). #27346's per-chunk coercion is undone by the existing Usage(**returned_usage.model_dump()) reconstruction in calculate_usage, which round-trips server_tool_use back to a plain dict; without this Usage.__init__ coercion the cost path still does attribute access on a dict and raises. The same prerequisite was bundled into the stable/1.88.x backport of #27346 (24b9655). Content-verified present on litellm_internal_staging via aggregator 32c88ca (Litellm oss staging 080626, #29932); this restores only the two-line coercion, not the rest of that aggregator. (cherry picked from commit 32c88ca)
|
|
Contributor
Greptile SummaryThis is a 23-cherry-pick backport onto
Confidence Score: 4/5Safe to merge; no correctness regressions found and all targeted fixes are well-reasoned with accompanying tests. The two findings are both non-blocking quality concerns: the per-request WARNING log will flood production logs when disable_budget_reservation is enabled, and the broad OSError classification could theoretically misclassify a non-DB I/O error as a DB outage. The DB-resilience fixes, deprecated-key return fix, MCP auth reorganization, atomic team-model append, and passthrough SSE hardening all look correct and carry new tests. litellm/proxy/auth/user_api_key_auth.py (per-request warning log) and litellm/proxy/db/exception_handler.py (OSError breadth in is_database_service_unavailable_error).
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/user_api_key_auth.py | Adds disable_budget_reservation general-settings flag that skips optimistic budget reservation; adds general_settings param to _reserve_budget_after_common_checks. Warning logged on every request when flag is enabled (log flood risk). |
| litellm/proxy/db/exception_handler.py | Adds is_database_service_unavailable_error and is_prisma_engine_internal_error (traceback-walking detector) to distinguish DB infrastructure failures from auth errors; enables 503 responses on DB outages instead of 401. Broad OSError catch could misclassify non-DB I/O errors. |
| litellm/proxy/utils.py | Replaces the SQL-comment cache-buster with a proper attempt_db_reconnect call on cached-plan errors; fixes deprecated-key lookup returning None instead of the resolved token by introducing deprecated_response and returning it directly before the dict-subscript normalization block that would crash on a Pydantic model object. |
| litellm/proxy/proxy_server.py | Publishes /v2/model/info in OpenAPI; refactors model_info_v1 to use llm_router.model_list directly and populate access_via_team_ids/direct_access; adds include_team_models/teamId params to /v1/model/info. |
| litellm/proxy/_experimental/mcp_server/auth/user_api_key_auth_mcp.py | Reorganizes MCP auth decision tree: explicit key first, then delegate_auth_to_upstream bypass, then OAuth2. Removes _target_servers_use_oauth2 - net security tightening. |
Reviews (1): Last reviewed commit: "fix(types): coerce dict server_tool_use ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
2450
to
+2462
| user_api_key_auth_obj.budget_reservation = None | ||
| if skip_budget_checks: | ||
| return | ||
| if general_settings.get("disable_budget_reservation") is True: | ||
| verbose_proxy_logger.warning( | ||
| "disable_budget_reservation is enabled: skipping optimistic budget " | ||
| "reservation. Budget enforcement is read-time only — concurrent " | ||
| "requests can each pass the spend check before their cost is recorded, " | ||
| "so a configured budget may be briefly exceeded under high concurrency. " | ||
| "Set disable_budget_reservation to False or remove it to restore " | ||
| "hard per-request budget enforcement." | ||
| ) | ||
| return |
Contributor
There was a problem hiding this comment.
disable_budget_reservation is enabled
Every authenticated request emits a WARNING-level log line when the flag is set. In production deployments handling thousands of requests per second, this generates millions of log lines per day and can overwhelm log aggregators, increase storage costs, and obscure genuinely actionable warnings. A one-time startup warning (or a single DEBUG log per request with the warning elevated once at initialization) would achieve the same operator reminder without the noise.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Comment on lines
+178
to
+185
| return True | ||
|
|
||
| # OSError already covers ConnectionError and (Py3.3+) TimeoutError. | ||
| # asyncio.TimeoutError is a distinct class before Py3.11. | ||
| if isinstance(e, (OSError, asyncio.TimeoutError)): | ||
| return True | ||
|
|
||
| try: |
Contributor
There was a problem hiding this comment.
OSError catch may misclassify non-DB errors as DB-unavailable
OSError is the base class for all POSIX I/O errors, including PermissionError, FileNotFoundError, and many others unrelated to network connectivity. If any of these bubble up through a code path that passes through the auth exception handler (e.g., a custom auth hook that reads a file), this function returns True and the caller responds with HTTP 503 instead of the correct 401 or 500. Restricting the match to connection-specific subclasses (ConnectionError, TimeoutError) would make the classification more precise without losing real DB-unavailability detection, since both are subclasses of OSError.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Contributor
Greptile SummaryThis PR backports 23 fixes and features from
Confidence Score: 4/5The backport is well-scoped with each pick carrying its own regression tests and zero new failures against the known baseline. The auth and DB-resilience paths are the highest-risk areas but the logic is well-reasoned and documented. The DB exception classifier (exception_handler.py) has two defensive gaps: traceback-walking doesn't follow exception chains (cause/context), and the broad OSError catch could reclassify unrelated OS errors as DB unavailability. Both are low-probability in the actual auth path but worth hardening. The disable_budget_reservation strict litellm/proxy/db/exception_handler.py (exception chain traversal and OSError scope), litellm/proxy/auth/user_api_key_auth.py (is True check for disable_budget_reservation), litellm/proxy/proxy_server.py (model_info_v1 full deep-copy on every request)
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/auth_exception_handler.py | Adds a DB-unavailability check before the default 401 raise, returning 503 when Prisma is unreachable. Logic is sound; the broad OSError catch is a minor concern. |
| litellm/proxy/db/exception_handler.py | New is_database_service_unavailable_error and is_prisma_engine_internal_error helpers. Traceback-walking approach doesn't follow exception chains, and the OSError catch is broader than strictly needed. |
| litellm/proxy/auth/user_api_key_auth.py | Adds general_settings param to _reserve_budget_after_common_checks to support the new disable_budget_reservation flag. The is True strict check could silently fail for string env-var values. |
| litellm/proxy/utils.py | Two fixes: (1) cached-plan recovery now recreates the Prisma client via attempt_db_reconnect instead of injecting a unique SQL comment; (2) deprecated-key path returns deprecated_response directly to avoid crashing dict normalization on a VerificationTokenView object. |
| litellm/proxy/proxy_server.py | Major refactor of /v1/model/info: now starts from llm_router.model_list instead of model-name strings, enabling team-scoped BYOK deployments to appear; adds include_team_models and teamId query params; exposes /v2/model/info in the OpenAPI spec. |
| litellm/proxy/_experimental/mcp_server/auth/user_api_key_auth_mcp.py | Refactors MCP auth: explicit litellm key branch runs first, delegate-auth branch skips all LiteLLM validation for upstream-delegated servers eliminating phantom 401 spans. 403 no longer falls back to anonymous for oauth2 servers. |
| litellm/proxy/_experimental/mcp_server/db.py | Extends server deletion cleanup to also remove per-user credential rows, iterating both tables independently. |
| litellm/types/utils.py | Adds dict→ServerToolUse coercion in Usage.init and new StandardCallbackDynamicParams fields for Datadog team-scoped credentials. |
| litellm/integrations/datadog/datadog_team_handler.py | New file: per-team DataDogLogger handler following the LangFuse pattern, with DynamicLoggingCache keyed by credentials for isolation. |
| litellm/proxy/management_endpoints/team_endpoints.py | Replaces in-memory array merge in team_model_add with an atomic SQL unnest + DISTINCT append to prevent concurrent BYOK creates from clobbering each other. |
| litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py | Adds model resolution from SSE chunks when model == unknown, skips [DONE] sentinels and non-JSON SSE frames, adds _resolve_costing_model fallback. |
| litellm/llms/bedrock_mantle/responses/transformation.py | Extends BedrockMantle from Bearer-only to Bearer-or-SigV4 auth. |
| litellm/proxy/hooks/batch_rate_limiter.py | Batch file auth now uses upload target_model_names from the managed file ID instead of body.model JSONL values. |
Reviews (2): Last reviewed commit: "fix(types): coerce dict server_tool_use ..." | Re-trigger Greptile
Comment on lines
2447
to
+2457
| end_user_id: Optional[str] = None, | ||
| end_user_object: Optional[LiteLLM_EndUserTable] = None, | ||
| ) -> None: | ||
| user_api_key_auth_obj.budget_reservation = None | ||
| if skip_budget_checks: | ||
| return | ||
| if general_settings.get("disable_budget_reservation") is True: | ||
| verbose_proxy_logger.warning( | ||
| "disable_budget_reservation is enabled: skipping optimistic budget " | ||
| "reservation. Budget enforcement is read-time only — concurrent " | ||
| "requests can each pass the spend check before their cost is recorded, " |
Contributor
There was a problem hiding this comment.
disable_budget_reservation may be silently ignored for string env-var values
The check general_settings.get("disable_budget_reservation") is True uses is True (identity, not truthiness). When the setting is injected via environment-variable substitution (e.g. disable_budget_reservation: ${DISABLE_BUDGET_RESERVATION}), the YAML resolver leaves a plain string "true" rather than a Python bool, causing the guard to never trigger. The sibling setting database_disable_prepared_statements in proxy_cli.py uses str_to_bool for exactly this reason — the same treatment would make both flags consistent.
Comment on lines
+167
to
+185
| unavailable; see ``is_prisma_engine_internal_error``. | ||
| """ | ||
| import asyncio | ||
|
|
||
| if PrismaDBExceptionHandler.is_database_connection_error(e): | ||
| return True | ||
| if PrismaDBExceptionHandler.is_database_transport_error(e): | ||
| return True | ||
| if PrismaDBExceptionHandler.is_prisma_engine_internal_error(e): | ||
| return True | ||
| if "cached plan must not change result type" in str(e).lower(): | ||
| return True | ||
|
|
||
| # OSError already covers ConnectionError and (Py3.3+) TimeoutError. | ||
| # asyncio.TimeoutError is a distinct class before Py3.11. | ||
| if isinstance(e, (OSError, asyncio.TimeoutError)): | ||
| return True | ||
|
|
||
| try: |
Contributor
There was a problem hiding this comment.
OSError catch in is_database_service_unavailable_error is broader than DB connectivity
isinstance(e, OSError) catches all OS-level errors — file-not-found, permission-denied, broken-pipe from unrelated I/O, etc. Any non-DB OSError raised during the auth processing path (e.g., from a custom auth hook doing file I/O or a socket operation unrelated to Prisma) would be classified as a DB unavailability event and return 503 instead of the correct error code. Narrowing to the specific ConnectionError / TimeoutError subclasses, or keeping the OSError branch conditional on the exception originating from a DB-adjacent frame (similar to how is_prisma_engine_internal_error already scopes by traceback), would reduce false positives.
Comment on lines
+113
to
+142
| def is_prisma_engine_internal_error(e: Exception) -> bool: | ||
| """True iff ``e`` is a non-``PrismaError`` exception raised from inside | ||
| prisma-client-py's query-engine layer. | ||
|
|
||
| During the instant a DB connection is torn down, the query engine can | ||
| return a malformed error payload (``user_facing_error.meta`` is | ||
| ``null``). prisma-client-py's ``handle_response_errors`` then crashes | ||
| with ``AttributeError: 'NoneType' object has no attribute 'get'`` | ||
| before it can raise the proper P1001 "can't reach database server" | ||
| error. That AttributeError carries no connection keyword, so it can't | ||
| be matched by message; identify it by its ``prisma.engine`` origin | ||
| instead. | ||
|
|
||
| Recognized ``PrismaError`` subclasses are excluded: connectivity ones | ||
| are already classified by type/keyword above, and data-layer ones | ||
| (the DB IS reachable) must stay 401. | ||
| """ | ||
| import prisma | ||
|
|
||
| if isinstance(e, prisma.errors.PrismaError): | ||
| return False | ||
| tb = getattr(e, "__traceback__", None) | ||
| while tb is not None: | ||
| if tb.tb_frame.f_globals.get("__name__", "").startswith("prisma.engine"): | ||
| return True | ||
| tb = tb.tb_next | ||
| return False | ||
|
|
||
| @staticmethod | ||
| def is_database_service_unavailable_error(e: Exception) -> bool: |
Contributor
There was a problem hiding this comment.
is_prisma_engine_internal_error does not follow exception chains
__traceback__ is walked linearly from the innermost frame, but Python's exception chaining (__cause__ and __context__) is not traversed. If the Prisma engine error is caught and re-raised inside a try/except (e.g. wrapped into a custom exception or surfaced via raise ... from e), the original prisma.engine frames will be on the chained __cause__.__traceback__, not on e.__traceback__. The detection would silently fail and the error would fall through to the default 401 path instead of the intended 503.
| # expanded model names from get_complete_model_list(). Team-scoped rows | ||
| # use internal routing keys (model_name_{team_id}_{uuid}) and were omitted | ||
| # when v1 resolved models only via public model_name strings. | ||
| all_models: List[dict] = copy.deepcopy(llm_router.model_list) |
Contributor
There was a problem hiding this comment.
/v1/model/info now directly deep-copies the full router model list
The new code uses copy.deepcopy(llm_router.model_list) as the base set and then filters, replacing the old pattern of building the set through get_complete_model_list → get_model_list. For large deployments with many team-scoped BYOK rows, deep-copying the entire model list on every request is more expensive than the previous targeted lookup. This isn't a correctness issue but worth flagging for operators with thousands of router entries.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
| def _resolve_region(params: dict) -> str: | ||
| region = params.get("aws_region_name") | ||
| if region: | ||
| return region |
Contributor
There was a problem hiding this comment.
High: SSRF via unvalidated AWS region
An authenticated caller can set aws_region_name to a value like x@169.254.169.254:80/ or attacker.com:443/; this value is later interpolated into https://bedrock-mantle.{region}.api.aws, and URL parsing treats the injected separators as part of the authority. In bearer-token deployments, the proxy will POST the Responses body to that host with the configured Bedrock Mantle bearer token, so validate the region against AWS region syntax before returning it or using it in get_complete_url.
Contributor
PR overviewThis pull request backports the 1.84.8 patch set along with MCP, model-info, database, and Bedrock Mantle response handling fixes to the stable/1.89.x branch for the 1.89.1 release. There is one open security issue in the Bedrock Mantle response URL construction: an authenticated caller can influence the AWS region value in a way that redirects outbound proxy requests to an unintended host. In deployments using a configured Bedrock Mantle bearer token, this could expose that token and request body to attacker-controlled infrastructure or enable access to internal metadata endpoints. No issues have been addressed yet, so the PR still has a significant open SSRF risk before release. Open issues (1)
Fixed/addressed: 0 · PR risk: 8/10 |
doonga
pushed a commit
to greyrock-labs/home-ops
that referenced
this pull request
Jun 16, 2026
![renovate[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
….1) (#298) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | patch | `v1.89.0` → `v1.89.1` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.89.1`](https://github.com/BerriAI/litellm/releases/tag/v1.89.1) [Compare Source](BerriAI/litellm@v1.89.1...v1.89.1) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.1/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - chore(release): backport 1.84.8 patch set + MCP/model-info/DB fixes to stable/1.89.x and cut 1.89.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30502](BerriAI/litellm#30502) **Full Changelog**: <BerriAI/litellm@v1.89.0...v1.89.1> ### [`v1.89.1`](https://github.com/BerriAI/litellm/releases/tag/v1.89.1) [Compare Source](BerriAI/litellm@v1.89.0...v1.89.1) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.1/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - chore(release): backport 1.84.8 patch set + MCP/model-info/DB fixes to stable/1.89.x and cut 1.89.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30502](BerriAI/litellm#30502) **Full Changelog**: <BerriAI/litellm@v1.89.0...v1.89.1> </details> --- ### Configuration 📅 **Schedule**: (in timezone America/New_York) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMjQuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIyNC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL3BhdGNoIl19--> Reviewed-on: https://git.greyrock.io/greyrock-labs/home-ops/pulls/298
hbjydev
pushed a commit
to hbjydev/phoebe
that referenced
this pull request
Jun 16, 2026
….1) (#133) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | patch | `v1.89.0` → `v1.89.1` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.89.1`](https://github.com/BerriAI/litellm/releases/tag/v1.89.1) [Compare Source](BerriAI/litellm@v1.89.1...v1.89.1) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.1/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - chore(release): backport 1.84.8 patch set + MCP/model-info/DB fixes to stable/1.89.x and cut 1.89.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30502](BerriAI/litellm#30502) **Full Changelog**: <BerriAI/litellm@v1.89.0...v1.89.1> ### [`v1.89.1`](https://github.com/BerriAI/litellm/releases/tag/v1.89.1) [Compare Source](BerriAI/litellm@v1.89.0...v1.89.1) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.1/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - chore(release): backport 1.84.8 patch set + MCP/model-info/DB fixes to stable/1.89.x and cut 1.89.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30502](BerriAI/litellm#30502) **Full Changelog**: <BerriAI/litellm@v1.89.0...v1.89.1> </details> --- ### Configuration 📅 **Schedule**: (in timezone Europe/London) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMjQuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIyNC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL3BhdGNoIl19--> Reviewed-on: https://forgejo.hayden.moe/hayden/phoebe/pulls/133
1 task
blake-hamm
added a commit
to blake-hamm/bhamm-lab
that referenced
this pull request
Jun 17, 2026
…to v1.89.1 (#204) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [https://github.com/BerriAI/litellm.git](https://github.com/BerriAI/litellm) | patch | `v1.89.0` → `v1.89.1` | --- ### Release Notes <details> <summary>BerriAI/litellm (https://github.com/BerriAI/litellm.git)</summary> ### [`v1.89.1`](https://github.com/BerriAI/litellm/releases/tag/v1.89.1) [Compare Source](BerriAI/litellm@v1.89.0...v1.89.1) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.1/cosign.pub \ ghcr.io/berriai/litellm:v1.89.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - chore(release): backport 1.84.8 patch set + MCP/model-info/DB fixes to stable/1.89.x and cut 1.89.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30502](BerriAI/litellm#30502) **Full Changelog**: <BerriAI/litellm@v1.89.0...v1.89.1> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMjAuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIyMC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=--> Co-authored-by: Renovate Bot <renovate@bhamm-lab.com> Reviewed-on: https://codeberg.org/blake-hamm/bhamm-lab/pulls/204
13 tasks
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Backports the patch set that went into the 1.84.8 build, plus the additional fixes requested for the 1.89 line, onto
stable/1.89.x, and cuts1.89.1. 1.89.0 was cut from staging on 2026-06-06; every change here merged into staging after that cut, so the line genuinely lacked all of them (verified per PR; #29490 was already present via the pre-cut oss-staging aggregator and is intentionally excluded). 1.89.0 has shipped (release/v1.89.0+ tagv1.89.0), so this PR bumps the line to 1.89.1.Linear ticket
N/A
What is included
Cherry-picks in original staging merge order (each
-xtraceable tolitellm_internal_staging, except the four oss-aggregator picks noted below):Plus a one-commit prerequisite and the version bump:
#29490 was excluded: its bedrock_mantle Responses API base already lives on the line via the pre-cut aggregator #29671, so re-picking it would conflict with code already present. #29788 (which extends that base with SigV4 auth) is included and applies on top.
Adaptation notes
ui/litellm-dashboard/src/lib/http/schema.d.tshunk; that file is produced by the UI build and carries no Python dependency. The Python changes apply unchanged.model = inputs.get("model"); ai_guard_payload["model"] = modelblock inapply_guardrailas pre-existing context (that block landed on staging via aggregator Litellm oss staging 080626 #29932, which is not on this line). On 1.89.x the block was absent, so resolving fix(guardrails): read CrowdStrike AIDR identity from both metadata bags #29991's conflict re-introduces it alongside fix(guardrails): read CrowdStrike AIDR identity from both metadata bags #29991's own metadata-bag change. The re-introduced block is byte-identical to Litellm oss staging 080626 #29932's and is required by fix(guardrails): read CrowdStrike AIDR identity from both metadata bags #29991's own test (assert payload["model"] == "gpt-4o"); content-verified against Litellm oss staging 080626 #29932.team_model_add's DB update to re-fetch withinclude={"object_permission": True}and call_refresh_cached_team, while fix(proxy): atomic merge for team model aliases and team.models on BYOK create #29528 rewrites the same statement to do an atomicexecute_rawdedup append plus anupdated_at-bump re-fetch. The resolution keeps both: the atomic append runs, then the re-fetch includesobject_permissionand the cache refresh follows. Both PRs' own tests were updated only in their setup to mock the other path's call (fix(proxy): atomic merge for team model aliases and team.models on BYOK create #29528's test mocks_refresh_cached_team; the line's LIT-3244 cache-refresh pin mocks the newexecute_raw); each test's behavioral assertion is unchanged and both pass.pyproject.toml(pypdf, tornado/aiohttp constraints) andui/litellm-dashboard/package.json(vitest 3.2.6) manifest hunks.uv.lockandpackage-lock.jsonwere regenerated on this line rather than cherry-picked, and they live in the version-bump commit alongside the 1.89.1 bump (pypdf resolves to 6.13.2). The line's conservative lock resolution kept brace-expansion at the advisory-affected 5.0.5, so an explicitbrace-expansion: 5.0.6override was added topackage.json(also in the bump commit) to match the patched transitive version that chore(deps): bump vitest, brace-expansion, pypdf and tornado #30220 ships on staging.ServerToolUsecoercion is round-tripped back to a plain dict by the pre-existingUsage(**returned_usage.model_dump())reconstruction incalculate_usage, so without a coercion inUsage.__init__the cost path still does attribute access on a dict and raises -- which is the exact bug fix: completion_cost AttributeError on streaming Anthropic web_search responses (#26153) #27346 fixes. The two-lineUsage.__init__coercion is restored verbatim from staging aggregator Litellm oss staging 080626 #29932 (content-verified), the same prerequisite the stable/1.88.x backport of fix: completion_cost AttributeError on streaming Anthropic web_search responses (#26153) #27346 bundled.litellm_internal_stagingonly inside the promoted aggregators Litellm oss staging 080626 #29932 and chore(oss): litellm oss staging 120626 #30292, so their discrete merge SHAs are not staging-reachable. Each pick's post-image was content-verified present in its aggregator's tree on staging. 973c7eb carries a proper-xfooter to the feat: litellm oss 110626 #30202 aggregator and is fully staging-traceable.Known noise on this line
tests/test_litellm/proxy/_experimental/mcp_server/test_discoverable_endpoints.pyhas 27 failing tests on the clean 1.89.0 tip (the test file expects X-Forwarded-Proto/Host base-URL resolution that the line's source predates). They fail identically before and after this PR; no pick touches that source. The targeted-test delta is 0 new failures against that baseline.Pre-Submission checklist
Screenshots / Proof of Fix
Live proxy on
localhost:4001built from this branch, hitting real Anthropic.P0 #30274 --
access_via_team_idspopulated on/v1/model/info:#29900 --
/v2/model/infopublished in the OpenAPI spec:Auth path (#29986) and key scoping:
Sanity completion (real Anthropic claude-haiku-4-5):
Targeted-test delta: 27 failed / 1184 passed, all 27 failures pre-existing on the clean tip (see Known noise); 0 new failures. A deep multi-agent review of the branch confirmed the operational properties: every referenced symbol resolves across the 33 touched modules, every pick's own tests pass as a delta against the known x-forwarded noise, no upstream caller of the modified functions is broken, and the #27346 prerequisite is load-bearing (removing it reproduces the original AttributeError). It also flagged two accounting gaps in the adaptation notes (the #29991 model block originating from #29932 and the #30220 lockfile/override commit location), both addressed above; neither is a behavioral regression.
Type
🐛 Bug Fix
Changes
23 backported fixes/features, one prerequisite coercion, and a deps + version bump to 1.89.1 on
stable/1.89.x. Details above.