chore(release): backport #30585, #30867, #31905, #32093, #32405, #34189, #36011 to stable/1.89.x and cut 1.89.7 - #36313
Conversation
…als (#30585) * fix: validate proxy request body and nested fields Ensure caller-supplied request fields cannot override server-side deployment configuration, and apply request-body validation consistently to nested structures. Adjusts router kwarg handling and client-side credential handling for base-url overrides * test: cover router strip ordering and advisor clientside credential gate * fix: clear deployment credentials on client base-url override When a request overrides api_base/base_url, recompute the deployment's litellm_params (clearing the deployment's own api_key) and drop the cached client built for the original endpoint, so the deployment credential is not reused for the client-supplied endpoint. Adds regression tests that assert the credentials actually forwarded to litellm.completion/acompletion. * fix(proxy): require api_key alongside api_base override A request that overrides api_base/base_url but supplies no api_key still left the proxy carrying a server credential: once the override clears the banned-param opt-in, the provider re-resolves a key from the environment (api_key or get_secret("OPENAI_API_KEY") and ~30 sibling chains in main.py) and forwards it to the caller-controlled URL. Popping the deployment api_key only changed which server key leaked. Gate is_request_body_safe so a permitted api_base/base_url override must also carry a non-empty caller api_key; reject otherwise. The env resolution in main.py is left as the provider boundary. * fix(proxy): extend request-body banlist with five additional credential and session targeting fields Yuneng's review found five deployment-owned request-body params still missing from the denylist and the router strip set. Each lets a caller reach the operator's provider credentials or retarget the outbound request: aws_profile_name selects a local AWS profile, oci_compartment_id and oci_region retarget the OCI request, litellm_credential_name selects any server-loaded credential by name with no ownership check, and runtimeSessionId resumes a Bedrock AgentCore runtime session (AWS does not enforce session-to-user mapping, so this is a cross-tenant session-resume vector). Add all five to _BANNED_REQUEST_BODY_PARAMS in auth_utils.py and to _DEPLOYMENT_OWNED_CREDENTIAL_KWARGS in router.py. Deployment litellm_params and SDK direct calls are unaffected: the banlist gates the request body only, and the router strip drops caller kwargs, never deployment["litellm_params"]. * test: rename arbitrary canary values in security tests to neutral placeholders * fix(proxy): apply api_key co-presence to nested base override and warn on Router credential strip P1-A: is_request_body_safe descended into _NESTED_CONFIG_KEYS (litellm_embedding_config, extra_body) for the banned-param check but not for the api_key co-presence check, so a base override smuggled into one of those nested dicts cleared the client-side-credentials opt-in without a paired api_key and let the provider re-resolve a server credential from the environment. Run _check_base_override_has_api_key on each nested config dict too, so the requirement applies wherever a base override is permitted. P1-B: the deployment-owned credential strip in the Router runs unconditionally on every _completion/_acompletion, which is security-correct but silently drops per-call api_version/vertex_project/etc. for SDK Router callers. Emit a single warning (key names only, never values) when the strip removes a non-empty value, so the backwards-incompatible behavior is visible without gating the strip on a context flag that does not exist. * fix(proxy): apply api_key co-presence to tool-entry base override is_request_body_safe scans three surfaces (root, _NESTED_CONFIG_KEYS, and tools[]); the previous commit extended the api_key co-presence rule to root and nested config dicts but not to tool entries. With allow_client_side_credentials enabled, a tool entry carrying api_base/base_url and no paired api_key cleared the gate, letting a provider interceptor fall back to a server-side credential for a caller-controlled URL. Add the same _check_base_override_has_api_key call to each tool dict and its nested function dict, mirroring the symmetry already applied to the nested config keys. The rule is unchanged: api_key must live in the same dict as the base override it accompanies. * test(proxy/auth): require paired api_key under extra_body opt-in * fix(router): gate deployment-owned kwarg strip on litellm.proxy_is_running * fix(advisor): narrow proxy-import guard to ImportError-family * fix(router): gate api_key clear on base override behind litellm.proxy_is_running * test(proxy/auth): scope proxy_is_running flag to dynamic-params class with autouse fixture * style: use built-in generics in PR-added type annotations * revert: drop proxy_is_running flag and router-level credential strip; rely on proxy gate * revert: scope PR to LIT-3828 + LIT-3834 only; drop LIT-3830/LIT-3833 changes * style: black-format advisor orchestration test (cherry picked from commit 1667b8f)
…30867) AWS auth parameters in the Bedrock and SageMaker path could be expanded against the process environment when credentials were built. Config-sourced references are already expanded at load time, so restrict expansion to that path: a reference still present at request time is treated as caller-supplied input and is left as-is, and the web-identity helper rejects environment-variable references before resolving the token. Also rework the ambient AWS_* fallback as a single pass that pairs each value with its own env-var name, fixing a latent index misalignment that left AWS_EXTERNAL_ID unresolved. Adds regression tests covering the resolution behavior. (cherry picked from commit 4ef7d08)
…ging override (LIT-3587) (#31905) The security fix in 34e9be1 removed turn_off_message_logging from _supported_callback_params to stop callers bypassing global redaction via the request body. That also killed the documented admin-only per-key or per-team override because both flows resolve through the same allowlist in initialize_standard_callback_dynamic_params. Put turn_off_message_logging back in _supported_callback_params so an admin-configured metadata.logging[].callback_vars.turn_off_message_logging survives into StandardCallbackDynamicParams and can override the global setting for that key or team, as documented at docs/proxy/team_logging#disableenable-message-redaction. Consolidate the metadata traversal so the extractor and the proxy strip walk the same set of client-controllable slots. iter_client_callback_metadata_dicts in litellm_core_utils/initialize_dynamic_callback_params.py is the single source of truth for metadata, litellm_metadata, and litellm_params.metadata; _strip_client_message_redaction_opt_out imports it so a future addition to one side automatically reaches the other. The extractor iterates the helper in reversed order so litellm_params.metadata keeps overriding metadata, matching the pre-refactor merge precedence. Client bypass stays blocked. Restoring the field re-enrolls it in the auth layer's _BANNED_REQUEST_BODY_PARAMS (derived from _supported_callback_params via _build_banned_observability_params), so client submissions at the top level, inside metadata, or inside a JSON-string litellm_metadata all 401 at ingress. is_request_body_safe also now descends into litellm_params.metadata for the same 401 defense against the nested-body attack vector, matching how the metadata and litellm_metadata slots are handled. _strip_client_message_redaction_opt_out runs after the litellm_metadata JSON parse and before the admin callback_vars unpack, so admin values survive while any leftover client-supplied opt-out is dropped when global redaction is on and the key or team lacks allow_client_message_redaction_opt_out. Flip the two dynamic-param e2e tests added by the security fix to reflect the restored override behavior, keeping the invariant that proxy client bypass is stopped by the auth layer 401 above. Co-authored-by: yucheng <yucheng@yuchengs-MBP.attlocal.net> Co-authored-by: Cursor Agent <cursoragent@cursor.com> (cherry picked from commit 8e6098a)
…advisor tool (#32093) * fix(anthropic): require caller api_key and SSRF-validate api_base in advisor tool The advisor_20260301 interceptor honored a caller-supplied api_base once allow_client_side_credentials was enabled, even without a caller-supplied api_key. AnthropicModelInfo.get_auth_header() then fell back to the proxy's own ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN, so the server's real credentials plus the conversation history got sent to a caller-chosen destination _resolve_advisor_credentials() now only honors api_base alongside a non-empty caller-supplied api_key, requires the https scheme, and validates api_base via validate_url() before use, mirroring check_complete_credentials in auth_utils.py. https is required because validate_url only DNS-pins the connection for http; for https with TLS verification on it returns the URL unchanged and relies on certificate validation to block DNS rebinding * fix(anthropic): also reject advisor api_base when ssl_verify is disabled validate_url only DNS-pins the connection for http, or for https with litellm.ssl_verify disabled; the previous https-only check missed the ssl_verify=False case, where validate_url's rewritten URL was still being discarded, per Greptile's review of this PR. Reject api_base outright when ssl_verify is False so the discarded rewrite can no longer matter (cherry picked from commit 07b9ea8)
Root cause: PR #30867 removed request-time os.environ/ expansion in BaseAWSLLM.get_credentials. That is only safe if config-load pre-resolves os.environ/ refs so the value reaching get_credentials is already the real secret. The YAML config path has always done this. The DB-load path (ProxyConfig._resolve_db_litellm_param) only re-expanded keys in a hardcoded whitelist (_DB_LITELLM_PARAM_ENV_REF_KEYS) plus short-circuited env-ref resolution entirely for team-scoped rows. PR #32256 extended that whitelist to 18 keys to unblock a customer whose Bedrock model with aws_role_name: os.environ/BEDROCK_ASSUME_ROLE_ARN broke on v1.90+, but the whitelist is structurally fragile: every future auth field breaks the same way until someone remembers to add it Fix: remove the whitelist and the team-scope short-circuit. The DB-load resolver now expands os.environ/ on every string field, matching the YAML path. Trust boundary stays on the write side: only PROXY_ADMIN can create team_id=None rows, only team admins of a team can create rows scoped to that team, and the request-body vector is still blocked by _BANNED_REQUEST_BODY_PARAMS. Team-scoped rows now resolve env refs — this is a deliberate LIT-3831 threat-model expansion trusting team admins for env-var reads Regression tests in tests/test_litellm/proxy/proxy_server/test_proxy_config.py: - test_ProxyConfig__add_deployment_resolves_env_refs_after_db_decrypt pins admin-scoped rows resolve every field (previously api_base stayed literal) - test_ProxyConfig__add_deployment_resolves_team_env_refs pins team rows resolve env refs (previously stayed literal) - test_ProxyConfig__add_deployment_resolves_env_refs_on_arbitrary_field pins the no-whitelist invariant against a made-up field name - test_ProxyConfig__add_deployment_resolves_env_refs_for_aws_bedrock_auth_params (from #32256) still passes - Path B counterparts (decrypt_model_list_from_db) mirror the above Left as followups (not fixed here): - /model/info and /v2/model/info still echo resolved values for fields not in the current pop-list (aws_role_name, aws_sts_endpoint, api_base, etc.). Fix is to extend remove_sensitive_info_from_deployment; separate PR - Master-key rotation reads DB rows via decrypt_model_list_from_db which now resolves universally, so rotation collapses env-refs into hardcoded values. Pre-existing bug for the 6 previously-whitelisted fields; wider surface after this PR. Separate PR (cherry picked from commit 5862be3)
…imeouts aiohttp 3.14.0 and 3.14.1 re-arm the sock_read timer on a keep-alive connection after it has already been returned to the idle pool. The stray timer stamps a SocketTimeoutError on the pooled connection without closing it, so the pool keeps handing it out and the next request to pick it up fails instantly on an error left behind by an earlier, unrelated request. Because a single pool is shared across providers, the failures appear simultaneously across Vertex AI, Bedrock, Anthropic and OpenAI-compatible deployments as sub-millisecond "Connection timed out" errors. uv.lock resolved aiohttp 3.14.1 and the published images install via `uv sync --frozen`, so every image built from that lock shipped the regression. The wheel's own metadata declared `aiohttp>=3.10,<4.0`, which also left pip consumers free to resolve into the same broken window, so both the runtime floor and the uv constraint move to >=3.14.2. Upstream fixed this in aio-libs/aiohttp#12954, released in aiohttp 3.14.2; the lock now resolves 3.14.3. Raising the floor rather than capping below 3.14 keeps the advisories that the existing 3.14.1 floor cleared, so no osv-scanner ignores are needed. litellm requires Python >=3.10 and aiohttp 3.14.2 requires >=3.10, so no supported interpreter loses support. Both new tests fail on the previous pins and pass on these. (cherry picked from commit ffd6ac5)
Greptile SummaryThis patch releases LiteLLM 1.89.7 with backported proxy security hardening, callback-redaction behavior, DB model parameter resolution, provider URL handling, AWS credential handling, and dependency updates.
Confidence Score: 5/5The PR appears safe to merge, with no concrete changed-code defect remaining after review. The request-validation, authorization, credential-isolation, provider, and dependency changes have matching regression coverage, and the security leads identified in unchanged dependency versions are not introduced or worsened by this patch.
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/auth_utils.py | Adds recursive validation for fallback destinations, nested form metadata, and forbidden request parameters without leaving a demonstrated supported-shape bypass. |
| litellm/proxy/auth/user_api_key_auth.py | Reuses the canonical fallback traversal for model-allowlist enforcement across supported fallback shapes. |
| litellm/llms/anthropic/experimental_pass_through/messages/interceptors/advisor.py | Gates caller-supplied advisor credentials and validates custom endpoints before use. |
| litellm/llms/bedrock/base_aws_llm.py | Stops expanding caller-supplied environment references while retaining ambient AWS fallback behavior. |
| litellm/proxy/litellm_pre_call_utils.py | Centralizes URL-destination rejection and strips unauthorized message-redaction opt-outs from all supported metadata locations. |
| litellm/proxy/health_endpoints/_health_endpoints.py | Prevents configured credentials from being combined with caller-overridden connection destinations unless explicitly enabled. |
| litellm/proxy/proxy_server.py | Consistently decrypts DB model parameters and resolves stored environment references before model construction. |
| pyproject.toml | Bumps the release and dependency constraints, including the aiohttp floor and ddtrace major version. |
| uv.lock | Regenerates the lockfile for the release and dependency maintenance; scanner-listed vulnerable package versions were unchanged from base. |
Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.89.7" | Re-trigger Greptile
| if isinstance(decrypted_value, str) and decrypted_value.startswith( | ||
| "os.environ/" | ||
| ): | ||
| return get_secret(decrypted_value) |
There was a problem hiding this comment.
Critical: Team models can resolve arbitrary server secrets
A team admin can create a team-scoped model with an arbitrary parameter such as some_future_field: os.environ/LITELLM_MASTER_KEY. This helper resolves it, and decrypt_model_list_from_db() returns the arbitrary field through model-info responses without generic redaction, allowing the team admin to retrieve the master key and authenticate as a proxy admin. Restrict environment-reference expansion to proxy-admin-authored rows or explicitly approved fields, and reject os.environ/ values when team models are created or updated.
PR overviewThis release PR backports seven changes to the stable/1.89.x branch and prepares version 1.89.7. The affected proxy code includes handling of model configuration loaded from the database. One critical security issue remains open, with none addressed so far. A team administrator can use a team-scoped model configuration to resolve and expose arbitrary server environment secrets, including the proxy master key, enabling escalation to proxy administrator privileges. Open issues (1)
Fixed/addressed: 0 · PR risk: 9/10 |
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🐛 Bug Fix
Changes
Patch release for the 1.89.x line, bumping 1.89.6 to 1.89.7. Every commit is either a
cherry-pick -xof a commit already onlitellm_internal_stagingor a tool-generated lock, version, or dependency commit. The only hand-written line in the whole branch is aruff.tomlentry described below. No merge commits, no schema or migration changes, no UI source, no workflow editsBackported fixes, in staging merge order, each carrying its
(cherry picked from commit ...)footer:2edb7501667b8f09b9a2a4ef7d0853fed7d8e6098af27028107b9ea8485d5c15862be3ca97194065faf6dae4679c898d3416a630fedc38ea29fb706ffd6ac5485d5c1is the one adapted pick. On staging it edits an existing_resolve_db_litellm_paramhelper, dropping the_DB_LITELLM_PARAM_ENV_REF_KEYSallowlist and the_db_model_is_team_scopedshort-circuit so that resolution applies to every parameter. This line has none of that machinery, its two decrypt loops inproxy_server.pyjust calldecrypt_value_helperinline, so the pick transposes staging's resulting three-line helper onto those two loops instead of trying to apply a diff against a pre-image that is not there. The resulting helper is logically identical to the one on staging today, which matters because09b9a2astops expandingos.environ/references at request time and would otherwise leave DB-stored Bedrock and SageMaker deployments holding literalos.environ/...strings.dae4679merged on staging as a six-commit merge commit and is picked with-m 1, so it lands squashed. Every other pick is a straight squash pickdae4679also adds one line toruff.toml. This line enablesPLR0915while staging does not, and the pick takescommon_processing_pre_call_logicfrom 50 statements to 51, soruff checkgoes red without it. The exemption follows the six entries this file already carries for the same rule, and it keeps the picked source byte-faithful instead of restructuring a function the pick only appends two lines to5d137edand3698031also touchpyproject.toml. The ddtrace pick collapses the version-split constraint this line predates, and the aiohttp pick raises the floor so a future relock cannot walk back below it. Both were already merged on staging and are picked, not re-derivedRoutine dependency maintenance on top, each a lock-only regeneration whose moved set is confined to the target package and its own new requirement floors: mcp 1.28.1, pypdf 6.14.2, pyasn1 0.6.4, gitpython 3.1.58, soupsieve 2.8.4, httplib2 0.32.0, h2 4.4.1, langgraph-checkpoint 4.1.1, setuptools 83.0.0, cryptography 50.0.0, Pillow 12.3.0. Every target is at or below what staging already resolves, so upgrading from this patch to a later release never moves backwards. cryptography and Pillow needed a pyproject range change to reach their target; the rest stayed inside the existing ranges
Finally
bump: version 1.89.6 → 1.89.7fromcz bumpand auv.lockrefresh whose only moved entry is litellm itselfQA runbook
Everything below was run on this branch against a baseline captured on the pristine
stable/1.89.xtip first, since an old line carries its own test noise and only the delta means anythingMapped test files for every touched module: 1045 passed against a 934-passed baseline, with the same three pre-existing failures on both sides and zero new ones. The three are
test_ProxyConfig_get_model_info_with_id_missing_model_id_raises,test_project_guardrails_only, andtest_request_guardrails_do_not_override_key_guardrails, all already red on the bare tipThe full
tests/test_litellmsuite was run on both trees the waymake test-unitruns it. The tip failed 67 tests out of 21536, and this branch failed 66 out of 21649, so the branch adds 113 passing tests and nets one fewer failure. Eight node ids appear on this branch's failure list that were not on the tip's, and all eight pass on both trees when the failing test, and again when its whole file, is run on its own, so they are cross-test pollution under-n 4rather than anything this branch changed. Nine node ids moved the other way for the same reasonSymbol closure: every changed module imports cleanly at runtime (15 of 15), and pyflakes reports exactly the same hard findings on this branch as on the tip, so no pick references an identifier the older code lacks and no path it adds is dead on arrival
Format and lint: this line gates on
black --checkat 88 columns rather than staging'sruff formatat 120, so every picked hunk was reflowed with the line's own black. Every target behindmake lintwas run on both trees and compared.black --checknames the same fifteen files on this branch as on the tip and none of them are files this branch touches,ruff checkpasses, the circular-import check passes,from litellm import *succeeds, anduv lock --checkreports the lock in syncProvenance: 22 commits, zero merge commits, zero
litellm/proxy/_experimental/out/paths. All nine picks carry a-xfooter resolving to a commit reachable fromlitellm_internal_staging, and each pick's author matches its staging counterpart. The thirteen footerless commits are exactly the tool-generated onesA live proxy on this branch was exercised against the same proxy booted from the pristine tip, using real provider traffic on both, and the DB-sourced model path was replayed against a single shared Postgres so both proxies read byte-identical rows