Skip to content

chore(release): backport #30585, #30867, #31905, #32093, #32405, #34189, #36011 to stable/1.89.x and cut 1.89.7 - #36313

Merged
yuneng-berri merged 22 commits into
stable/1.89.xfrom
litellm_backport_1_89_x_bp-189x-0808sec
Aug 8, 2026
Merged

chore(release): backport #30585, #30867, #31905, #32093, #32405, #34189, #36011 to stable/1.89.x and cut 1.89.7#36313
yuneng-berri merged 22 commits into
stable/1.89.xfrom
litellm_backport_1_89_x_bp-189x-0808sec

Conversation

@yuneng-berri

Copy link
Copy Markdown
Collaborator

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Screenshots / Proof of Fix

Type

🐛 Bug Fix

Changes

Patch release for the 1.89.x line, bumping 1.89.6 to 1.89.7. Every commit is either a cherry-pick -x of a commit already on litellm_internal_staging or a tool-generated lock, version, or dependency commit. The only hand-written line in the whole branch is a ruff.toml entry described below. No merge commits, no schema or migration changes, no UI source, no workflow edits

Backported fixes, in staging merge order, each carrying its (cherry picked from commit ...) footer:

Commit on this branch Staging commit Subject
2edb750 1667b8f fix: reject model_list in proxy body and gate advisor client credentials
09b9a2a 4ef7d08 fix(bedrock): only expand config-sourced AWS credential references
53fed7d 8e6098a fix(proxy): restore admin key/team callback_vars.turn_off_message_logging override
f270281 07b9ea8 fix(anthropic): require caller api_key and SSRF-validate api_base in advisor tool
485d5c1 5862be3 fix(proxy): resolve os.environ/ refs universally in DB-sourced models
ca97194 065faf6 chore(proxy): clean up request parameter validation and provider destination handling
dae4679 c898d34 fix(proxy)!: apply request-parameter checks consistently across body, path and form inputs
16a630f edc38ea build(deps): update ddtrace to the 4.x line
29fb706 ffd6ac5 fix(deps): raise aiohttp floor to 3.14.2 to clear pooled-connection timeouts

485d5c1 is the one adapted pick. On staging it edits an existing _resolve_db_litellm_param helper, dropping the _DB_LITELLM_PARAM_ENV_REF_KEYS allowlist and the _db_model_is_team_scoped short-circuit so that resolution applies to every parameter. This line has none of that machinery, its two decrypt loops in proxy_server.py just call decrypt_value_helper inline, so the pick transposes staging's resulting three-line helper onto those two loops instead of trying to apply a diff against a pre-image that is not there. The resulting helper is logically identical to the one on staging today, which matters because 09b9a2a stops expanding os.environ/ references at request time and would otherwise leave DB-stored Bedrock and SageMaker deployments holding literal os.environ/... strings. dae4679 merged on staging as a six-commit merge commit and is picked with -m 1, so it lands squashed. Every other pick is a straight squash pick

dae4679 also adds one line to ruff.toml. This line enables PLR0915 while staging does not, and the pick takes common_processing_pre_call_logic from 50 statements to 51, so ruff check goes red without it. The exemption follows the six entries this file already carries for the same rule, and it keeps the picked source byte-faithful instead of restructuring a function the pick only appends two lines to

5d137ed and 3698031 also touch pyproject.toml. The ddtrace pick collapses the version-split constraint this line predates, and the aiohttp pick raises the floor so a future relock cannot walk back below it. Both were already merged on staging and are picked, not re-derived

Routine dependency maintenance on top, each a lock-only regeneration whose moved set is confined to the target package and its own new requirement floors: mcp 1.28.1, pypdf 6.14.2, pyasn1 0.6.4, gitpython 3.1.58, soupsieve 2.8.4, httplib2 0.32.0, h2 4.4.1, langgraph-checkpoint 4.1.1, setuptools 83.0.0, cryptography 50.0.0, Pillow 12.3.0. Every target is at or below what staging already resolves, so upgrading from this patch to a later release never moves backwards. cryptography and Pillow needed a pyproject range change to reach their target; the rest stayed inside the existing ranges

Finally bump: version 1.89.6 → 1.89.7 from cz bump and a uv.lock refresh whose only moved entry is litellm itself

QA runbook

Everything below was run on this branch against a baseline captured on the pristine stable/1.89.x tip first, since an old line carries its own test noise and only the delta means anything

Mapped test files for every touched module: 1045 passed against a 934-passed baseline, with the same three pre-existing failures on both sides and zero new ones. The three are test_ProxyConfig_get_model_info_with_id_missing_model_id_raises, test_project_guardrails_only, and test_request_guardrails_do_not_override_key_guardrails, all already red on the bare tip

The full tests/test_litellm suite was run on both trees the way make test-unit runs it. The tip failed 67 tests out of 21536, and this branch failed 66 out of 21649, so the branch adds 113 passing tests and nets one fewer failure. Eight node ids appear on this branch's failure list that were not on the tip's, and all eight pass on both trees when the failing test, and again when its whole file, is run on its own, so they are cross-test pollution under -n 4 rather than anything this branch changed. Nine node ids moved the other way for the same reason

Symbol closure: every changed module imports cleanly at runtime (15 of 15), and pyflakes reports exactly the same hard findings on this branch as on the tip, so no pick references an identifier the older code lacks and no path it adds is dead on arrival

Format and lint: this line gates on black --check at 88 columns rather than staging's ruff format at 120, so every picked hunk was reflowed with the line's own black. Every target behind make lint was run on both trees and compared. black --check names the same fifteen files on this branch as on the tip and none of them are files this branch touches, ruff check passes, the circular-import check passes, from litellm import * succeeds, and uv lock --check reports the lock in sync

Provenance: 22 commits, zero merge commits, zero litellm/proxy/_experimental/out/ paths. All nine picks carry a -x footer resolving to a commit reachable from litellm_internal_staging, and each pick's author matches its staging counterpart. The thirteen footerless commits are exactly the tool-generated ones

A live proxy on this branch was exercised against the same proxy booted from the pristine tip, using real provider traffic on both, and the DB-sourced model path was replayed against a single shared Postgres so both proxies read byte-identical rows

yucheng-berri and others added 22 commits August 8, 2026 00:47
…als (#30585)

* fix: validate proxy request body and nested fields

Ensure caller-supplied request fields cannot override server-side deployment
configuration, and apply request-body validation consistently to nested
structures. Adjusts router kwarg handling and client-side credential handling
for base-url overrides

* test: cover router strip ordering and advisor clientside credential gate

* fix: clear deployment credentials on client base-url override

When a request overrides api_base/base_url, recompute the deployment's
litellm_params (clearing the deployment's own api_key) and drop the cached
client built for the original endpoint, so the deployment credential is not
reused for the client-supplied endpoint. Adds regression tests that assert the
credentials actually forwarded to litellm.completion/acompletion.

* fix(proxy): require api_key alongside api_base override

A request that overrides api_base/base_url but supplies no api_key still
left the proxy carrying a server credential: once the override clears the
banned-param opt-in, the provider re-resolves a key from the environment
(api_key or get_secret("OPENAI_API_KEY") and ~30 sibling chains in
main.py) and forwards it to the caller-controlled URL. Popping the
deployment api_key only changed which server key leaked.

Gate is_request_body_safe so a permitted api_base/base_url override must
also carry a non-empty caller api_key; reject otherwise. The env
resolution in main.py is left as the provider boundary.

* fix(proxy): extend request-body banlist with five additional credential and session targeting fields

Yuneng's review found five deployment-owned request-body params still missing
from the denylist and the router strip set. Each lets a caller reach the
operator's provider credentials or retarget the outbound request:
aws_profile_name selects a local AWS profile, oci_compartment_id and oci_region
retarget the OCI request, litellm_credential_name selects any server-loaded
credential by name with no ownership check, and runtimeSessionId resumes a
Bedrock AgentCore runtime session (AWS does not enforce session-to-user
mapping, so this is a cross-tenant session-resume vector).

Add all five to _BANNED_REQUEST_BODY_PARAMS in auth_utils.py and to
_DEPLOYMENT_OWNED_CREDENTIAL_KWARGS in router.py. Deployment litellm_params and
SDK direct calls are unaffected: the banlist gates the request body only, and
the router strip drops caller kwargs, never deployment["litellm_params"].

* test: rename arbitrary canary values in security tests to neutral placeholders

* fix(proxy): apply api_key co-presence to nested base override and warn on Router credential strip

P1-A: is_request_body_safe descended into _NESTED_CONFIG_KEYS
(litellm_embedding_config, extra_body) for the banned-param check but not for
the api_key co-presence check, so a base override smuggled into one of those
nested dicts cleared the client-side-credentials opt-in without a paired
api_key and let the provider re-resolve a server credential from the
environment. Run _check_base_override_has_api_key on each nested config dict
too, so the requirement applies wherever a base override is permitted.

P1-B: the deployment-owned credential strip in the Router runs unconditionally
on every _completion/_acompletion, which is security-correct but silently
drops per-call api_version/vertex_project/etc. for SDK Router callers. Emit a
single warning (key names only, never values) when the strip removes a
non-empty value, so the backwards-incompatible behavior is visible without
gating the strip on a context flag that does not exist.

* fix(proxy): apply api_key co-presence to tool-entry base override

is_request_body_safe scans three surfaces (root, _NESTED_CONFIG_KEYS, and
tools[]); the previous commit extended the api_key co-presence rule to root
and nested config dicts but not to tool entries. With
allow_client_side_credentials enabled, a tool entry carrying api_base/base_url
and no paired api_key cleared the gate, letting a provider interceptor fall
back to a server-side credential for a caller-controlled URL. Add the same
_check_base_override_has_api_key call to each tool dict and its nested function
dict, mirroring the symmetry already applied to the nested config keys. The
rule is unchanged: api_key must live in the same dict as the base override it
accompanies.

* test(proxy/auth): require paired api_key under extra_body opt-in

* fix(router): gate deployment-owned kwarg strip on litellm.proxy_is_running

* fix(advisor): narrow proxy-import guard to ImportError-family

* fix(router): gate api_key clear on base override behind litellm.proxy_is_running

* test(proxy/auth): scope proxy_is_running flag to dynamic-params class with autouse fixture

* style: use built-in generics in PR-added type annotations

* revert: drop proxy_is_running flag and router-level credential strip; rely on proxy gate

* revert: scope PR to LIT-3828 + LIT-3834 only; drop LIT-3830/LIT-3833 changes

* style: black-format advisor orchestration test

(cherry picked from commit 1667b8f)
…30867)

AWS auth parameters in the Bedrock and SageMaker path could be expanded against
the process environment when credentials were built. Config-sourced references
are already expanded at load time, so restrict expansion to that path: a
reference still present at request time is treated as caller-supplied input and
is left as-is, and the web-identity helper rejects environment-variable
references before resolving the token.

Also rework the ambient AWS_* fallback as a single pass that pairs each value
with its own env-var name, fixing a latent index misalignment that left
AWS_EXTERNAL_ID unresolved.

Adds regression tests covering the resolution behavior.

(cherry picked from commit 4ef7d08)
…ging override (LIT-3587) (#31905)

The security fix in 34e9be1 removed turn_off_message_logging from
_supported_callback_params to stop callers bypassing global redaction via
the request body. That also killed the documented admin-only per-key or
per-team override because both flows resolve through the same allowlist
in initialize_standard_callback_dynamic_params.

Put turn_off_message_logging back in _supported_callback_params so an
admin-configured metadata.logging[].callback_vars.turn_off_message_logging
survives into StandardCallbackDynamicParams and can override the global
setting for that key or team, as documented at
docs/proxy/team_logging#disableenable-message-redaction.

Consolidate the metadata traversal so the extractor and the proxy strip
walk the same set of client-controllable slots. iter_client_callback_metadata_dicts
in litellm_core_utils/initialize_dynamic_callback_params.py is the single
source of truth for metadata, litellm_metadata, and litellm_params.metadata;
_strip_client_message_redaction_opt_out imports it so a future addition
to one side automatically reaches the other. The extractor iterates the
helper in reversed order so litellm_params.metadata keeps overriding
metadata, matching the pre-refactor merge precedence.

Client bypass stays blocked. Restoring the field re-enrolls it in the
auth layer's _BANNED_REQUEST_BODY_PARAMS (derived from
_supported_callback_params via _build_banned_observability_params), so
client submissions at the top level, inside metadata, or inside a
JSON-string litellm_metadata all 401 at ingress. is_request_body_safe
also now descends into litellm_params.metadata for the same 401 defense
against the nested-body attack vector, matching how the metadata and
litellm_metadata slots are handled. _strip_client_message_redaction_opt_out
runs after the litellm_metadata JSON parse and before the admin callback_vars
unpack, so admin values survive while any leftover client-supplied
opt-out is dropped when global redaction is on and the key or team
lacks allow_client_message_redaction_opt_out.

Flip the two dynamic-param e2e tests added by the security fix to
reflect the restored override behavior, keeping the invariant that
proxy client bypass is stopped by the auth layer 401 above.

Co-authored-by: yucheng <yucheng@yuchengs-MBP.attlocal.net>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
(cherry picked from commit 8e6098a)
…advisor tool (#32093)

* fix(anthropic): require caller api_key and SSRF-validate api_base in advisor tool

The advisor_20260301 interceptor honored a caller-supplied api_base once
allow_client_side_credentials was enabled, even without a caller-supplied
api_key. AnthropicModelInfo.get_auth_header() then fell back to the proxy's
own ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN, so the server's real credentials
plus the conversation history got sent to a caller-chosen destination

_resolve_advisor_credentials() now only honors api_base alongside a
non-empty caller-supplied api_key, requires the https scheme, and validates
api_base via validate_url() before use, mirroring check_complete_credentials
in auth_utils.py. https is required because validate_url only DNS-pins the
connection for http; for https with TLS verification on it returns the URL
unchanged and relies on certificate validation to block DNS rebinding

* fix(anthropic): also reject advisor api_base when ssl_verify is disabled

validate_url only DNS-pins the connection for http, or for https with
litellm.ssl_verify disabled; the previous https-only check missed the
ssl_verify=False case, where validate_url's rewritten URL was still being
discarded, per Greptile's review of this PR. Reject api_base outright when
ssl_verify is False so the discarded rewrite can no longer matter

(cherry picked from commit 07b9ea8)
Root cause: PR #30867 removed request-time os.environ/ expansion in
BaseAWSLLM.get_credentials. That is only safe if config-load pre-resolves
os.environ/ refs so the value reaching get_credentials is already the real
secret. The YAML config path has always done this. The DB-load path
(ProxyConfig._resolve_db_litellm_param) only re-expanded keys in a hardcoded
whitelist (_DB_LITELLM_PARAM_ENV_REF_KEYS) plus short-circuited env-ref
resolution entirely for team-scoped rows. PR #32256 extended that whitelist
to 18 keys to unblock a customer whose Bedrock model with aws_role_name:
os.environ/BEDROCK_ASSUME_ROLE_ARN broke on v1.90+, but the whitelist is
structurally fragile: every future auth field breaks the same way until
someone remembers to add it

Fix: remove the whitelist and the team-scope short-circuit. The DB-load
resolver now expands os.environ/ on every string field, matching the YAML
path. Trust boundary stays on the write side: only PROXY_ADMIN can create
team_id=None rows, only team admins of a team can create rows scoped to
that team, and the request-body vector is still blocked by
_BANNED_REQUEST_BODY_PARAMS. Team-scoped rows now resolve env refs — this
is a deliberate LIT-3831 threat-model expansion trusting team admins for
env-var reads

Regression tests in tests/test_litellm/proxy/proxy_server/test_proxy_config.py:
- test_ProxyConfig__add_deployment_resolves_env_refs_after_db_decrypt pins
  admin-scoped rows resolve every field (previously api_base stayed literal)
- test_ProxyConfig__add_deployment_resolves_team_env_refs pins team rows
  resolve env refs (previously stayed literal)
- test_ProxyConfig__add_deployment_resolves_env_refs_on_arbitrary_field pins
  the no-whitelist invariant against a made-up field name
- test_ProxyConfig__add_deployment_resolves_env_refs_for_aws_bedrock_auth_params
  (from #32256) still passes
- Path B counterparts (decrypt_model_list_from_db) mirror the above

Left as followups (not fixed here):
- /model/info and /v2/model/info still echo resolved values for fields not
  in the current pop-list (aws_role_name, aws_sts_endpoint, api_base, etc.).
  Fix is to extend remove_sensitive_info_from_deployment; separate PR
- Master-key rotation reads DB rows via decrypt_model_list_from_db which
  now resolves universally, so rotation collapses env-refs into hardcoded
  values. Pre-existing bug for the 6 previously-whitelisted fields; wider
  surface after this PR. Separate PR

(cherry picked from commit 5862be3)
… path and form inputs (#36011)

Backport of the #36011 batch; picked from its merge commit on internal staging.

(cherry picked from commit c898d34)
A single ddtrace constraint now covers every supported Python version, so this collapses the version split introduced in #33438. Also aligns the build_from_pip image pin and updates the type-only Tracer import to its current module path

(cherry picked from commit edc38ea)
…imeouts

aiohttp 3.14.0 and 3.14.1 re-arm the sock_read timer on a keep-alive
connection after it has already been returned to the idle pool. The stray
timer stamps a SocketTimeoutError on the pooled connection without closing
it, so the pool keeps handing it out and the next request to pick it up
fails instantly on an error left behind by an earlier, unrelated request.
Because a single pool is shared across providers, the failures appear
simultaneously across Vertex AI, Bedrock, Anthropic and OpenAI-compatible
deployments as sub-millisecond "Connection timed out" errors.

uv.lock resolved aiohttp 3.14.1 and the published images install via
`uv sync --frozen`, so every image built from that lock shipped the
regression. The wheel's own metadata declared `aiohttp>=3.10,<4.0`, which
also left pip consumers free to resolve into the same broken window, so
both the runtime floor and the uv constraint move to >=3.14.2.

Upstream fixed this in aio-libs/aiohttp#12954, released in aiohttp 3.14.2;
the lock now resolves 3.14.3. Raising the floor rather than capping below
3.14 keeps the advisories that the existing 3.14.1 floor cleared, so no
osv-scanner ignores are needed. litellm requires Python >=3.10 and aiohttp
3.14.2 requires >=3.10, so no supported interpreter loses support.

Both new tests fail on the previous pins and pass on these.

(cherry picked from commit ffd6ac5)
@yuneng-berri
yuneng-berri requested a review from a team August 8, 2026 23:06
@greptile-apps

greptile-apps Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This patch releases LiteLLM 1.89.7 with backported proxy security hardening, callback-redaction behavior, DB model parameter resolution, provider URL handling, AWS credential handling, and dependency updates.

  • Applies request validation consistently to model destinations, fallback targets, nested metadata, and connection-test overrides.
  • Gates and validates caller-provided Anthropic advisor credentials.
  • Resolves environment references in DB-sourced model parameters while preventing request-time AWS environment expansion.
  • Updates aiohttp, ddtrace, cryptography, Pillow, and regenerated lockfile dependencies.
  • Adds regression coverage for the changed proxy, provider, callback, and dependency behavior.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code defect remaining after review.

The request-validation, authorization, credential-isolation, provider, and dependency changes have matching regression coverage, and the security leads identified in unchanged dependency versions are not introduced or worsened by this patch.

Important Files Changed

Filename Overview
litellm/proxy/auth/auth_utils.py Adds recursive validation for fallback destinations, nested form metadata, and forbidden request parameters without leaving a demonstrated supported-shape bypass.
litellm/proxy/auth/user_api_key_auth.py Reuses the canonical fallback traversal for model-allowlist enforcement across supported fallback shapes.
litellm/llms/anthropic/experimental_pass_through/messages/interceptors/advisor.py Gates caller-supplied advisor credentials and validates custom endpoints before use.
litellm/llms/bedrock/base_aws_llm.py Stops expanding caller-supplied environment references while retaining ambient AWS fallback behavior.
litellm/proxy/litellm_pre_call_utils.py Centralizes URL-destination rejection and strips unauthorized message-redaction opt-outs from all supported metadata locations.
litellm/proxy/health_endpoints/_health_endpoints.py Prevents configured credentials from being combined with caller-overridden connection destinations unless explicitly enabled.
litellm/proxy/proxy_server.py Consistently decrypts DB model parameters and resolves stored environment references before model construction.
pyproject.toml Bumps the release and dependency constraints, including the aiohttp floor and ddtrace major version.
uv.lock Regenerates the lockfile for the release and dependency maintenance; scanner-listed vulnerable package versions were unchanged from base.

Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.89.7" | Re-trigger Greptile

@yuneng-berri
yuneng-berri enabled auto-merge August 8, 2026 23:10
if isinstance(decrypted_value, str) and decrypted_value.startswith(
"os.environ/"
):
return get_secret(decrypted_value)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critical: Team models can resolve arbitrary server secrets

A team admin can create a team-scoped model with an arbitrary parameter such as some_future_field: os.environ/LITELLM_MASTER_KEY. This helper resolves it, and decrypt_model_list_from_db() returns the arbitrary field through model-info responses without generic redaction, allowing the team admin to retrieve the master key and authenticate as a proxy admin. Restrict environment-reference expansion to proxy-admin-authored rows or explicitly approved fields, and reject os.environ/ values when team models are created or updated.

@veria-ai

veria-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

PR overview

This release PR backports seven changes to the stable/1.89.x branch and prepares version 1.89.7. The affected proxy code includes handling of model configuration loaded from the database.

One critical security issue remains open, with none addressed so far. A team administrator can use a team-scoped model configuration to resolve and expose arbitrary server environment secrets, including the proxy master key, enabling escalation to proxy administrator privileges.

Open issues (1)

Fixed/addressed: 0 · PR risk: 9/10

@yuneng-berri
yuneng-berri merged commit bcff771 into stable/1.89.x Aug 8, 2026
4 of 5 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_backport_1_89_x_bp-189x-0808sec branch August 8, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants