Skip to content

chore(release): backport proxy request-handling fixes and refresh runtime deps for 1.93.2 - #36318

Merged
yuneng-berri merged 7 commits into
stable/1.93.xfrom
litellm_backport_1_93_x_bp-193x-0808sec2
Aug 8, 2026
Merged

chore(release): backport proxy request-handling fixes and refresh runtime deps for 1.93.2#36318
yuneng-berri merged 7 commits into
stable/1.93.xfrom
litellm_backport_1_93_x_bp-193x-0808sec2

Conversation

@yuneng-berri

Copy link
Copy Markdown
Collaborator

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Screenshots / Proof of Fix

Type

🐛 Bug Fix

Changes

Patch release 1.93.2 on top of v1.93.1. Every code commit is a cherry-pick -x of a commit already merged to litellm_internal_staging, so nothing here is new work written against the stable line.

Backported Staging commit Notes
chore(proxy): clean up request parameter validation and provider destination handling (#34189) 065faf6e69 adapted, see below
fix(proxy)!: apply request-parameter checks consistently across body, path and form inputs (#36011) c898d341c0 adapted, see below
fix(deps): raise aiohttp floor to 3.14.2 to clear pooled-connection timeouts ffd6ac52c5 source + tests picked, lock re-derived
chore(deps): upgrade cryptography to 50.0.0 (#35803) 49eb19c39f source picked, lock re-derived

Five PRs considered for this patch — #30585, #30867, #31905, #32093 and #32405 — are already on this line by direct ancestry. stable/1.93.x diverged from staging on 2026-07-11, after all five had merged, so they shipped in v1.93.0. Only #34189 and #36011 postdated the branch point.

#36011 is a merge commit carrying six underlying commits, so it is picked as cherry-pick -x -m 1. #32405's PR was single-commit and its discrete commit was already present.

Dependencies

Package 1.93.1 1.93.2 Staging
aiohttp 3.14.1 3.14.3 3.14.3
cryptography 48.0.1 50.0.0 50.0.0
mlflow (trio) 3.14.0 3.15.0 3.15.0
gitpython 3.1.54 3.1.58 3.1.58
h2 4.3.0 4.4.1 4.4.1
hpack 4.1.0 4.2.0 4.2.0

Lock deltas were re-derived on this line rather than taking the source commits' lock hunks. Accounting is complete and closed: 406 packages before and after, none added, none removed, exactly nine changed — the six above plus litellm itself for the version bump. The resulting lock carries the same advisory set as staging itself.

aiohttp moves both the runtime floor and the uv constraint to >=3.14.2, matching staging, because 3.14.0/3.14.1 re-arm the socket read timer on a keep-alive connection after it has returned to the idle pool — the stray timer marks a pooled connection timed out without closing it, so the next request to reuse it fails instantly on an unrelated request's error. One pool is shared across providers, so it surfaced as simultaneous sub-millisecond "Connection timed out" failures across Vertex AI, Bedrock, Anthropic and OpenAI-compatible deployments.

cryptography needs both halves of the staging change: the proxy extra's published range widens to >=49.0.0,<51.0 so pip install "litellm[proxy,mlflow]" stays resolvable downstream, while a [tool.uv] override-dependencies entry — a workspace setting that never reaches published metadata — holds our own lock and images at 50.0.0. mlflow moves to 3.15.0 deliberately rather than as incidental drift: 3.14.0 declares cryptography<49 while 3.15.0 declares <50, so holding it would have placed the lock two majors past mlflow's declared cap instead of one.

gitpython, h2 and hpack are lock-only and transitive. hpack is forced by h2 4.4.1's own metadata (4.3.0 required hpack>=4.1, 4.4.1 requires >=4.2), not incidental drift.

Adaptation notes

#34189 — the recursive_detector.py allowlist hunk also carried _redact_scanned_content, which does not exist on this line. Kept _iter_fallback_targets, dropped the other.

#36011 — three adaptations in the conflict resolution:

  • The conflict region carried a LITELLM_PASS_THROUGH_ENDPOINT_MARKER import that is pre-existing staging context, not something the pick adds (both upstream diffs render it without a leading +). This line has no pass-through marker machinery, so taking it verbatim is a hard ImportError on every proxy auth path. Dropped.
  • litellm_pre_call_utils.py needed Final added to its typing import, since the pick annotates allowed_hosts: Final.
  • _health_endpoints.py takes Mapping from typing rather than staging's collections.abc form, because this line already imports Iterable from typing and the verbatim line would produce a duplicate binding. Final is included in that same import — the pick adds a module-level _CONFIG_CONNECTION_FIELDS: Final[frozenset[str]], and since module-level annotations evaluate eagerly on Python ≤ 3.13, omitting it would fail at import and take the whole proxy down on four of the five supported interpreters.

All touched files are clean under ruff format --check, and ruff check --select F821 over litellm/ reports no undefined names.

Verification

Both picks were exercised behaviourally against the pre-patch tree, not just by test count:

  • Metadata supplied in bracket notation (metadata[api_base]) carrying a restricted parameter was accepted before this patch and is rejected after, matching its JSON-body equivalent. A clean-metadata control is unaffected on both.
  • #34189's fallback-destination helpers are absent before and present and functional after. A differential over 13 fallback shapes shows the patched validation is a strict superset of the previous behaviour, with no model-access coverage lost. Ordinary names — gpt-4, openai/gpt-4, bedrock/anthropic.claude-3-5-sonnet, azure/my-deployment, multi-slash fireworks_ai/accounts/fireworks/models/llama-v3 — are unaffected.

Regression checks against v1.93.1:

  • Broad matched pair over tests/test_litellm/proxy/ and llms/huggingface/, 12,700+ tests: seven patched-only failures, all seven reproducing identically on both trees in isolation — xdist ordering artifacts, pre-existing. Note that python-dotenv walks up from a worktree, so both trees must see the same .env or the comparison is invalid.
  • Picked test set: 370 passed on both Python 3.13 and 3.14, with three health tests erroring at setup on both trees because the Prisma client is not generated locally.
  • Removed symbols iter_router_fallback_model_names, ROUTER_FALLBACK_FIELDS and HuggingFaceEmbeddingConfig.get_api_base have no remaining references in litellm/, enterprise/ or tests/.
  • The two aiohttp guard tests fail on the previous pins and pass on these.

One behaviour change worth calling out for reviewers: a model identifier of the form oobabooga/http://host supplied together with an explicit api_base now routes to the model URL rather than to api_base, because the previous predicate tested for https and was False for plain http://. Other shapes are unchanged or strictly improved — the previous code produced schemeless URLs for names that merely contained the substring https. The same predicate change applies in the HuggingFace embedding handler. This is an SDK path; the destination guard itself is proxy-only, and every call site lives under litellm/proxy/.

yucheng-berri and others added 7 commits August 8, 2026 11:49
fix(proxy)!: apply request-parameter checks consistently across body, path and form inputs

(cherry picked from commit c898d34)
…imeouts

aiohttp 3.14.0 and 3.14.1 re-arm the sock_read timer on a keep-alive
connection after it has already been returned to the idle pool. The stray
timer stamps a SocketTimeoutError on the pooled connection without closing
it, so the pool keeps handing it out and the next request to pick it up
fails instantly on an error left behind by an earlier, unrelated request.
Because a single pool is shared across providers, the failures appear
simultaneously across Vertex AI, Bedrock, Anthropic and OpenAI-compatible
deployments as sub-millisecond "Connection timed out" errors.

uv.lock resolved aiohttp 3.14.1 and the published images install via
`uv sync --frozen`, so every image built from that lock shipped the
regression. The wheel's own metadata declared `aiohttp>=3.10,<4.0`, which
also left pip consumers free to resolve into the same broken window, so
both the runtime floor and the uv constraint move to >=3.14.2.

Upstream fixed this in aio-libs/aiohttp#12954, released in aiohttp 3.14.2;
the lock now resolves 3.14.3. Raising the floor rather than capping below
3.14 keeps the advisories that the existing 3.14.1 floor cleared, so no
osv-scanner ignores are needed. litellm requires Python >=3.10 and aiohttp
3.14.2 requires >=3.10, so no supported interpreter loses support.

Both new tests fail on the previous pins and pass on these.

uv.lock was re-derived on this line rather than taking the source lock
hunks; the resulting delta is aiohttp 3.14.1 -> 3.14.3 and nothing else.

(cherry picked from commit ffd6ac5)
Moves the proxy extra's cryptography floor from 48.0.1 to 49.0.0 and widens the
ceiling to <51, then holds the lock at 50.0.0 with a uv override

mlflow caps cryptography at <50 even in its newest release, so publishing a
plain >=50.0.0,<51.0 range would make `pip install "litellm[proxy,mlflow]"`
unresolvable for downstream consumers. Publishing >=49.0.0,<51.0 keeps that
combination installable (it resolves to 49.0.0), while the
override-dependencies entry, which is a uv workspace setting and never reaches
published metadata, keeps our own lock and Docker images on 50.0.0

mlflow only uses PBKDF2HMAC, AESGCM, Fernet, InvalidTag and hashes from
cryptography; none of those are affected by the 49 or 50 breaking changes, so
overriding its ceiling is safe in practice

cryptography 49 dropped its x86_64 macOS and 32-bit Windows wheels. Linux CI
and the Docker images are unaffected; developers on Intel Macs will build from
source

uv.lock was re-derived on this line rather than taking the source lock hunks.
The delta is cryptography 48.0.1 -> 50.0.0 and the mlflow trio 3.14.0 ->
3.15.0, matching the source commit. mlflow is moved deliberately and not as
incidental drift: 3.14.0 declares cryptography<49 while 3.15.0 declares <50, so
holding mlflow would have placed the lock two majors past its declared cap
instead of one. msal did not need to follow here and stays at 1.36.0.

(cherry picked from commit 49eb19c)
Lock-only move; gitpython reaches this tree transitively through
mlflow-skinny and is not a declared litellm dependency, so no range
needed widening. Re-derived on this line; the delta is gitpython
3.1.54 -> 3.1.58 and nothing else.
Lock-only move; h2 reaches this tree transitively through hypercorn and
is not a declared litellm dependency, so no range needed widening.

hpack moves 4.1.0 -> 4.2.0 alongside it. That is h2 4.4.1's own new
floor (4.3.0 required hpack>=4.1, 4.4.1 requires hpack>=4.2), not
incidental drift; the delta is confined to those two packages.
@yuneng-berri
yuneng-berri requested a review from a team August 8, 2026 23:23
@greptile-apps

greptile-apps Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This patch release backports proxy request-validation and destination-handling improvements while refreshing selected runtime dependencies and the lockfile

  • Extends validation across nested form metadata, fallback targets, path models, and health-check connection parameters
  • Corrects URL-model handling in the Hugging Face and Oobabooga adapters
  • Raises aiohttp and cryptography constraints and re-resolves related transitive packages
  • Adds regression coverage for the changed proxy and provider behavior

Confidence Score: 4/5

The PR appears safe to merge after addressing the two non-blocking inline comments that violate repository policy

The request-handling and dependency changes have no accepted blocking defect; the only retained issue is the addition of prohibited source-code comments

Files Needing Attention: litellm/proxy/health_endpoints/_health_endpoints.py

Important Files Changed

Filename Overview
litellm/proxy/auth/auth_utils.py Adds shared traversal and validation for nested metadata and fallback destinations without an accepted functional defect
litellm/proxy/auth/user_api_key_auth.py Reuses the shared fallback traversal for key-level model authorization
litellm/proxy/litellm_pre_call_utils.py Centralizes URL-valued destination validation for request identifiers
litellm/proxy/health_endpoints/_health_endpoints.py Prevents configured credentials from carrying into caller-overridden connections, but adds two comments prohibited by repository policy
litellm/proxy/common_request_processing.py Validates URL-valued path models before selecting the effective request model
litellm/llms/huggingface/embedding/handler.py Restricts direct model URL detection to actual HTTP and HTTPS prefixes
litellm/llms/oobabooga/chat/oobabooga.py Applies exact URL-prefix detection to chat and embedding destinations
pyproject.toml Publishes version 1.93.2 and updates aiohttp and cryptography dependency constraints
uv.lock Re-resolves the selected direct and transitive dependency versions for the patch release

Reviews (1): Last reviewed commit: "bump: version 1.93.1 → 1.93.2" | Re-trigger Greptile

UserAPIKeyAuth,
WebhookEvent,
)
from litellm.proxy.auth.auth_utils import (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 New inline suppression comments

The private imports on this line and line 59 add new inline suppression comments despite the repository policy prohibiting new source-code comments; remove both comments or avoid the private imports that require them

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@yuneng-berri
yuneng-berri merged commit cd1bd0f into stable/1.93.x Aug 8, 2026
6 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_backport_1_93_x_bp-193x-0808sec2 branch August 8, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants