chore(release): backport #31036, #31342, #31653 to stable/1.90.x and cut 1.90.1 (litellm-enterprise 0.1.43.post1)

[yuneng-berri]

Relevant issues

Backports three already-merged staging fixes onto stable/1.90.x and cuts 1.90.1, plus the litellm-enterprise 0.1.43.post1 required because one pick touches an enterprise source file. The headline reason is feature parity on upgrade: stable/1.88.x and stable/1.89.x carry the streaming Vertex batch-upload feature (#31036) but stable/1.90.x does not, so a user upgrading from 1.89.x to 1.90.x silently loses large-file Vertex batch uploads. This brings #31036 to the line together with its follow-up #31653, so 1.90.x gets the fixed version of the feature (a single streaming media upload, not the chunked-resumable path that caused 499s) rather than the buggy interim state the older lines still carry. It also picks an unrelated client-side reliability fix (#31342) that scrubs a secret-bearing query param out of key/info error strings.

There is no litellm double-bump risk: 1.90.0 is already released (tag v1.90.0 and release/v1.90.0 both exist), so the picks cut 1.90.1. 0.1.43.post1 is the correct enterprise increment: 0.1.43 is published and 0.1.44 is staging's next, so the post-release sorts 0.1.43 < 0.1.43.post1 < 0.1.44.

What is included

In cherry-pick (merge) order:

• fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads #31036 fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (cherry-picked with -x from staging 56825926af7)
• fix(proxy/client): redact api key from key/info client error messages #31342 fix(proxy/client): redact api key from key/info client error messages (-x from 71ee1a8)
• fix(vertex_ai/files): single media upload for batch files to fix 499s on large uploads #31653 fix(vertex_ai/files): single media upload for batch files to fix 499s on large uploads (-x from 85840ae)
• chore(release): bump litellm-enterprise 0.1.43 -> 0.1.43.post1
• bump: version 1.90.0 → 1.90.1
• chore: refresh uv.lock for 1.90.1 and litellm-enterprise 0.1.43.post1

Adaptation notes

Two picks are ADAPTED; #31342 applied verbatim.

#31036: the OOM-fix core (the vertex transformation and handler, the files endpoint, the router and batch utilities, the types and the enterprise managed-files hook) and all test files, including the new 696-line streaming suite, applied as on staging. One file diverges: litellm/llms/custom_httpx/llm_http_handler.py had an import-block conflict from the pick's import-sort. Resolved by keeping the streaming body and dropping three imports the sort carried but that this line's code never references: functools.lru_cache, litellm.litellm_core_utils.asyncify.run_async_function, and a duplicate urllib.parse line. Their only consumer on staging, _responses_api_optional_request_param_names, is not part of #31036 and is absent on this line, so importing them here would be unused-import violations. Unlike the 1.88.x/1.89.x backports, RealtimeQueryParams is genuinely used on this line, so it was kept.

#31653: the upload-path rewrite conflicted against #31036's just-landed resumable code (ten regions in llm_http_handler.py and one in transformation.py), because #31653's upstream base also carried intermediate staging commits this line does not (a repo-wide ruff reformat and unrelated realtime/websearch fixes). Resolved by taking #31653's streaming_media_upload rewrite for every region; the removed resumable code is wider-wrapped on this line, which accounts for the higher deletion count versus the upstream commit. The StreamingMediaUploadConfig and BaseFileUploadStream imports auto-merged. Verified no resumable-upload symbol remains referenced and the added test functions exactly match the upstream PR's own (no absorbed neighbor tests).

Known noise on this line

The targeted test set (each picked PR's own test files plus the mirrored tests for every touched source) was fully green on the line tip before any pick once the proxy extras are synced: 242 passed, 0 failed. The only pre-existing lint nit on a touched file is a bare # noqa (PGH004) in litellm/types/router.py, present on the line tip before the picks; ruff check on every touched module otherwise passes. The line's ruff format baseline is already dirty against its line-length = 120 config (the bulk of the code predates the width change), so no reformat was applied; ruff check, the enforced gate, passes.

Screenshots / Proof of Fix

Proxy booted on the worktree against real provider keys, before and after the picks.

Before the picks:

$ curl -sf "http://localhost:4010/health/liveliness"
"I'm alive!"
$ curl -s "http://localhost:4010/v1/chat/completions" -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"reply with exactly: baseline ok"}],"max_tokens":20}'
-> content: "baseline ok"

After the picks (proxy auto-reloaded on the picked code; confirms the streaming rewrite and the client redaction do not regress startup or serving):

$ curl -sf "http://localhost:4010/health/liveliness"
"I'm alive!"
$ curl -s "http://localhost:4010/v1/chat/completions" -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"reply with exactly: post-pick ok"}],"max_tokens":20}'
-> content: "post-pick ok"

Targeted test deltas against the 242-passing baseline: 329 passed / 2 skipped after #31036 (including the new streaming suite), 29 passed for #31342's client redaction suite, 326 passed / 2 skipped after #31653 (the net decrease is the resumable-upload tests being replaced by the media-upload tests; zero failures throughout). A behavioral verification pass with adversarial subagents returned a clean verdict on all three sub-claims: every identifier the picked diffs reference resolves on this line, each pick's own tests pass, and no existing caller of the modified functions (the vertex transformation, the handler create-file paths, the proxy client) regresses. The only dissent was a cosmetic stale docstring on StreamingMediaUploadConfig that exists upstream as well, not a behavioral defect.

The end-to-end large (1GB+) Vertex upload itself needs live GCS and Vertex service-account credentials, which were not available in this environment, so it is covered by the streaming suite rather than a live replay. To confirm on a real deployment: configure a Vertex AI model and a GCS bucket, then POST a >500MB batch JSONL to /v1/files with purpose=batch and submit it via /v1/batches; before #31653 the chunked-resumable upload accumulates hundreds of sequential round-trips and surfaces as a 499, after it the upload streams in one continuous media request and completes.

Type

🐛 Bug Fix
🚄 Infrastructure

Changes

Brings the streaming Vertex batch-upload feature (#31036) and its single-media-upload follow-up (#31653) to stable/1.90.x so upgrading from 1.89.x does not lose the feature, plus a client-side error-redaction fix (#31342), cutting 1.90.1 and the litellm-enterprise 0.1.43.post1 required because #31036 touches an enterprise source file.

[mateo-berri]

LGTM; thanks!

[greptile-apps]

Greptile Summary

This PR backports Vertex batch upload and proxy client reliability fixes to stable/1.90.x. The main changes are:

• Streams OpenAI batch JSONL uploads into Vertex AI GCS media uploads without buffering the full file.
• Adds metadata-only file handling and lazy JSONL iteration for large batch files.
• Updates batch rate limiting to count streamed rows, estimate uncountable rows, and validate embedded batch models.
• Redacts sensitive values from proxy client key error messages.
• Bumps litellm to 1.90.1 and litellm-enterprise to 0.1.43.post1 with refreshed lock metadata.
• Adds targeted tests for streaming Vertex uploads, proxy file endpoints, batch validation, router utilities, and client redaction.

Confidence Score: 5/5

Safe to merge based on the reviewed changes.

The changes are targeted backports with focused test coverage for the streaming upload paths, proxy file handling, batch utilities, and key redaction behavior. No blocking correctness or security issues were identified.

T-Rex Logs

What T-Rex did

• Recorded the vertex streaming upload workflow, capturing the base run and the head run, plus the helper script used for both runs.
• Executed two HTTP GET checks against the local server for key/info, observing 404 Not Found and 401 Unauthorized responses with a redacted secret, and both checks exited with code 0.
• Compared build configuration by inspecting pyproject.toml and uv.lock across base and head runs, noting the pyproject version bump from 1.90.0 to 1.90.1 and the enterprise version from 0.1.43 to 0.1.43.post1, with uv.lock entries aligned accordingly.

_{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.90.1 and li..." | Re-trigger Greptile}