chore(release): backport #33592, #33853 to stable/1.91.x and cut 1.91.4 - #33891
Conversation
|
|
Greptile SummaryThis backport restores two Docker fixes onto stable/1.91.x and cuts version 1.91.4. It carries zero Python source changes — the proxy's runtime behavior is identical to v1.91.3.
Confidence Score: 5/5Safe to merge — all changes are confined to Dockerfiles and the version/lock files, with no Python source modifications. The patch is a clean cherry-pick of two Docker fixes onto a stable branch. The prisma-baking approach is self-verifying: build-time No files require special attention.
|
| Filename | Overview |
|---|---|
| Dockerfile | Bakes prisma CLI and engines at /opt/prisma in the builder and copies to runtime; restores /app/litellm-proxy-extras; adds build-time assertions to guard correctness |
| docker/Dockerfile.database | Same prisma-baking and litellm-proxy-extras restore as the main Dockerfile; identical pattern applied to the database-specific image |
| docker/Dockerfile.non_root | Only adds the /app/litellm-proxy-extras restore; intentionally keeps the existing runtime-generation prisma approach for the non-root uid image |
| pyproject.toml | Version bump from 1.91.3 to 1.91.4; no logic changes |
| uv.lock | Lock-only refresh for mcp 1.28.1 and soupsieve 2.8.4 bumps plus version bump regeneration |
Reviews (2): Last reviewed commit: "chore: refresh uv.lock for 1.91.4" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if model_cost_info is not None: | ||
| cost_map_input = model_cost_info.get("max_input_tokens") | ||
| if cost_map_input is not None: | ||
| max_input_tokens = int(cost_map_input) | ||
| cost_map_output = model_cost_info.get("max_output_tokens") | ||
| if cost_map_output is not None: | ||
| max_output_tokens = int(cost_map_output) |
There was a problem hiding this comment.
int() casts on cost-map values are unguarded, unlike the _as_int helper introduced in router.py. The static cost map is well-controlled, but a custom pricing override via environment variable or a future entry with an unexpected type (e.g. a float string "128,000") would propagate a ValueError all the way up and turn the /v1/models listing into a 500 — the exact failure mode the rest of this PR hardens against.
| if model_cost_info is not None: | |
| cost_map_input = model_cost_info.get("max_input_tokens") | |
| if cost_map_input is not None: | |
| max_input_tokens = int(cost_map_input) | |
| cost_map_output = model_cost_info.get("max_output_tokens") | |
| if cost_map_output is not None: | |
| max_output_tokens = int(cost_map_output) | |
| if model_cost_info is not None: | |
| cost_map_input = model_cost_info.get("max_input_tokens") | |
| if cost_map_input is not None: | |
| try: | |
| max_input_tokens = int(cost_map_input) | |
| except (TypeError, ValueError): | |
| pass | |
| cost_map_output = model_cost_info.get("max_output_tokens") | |
| if cost_map_output is not None: | |
| try: | |
| max_output_tokens = int(cost_map_output) | |
| except (TypeError, ValueError): | |
| pass |
|
Converting to draft: the live replay found a case where this branch regresses the listing relative to v1.91.3. A deployment whose model_info carries a malformed token limit (for example max_input_tokens "128,000") gets registered into the model cost map by the router, so create_model_info_response now reaches the bare int() in its cost-map branch and the whole /v1/models listing returns 500; on the 1.91.3 tip the same config returns 200 and simply omits that deployment's limits. #33864 hardened the router index path but not this cost-map path, and the same latent issue exists on litellm_internal_staging with the same config. Per the staging-first rule the fix lands on staging and gets cherry-picked here before this PR leaves draft |
#33592) * fix(docker): restore litellm-proxy-extras source dir in runtime images #30243 narrowed the runtime stage to an allowlist COPY, which dropped /app/litellm-proxy-extras from the published images. Downstream migration jobs point prisma migrate deploy at that path; with the schema gone (or a schema with no adjacent migrations dir, where prisma exits 0 without applying anything) those jobs went green while never migrating the database. Restore the folder in all three runtime stages and assert in image-scan that the schema and a non-empty migrations dir ship at the source path * chore(ci): drop image-scan migration-assets assertion (cherry picked from commit 111d447)
…migrations work for any uid offline (#33853) * fix(docker): bake prisma CLI and engines at a fixed path so fresh-DB migrations work for any uid offline The runtime image shipped the prisma CLI and engines under /root/.cache, the default HOME-derived prisma-python cache location. Any deployment whose runtime HOME is not /root (kubernetes runAsUser, docker --user, HOME overrides) missed that cache on a fresh database, fell back to a nodeenv Node download that crashes on Wolfi (libatomic.so.1), and started the proxy with zero tables while every DB-backed endpoint returned 500 The bake now lives at /opt/prisma, a path no HOME resolution or cache volume mount can shadow. The builder records the engine paths there at generate time, and the runtime stage pins PRISMA_BINARY_CACHE_DIR, PRISMA_CLI_PATH, PRISMA_CLI_QUERY_ENGINE_TYPE=binary and PRISMA_OFFLINE_MODE so both litellm-proxy-extras and prisma-python resolve the baked CLI and engines directly. prisma migrate deploy on a fresh database now needs no npm and no network access for any runtime uid, including readOnlyRootFilesystem deployments Verified against live containers: fresh and existing databases as root, uid 12345, HOME overridden, on an internal-only docker network, and with a read-only root filesystem all migrate and serve /team/new successfully Fixes #33650, #24554 * chore(docker): fail the image build if the baked prisma CLI layout drifts Asserts the baked CLI shim is executable and its entrypoint exists in the runtime stage after the COPY and chmod, so a layout change in a future prisma-python release breaks the image build loudly instead of silently degrading the migration path at container startup (cherry picked from commit 567ebcb)
6e853da to
2829cab
Compare
yuneng-berri
changed the title
3 participants
Relevant issues
Backports two Docker fixes onto stable/1.91.x and cuts 1.91.4 (v1.91.3 is already published on DockerHub and GHCR, so the line bumps). #33592 restores the /app/litellm-proxy-extras source dir that the runtime-stage allowlist COPY dropped; downstream migration jobs point prisma migrate deploy at that path, and with the schema gone they went green while never migrating the database. #33853 bakes the prisma CLI and engines at /opt/prisma so fresh-database migrations work for any runtime uid with no network access (#33650, #24554). Two dependency maintenance bumps ride along. This PR carries zero Python source changes, so the proxy's runtime behavior is byte-for-byte that of v1.91.3
Scope note: #33721 and #33864 were part of the original pick set and were dropped after live verification on this line found that a deployment with a malformed configured token limit turns the whole /v1/models listing into a 500 through the cost-map path (details in the PR comments). The same gap exists on litellm_internal_staging; those two picks return in a follow-up backport once the staging-side fix lands
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)What is included
Both picks are patch-identical to their litellm_internal_staging squashes and carry
-xfooters. The dep bumps are lock-only regenerations produced with the line's own uv (0.11.7, the Dockerfile UV_IMAGE pin); each moved exactly its target package and nothing elseKnown noise on this line
tests/test_litellm/proxy/test_proxy_utils.py::test_get_custom_url fails on the bare stable/1.91.x tip before any pick (custom URL assertion, environment dependent). It is the only failure in both the baseline and post-pick targeted runs
Screenshots / Proof of Fix
The branch has no Python diff against the line tip, and the live proxy confirms behavior preservation. GET /v1/models on a proxy running this branch is identical to the v1.91.3 baseline (41 models, same token limits on every entry), a live chat completion succeeds, and a config carrying a deployment with malformed token limits (max_input_tokens "128,000") returns HTTP 200 with that deployment's limits omitted, matching the 1.91.3 tip
Targeted suite (5 files covering the proxy model-listing surface plus the router): baseline 195 passed / 1 failed (the known-noise test), post-pick 195 passed / 1 failed (same test), identical failure set
Dependency bumps verified in the line's environment: mcp resolves 1.28.1, soupsieve resolves 2.8.4
All three runtime images (Dockerfile, docker/Dockerfile.database, docker/Dockerfile.non_root) were built locally from this branch and the shipped paths asserted inside each image: /app/litellm-proxy-extras/litellm_proxy_extras/schema.prisma present with 128 entries in the adjacent migrations dir in all three, and in the Dockerfile and Dockerfile.database images /opt/prisma/binaries/node_modules/.bin/prisma is executable, prisma/build/index.js is present, and the four PRISMA_* env pins are set in the image config. #33853's build additionally bakes
test -xassertions that fail the image build if the prisma bake is wrong, and all three builds passedA full mirrored-suite delta against the line baseline and a behavioral gauntlet run are completing; their results will be appended here
Type
Bug Fix (backport) + dependency maintenance
Changes
Cherry-picks of the two staging Docker squashes above onto stable/1.91.x, two lock-only dependency regenerations, and the 1.91.4 version bump with its lock refresh
QA runbook
prisma migrate deployfrom the database image as a non-root uid with no network; expect migrations to applyFinal Attestation