[Infra] Promote internal staging to main - #26211
Merged
Merged
Conversation
Opt-in CLI flag, off by default, no env var. Only affects the uvicorn run path; gunicorn/hypercorn paths and prod (which doesn't pass the flag) are unaffected.
…itellm_feat-add-reload-flag-proxy
Adds a CI job that rebuilds the admin UI from source and fails if the committed static export at litellm/proxy/_experimental/out/ has drifted from what npm run build produces. This prevents silently shipping stale UI bytes and is a prerequisite for the non_root Dockerfile streamlining work, which will stage the UI from _experimental/out/ directly instead of rebuilding it inside the image. Also regenerates litellm/proxy/_experimental/out/ to match a fresh npm run build (Node 20.20.2) — the committed tree had drifted from source prior to this commit. Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
The checked-in Next.js static export at litellm/proxy/_experimental/out/ is kept fresh by the UI Drift Guard CI workflow. Stage it directly instead of re-running npm ci + npm run build inside the image. This removes: nvm install, node 20.20.2 install, npm ci (801 pkgs), next build, and the resulting intermediate node_modules/out tree. Build time: ~6m25s -> ~2m (fuse-overlayfs DinD); image 6.57GB -> 5.0GB. Behavior parity verified: API endpoints, UI screenshots (all 10 routes pixel-perfect), and Trivy HIGH/CRITICAL CVE count (6 -> 5, one npm GHSA removed) all match or improve over baseline. Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
npm was installed in the runtime only to globally install vulnerability patched versions of tar/glob/brace-expansion/minimatch/diff and to in-place rewrite npm's own bundled package.json. Both were to silence CVE scanners against modules that ship with npm itself. Since we no longer run npm anywhere in the runtime (Prisma uses the node binary directly for migrate deploy and generate), we can just skip installing npm in the first place. This eliminates both the ~25-line CVE-patch shuffle AND the underlying CVE surface. Kept: nodejs (needed by prisma-python's CLI and migrate deploy). Removed: npm apk package, all 'npm install -g', all find+sed patching, the redundant 'apk upgrade --no-cache nodejs' (already covered by the preceding 'apk upgrade'). Image: 4.97GB (opt-1) -> 4.97GB (opt-2); the real win is that two CVEs (CVE-2026-33671 and GHSA-q4gf-8mx6-v5v3) drop off the Trivy HIGH/CRITICAL list. No new CVEs introduced. API parity and UI visual regression both match baseline. Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
After Task 2.1 removed the in-image Next.js build, the builder stage no longer needs a full C/C++ + Clang toolchain. Keep gcc + python3-dev (required to compile ml-dtypes 0.4.1 from source — no wheel published for Python 3.13 yet). Drop everything else. Removed from apk: clang, llvm, lld, linux-headers, build-base, openssl-dev, npm. Removed NVM_DIR env and /root/.nvm from PATH (no nvm-based Node install anymore). Kept: python3, python3-dev, gcc, bash, coreutils, curl, openssl, libsndfile, nodejs. gcc (15.2) serves both C and C++; the separate g++ package doesn't exist in Wolfi. Image size unchanged (builder stage doesn't end up in the runtime); cold builds slightly slower due to ml-dtypes source compile, but that will be recovered in the next task via a BuildKit uv cache mount. API parity and UI visual regression both match baseline, Trivy HIGH/CRITICAL CVE count unchanged from opt-2 (4 CVEs, none new). Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
Mount /app/.cache/uv as a BuildKit type=cache on both 'uv sync' steps. The cache persists across builds on the same builder (and, when used with type=gha in CI, across CI runs) so repeat builds don't re-download every wheel. Side-effect: because the cache lives outside the image layer, the ~742MB of downloaded wheel archives that were previously baked into /app/.cache/uv drop out of the final image. Compressed image size goes from ~5.0GB to ~3.7GB, and the 'USER nobody' prisma-generate layer is 1.7GB vs 2.4GB. Warm-build timing: a uv-sync-invalidating edit now takes ~1m30s vs ~2m39s without the cache mount, on this dev VM. API parity and UI visual regression continue to match baseline. Trivy HIGH/CRITICAL: 6 at baseline -> 2 now, no new CVEs. Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
Five small, individually-verified cleanups collected into one commit: - Drop 'prisma migrate diff --from-empty ... > /dev/null 2>&1 || true' from the builder. Stdout/stderr/exit-status all discarded; nothing reads the output. Dead line. - Drop 'mkdir -p /app/.cache/npm' from the same RUN. npm is gone. - Drop the runtime's redundant 'sed -i' + 'chmod +x' on the entrypoint scripts. The builder already does the same three lines, and the runtime copies /app from the builder via COPY --from=builder, so the normalized files (and exec bits, which buildkit preserves) are already in place. - Drop NPM_CONFIG_CACHE and NPM_CONFIG_PREFER_OFFLINE from the runtime ENV — nothing reads them after Task 2.2 removed npm. - Drop '/.npm' and '/tmp/.npm' from the runtime's mkdir + chown. These directories only existed as npm's writable dirs for the non-root user; npm is gone. .dockerignore: add 'ui/'. After Task 2.1 the non_root image sources its UI bytes from litellm/proxy/_experimental/out/, so the whole ui/litellm-dashboard/ source tree is dead weight when the blanket 'COPY . .' pulls it into /app. Verified (with ripgrep) that no Python code under litellm/ opens any file under ui/. All string references to 'ui/...' are URL paths, not filesystem paths. Final image size: 6.57GB baseline -> 1.96GB. API parity and UI visual regression match baseline across all 12 API scenarios and 10 UI routes. Trivy HIGH/CRITICAL: 6 -> 2, no new CVEs introduced. Co-authored-by: yuneng-jiang <yuneng-berri@users.noreply.github.com>
…roxy feat(proxy): add --reload flag for uvicorn hot reload (dev only)
Revert the .dockerignore ui/ exclusion and remove the UI Drift Guard workflow. _experimental/out/ refresh is already handled by the release runbook; the global .dockerignore change also broke Dockerfile.custom_ui (explicit COPY ./ui/litellm-dashboard) and the enterprise-colors inline rebuild path in Dockerfile, Dockerfile.database, and Dockerfile.dev. Dockerfile.non_root itself is unchanged functionally — still stages the UI from the checked-in _experimental/out/. Only the companion workflow and global dockerignore exclusion are dropped.
prisma --version invokes the Schema Engine, which has no binary for the Wolfi base image (only debian). In the baseline this was silenced by a trailing || true wrapping the whole prisma chain; removing that wrapper uncovered the failure on arm64 builds. The main Dockerfile does not call prisma --version at all, so drop it here to match — prisma generate is sufficient to validate the toolchain.
* feat(scaleway): add SCALEWAY to LlmProviders enum * feat(scaleway): add audio transcription config and dispatch wiring Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(scaleway): add behavior tests for audio transcription config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(scaleway): advertise audio_transcriptions in endpoint-support JSON * docs(scaleway): document audio transcription support * fix(scaleway): address PR review — plain-text response_format + missing-key fail-fast Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(scaleway): cover new response paths, drop gettysburg.wav coupling Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor: new agentic loop event hook simplifies how to create logic for tool based multi llm calls * fix: compress - make it work on anthropic input as well * fix(compress.py): working prompt compression for claude code ensures claude code messages can run through proxy easily * docs: add agentic loop hook guide * docs: add agentic_loop_hook to sidebar * fix: fix multiple arguments error * fix: fix tool call loop for compression on streaming /v1/messages * fix: fix linting errors * fix: fix ci/cd errors * feat(litellm_pre_call_utils.py): use claude code session for litellm session id allows claude code logs to be stitched together, making it easy to know they were all part of the same conversation * fix: suppress incorrect mypy warning rE: module * revert: drop PR's changes to litellm/proxy/_experimental/out/ Restores the 34 HTML files under _experimental/out/ to their pre-PR paths (X/index.html -> X.html). All renames are R100 (content unchanged); no other files are touched. * fix: address greptile review comments on PR #25729 - Skip ``kwargs["tools"] = []`` injection when compression is a no-op — Anthropic Messages rejects empty tool arrays on requests that did not originally declare tools. - Move agentic-loop safety guards (fingerprint cycle / max depth) out of the per-callback try/except so they propagate instead of being swallowed by the generic exception handler. Extracted _check_agentic_loop_safety. - Gate generic ``x-<vendor>-session-id`` capture behind the LITELLM_CAPTURE_VENDOR_SESSION_HEADERS env var (off by default) to preserve backwards compatibility; explicit x-litellm-* headers are unaffected. - Fix monkeypatch target in pre-call-hook test to patch the actual module-level binding (litellm.integrations.compression_interception.handler.compress). - Add regression tests for empty-tools skip and opt-in session capture. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert: drop LITELLM_CAPTURE_VENDOR_SESSION_HEADERS flag Generic x-<vendor>-session-id header capture is a new feature and only runs *after* the explicit x-litellm-trace-id / x-litellm-session-id checks, so it does not change behavior for any existing caller that was already using the LiteLLM headers — no backwards-incompatibility to gate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(compress): replace input_type with CallTypes call_type Drop the bespoke ``CompressionInputType`` literal and use the existing ``litellm.types.utils.CallTypes`` enum instead. ``litellm.compress()`` now takes ``call_type: Union[CallTypes, str]`` (default ``CallTypes.completion``) — no new concept to learn, and the enum is already the way the rest of the codebase talks about request shapes. Supported values: ``completion`` / ``acompletion`` (OpenAI chat-completions shape) and ``anthropic_messages`` (Anthropic structured content blocks). Updated: compress(), the compression_interception handler, tests, docs, and the two eval scripts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add supported providers to prompt caching doc * Move Z.ai / GLM to cache_control marker list * Mark xAI models as supporting prompt caching * Narrow xAI prompt caching flag to models with documented cache pricing * Add prompt caching flag to grok-4, grok-4-0709, grok-4-latest --------- Co-authored-by: Michael Riad Zaky <michaelr@Michaels-MacBook-Air.local>
* fix: /health/readiness returns 503 when DB is unreachable due to handle_db_exception re-raising handle_db_exception() re-raises the Prisma exception inside _db_health_readiness_check's except block, which propagates out to health_readiness() and gets wrapped in a 503. The health endpoint never reached the reconnect path and the service never recovered. Fix: - Remove handle_db_exception() call from _db_health_readiness_check — that helper is for API request handlers (allow_requests_on_db_unavailable flag), not health checks - Replace raw disconnect()+connect() with attempt_db_reconnect(), which uses the proper lock, cooldown, escalation, and heavy-reconnect (recreate_prisma_client) machinery * test: update health readiness tests for handle_db_exception removal - Remove tests that expected handle_db_exception to re-raise (old buggy behaviour) - Remove tests asserting disconnect()/connect() calls (replaced by attempt_db_reconnect) - Add regression tests covering the 503 loop fix: - transport errors never raise (ClientNotConnectedError, httpx.ConnectError, etc.) - reconnect success path returns 'connected' - reconnect failure path returns 'disconnected' without raising - non-transport errors return 'disconnected', skip reconnect --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai>
…nbook The UI rebuild bundled into this PR is not needed for the Dockerfile change — the image simply copies whatever _experimental/out/ is in the tree. Regenerating here conflicts with the release-time refresh policy and adds ~100 files of review noise / merge-conflict risk for any concurrent UI PR.
…1 in anthropic messages passthrough test Anthropic retired claude-3-haiku-20240307 on 2026-04-20, causing the test_anthropic_messages_litellm_router_non_streaming_with_logging test to 404. Update the model references in this file to the current pinned haiku version.
The previous revert left three files under _experimental/out/_next/static/8Gn6tA2K4jsPxzCCMwObH/ still staged (git checkout origin/main -- only restores files present in main, it does not delete files that exist only on the PR branch). Remove them so the PR no longer touches _experimental/out/ at all.
[Fix] Apply black formatting to fix CI lint failures
…1 in streaming tests
…1 in local_testing part1 and router fallback tests
Run auth_ui_unit_tests against a per-job cimg/postgres:16.0 sidecar with DATABASE_URL pointing at localhost:5432, matching the pattern used by e2e_ui_testing. Seed the schema via 'litellm --skip_server_startup --use_prisma_db_push' so each run starts on a clean DB with the current schema.prisma.
* feat(router): add auto_router/quality_router for quality-tier routing (#25987) * feat(router): add auto_router/quality_router for quality-tier routing Adds a new auto-router type that routes a request to a model at a target quality tier. The quality tier is inferred by re-using the existing ComplexityRouter's classification, then mapped through an admin-configured complexity_to_quality table. Each candidate model declares its own quality_tier in model_info.litellm_routing_preferences. Resolution strategy: exact tier match, else round up to the next higher tier, else fall back to default_model. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com> * feat(quality_router): add capability-based filtering Each deployment can declare a `capabilities: List[str]` field in `model_info.litellm_routing_preferences` (e.g. ["vision", "function_calling"]). Requests can pass `litellm_capabilities` in `request_kwargs` to require specific capabilities — the router will only route to deployments whose declared capabilities are a superset. Resolution still walks tier (exact → round up), but at each tier filters by capability before picking. Falls back to default_model only when it also satisfies the required capabilities; otherwise raises rather than silently routing to a model that lacks a required capability. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com> * feat(quality_router): expose routing decision in response headers For transparency, expose the QualityRouter's routing decision in the proxy response headers: x-litellm-quality-router-model → picked model_name (e.g. "haiku-vision") x-litellm-quality-router-tier → resolved quality tier (e.g. "1") x-litellm-quality-router-complexity → ComplexityTier name (e.g. "SIMPLE") Mechanism: the pre-routing hook stashes the decision in request_kwargs["metadata"]["quality_router_decision"]. After the call returns, Router.set_response_headers lifts the decision into response._hidden_params["additional_headers"] alongside the existing x-litellm-model-group / x-litellm-model-id headers. Existing metadata keys (trace_id, user_id, etc.) are preserved. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com> * feat(quality_router): replace capabilities with keyword override Drops the capability-based filtering in favor of a keyword-based override for v0: - RoutingPreferences.keywords: List[str] (replaces capabilities) — each deployment can declare substring keywords. - If any declared keyword (case-insensitive) appears in the user message, the router short-circuits the complexity-classification flow and routes to the matching deployment. - Tiebreaker for overlapping keyword matches: quality_tier DESC, then cheapest model_info.input_cost_per_token ASC. Unpriced models lose ties to priced ones. Decision metadata + headers now expose the override: x-litellm-quality-router-via → "keyword" | "quality_tier" x-litellm-quality-router-keyword → matched keyword (only on keyword route) x-litellm-quality-router-complexity → complexity tier (only on tier route) Removes: - request_kwargs["litellm_capabilities"] reading - _model_capabilities, _model_supports_capabilities, _first_capable_model_at_tier, capability filter in _resolve_model_for_quality_tier Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com> * feat(quality_router): add explicit `order` to RoutingPreferences Adds an explicit priority field to RoutingPreferences for resolving collisions deterministically: RoutingPreferences.order: Optional[int] # lower wins; unset = +inf Used as the PRIMARY tiebreaker in two places: 1. Keyword overlap: when multiple deployments declare the same matching keyword, sort by (order ASC, quality_tier DESC, input_cost_per_token ASC, model_name ASC). Explicit always beats implicit. 2. Tier resolution: when multiple deployments share a quality tier, `_resolve_model_for_quality_tier` picks the one with the lowest order. The tier list is now sorted at index-build time. This lets admins make routing decisions explicit when the natural quality-and-price ordering would pick the wrong model. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com> * feat(quality_router): reorder tiebreak to (quality, order, price) Changes the tiebreak ordering so quality_tier always wins first, then explicit `order` is used to break ties within the same tier, then price breaks the rest: 1. quality_tier DESC ← best model wins first 2. order ASC ← explicit priority within a tier 3. input_cost_per_token ASC 4. model_name ASC Previously `order` was the primary key — that meant a tier-2 model with `order=1` would beat a tier-3 model with no `order`, which is the wrong default. Now `order` only resolves collisions among same-tier candidates. Tier resolution (within a single tier) keeps the same key minus quality: (order ASC, cost ASC, name). Test renames + flips: - test_explicit_order_overrides_quality_tier → test_quality_wins_over_explicit_order - new: test_order_breaks_tie_within_same_quality_tier Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com> * fix(quality_router): resolve Greptile review feedback Addresses four P1 findings from PR review plus test coverage: 1. set_model_list missing quality_routers reset - Hot-reloading the Router would leave stale QualityRouter instances pointing at the old model_list. `set_model_list` now clears `self.quality_routers` alongside the other indices. 2. Round-down fallback before default_model - `_resolve_model_for_quality_tier` now rounds DOWN to the closest lower tier after round-up fails, before falling back to `default_model`. Degrades gracefully rather than jumping straight off-tier. 3. RoutingPreferences validation bypass - `_build_tier_index` now instantiates `RoutingPreferences(**prefs)` so invalid shapes (e.g. non-int quality_tier) raise a clear ValueError instead of silently succeeding. 4. Config-ordering dependency - `_tier_to_models` is now built lazily on first access. Previously, eager construction in `__init__` meant a QualityRouter deployment had to appear AFTER all its referenced models in config.yaml, because `Router._create_deployment` populates `model_list` incrementally. Any `available_models` defined after the router entry would silently be reported as missing. Also adds 6 new tests covering each fix: - test_invalid_quality_tier_type_raises_clear_error - test_router_can_be_instantiated_before_its_targets_exist - test_set_model_list_clears_quality_routers_registry - test_rounds_down_when_no_higher_tier_exists - test_rounds_down_prefers_closest_lower_tier - test_prefers_round_up_over_round_down Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com> * style: apply black 24.10.0 formatting to pre-existing offenders Unblocks the LiteLLM Linting check for this PR — these 12 files are already failing `black --check` on main (the lint workflow only runs on PRs, so main drifts). No behavior changes; formatting-only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Update litellm/router.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4 (1M context) <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Support /v1/responses in complexity router (#26137) * feat(proxy): add --reload flag for uvicorn hot reload (dev only) Opt-in CLI flag, off by default, no env var. Only affects the uvicorn run path; gunicorn/hypercorn paths and prod (which doesn't pass the flag) are unaffected. * Feature/add audio support for scaleway (#26110) * feat(scaleway): add SCALEWAY to LlmProviders enum * feat(scaleway): add audio transcription config and dispatch wiring Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(scaleway): add behavior tests for audio transcription config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(scaleway): advertise audio_transcriptions in endpoint-support JSON * docs(scaleway): document audio transcription support * fix(scaleway): address PR review — plain-text response_format + missing-key fail-fast Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(scaleway): cover new response paths, drop gettysburg.wav coupling Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * Prompt Compression - add it to the proxy (#25729) * refactor: new agentic loop event hook simplifies how to create logic for tool based multi llm calls * fix: compress - make it work on anthropic input as well * fix(compress.py): working prompt compression for claude code ensures claude code messages can run through proxy easily * docs: add agentic loop hook guide * docs: add agentic_loop_hook to sidebar * fix: fix multiple arguments error * fix: fix tool call loop for compression on streaming /v1/messages * fix: fix linting errors * fix: fix ci/cd errors * feat(litellm_pre_call_utils.py): use claude code session for litellm session id allows claude code logs to be stitched together, making it easy to know they were all part of the same conversation * fix: suppress incorrect mypy warning rE: module * revert: drop PR's changes to litellm/proxy/_experimental/out/ Restores the 34 HTML files under _experimental/out/ to their pre-PR paths (X/index.html -> X.html). All renames are R100 (content unchanged); no other files are touched. * fix: address greptile review comments on PR #25729 - Skip ``kwargs["tools"] = []`` injection when compression is a no-op — Anthropic Messages rejects empty tool arrays on requests that did not originally declare tools. - Move agentic-loop safety guards (fingerprint cycle / max depth) out of the per-callback try/except so they propagate instead of being swallowed by the generic exception handler. Extracted _check_agentic_loop_safety. - Gate generic ``x-<vendor>-session-id`` capture behind the LITELLM_CAPTURE_VENDOR_SESSION_HEADERS env var (off by default) to preserve backwards compatibility; explicit x-litellm-* headers are unaffected. - Fix monkeypatch target in pre-call-hook test to patch the actual module-level binding (litellm.integrations.compression_interception.handler.compress). - Add regression tests for empty-tools skip and opt-in session capture. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert: drop LITELLM_CAPTURE_VENDOR_SESSION_HEADERS flag Generic x-<vendor>-session-id header capture is a new feature and only runs *after* the explicit x-litellm-trace-id / x-litellm-session-id checks, so it does not change behavior for any existing caller that was already using the LiteLLM headers — no backwards-incompatibility to gate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(compress): replace input_type with CallTypes call_type Drop the bespoke ``CompressionInputType`` literal and use the existing ``litellm.types.utils.CallTypes`` enum instead. ``litellm.compress()`` now takes ``call_type: Union[CallTypes, str]`` (default ``CallTypes.completion``) — no new concept to learn, and the enum is already the way the rest of the codebase talks about request shapes. Supported values: ``completion`` / ``acompletion`` (OpenAI chat-completions shape) and ``anthropic_messages`` (Anthropic structured content blocks). Updated: compress(), the compression_interception handler, tests, docs, and the two eval scripts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Support /v1/responses in complexity router Adds cross-format support to the complexity router via the guardrail translation handler dispatch. Adds get_structured_messages to base translation plus OpenAI chat, Responses, and Anthropic handlers. Auto-router helper _extract_text_from_messages handles tool-call and multimodal messages. Widens async_pre_routing_hook messages type to Dict[str, Any]. Fixes #25134 * chore: apply black formatting * fix: fallback to trying each handler when route inference fails --------- Co-authored-by: Ryan Crabbe <ryan@berri.ai> Co-authored-by: nhyy244 <106547304+nhyy244@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * test: cover _is_quality_router_deployment and init_quality_router_deployment * fix: reset auto_routers on set_model_list to prevent hot-reload ValueError * style: apply black formatting to websearch_interception and agentic_streaming_iterator --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Claude Opus 4 (1M context) <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Ryan Crabbe <ryan@berri.ai> Co-authored-by: nhyy244 <106547304+nhyy244@users.noreply.github.com>
chore: update retired claude-3-haiku-20240307 to claude-haiku-4-5-20251001
[Fix] CI - auth_ui_unit_tests: use Postgres sidecar instead of shared DB
…ile-optimization-31b6
The "remaining" proxy-db job was consistently timing out at ~98% because --dist=loadscope pins every test in test_proxy_utils.py (168+ parametrized tests) to a single xdist worker. 7 workers finished their files in ~15 minutes, then one worker ran alone for another 8+ minutes and hit the 30-minute job cap. Give test_proxy_utils.py its own matrix entry so its tests spread across all 8 workers, and add it to the "remaining" ignore list.
…lures Addresses two further Greptile findings: - `_warn_if_db_ahead_of_head` only caught `psycopg.OperationalError`. Non-connection DB errors (e.g. `InsufficientPrivilege` / 42501 if the runtime DB user lacks SELECT on `_prisma_migrations`) would propagate uncaught and crash startup — contradicting the docstring's "informational only, never blocks" guarantee. Widen the catch to `psycopg.DatabaseError` so all DB-layer errors are swallowed. - In the P3009 and P3018 idempotent-recovery paths, the call to `_resolve_specific_migration(name)` was not wrapped in its own try/except. Being inside an active `except CalledProcessError` handler, a new `CalledProcessError` from the resolve call would NOT re-enter the same handler — it would propagate out as `CalledProcessError`, past `proxy_cli.py`'s `except RuntimeError`, crashing startup with an unhandled traceback instead of the intended clean `sys.exit(2)`. Wrap both call sites to convert to RuntimeError. Adds unit tests for both behaviors.
[Infra] Merge dev branch
[Feature] Proxy: opt-in v2 migration resolver
fix(bedrock_guardrails): use Bedrock OUTPUT source for apply_guardrail when scanning model responses
…he team default Previously, members added to a team without an explicit per-member budget were all linked to the same `litellm_budgettable` row referenced by the team's `metadata.team_member_budget_id`. Updating one member's budget via `/team/member_update` mutated the shared row and silently changed every other member's budget too. Now both write paths produce a private, per-member budget: - `add_new_member` clones the team's default budget into a fresh row when a member is added without `max_budget_in_team`/`allowed_models`. If no team default exists, the membership is created with no budget. - `_upsert_budget_and_membership` detects when an existing membership still points at the team's default budget id and clones-on-write, relinking the membership to the new private budget before applying the update. - `team_member_update` reads `team_member_budget_id` from team metadata and passes it through so the helper can make this distinction. Adds unit tests for clone-on-write, in-place update of a private budget, and the no-default-no-budget add path. Made-with: Cursor
[Infra] Bump version
Contributor
|
Too many files changed for review. ( |
|
|
ryan-crabbe-berri
approved these changes
Apr 22, 2026
Contributor
…-budgets Litellm individual team member budgets
yuneng-berri
temporarily deployed
to
integration-postgres
April 22, 2026 01:38 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
yuneng-berri
temporarily deployed
to
integration-postgres
April 22, 2026 01:38 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
yuneng-berri
temporarily deployed
to
integration-postgres
April 22, 2026 01:38 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
yuneng-berri
temporarily deployed
to
integration-postgres
April 22, 2026 01:38 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
yuneng-berri
temporarily deployed
to
integration-postgres
April 22, 2026 01:38 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
shin-berri
approved these changes
Apr 22, 2026
ishaan-berri
had a problem deploying
to
integration-postgres
April 22, 2026 02:23 — with
GitHub Actions
Error
GitHub Actions
Error
ishaan-berri
had a problem deploying
to
integration-postgres
April 22, 2026 02:23 — with
GitHub Actions
Error
GitHub Actions
Error
ishaan-berri
had a problem deploying
to
integration-postgres
April 22, 2026 02:23 — with
GitHub Actions
Error
GitHub Actions
Error
ishaan-berri
had a problem deploying
to
integration-postgres
April 22, 2026 02:23 — with
GitHub Actions
Error
GitHub Actions
Error
ishaan-berri
had a problem deploying
to
integration-postgres
April 22, 2026 02:23 — with
GitHub Actions
Error
GitHub Actions
Error
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
[Infra] Promote internal staging to main
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🚄 Infrastructure
Changes