chore(ci): promote internal staging to main - #33640
Merged
Merged
Conversation
Port of #16162 by @dhruvyad onto litellm_internal_staging. OpenRouter sends a usage chunk (including a provider-reported cost field) after the finish_reason chunk. Previously the stream handler raised StopIteration on the first post-finish chunk, so that usage/cost never reached the assembled response and cost tracking fell back to token-based estimates. Carry usage.cost through chunk accumulation, preserve stripped usage in _hidden_params, and propagate the provider cost into _hidden_params["additional_headers"]["llm_provider-x-litellm-response-cost"] so the cost calculator uses it.
The /guardrails/usage/{overview,detail,logs} endpoints resolved guardrails only
from the litellm_guardrailstable Prisma table, so guardrails defined in
config.yaml (which live only in IN_MEMORY_GUARDRAIL_HANDLER) were invisible:
detail 404'd, overview omitted them or rendered them as Custom/Guardrail
orphans, and logs missed their logical-name alias.
Add config-owned accessors (list_config_guardrails, get_config_guardrail_by_id)
to the in-memory handler and use them in the usage endpoints, mirroring the
union/fallback already used by list_guardrails_v2 and get_guardrail_info. Also
preserve guardrail_info when storing a config guardrail (type/description were
dropped at initialize time) and read the Prisma-row / dict / LitellmParams
shapes uniformly.
Resolves LIT-2529
Addresses PR review: _to_dict special-cased LitellmParams and returned an empty dict for any other pydantic model. Switch to isinstance(value, BaseModel) so it coerces any pydantic model uniformly (BaseModel is already imported for the response models), which also drops the now-unused LitellmParams import. Behavior is unchanged for the current call sites.
…ilter startup index The semantic tool filter builds its index by listing every MCP server without per-user credentials, so servers needing per-user auth (interactive OAuth tokens, user-scoped env vars) reject the anonymous tools/list and contribute zero routes. Request-time expansion resolves that auth, so filter_tools received tools the router could never select: an empty index failed open N->N (past 128 tools OpenAI rejects the request outright) and a partial index matched only unavailable tools, stripping every tool from the request. filter_tools now syncs missing tools into the router before matching (building the router when absent) behind an asyncio lock so each tool embeds once, and falls back to the full tool list when matches map to no available tool, consistent with the zero-match fallback. Context-window overflows keep failing closed.
…dict allocation in the fast path
…equesting call Match candidates are restricted to the request's own tool names via route_filter, so routes learned from other principals' listings cannot displace the caller's tools from top_k. Lazy indexing now uses the async aadd flow exclusively; an embedding failure, including a context-window overflow on an oversized description, raises for the requesting call only and never writes the shared context_window_error, so one request cannot poison the filter for every user on the worker. The router is also sized to the configured top_k, which the semantic-router index layer otherwise silently caps at its default of 5.
…shared DataTable Rewrites the three Batch A tables from hand-rolled TanStack + tremor renderers into thin DataTable consumers with a separate ColumnDef module each, matching the Guardrails and Tags migrations. Row actions move into a per-row overflow menu (edit/copy/delete for vector stores; copy for everyone plus admin-gated delete for prompts and skills). The vector stores parent gains an isLoading flag resolved on every exit path so the table shows the shared skeleton instead of flashing the empty state. Table files are renamed to PascalCase and stale eslint bulk-suppressions for the rewritten files are pruned.
Only the name cell and the overflow menu act on a row, matching the unified table pattern; the previous table navigated on any row click
…itellm_fix_stream_reset_empty_200
…r OpenAI, Azure OpenAI, and Bedrock Mantle
…st anchor Adds an admin-configured issuer to MCP servers. When set, OAuth metadata is fetched from the issuer's own origin and adopted only when the document self-attests that same issuer (RFC 8414 §3.3), making token_endpoint, registration_endpoint, and scopes authoritative for the pinned issuer instead of a document the MCP resource server chose. This closes the mix-up where a compromised resource echoes a pinned authorization_url to smuggle its own token endpoint and inflated scopes past the corroboration gate. Discovery is same-authority against the issuer origin, fails closed on a §3.3 mismatch, and does not fall back to resource-rooted discovery. Rows without an issuer keep the existing corroboration-gate behavior unchanged. Backend + schema only; UI field and live-proxy proof follow.
…ear on url/auth_type change + UI Discover the issuer from the upstream and persist it trust-on-first-use (fill-empty-only, frozen thereafter), so admins do not have to type it; an admin-configured issuer always wins and is never overwritten. Re-pointing the server url now clears the discovered issuer and endpoints (matching the existing auth_type-change clearing) so a new upstream re-discovers instead of anchoring on the previous upstream's issuer. Adds the issuer to the per-user OAuth token identity so re-pointing it purges stale tokens. Surfaces the issuer as an optional, auto-discovered, overridable field in the create and edit MCP server forms. C901 gate shows +1 vs staging; that is inherited from the #33317 stack base (delta 0 against
The issuer anchor is for the token/registration endpoints only (the RFC 9700 mix-up). Scope selection stays resource-driven per the MCP authorization spec Scope Selection Strategy: _fetch_issuer_anchored_oauth_metadata now validates the issuer document (RFC 8414 §3.3) for the endpoints and separately fetches the resource's advertised scopes (WWW-Authenticate challenge, else RFC 9728 scopes_supported) for the scope value, instead of using the issuer document's own scopes_supported. The resource can influence only the requested scope, which the authorization server and user consent bound (RFC 6749 §3.3), never the token endpoint.
…site When an admin pins an issuer, RFC 8414 section 3.3 makes that issuer the sole authoritative source of the authorization and token endpoints, so a compromised or misconfigured upstream cannot smuggle a token endpoint by echoing the pinned authorize URL. The first cut enforced that only on the database build path; the carry-forward, persistence, config-load, serialization and sanitization paths could still restore or emit upstream-derived endpoints for an issuer-anchored server, which is the class of gap the review flagged. Every site now routes through one predicate. _endpoints_yield_to_issuer returns all-None whenever the issuer is the anchor, so both build paths, has_all_upstream_oauth_fields, needs_discovery and the endpoint merge defer to the issuer. _carry_forward_resolved_oauth_endpoints carries only scopes for an issuer-anchored server and fails closed on endpoints. _persist_discovered_oauth_endpoints skips endpoint writes under the anchor. The two table serializers round-trip the issuer and both non-admin sanitizers redact it. Scope selection stays resource-driven per the MCP authorization spec: _fetch_issuer_anchored_oauth_metadata takes endpoints from the issuer document and scopes from the resource document. The OAuth metadata resolution and corroboration gating for the database build path move into _resolve_table_oauth_metadata so build_mcp_server_from_table stays within the cyclomatic-complexity budget without changing behavior. Regression tests pin the invariant at each site: the issuer overrides stored endpoints even when they are populated, carry-forward does not restore endpoints under the anchor, persistence does not write endpoints under the anchor, a url or auth_type change clears stale issuer-scoped fields even when resubmitted unchanged, the Azure heuristic stays reachable under a required issuer, and anchored metadata takes endpoints from the issuer while scopes come from the resource
…covered Two lifecycle gaps let the issuer trust anchor drift out of sync with the endpoints it governs. Changing or clearing a previously pinned issuer left the authorization_url and token_url that were resolved under the old issuer in the row, so clearing the anchor could revive stale, possibly untrusted endpoints instead of re-discovering. And a build that discovered an issuer trust-on-first-use persisted it to the row while the returned in-memory server kept the issuer unset, so the registry and the row disagreed and the per-user OAuth token identity, which includes the issuer, differed between that build and the next rebuild and forced a spurious re-auth. update_mcp_server now treats a change to a previously pinned issuer the same as a url or auth_type change and clears the auth-flow-scoped endpoint fields that were resolved under it. The trigger fires only when an issuer was already pinned and is now changed or cleared, so establishing one for the first time, including the trust-on-first-use discovery write-back, does not wipe the fields it just resolved. Both build paths, build_mcp_server_from_table and load_servers_from_config, now construct the server with effective_issuer = manual_issuer or the discovered issuer, skipping an origin-fallback guess exactly as the persistence does, so the in-memory object always reflects what the row will hold. Regression tests pin each case: clearing and re-pointing a pinned issuer clear the stale endpoints, a first-time establish preserves the discovered fields, and a build reflects the discovered issuer while an origin-fallback guess is not reflected
…est_connection (#33498) * fix(ui): stop sending the complexity-router pseudo-model to /health/test_connection Test Connection on a saved auto-router model sent the raw "auto_router/complexity_router" model string to the generic health-check endpoint, which always failed with "Unmapped LLM provider" since it's a routing-strategy config, not a real completion endpoint. Reuse the per-tier connection test already built for the Add Auto Router wizard: for complexity-router models, test each configured tier's underlying model group instead of the router pseudo-model. Semantic-type auto routers (auto_router_config) have no equivalent tier-based test yet, so the button is hidden for them instead of guaranteed to fail. * fix(ui): address review feedback on auto-router test connection fix Type the complexity-router config parsing instead of using `any`, use NotificationsManager.warning instead of fromBackend for the client-generated "no tiers configured" message, remove comments added in the previous commit, and also test the deployment's configured complexity_router_default_model as a fallback target when it isn't already covered by a configured tier (matches the fallback Router itself uses for unconfigured tiers).
…e proxy (#33231) * feat(cli): add `lite up`/`lite down` to ambiently route Claude Code through the proxy Patches ~/.claude/settings.json in place (env.ANTHROPIC_BASE_URL + apiKeyHelper via `lite auth print-token`) so any `claude` session started afterward, from any terminal, routes through the local LiteLLM proxy with no wrapper command needed, unlike the existing `lite claude` subprocess-exec approach. Backs up the original file first and restores it on Ctrl-C/SIGTERM, or via `lite down` after an unclean exit. Cursor is not supported: no equivalent file-based config to patch. * feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy (#33249) * feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy Lets a customer try litellm's complexity_router against models they already have on their existing, unmodified production proxy, with no config.yaml edits and no new infra. lite autoroute configure discovers accessible models via /model_group/info and walks through tier assignment (plus optional LLM classifier / semantic matching / adaptive selection); every referenced model becomes its own litellm_proxy/<name> deployment forwarding back to the real proxy with the real key, so every actual call, routed completions, classifier calls, embedding calls, still lands on their real proxy. lite autoroute up launches that generated config as an ephemeral local proxy, patches ~/.claude/settings.json to point Claude Code at it, and streams routing decisions live; Ctrl-C/SIGTERM (or lite autoroute down after an unclean exit) restores everything. Also adds lite model-groups list (a thin CLI wrapper over the existing ModelGroupsManagementClient), and generalizes up.py's settings-backup/restore helpers to take explicit paths so this feature can reuse them instead of duplicating the logic. Depends on litellm_lite_up_down (#33231) for that generalization. * feat(cli): allow multiple models per autoroute tier complexity_router already supports a pool of models per tier (randomly picked per request; adaptive mode specifically needs a pool to choose within), but the configure wizard only ever let you assign one. Tiers are now a tuple of model names; the wizard prompt accepts comma-separated indices to pick more than one per tier. * feat(cli): fuzzy model picker and auto-route Claude Code to autorouter Numbered-index selection didn't scale past a handful of models, so switch the tier picker to InquirerPy's fzf-style fuzzy search. Also set ANTHROPIC_DEFAULT_{SONNET,HAIKU,OPUS}_MODEL to "autorouter" in Claude Code's settings, since Router resolves auto-router deployments by literal model name with no wildcard support, so a "*" catch-all model_name would never match real traffic. * feat(cli): allow installing lite CLI from source via LITELLM_CLI_REF Lets testers try an unreleased branch's CLI changes with the same curl-piped installer, instead of waiting for a PyPI release. * fix(ci): modernize type hints to clear ruff strict-rule budget * fix(ci): bump httplib2 and setuptools to patched versions Clears osv-scan findings for PYSEC-2026-3444 and PYSEC-2026-3447. * fix(cli): write autoroute's secret-bearing files with mode 0600 commands.py wrote config.yaml (embeds the real proxy key) and Claude Code's settings.json (embeds the ephemeral proxy's master key) with plain open(), landing at the umask-derived default (commonly 0644) until a later chmod call caught up. That window, and the missed case where settings.json already exists (chmod never ran at all there), left a credential-bearing file readable by another local account. secure_create() fixes the mode via fchmod on the fd before any content is written, covering both the brand-new-file and already-exists cases, and commands.py/wizard.py now route their sensitive writes through it. * docs(cli): warn that a stale Claude Code session can leak to a squatted port lite autoroute up's master key is embedded statically (unlike lite up's apiKeyHelper, resolved per request), so a Claude Code session still running after teardown keeps sending it, along with prompt content, to a now-unbound loopback port that another local account can bind. This is the same one-time-patch tradeoff lite up already accepts, just with a static secret instead of a re-resolved one -- document it in the README's Caveats section and surface it in the teardown message itself. * fix(cli): address greptile review feedback on autoroute PR - terminate the ephemeral proxy child process when its health check fails, instead of leaking an orphaned, unrecoverable process bound to the port - replace bare assert isinstance checks (no-ops under python -O) with click.ClickException in the model-groups list and configure wizard code paths - close launch_proxy's log file handle once the child process has inherited its fd, instead of leaking it - add build_generated_proxy_config to config.py's __all__ * fix(cli): close TOCTOU window in lite up's settings backup write write_backup wrote the backup (which can embed the original apiKeyHelper/settings content) with plain open() + a chmod call after the fact -- the same permissive-until-corrected window already fixed for autoroute's config.yaml and Claude settings writes, and missed entirely when the backup file already exists with broader permissions. Moves secure_create (atomic-enough 0600 via fchmod before any content is written) to up.py, the module both lite up and lite autoroute share, and has autoroute/process.py import it from there instead of keeping its own copy. * fix(cli): refuse autoroute up when a stale backup exists from a crash The pid-record check only catches a still-live duplicate process; a SIGKILL'd `up` leaves no live pid but does leave AUTOROUTE_BACKUP_PATH behind. Without this guard, a fresh `up` overwrote that backup with the currently-patched Claude settings instead of the true originals, so `down`/Ctrl-C would restore the wrong content permanently. up.py's `lite up` already guards the analogous case; mirror it here. * fix(cli): bind the ephemeral autoroute proxy to loopback only proxy_cli.py defaults --host to 0.0.0.0 when not passed explicitly. launch_proxy never passed it, so the ephemeral proxy -- despite every base_url in this module being built from 127.0.0.1 -- was actually reachable from other hosts on the network, including its unauthenticated-until-config-lands routes before the master key is wired in. * docs(cli): show curl install for the autoroute QA flow Points readers at scripts/install-cli.sh's curl one-liner instead of assuming uv/pip is already set up, and documents the LITELLM_CLI_REF override for trying an unreleased branch or commit. * fix(cli): surface a clean error on an empty or corrupt autoroute config A configure run killed between secure_create's O_TRUNC and the write completing leaves an empty config.yaml on disk. The next up read that via yaml.safe_load (None) into the generated-config TypeAdapter uncaught, surfacing a raw pydantic.ValidationError instead of pointing the user back at `lite autoroute configure`. * fix(cli): bind lite up's apiKeyHelper to the proxy it was started against _ensure_fresh_login only checked token freshness, not which proxy the cached token belonged to, and resolve_api_key_helper built a bare `lite auth print-token` command with no --base-url. A user logged into proxy A who ran `up --base-url proxy-b` (or LITELLM_PROXY_URL=proxy-b) would silently get proxy A's real token wired into Claude Code's apiKeyHelper; since apiKeyHelper is invoked bare, print-token's existing origin check never engaged, so proxy B -- attacker-controlled or not -- received every subsequent request's Authorization header carrying proxy A's credential. _ensure_fresh_login now requires the cached token's base_url to match before treating it as usable, forcing a fresh login for the selected proxy otherwise. resolve_api_key_helper now takes that base_url and threads it through as an explicit --base-url, so print-token's existing (but previously unreachable in the apiKeyHelper flow) base_url_explicit check actually enforces the match at request time too. * fix(cli): surface clean errors instead of raw tracebacks in lite up/down load_json_or_empty and read_backup both delegate to pydantic's validate_json, which raises ValidationError on invalid JSON or a non-object root -- neither up() nor down() caught it, so a corrupt settings or backup file surfaced an unformatted Python traceback instead of a clean CLI error. Both now convert to UpError, and down() (previously uncaught entirely) and up()'s teardown path now handle it. restore_claude_settings also gained a parent.mkdir guard before rewriting CLAUDE_SETTINGS_PATH: if ~/.claude/ was removed while `lite up` was running, the restore would crash before deleting the backup file, permanently stranding it and breaking every future `lite down`. * docs(cli): call out env-var auth for autoroute commands * fix(cli): clean up leaked proxy and surface clean errors in autoroute Three related gaps, all following an UpError getting raised somewhere that wasn't catching it yet: - up() left the just-launched ephemeral proxy running with no pid record if load_json_or_empty/write_backup/secure_create raised after the health check passed, mirroring the existing ProcessLaunchError cleanup for the health-check-failure branch. - _teardown() didn't catch restore_claude_settings raising UpError (e.g. a corrupt backup at stop time), which would otherwise escape to Click as an unhandled error in the normal-exit path, or print "Error in atexit" in the atexit path. up.py's own _restore_once handles the identical case the same way. - read_pid_record let a corrupt PID file surface a raw pydantic.ValidationError instead of a clean message, and did so in down(), the command specifically meant for crash recovery. down() now clears an unreadable pid record and continues cleanup instead of aborting, since a corrupt pid file must never block the one command meant to recover from exactly this kind of crash. * docs(cli): warn against running lite up and lite autoroute up together
Pin a session's first-turn model for the rest of the session by default instead of reclassifying every turn. Keeps multi-turn sessions on a single model, preserving provider prompt caches and avoiding cross-model conversation-history errors (e.g. Anthropic rejecting a thinking block produced by a different model). Requests without a resolvable session_id are unaffected. Set session_affinity: false to opt out. Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…eps endpoints Making the in-memory issuer reflect a trust-on-first-use discovered value fixed the registry/row token-identity drift, but it overloaded a single field: the carry-forward gate keyed on issuer truthiness as a proxy for "endpoints are anchored to a pinned issuer, fail-closed". A discovered issuer is truthy yet not anchored, so a resource-rooted server that had learned its issuer would drop its last-known-good endpoints on a transient discovery blip instead of carrying them forward. Anchoring is now a first-class property rather than a proxy. MCPServer carries issuer_is_anchored, set at both build paths from the single _uses_issuer_anchor definition (a pinned issuer on a discovery auth type). issuer stays the identity value used by the token-identity tuple and the serializers; issuer_is_anchored is the provenance value the carry-forward gate reads to decide fail-closed. The two properties can no longer be conflated, so a discovered issuer keeps its resource-rooted endpoints carrying forward while a pinned issuer still fails closed. Regression tests pin both directions: a discovered-but-not-anchored server restores its endpoints on a discovery blip, an anchored server does not, and the build sets issuer_is_anchored true only when the issuer is pinned
Co-authored-by: shivam <shivam@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
feat(scim): ingest and round-trip SCIM entitlements and roles user attributes
Migrate Projects, Project Keys, Vector-store Documents, Logging Callbacks, and Pass-through Endpoints onto the shared DataTable and cell library, each split into a thin container plus a columns file. Row actions move into the unified overflow menu, empty states get the rich icon+title+body treatment, and parents that never tracked loading gain an initial-load-only isLoading flag so tables render skeletons instead of flashing the empty state. Project Keys switches from a hand-rolled antd Pagination header to server pagination through the shared footer, and its section is rewired into ProjectDetailsPage, which had regressed to a hardcoded "No keys to display" empty card while the section sat orphaned. pass_through_settings.tsx moves to PascalCase under components/PassThroughSettings/ and drops its dead modelData prop and tremor imports; its stale eslint bulk-suppressions are pruned.
…attachments and remove the attachment count cap (#33554) * fix(model_armor): add skip_unscannable_attachments to allow reference-only attachments through * fix(model_armor): wire skip_unscannable_attachments through guardrail config * fix(model_armor): make max_file_attachments configurable and scan overflow instead of dropping * fix(model_armor): remove the per-request attachment count cap and scan all attachments --------- Co-authored-by: yucheng <yucheng@berri.ai>
…_tool_format fix(mcp): keep the MCP reference intact when the semantic filter narrows tools
…, gate only bedrock_mantle The openai and azure_openai columns have working credentials in every suite runner, so only the bedrock_mantle column still needs an opt-in flag while the AWS account waits on the Mantle allowlist; COMPAT_GPT_CELLS becomes COMPAT_MANTLE_CELLS. The six tool_use cells now resolve the proxy through claude_code._env like the basic_messaging cells instead of hardcoding LITELLM_PROXY_BASE_URL/LITELLM_PROXY_API_KEY.
The preemptive-401 gate for auth_type=oauth2 MCP servers keyed the challenge on whether an Authorization header was present (not oauth2_headers). Because the header parser classifies any Authorization bearer as an OAuth token before the target server is resolved, a LiteLLM virtual key presented as Authorization: Bearer sk-... suppressed the challenge on a gateway-managed authorization_code server; the session then opened with no upstream token and tools/list masked the failure as 200 with an empty tool list. The same gate also wrongly challenged client_credentials (M2M) servers, which the gateway authenticates by minting its own token at egress. The decision is per oauth2 sub-mode, not per header. Gateway-managed modes never receive a client-supplied upstream token: client_credentials mints at egress so it is never challenged, and gateway-managed interactive (authorization_code, non-delegate) is challenged whenever no stored per-user token exists, regardless of any bearer. Only the delegate/upstream-PKCE mode, where a present bearer genuinely is the upstream token, keeps keying on the Authorization header. oauth2_headers itself is left untouched so the delegate/passthrough egress paths that forward the client bearer are unchanged.
…e-401 gate Grafted from PR #33582 (closing as superseded by this PR): drives handle_streamable_http_mcp with real MCPServer objects, parametrized over a stamped client_credentials row and a legacy unstamped M2M-shape row; both must reach the session manager without the per-user token store being consulted
…3312) A `lite login` token 429'd with "Budget has been exceeded! Max budget: 0.25" even when no budget was configured anywhere. cli_poll_key stamped the minted CLI session token with litellm.max_ui_session_budget ($0.25) as a fallback whenever the user and team had no budget of their own. That cap was designed for the Admin UI "Test Key" chat pane; the CLI reused the same session-token machinery, so it inherited a playground-sized budget baked into the encrypted token at login (unchangeable without re-login), which trips fast under real CLI/agent use. The cap is also redundant: the token already carries user_id and team_id, so the real user/team budgets are enforced independently at request time. Pass max_budget=None so the CLI token is governed only by those real budgets, and drop the now-dead user/team budget lookups. The UI login token's guard (get_experimental_ui_login_jwt_auth_token) is untouched.
* test(e2e): read datadog log delivery back from the real datadog api (#33604) * test(e2e): read datadog log delivery back from the real datadog api * test(e2e): compare datadog-read cost with math.isclose, not bit-equality The response_cost now round-trips through DataDog's attribute indexing pipeline, whose float serialization is not guaranteed to preserve the exact bit pattern the proxy shipped. rel_tol=1e-9 (equal to 9 significant digits) still fails on any real cost discrepancy while tolerating representation drift. Addresses the Greptile P2 on this PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): widen the duplicate-settle window to 30s for real DataDog Against the local sink one poll interval (5s) after the first hit was enough to catch a same-call duplicate, because both events arrived in the same flush batch. Against real DataDog, ingestion jitter can make one call's two events searchable tens of seconds apart, so a 5s settle could let the LIT-4447 duplicate slip past the exactly-one assertion. The reader now keeps re-reading for DD_SETTLE_SECONDS (default 30s, env-overridable via E2E_DD_SETTLE_SECONDS) after the first event appears, returning early only when a duplicate is already visible - more waiting cannot clear it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(e2e): point UI tests at dashboard service; register complexity router Stage gateway 404s /ui; the Next.js dashboard is litellm-ui:3000. Drive playwright against E2E_UI_BASE_URL and wait on login placeholders after client render. Register complexity-smart-router via /model/new when the proxy does not already list it so stage matches compose config * docs(e2e): clarify E2E_UI_BASE_URL should be ALB when ingress splits UI * docs(e2e): prefer single path-routing host for control plane and UI CONTROL_PLANE and UI already default to PROXY_BASE_URL; clarify that stage should set one ALB host rather than three endpoints * fix(e2e): always capture complexity router model_id for teardown Split /model/new from the data-plane wait so a propagation timeout still deletes the control-plane registration (greptile orphan-model concern) * fix(e2e): click exact Login button so SSO control is not matched Playwright strict mode matched both Login and Login with SSO * fix(router): score complexity by difficulty not request length The LLM classifier prompt treated short wording as SIMPLE, so probes like "Is P equal to NP?" stayed on the SIMPLE backend even though the classifier ran. Judge intellectual difficulty so short hard questions route higher * fix(e2e): open key edit via Key ID and wait for team models Key Alias text is not the row open control on the virtual keys table; KeyInfoView opens from the Key ID button in that row. Also wait for a real team model in the edit Models dropdown so we do not race the async availableModels fetch that only has All Team Models on first paint * fix(e2e): keep settled DD events on empty search; bump mcp for OSV Do not let a transient empty DataDog search wipe events already seen in the settle window (Greptile P1). Make the logs-search from window env-overridable via E2E_DD_SEARCH_FROM (Greptile P2). Prefer the mono Key ID button when opening key edit. Bump mcp 1.26.0 -> 1.28.1 so OSV clears the three high GHSA findings on the staging PR * revert: drop mcp lock bump from e2e staging PR OSV mcp upgrade is unrelated to the e2e fixes; leave the dep pin alone --------- Co-authored-by: yucheng-berri <yucheng@berri.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test(e2e/claude_code): add GPT-5.6 Sol/Terra/Luna columns for OpenAI, Azure OpenAI, and Bedrock Mantle
…de_aware fix(mcp): make the preemptive-401 OAuth challenge decision mode-aware
The policy attachment form fetched /team/list with the caller's own user_id, which the backend treats as a membership filter even for proxy admins. Admins only saw teams they were personally a member of, and the scope validation added in #32131 then rejected every other valid team alias as nonexistent. Drop the user_id filter; the policies page is admin-only and /team/list without user_id returns all teams for admin roles. Fixes LIT-4199
…o shared DataTable (#33629) * refactor(ui): migrate AI Hub, public hub, and MCP Toolsets tables onto shared DataTable * test(ui): stub skillHubPublicCall in the public model hub networking mock
Contributor
|
Too many files changed for review. ( Bypass the limit by tagging |
|
|
ryan-crabbe-berri
approved these changes
Jul 17, 2026
…he real datadog api (#33566) * fix(e2e): make the datadog read-back find what DataDog actually indexes Live verification of the merged #33604 against real DataDog (us5) exposed three read-back defects that the local-sink tests could never see; all three fixes are verified against the real API: - Marker search: DataDog consumes the shipped JSON message into the event's attributes and leaves the indexed message EMPTY, so the full-text '"marker"' query matched nothing and every test failed with zero events. The query is now '*:*marker*', which scans all attributes (the marker sits in messages.content); verified to return exactly the event for the call. - Rate limit: the Logs Search API budget is 2 requests per 10s org-wide (x-ratelimit-name logs_public_search_api). Polling at POLL_INTERVAL=5s sat exactly at the limit and the reader hard-failed on the first 429. Searches now pace at DD_SEARCH_INTERVAL (10s default) and a 429 backs off and retries up to 5 times; only non-429 failures stay hard fails. - Envelope status: DataDog re-derives the indexed event status from the parsed payload's status attribute ('success') and normalizes it to its OK severity, so the assertion expects 'ok', not the shipped 'info'. Live run: chat_completions and responses pass every assertion including the exact response-cost cross-check; messages red-pins the LIT-4447 duplicate for real (one call -> two sync-sweep copies + one async batch copy, same request id, confirmed in proxy debug logs). The duplicate is race-dependent, so the pin flickers until #33589 lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): datadog log delivery for streamed chat, messages, and responses Rewritten from the dd-sink version (original #33566) to judge delivery on what real DataDog ingested, matching the merged #33604 conversion: the dd_logs reader searches events back through the Logs Search API and the assertions validate the indexed envelope (source:litellm tag, ok status) and the StandardLoggingPayload fields under the event's attributes. Each streamed test drives one STREAMED call per route, asserts the stream actually streamed (event-stream content type, >0 chunks, no upstream error event), then pins exactly one DataDog event whose payload records stream=true, the aggregated token count, and a response_cost equal to the /spend/logs row for the call - a stream's headers ship before its cost exists, so the spend row is the cross-check anchor, and the spend row and DataDog event must also agree on total_tokens. Coverage registry: adds logging.datadog.stream.exports_metric exercised on chat_completions, messages, and responses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Update test_datadog_log_e2e.py --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tin-berri
approved these changes
Jul 17, 2026
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
| metadata["_model_armor_status"] = "blocked" | ||
| raise self._unscannable_block_error(reason) | ||
|
|
||
| if len(attachments) > MAX_FILE_ATTACHMENTS_PER_REQUEST: |
Contributor
There was a problem hiding this comment.
Medium: Unbounded guardrail request fan-out
Removing this cap lets an authenticated caller force one outbound Model Armor request per supplied attachment. The per-file size check does not limit attachment count, so many small files can exhaust guardrail quota and hold proxy resources; retain a bounded attachment count or enforce a comparable aggregate scan limit before entering the loop.
Contributor
PR overviewThis PR promotes internal staging changes to the main branch, including updates in the LiteLLM proxy’s Model Armor guardrail hook handling for attachments. The touched code is part of request processing that sends attachment content through guardrail scanning. There is one open security concern around attachment processing in the Model Armor guardrail path. An authenticated caller could submit many small attachments and trigger excessive outbound guardrail requests, consuming quota and proxy resources. No issues have been addressed yet, so the PR still carries a moderate resource-exhaustion risk until an attachment-count or aggregate-scan bound is restored. Open issues (1)
Fixed/addressed: 0 · PR risk: 6/10 |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
doonga
pushed a commit
to greyrock-labs/home-ops
that referenced
this pull request
Jul 29, 2026
![renovate[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
…4.0) (#201) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.93.0` → `v1.94.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.94.0`](https://github.com/BerriAI/litellm/releases/tag/v1.94.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.94.0...v1.94.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.94.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.94.0/cosign.pub \ ghcr.io/berriai/litellm:v1.94.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - feat(ui): working Test Connection for the complexity auto router by [@​akapur99](https://github.com/akapur99) in [#​32950](https://github.com/BerriAI/litellm/pull/32950) - fix(xecguard): use StandardLoggingGuardrailInformation in logging hook by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​32911](https://github.com/BerriAI/litellm/pull/32911) - feat(ui): adopt openapi-react-query ($api) and convert useCustomers by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32949](https://github.com/BerriAI/litellm/pull/32949) - refactor(ui): colocate the mcp-servers view, keeping the shared mcp\_tools surface by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32968](https://github.com/BerriAI/litellm/pull/32968) - refactor(ui): convert endpoint usage charts to shadcn/recharts by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32723](https://github.com/BerriAI/litellm/pull/32723) - fix(proxy-auth): stop unrecognized model namespaces slipping through provider wildcard keys by [@​mateo-berri](https://github.com/mateo-berri) in [#​32979](https://github.com/BerriAI/litellm/pull/32979) - feat(router): random-pick multi-model complexity tiers by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32967](https://github.com/BerriAI/litellm/pull/32967) - fix(xecguard): sanitize scan result before recording it for logging by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​32935](https://github.com/BerriAI/litellm/pull/32935) - fix(auto\_router): filter embedding models in complexity tab dropdowns, require all tiers, inline validation by [@​akapur99](https://github.com/akapur99) in [#​32978](https://github.com/BerriAI/litellm/pull/32978) - fix(anthropic): translate raw adaptive thinking for pre-4.6 models on chat completions and Bedrock Converse by [@​akapur99](https://github.com/akapur99) in [#​32944](https://github.com/BerriAI/litellm/pull/32944) - feat(router): add Router(plugins=\[...]) routing-plugin pipeline by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32972](https://github.com/BerriAI/litellm/pull/32972) - feat(router): soft-floor adaptive mode for complexity router by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32947](https://github.com/BerriAI/litellm/pull/32947) - docs(github): add QA runbook section to the PR template by [@​mateo-berri](https://github.com/mateo-berri) in [#​32965](https://github.com/BerriAI/litellm/pull/32965) - fix(model\_cost): add supports\_reasoning: false to Gemini image generation models by [@​mateo-berri](https://github.com/mateo-berri) in [#​32836](https://github.com/BerriAI/litellm/pull/32836) - build(dev-env): add make bootstrap and unprovisioned-checkout preflight to pre-commit by [@​mateo-berri](https://github.com/mateo-berri) in [#​32981](https://github.com/BerriAI/litellm/pull/32981) - ci(ui): report only error-level knip findings in CI by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32971](https://github.com/BerriAI/litellm/pull/32971) - feat(batches): track cost for unmanaged Bedrock batches, generalize the flag by [@​Sameerlite](https://github.com/Sameerlite) in [#​32315](https://github.com/BerriAI/litellm/pull/32315) - fix(guardrails): walk custom\_tool\_call\_output items in \_content\_utils by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​32969](https://github.com/BerriAI/litellm/pull/32969) - fix: show and allow editing team model aliases after team creation by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33047](https://github.com/BerriAI/litellm/pull/33047) - chore(deps): bump pillow to 12.3.0 to resolve osv-scan CVEs by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33093](https://github.com/BerriAI/litellm/pull/33093) - feat(mcp): mint gateway-bound envelope at the token endpoint for dcr\_bridge oauth\_delegate by [@​tin-berri](https://github.com/tin-berri) in [#​32828](https://github.com/BerriAI/litellm/pull/32828) - fix(mcp): surface rejected delegate-auth upstream tokens as connect-time 401 by [@​tin-berri](https://github.com/tin-berri) in [#​32741](https://github.com/BerriAI/litellm/pull/32741) - fix(proxy): track unauthenticated pass-through requests in spend logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​32410](https://github.com/BerriAI/litellm/pull/32410) - feat(lasso): send source.type for Used By attribution by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33090](https://github.com/BerriAI/litellm/pull/33090) - fix(responses): continue MCP gateway tool turns from the final response and surface failures by [@​thibault-linktree](https://github.com/thibault-linktree) in [#​33025](https://github.com/BerriAI/litellm/pull/33025) - fix(responses): continue MCP gateway tool turns from the final response and surface failures by [@​tin-berri](https://github.com/tin-berri) in [#​33099](https://github.com/BerriAI/litellm/pull/33099) - fix(completion): forward aws credential kwargs into litellm\_params so the responses bridge keeps WIF auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​32956](https://github.com/BerriAI/litellm/pull/32956) - fix(ui): respect litellm\_key\_header\_name in BYOK credential save and workflow runs fetches by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33103](https://github.com/BerriAI/litellm/pull/33103) - refactor(ui): standardize debounce waits behind shared DEBOUNCE\_WAIT\_MS constant by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33040](https://github.com/BerriAI/litellm/pull/33040) - feat(ui): rebuild the Virtual Keys table on the shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​32991](https://github.com/BerriAI/litellm/pull/32991) - fix: redact async complete streaming response for custom callbacks by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33106](https://github.com/BerriAI/litellm/pull/33106) - build(ui): bump [@​tanstack/react-pacer](https://github.com/tanstack/react-pacer) from 0.2.0 to 0.22.1 by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33041](https://github.com/BerriAI/litellm/pull/33041) - fix(ui): address Virtual Keys redesign review nits by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33112](https://github.com/BerriAI/litellm/pull/33112) - fix(openai/responses): clamp max\_output\_tokens below API minimum by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33098](https://github.com/BerriAI/litellm/pull/33098) - fix(prometheus): read v3 rate limiter remaining values for per-key model gauges by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33119](https://github.com/BerriAI/litellm/pull/33119) - fix(ui): drop w-full from page-content wrappers to remove 32px horizontal overflow by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33118](https://github.com/BerriAI/litellm/pull/33118) - refactor(ui): migrate straightforward value debounces to react-pacer by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33042](https://github.com/BerriAI/litellm/pull/33042) - feat(mcp): interactive SSO sign-in for dcr\_bridge oauth\_delegate DCR clients by [@​tin-berri](https://github.com/tin-berri) in [#​32946](https://github.com/BerriAI/litellm/pull/32946) - test(proxy): add regression tests for management\_endpoints edge cases by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​32976](https://github.com/BerriAI/litellm/pull/32976) - fix(auto-router): correct Responses API tool\_choice shape and propagate alias litellm\_params by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32974](https://github.com/BerriAI/litellm/pull/32974) - fix(ui): render the sidebar scrollbar with shadcn ScrollArea by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33124](https://github.com/BerriAI/litellm/pull/33124) - refactor(ui): migrate callback debounce sites to react-pacer with regression tests by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33043](https://github.com/BerriAI/litellm/pull/33043) - chore: add CODEOWNERS for ui and proxy UI build artifacts by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33131](https://github.com/BerriAI/litellm/pull/33131) - feat(mcp): client-held refresh envelope for the dcr\_bridge oauth\_delegate flow by [@​tin-berri](https://github.com/tin-berri) in [#​32980](https://github.com/BerriAI/litellm/pull/32980) - feat(ui): rebuild the Teams table on the shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33128](https://github.com/BerriAI/litellm/pull/33128) - fix(mcp): relay upstream OAuth token and DCR rejections instead of a generic 500 by [@​tin-berri](https://github.com/tin-berri) in [#​33113](https://github.com/BerriAI/litellm/pull/33113) - fix(keys): persist key\_type so the UI shows correct key scope instead of "All Proxy Models" by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33115](https://github.com/BerriAI/litellm/pull/33115) - feat(router): opt-in session affinity for complexity router by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​33126](https://github.com/BerriAI/litellm/pull/33126) - feat(prometheus): expose video duration and image count consumption metrics by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33138](https://github.com/BerriAI/litellm/pull/33138) - test(e2e): otel trace completeness on /chat/completions by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33132](https://github.com/BerriAI/litellm/pull/33132) - fix(sso): paginate through all pages when fetching service principal group assignments by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33149](https://github.com/BerriAI/litellm/pull/33149) - test(e2e): otel trace completeness on /v1/messages by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33133](https://github.com/BerriAI/litellm/pull/33133) - feat(ui): add adaptive routing settings to Auto-Router v2 by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​33146](https://github.com/BerriAI/litellm/pull/33146) - refactor(mcp): extract the dcr\_bridge token flow into bridge\_token\_flow\.py by [@​tin-berri](https://github.com/tin-berri) in [#​33141](https://github.com/BerriAI/litellm/pull/33141) - chore: bump litellm 1.93.0 -> 1.94.0, litellm-enterprise 0.1.49 -> 0.1.50, litellm-proxy-extras 0.4.76 -> 0.4.77 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33229](https://github.com/BerriAI/litellm/pull/33229) - fix(proxy): route master key to team-scoped models by [@​kunal2002](https://github.com/kunal2002) in [#​32926](https://github.com/BerriAI/litellm/pull/32926) - chore(deps): pin httplib2 and setuptools transitive floors by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33233](https://github.com/BerriAI/litellm/pull/33233) - feat(ui): left-anchor the Create Key and Create Team CTAs by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33248](https://github.com/BerriAI/litellm/pull/33248) - fix(anthropic/passthrough): drop incompatible temperature when downgrading adaptive thinking for pre-4.6 models by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33244](https://github.com/BerriAI/litellm/pull/33244) - fix(guardrails): run apply\_guardrail-style model-level pre\_call guardrails at deployment hook by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33136](https://github.com/BerriAI/litellm/pull/33136) - fix(proxy)!: enforce user budget on team keys (read-time + reservation) with UI opt-out by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​32005](https://github.com/BerriAI/litellm/pull/32005) - fix(e2e): bound spend-log snapshots to a /spend/logs/v2 window by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33265](https://github.com/BerriAI/litellm/pull/33265) - test(e2e): cover key rpm/tpm rate limiting, window reset, and pacing headers by [@​mateo-berri](https://github.com/mateo-berri) in [#​32914](https://github.com/BerriAI/litellm/pull/32914) - fix(anthropic): use native output capability by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​33235](https://github.com/BerriAI/litellm/pull/33235) - fix(ci): retry setup-uv installs to survive transient manifest fetch failures by [@​mateo-berri](https://github.com/mateo-berri) in [#​33279](https://github.com/BerriAI/litellm/pull/33279) - fix(proxy): never log raw virtual keys in key insertion debug output by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33268](https://github.com/BerriAI/litellm/pull/33268) - fix(bedrock\_mantle): route xai.grok-4.3 via /openai/v1 frontier path by [@​marty-sullivan](https://github.com/marty-sullivan) in [#​33027](https://github.com/BerriAI/litellm/pull/33027) - feat(pricing): add gemini-omni-flash-preview with video output token pricing by [@​mateo-berri](https://github.com/mateo-berri) in [#​33274](https://github.com/BerriAI/litellm/pull/33274) - fix(auth): scope the JWT enterprise gate to actual JWTs by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33296](https://github.com/BerriAI/litellm/pull/33296) - fix(s3): sanitize slashes in response-id-derived object key file name by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33271](https://github.com/BerriAI/litellm/pull/33271) - refactor(ui): migrate guardrails table onto shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33303](https://github.com/BerriAI/litellm/pull/33303) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33308](https://github.com/BerriAI/litellm/pull/33308) - feat(guardrails): streaming text transformation in generic\_guardrail\_api by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33110](https://github.com/BerriAI/litellm/pull/33110) - test(e2e): cover model-aware mid-conversation system handling on Bedrock Invoke /v1/messages by [@​mateo-berri](https://github.com/mateo-berri) in [#​32963](https://github.com/BerriAI/litellm/pull/32963) - test(claude\_code): move the Claude Code compatibility matrix under tests/e2e by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​32548](https://github.com/BerriAI/litellm/pull/32548) - chore(ci): sync litellm\_internal\_staging into daily OSS branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33337](https://github.com/BerriAI/litellm/pull/33337) - feat(bedrock guardrails): add resource-less InvokeGuardrailChecks (detect-only) mode by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33299](https://github.com/BerriAI/litellm/pull/33299) - Revert "chore(ci): sync litellm\_internal\_staging into daily OSS branch" by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33339](https://github.com/BerriAI/litellm/pull/33339) - fix(websearch): intercept web search on the Responses API by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33129](https://github.com/BerriAI/litellm/pull/33129) - fix(anthropic-adapter): drop empty content\_block\_delta events by [@​mateo-berri](https://github.com/mateo-berri) in [#​33315](https://github.com/BerriAI/litellm/pull/33315) - fix(mcp): persist discovered OAuth endpoints and keep last known good on failed re-discovery by [@​tin-berri](https://github.com/tin-berri) in [#​33286](https://github.com/BerriAI/litellm/pull/33286) - test(e2e): otel trace completeness on streaming chat, messages, and responses (LIT-3787) by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33234](https://github.com/BerriAI/litellm/pull/33234) - feat(router): resolve auto-router routing plugins from proxy YAML config by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​33251](https://github.com/BerriAI/litellm/pull/33251) - test(e2e): failed request error span carries the full untruncated message and status (LIT-4179) by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33304](https://github.com/BerriAI/litellm/pull/33304) - refactor(ui): migrate tags table onto shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33314](https://github.com/BerriAI/litellm/pull/33314) - fix(cli): surface actionable CLI SSO errors when CLI and proxy versions skew by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33309](https://github.com/BerriAI/litellm/pull/33309) - feat(bedrock\_mantle): add GPT-5.6 sol/terra/luna to model cost map by [@​mateo-berri](https://github.com/mateo-berri) in [#​33412](https://github.com/BerriAI/litellm/pull/33412) - chore(codeowners): exempt generated schema.d.ts from UI ownership by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33411](https://github.com/BerriAI/litellm/pull/33411) - feat(proxy): push-based OTLP billable-request metering for enterprise deployments by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​31592](https://github.com/BerriAI/litellm/pull/31592) - fix(mcp): cap per-user OAuth token cache TTL at the token's own lifetime by [@​tin-berri](https://github.com/tin-berri) in [#​33346](https://github.com/BerriAI/litellm/pull/33346) - feat(ui): move Caching out of Experimental into Developer Tools by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33432](https://github.com/BerriAI/litellm/pull/33432) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33425](https://github.com/BerriAI/litellm/pull/33425) - chore(ci): merge daily internal staging branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33335](https://github.com/BerriAI/litellm/pull/33335) - feat(guardrails): add Compresr guardrail for query-aware context compression by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33295](https://github.com/BerriAI/litellm/pull/33295) - fix(logging): preserve callback order in get\_combined\_callback\_list by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33005](https://github.com/BerriAI/litellm/pull/33005) - fix(logging): redact assistant tool call arguments in spend logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33111](https://github.com/BerriAI/litellm/pull/33111) - fix(anthropic): honor messages request timeout by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33418](https://github.com/BerriAI/litellm/pull/33418) - fix(llm\_guard): apply sanitized prompt returned by moderation API to request by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33331](https://github.com/BerriAI/litellm/pull/33331) - fix(logging): stop pinning large request payloads past request end by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33455](https://github.com/BerriAI/litellm/pull/33455) - feat(guardrails): forward optional metadata on POST /guardrails/apply\_guardrail by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33067](https://github.com/BerriAI/litellm/pull/33067) - build: raise requires-python cap to <3.15 so Python 3.14 installs current releases by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33438](https://github.com/BerriAI/litellm/pull/33438) - feat(ui): add reusable BetaBadge and use it for Projects sidebar item by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33449](https://github.com/BerriAI/litellm/pull/33449) - fix(mcp): discover missing OAuth scopes and token\_url when authorization\_url is set manually by [@​tin-berri](https://github.com/tin-berri) in [#​33317](https://github.com/BerriAI/litellm/pull/33317) - feat(ui): show exact license expiration date in usage cards by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33478](https://github.com/BerriAI/litellm/pull/33478) - test(claude\_code): rename misleading REPO\_ROOT to SUITE\_ROOT in test\_v0\_layout by [@​mateo-berri](https://github.com/mateo-berri) in [#​33472](https://github.com/BerriAI/litellm/pull/33472) - build(deps): update ddtrace to the 4.x line by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33484](https://github.com/BerriAI/litellm/pull/33484) - fix(complexity\_router): return empty dict from \_classifier\_call\_metadata when metadata is absent by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33452](https://github.com/BerriAI/litellm/pull/33452) - fix(ui/chat): resolve chat routes at render time so navigation works under server\_root\_path by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33446](https://github.com/BerriAI/litellm/pull/33446) - fix(key management): enforce minimum custom key length and mask short keys in key\_name by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33462](https://github.com/BerriAI/litellm/pull/33462) - chore(ui): remove unmounted UsageIndicator and the Hide Usage Indicator flag by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33482](https://github.com/BerriAI/litellm/pull/33482) - test(e2e/claude\_code): add passthrough matrix row for the big-3 clouds and Anthropic API by [@​mateo-berri](https://github.com/mateo-berri) in [#​33473](https://github.com/BerriAI/litellm/pull/33473) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33491](https://github.com/BerriAI/litellm/pull/33491) - fix(ui): stop sending the complexity-router pseudo-model to /health/test\_connection by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​33498](https://github.com/BerriAI/litellm/pull/33498) - feat(cli): add lite up/down to ambiently route Claude Code through the proxy by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​33231](https://github.com/BerriAI/litellm/pull/33231) - feat(complexity\_router): enable session\_affinity by default by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33500](https://github.com/BerriAI/litellm/pull/33500) - fix(anthropic): stop 500 on combined thinking+signature streaming chunk by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33505](https://github.com/BerriAI/litellm/pull/33505) - feat(autoroute): prompt for semantic keywords per tier in configure wizard by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33508](https://github.com/BerriAI/litellm/pull/33508) - fix(cli/anthropic): unblock lite autoroute proxy deps, adaptive thinking, and thinking+signature streaming by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33507](https://github.com/BerriAI/litellm/pull/33507) - test(ocr): use mistral-document-ai-2512 in azure\_ai OCR tests by [@​mateo-berri](https://github.com/mateo-berri) in [#​33489](https://github.com/BerriAI/litellm/pull/33489) - fix(guardrails): show YAML-defined guardrails in the Guardrail Monitor by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​32853](https://github.com/BerriAI/litellm/pull/32853) - refactor(ui): migrate policies, deleted keys, deleted teams, budgets, and search tools tables onto shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33357](https://github.com/BerriAI/litellm/pull/33357) - refactor(ui): migrate vector stores, prompts, and skills tables onto shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33343](https://github.com/BerriAI/litellm/pull/33343) - test(e2e): datadog log delivery for successful chat, messages, and responses (LIT-4447) by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33415](https://github.com/BerriAI/litellm/pull/33415) - fix(cli): make CLI output ASCII-only so it doesn't crash legacy Windows consoles by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33465](https://github.com/BerriAI/litellm/pull/33465) - fix: remove dead user-cache lookup with None key in spend-update path by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33555](https://github.com/BerriAI/litellm/pull/33555) - feat(helm): add per-component PodDisruptionBudget and topologySpreadConstraints to componentized chart by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33430](https://github.com/BerriAI/litellm/pull/33430) - fix(e2e/claude\_code): unblock stage collection, align proxy env names, register compat models by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33433](https://github.com/BerriAI/litellm/pull/33433) - fix(mcp): index authed request-time tools missing from the semantic filter startup index by [@​tin-berri](https://github.com/tin-berri) in [#​33318](https://github.com/BerriAI/litellm/pull/33318) - fix(streaming): use provider-reported usage cost for OpenRouter streams by [@​mateo-berri](https://github.com/mateo-berri) in [#​32255](https://github.com/BerriAI/litellm/pull/32255) - feat(mcp): issuer-anchored OAuth discovery (RFC 8414 §3.3) to close the authorization-server mix-up by [@​tin-berri](https://github.com/tin-berri) in [#​33450](https://github.com/BerriAI/litellm/pull/33450) - feat(logging): add user and team level spend and budget to StandardLoggingPayload metadata by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33459](https://github.com/BerriAI/litellm/pull/33459) - fix(router): cast model\_info cost values to float in \_set\_model\_group\_info by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33556](https://github.com/BerriAI/litellm/pull/33556) - chore(e2e): establish litellm\_e2e\_staging integration line by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33502](https://github.com/BerriAI/litellm/pull/33502) - fix(ui): navigate to /ui/login/ with trailing slash via hard navigation by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33561](https://github.com/BerriAI/litellm/pull/33561) - feat(logging): add structured budget fields to budget rejection failure logs by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33460](https://github.com/BerriAI/litellm/pull/33460) - fix(streaming): surface upstream connection resets instead of empty 200 streams by [@​mateo-berri](https://github.com/mateo-berri) in [#​33222](https://github.com/BerriAI/litellm/pull/33222) - fix(proxy\_cli): reap orphaned prisma query-engine processes when a worker dies by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33424](https://github.com/BerriAI/litellm/pull/33424) - test(reasoning\_effort\_grid): enable azure fable-5 and opus-4-8 grid cells by [@​mateo-berri](https://github.com/mateo-berri) in [#​33485](https://github.com/BerriAI/litellm/pull/33485) - build(deps): bump uvicorn lock to 0.51.0 so worker health-check and jitter flags take effect by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33574](https://github.com/BerriAI/litellm/pull/33574) - fix(proxy): coerce default\_internal\_user\_params.max\_budget to float on config load by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​32434](https://github.com/BerriAI/litellm/pull/32434) - fix(router): honor per-request routing\_strategy from key/team router\_settings by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33429](https://github.com/BerriAI/litellm/pull/33429) - fix(redis): honor ssl value instead of key presence when building async connection pool by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​32590](https://github.com/BerriAI/litellm/pull/32590) - fix(langfuse\_otel): build per-request OTLP exporter from key and team dynamic Langfuse credentials by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​32437](https://github.com/BerriAI/litellm/pull/32437) - ci: run zizmor and proxy-db unit tests on PRs targeting litellm\_ branches by [@​mateo-berri](https://github.com/mateo-berri) in [#​33568](https://github.com/BerriAI/litellm/pull/33568) - fix(router): apply team/key enable\_tag\_filtering to tag routing by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33436](https://github.com/BerriAI/litellm/pull/33436) - feat(proxy): add disable\_auto\_add\_proxy\_admin\_to\_teams flag by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33563](https://github.com/BerriAI/litellm/pull/33563) - fix(proxy): stop stale auth cache re-publish to Redis so key updates propagate across replicas by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33565](https://github.com/BerriAI/litellm/pull/33565) - feat(e2e): emit structured E2E\_RESULT lines for package status history by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33578](https://github.com/BerriAI/litellm/pull/33578) - chore: bump litellm-enterprise 0.1.50 -> 0.1.51, litellm-proxy-extras 0.4.77 -> 0.4.78 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33571](https://github.com/BerriAI/litellm/pull/33571) - fix(docker): restore litellm-proxy-extras source dir in runtime images by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33592](https://github.com/BerriAI/litellm/pull/33592) - feat(ui): require embedding model for semantic auto router by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33313](https://github.com/BerriAI/litellm/pull/33313) - feat(scim): ingest and round-trip SCIM entitlements and roles user attributes by [@​tin-berri](https://github.com/tin-berri) in [#​33587](https://github.com/BerriAI/litellm/pull/33587) - refactor(ui): migrate 5 simple tables onto shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33548](https://github.com/BerriAI/litellm/pull/33548) - fix(model\_armor): restore reference attachments via skip\_unscannable\_attachments and remove the attachment count cap by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33554](https://github.com/BerriAI/litellm/pull/33554) - fix(mcp): keep the MCP reference intact when the semantic filter narrows tools by [@​tin-berri](https://github.com/tin-berri) in [#​33584](https://github.com/BerriAI/litellm/pull/33584) - fix(sso): stop enforcing UI session budget on CLI login tokens by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33312](https://github.com/BerriAI/litellm/pull/33312) - test: e2e staging leftovers by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33613](https://github.com/BerriAI/litellm/pull/33613) - test(e2e/claude\_code): add GPT-5.6 Sol/Terra/Luna columns for OpenAI, Azure OpenAI, and Bedrock Mantle by [@​mateo-berri](https://github.com/mateo-berri) in [#​33474](https://github.com/BerriAI/litellm/pull/33474) - test(e2e): otel streaming spans record a real ttft below span duration by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33588](https://github.com/BerriAI/litellm/pull/33588) - fix(mcp): make the preemptive-401 OAuth challenge decision mode-aware by [@​tin-berri](https://github.com/tin-berri) in [#​33586](https://github.com/BerriAI/litellm/pull/33586) - fix(ui): show all teams in policy attachment form for admins by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33628](https://github.com/BerriAI/litellm/pull/33628) - refactor(ui): migrate AI Hub, public hub, and MCP Toolsets tables onto shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33629](https://github.com/BerriAI/litellm/pull/33629) - test(e2e): datadog log delivery for streamed routes, read back from the real datadog api by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33566](https://github.com/BerriAI/litellm/pull/33566) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33640](https://github.com/BerriAI/litellm/pull/33640) - fix(vertex\_ai): surface Gemini grounding toolUsePromptTokenCount in Usage by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33533](https://github.com/BerriAI/litellm/pull/33533) - test(e2e): harness fixes for stage job green (skips + router/UI/budget) by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33634](https://github.com/BerriAI/litellm/pull/33634) - fix(router): resolve prompt cache minimum per model instead of a flat 1024 by [@​tin-berri](https://github.com/tin-berri) in [#​33637](https://github.com/BerriAI/litellm/pull/33637) - fix(logging): classify async anthropic\_messages and generate\_content as async by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33589](https://github.com/BerriAI/litellm/pull/33589) - fix(ui): remove Chat item from dashboard leftnav by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33647](https://github.com/BerriAI/litellm/pull/33647) - fix(router): tag-aware pre-routing strategy selection for shared model\_name by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33691](https://github.com/BerriAI/litellm/pull/33691) - fix(proxy): enforce max\_parallel\_requests as a per-slot concurrency gauge by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​32441](https://github.com/BerriAI/litellm/pull/32441) - fix(proxy): stop treating upstream model body field as a LiteLLM model on auth-enforced pass-through routes by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33710](https://github.com/BerriAI/litellm/pull/33710) - fix(mcp): expand toolset grants in shared permission primitives so tools/call honors them by [@​tin-berri](https://github.com/tin-berri) in [#​33612](https://github.com/BerriAI/litellm/pull/33612) - feat(complexity-router): user-triggered escalation keywords by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33656](https://github.com/BerriAI/litellm/pull/33656) - fix(fireworks\_ai): bill prompt-cache hits at cache\_read rate by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33714](https://github.com/BerriAI/litellm/pull/33714) - fix(pricing): mark realtime-only gpt-realtime models as mode realtime by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33728](https://github.com/BerriAI/litellm/pull/33728) - fix(rag): track LLM completion usage and spend for /v1/rag/query by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​32438](https://github.com/BerriAI/litellm/pull/32438) - feat(anthropic): add enable\_anthropic\_prompt\_caching for automatic cache\_control injection by [@​tin-berri](https://github.com/tin-berri) in [#​33573](https://github.com/BerriAI/litellm/pull/33573) - fix(anthropic): self-heal on missing thinking-signature errors from Bedrock/Vertex by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33719](https://github.com/BerriAI/litellm/pull/33719) - fix(proxy): resolve router\_settings.plugins dotted paths and load plugins from installed packages by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33644](https://github.com/BerriAI/litellm/pull/33644) - test(e2e): budget refusals are 429 for bare keys and team caps block every team key by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33632](https://github.com/BerriAI/litellm/pull/33632) - feat(router): add router plugin reference catalog by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33746](https://github.com/BerriAI/litellm/pull/33746) - test(e2e): assert an org budget block is a 429 naming the organization by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33638](https://github.com/BerriAI/litellm/pull/33638) - fix(proxy): bill partial streamed spend when the client disconnects mid-stream by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33736](https://github.com/BerriAI/litellm/pull/33736) - test(e2e): delete unreferenced Grafana panel docs by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33743](https://github.com/BerriAI/litellm/pull/33743) - docs(tests/e2e): align skip-vs-fail docs with the hard-fail contract and scope the no-unit-tests rule by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33755](https://github.com/BerriAI/litellm/pull/33755) - refactor(e2e): replace bespoke result reporter with standard JUnit report by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33758](https://github.com/BerriAI/litellm/pull/33758) - test(e2e): user budget across keys and team member budget isolation by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33745](https://github.com/BerriAI/litellm/pull/33745) - refactor(e2e): remove bob\_the\_builder; drive remediation from a Grafana alert (provisioned outside the repo) by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33749](https://github.com/BerriAI/litellm/pull/33749) - feat(mcp): per-server outcomes for aggregate tools/list and truthful single-server REST statuses by [@​tin-berri](https://github.com/tin-berri) in [#​33153](https://github.com/BerriAI/litellm/pull/33153) - test(e2e): mcp suite for key-without-access denial by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33752](https://github.com/BerriAI/litellm/pull/33752) - chore(ci): merge oss branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33784](https://github.com/BerriAI/litellm/pull/33784) - chore(ci): merge oss branch - July 17th by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33793](https://github.com/BerriAI/litellm/pull/33793) - fix(ui): migrate tag deletion to shared DeleteResourceModal by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33795](https://github.com/BerriAI/litellm/pull/33795) - build(rust): raise pyo3 to 0.29 so the native bridge compiles on Python 3.14 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33798](https://github.com/BerriAI/litellm/pull/33798) - chore(guardrails): remove docstring from singulr module for consistency by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33800](https://github.com/BerriAI/litellm/pull/33800) - fix(ui): stop credential edit from persisting the masked api key by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33797](https://github.com/BerriAI/litellm/pull/33797) - build(deps): allow redisvl, pypdf, and openapi-core on Python 3.14 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33801](https://github.com/BerriAI/litellm/pull/33801) - test(proxy): make streaming-cancel mocks awaitable for the disconnect slot release by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33802](https://github.com/BerriAI/litellm/pull/33802) - test(e2e): a member's team budget cuts off only that member's key by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33718](https://github.com/BerriAI/litellm/pull/33718) - test(e2e): a user's max\_budget follows the person across personal and team keys by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33762](https://github.com/BerriAI/litellm/pull/33762) - test(e2e): skip flaky OpenAI GPT cells; raise multi-window max\_tokens by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33799](https://github.com/BerriAI/litellm/pull/33799) - chore: remove accidentally committed dist tarball and ignore dist/ by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33805](https://github.com/BerriAI/litellm/pull/33805) - fix(passthrough): stop classifying plain 'predict'/'search' paths as Vertex by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33658](https://github.com/BerriAI/litellm/pull/33658) - build(deps): bump mcp lock to 1.28.1 to clear image-scan findings by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33803](https://github.com/BerriAI/litellm/pull/33803) - test(pricing): pin the realtime mode assertion to the bundled cost map by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33806](https://github.com/BerriAI/litellm/pull/33806) - fix(proxy): derive session id from Anthropic metadata.user\_id for session affinity by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33723](https://github.com/BerriAI/litellm/pull/33723) - test(e2e): budget reset diagonal for team, org, user, and [#​32005](https://github.com/BerriAI/litellm/issues/32005) team-member keys by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33771](https://github.com/BerriAI/litellm/pull/33771) - fix(proxy): source /v1/models token limits from the cost map instead of Router.get\_model\_group\_info by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33721](https://github.com/BerriAI/litellm/pull/33721) - fix(fireworks\_ai): correct glm-5p2 prompt-cache read price to $0.14/1M by [@​tin-berri](https://github.com/tin-berri) in [#​33796](https://github.com/BerriAI/litellm/pull/33796) - feat(proxy): add x-litellm-model-name response header with deployment model string by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33698](https://github.com/BerriAI/litellm/pull/33698) - feat: add Straiker guardrail integration by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33781](https://github.com/BerriAI/litellm/pull/33781) - fix(vertex\_ai): exclude Gemini Google Search grounding tokens from input token billing by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33742](https://github.com/BerriAI/litellm/pull/33742) - feat(fireworks\_ai): map litellm session id to x-session-affinity header for prompt caching by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33717](https://github.com/BerriAI/litellm/pull/33717) - feat(ui): configure Anthropic automatic prompt caching from the Admin UI by [@​tin-berri](https://github.com/tin-berri) in [#​33581](https://github.com/BerriAI/litellm/pull/33581) - fix(router): enforce context-window pre-call checks for Responses API input by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33706](https://github.com/BerriAI/litellm/pull/33706) - fix(otel): restore proxy-level error.\* attributes on v2 failure spans (LIT-4179) by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33664](https://github.com/BerriAI/litellm/pull/33664) - fix(mcp): persist config.yaml DCR clients in a server-scoped store so refresh survives token expiry by [@​tin-berri](https://github.com/tin-berri) in [#​33768](https://github.com/BerriAI/litellm/pull/33768) - refactor(ui): consolidate Add/Edit credential modals into one CredentialModal by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32572](https://github.com/BerriAI/litellm/pull/32572) - feat(mcp): add ID-JAG (identity assertion authorization grant) support for MCP egress by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​31516](https://github.com/BerriAI/litellm/pull/31516) - refactor(ui): migrate policy attachments table onto shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33827](https://github.com/BerriAI/litellm/pull/33827) - docs(litellm-rust): add provider coding standards by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33833](https://github.com/BerriAI/litellm/pull/33833) - test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped pytest fixture by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33750](https://github.com/BerriAI/litellm/pull/33750) - feat(messages): route Azure Anthropic /messages through Rust behind rust:true by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33616](https://github.com/BerriAI/litellm/pull/33616) - test(e2e): add Locust throughput load test that runs last by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33748](https://github.com/BerriAI/litellm/pull/33748) - fix(proxy): resolve team wildcard credentials for vector store files by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​33649](https://github.com/BerriAI/litellm/pull/33649) - refactor(e2e): fold claude\_code HTTP probes onto shared ProxyClient methods by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33760](https://github.com/BerriAI/litellm/pull/33760) - test(e2e): harden stage flakes for batches, UI, and MCP by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33831](https://github.com/BerriAI/litellm/pull/33831) - fix(e2e): migrate load suite from e2e\_gateway to ProxyClient by [@​mubashir1osmani](https://github.com/mubashir1osmani) in [#​33839](https://github.com/BerriAI/litellm/pull/33839) - chore(e2e): remove tests/e2e/docker-compose.yml by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33837](https://github.com/BerriAI/litellm/pull/33837) - test(e2e): cover /v1/responses openai basic nonstream and stream by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33830](https://github.com/BerriAI/litellm/pull/33830) - test(e2e): cover /v1/responses openai cost\_logged and tool\_use by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33835](https://github.com/BerriAI/litellm/pull/33835) - test(e2e): cover /v1/responses OpenAI vision and Anthropic basic by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33838](https://github.com/BerriAI/litellm/pull/33838) - test(e2e): spendlog cost for streaming /v1/messages via responses bridge by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​33753](https://github.com/BerriAI/litellm/pull/33753) - fix(docker): bake prisma CLI and engines at a fixed path so fresh-DB migrations work for any uid offline by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33853](https://github.com/BerriAI/litellm/pull/33853) - feat(chat-ui): add personal Logs view scoped to the current user by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33829](https://github.com/BerriAI/litellm/pull/33829) - chore: bump litellm-proxy-extras 0.4.78 -> 0.4.79 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33855](https://github.com/BerriAI/litellm/pull/33855) - docs(litellm-rust): require the official Rust Style Guide in agent rules by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33867](https://github.com/BerriAI/litellm/pull/33867) - fix(router): treat malformed configured token limits as absent on /v1/models by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33864](https://github.com/BerriAI/litellm/pull/33864) - chore: rebuild admin UI bundle for the rc release by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33857](https://github.com/BerriAI/litellm/pull/33857) - docs(rust): add provider abstraction standards by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33865](https://github.com/BerriAI/litellm/pull/33865) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33868](https://github.com/BerriAI/litellm/pull/33868) - chore(release): backport [#​33929](https://github.com/BerriAI/litellm/issues/33929) to rc/1.94.0 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34033](https://github.com/BerriAI/litellm/pull/34033) - chore(release): backport [#​33810](https://github.com/BerriAI/litellm/issues/33810), [#​33733](https://github.com/BerriAI/litellm/issues/33733) to rc/1.94.0 and bump litellm-proxy-extras to 0.4.79.post1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34215](https://github.com/BerriAI/litellm/pull/34215) - chore(ui): rebuild Next.js bundle on rc/1.94.0 so the Cost Optimization page ships by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34216](https://github.com/BerriAI/litellm/pull/34216) - chore(release): backport auth, CLI SSO and guardrail fixes to rc/1.94.0 and refresh flagged dependencies by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34640](https://github.com/BerriAI/litellm/pull/34640) - chore(release): backport [#​33899](https://github.com/BerriAI/litellm/issues/33899), [#​33978](https://github.com/BerriAI/litellm/issues/33978), [#​34582](https://github.com/BerriAI/litellm/issues/34582), [#​34675](https://github.com/BerriAI/litellm/issues/34675) to rc/1.94.0 and bump litellm-proxy-extras to 0.4.79.post2 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34855](https://github.com/BerriAI/litellm/pull/34855) - fix(ui): backport cache leakage card layout fix to rc/1.94.0 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34964](https://github.com/BerriAI/litellm/pull/34964) - fix(ui): add missing cost-optimization page description on rc/1.94.0 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34967](https://github.com/BerriAI/litellm/pull/34967) - feat(ui): mark Cost Optimization as beta in the left nav ([#​34984](https://github.com/BerriAI/litellm/issues/34984)) by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34987](https://github.com/BerriAI/litellm/pull/34987) - chore: rebuild Admin UI bundle for v1.94.0 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34982](https://github.com/BerriAI/litellm/pull/34982) - fix(cost-optimization): backport the savings chart axis fix and methodology popovers to rc/1.94.0 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34994](https://github.com/BerriAI/litellm/pull/34994) - chore: rebuild Admin UI bundle for rc/1.94.0 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​34995](https://github.com/BerriAI/litellm/pull/34995) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.93.0...v1.94.0> ### [`v1.94.0`](https://github.com/BerriAI/litellm/releases/tag/v1.94.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.94.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.94.0/cosign.pub \ ghcr.io/berriai/litellm:v1.94.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - feat(ui): working Test Connection for the complexity auto router by [@​akapur99](https://github.com/akapur99) in [#​32950](https://github.com/BerriAI/litellm/pull/32950) - fix(xecguard): use StandardLoggingGuardrailInformation in logging hook by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​32911](https://github.com/BerriAI/litellm/pull/32911) - feat(ui): adopt openapi-react-query ($api) and convert useCustomers by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32949](https://github.com/BerriAI/litellm/pull/32949) - refactor(ui): colocate the mcp-servers view, keeping the shared mcp\_tools surface by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32968](https://github.com/BerriAI/litellm/pull/32968) - refactor(ui): convert endpoint usage charts to shadcn/recharts by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32723](https://github.com/BerriAI/litellm/pull/32723) - fix(proxy-auth): stop unrecognized model namespaces slipping through provider wildcard keys by [@​mateo-berri](https://github.com/mateo-berri) in [#​32979](https://github.com/BerriAI/litellm/pull/32979) - feat(router): random-pick multi-model complexity tiers by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32967](https://github.com/BerriAI/litellm/pull/32967) - fix(xecguard): sanitize scan result before recording it for logging by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​32935](https://github.com/BerriAI/litellm/pull/32935) - fix(auto\_router): filter embedding models in complexity tab dropdowns, require all tiers, inline validation by [@​akapur99](https://github.com/akapur99) in [#​32978](https://github.com/BerriAI/litellm/pull/32978) - fix(anthropic): translate raw adaptive thinking for pre-4.6 models on chat completions and Bedrock Converse by [@​akapur99](https://github.com/akapur99) in [#​32944](https://github.com/BerriAI/litellm/pull/32944) - feat(router): add Router(plugins=\[...]) routing-plugin pipeline by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32972](https://github.com/BerriAI/litellm/pull/32972) - feat(router): soft-floor adaptive mode for complexity router by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32947](https://github.com/BerriAI/litellm/pull/32947) - docs(github): add QA runbook section to the PR template by [@​mateo-berri](https://github.com/mateo-berri) in [#​32965](https://github.com/BerriAI/litellm/pull/32965) - fix(model\_cost): add supports\_reasoning: false to Gemini image generation models by [@​mateo-berri](https://github.com/mateo-berri) in [#​32836](https://github.com/BerriAI/litellm/pull/32836) - build(dev-env): add make bootstrap and unprovisioned-checkout preflight to pre-commit by [@​mateo-berri](https://github.com/mateo-berri) in [#​32981](https://github.com/BerriAI/litellm/pull/32981) - ci(ui): report only error-level knip findings in CI by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​32971](https://github.com/BerriAI/litellm/pull/32971) - feat(batches): track cost for unmanaged Bedrock batches, generalize the flag by [@​Sameerlite](https://github.com/Sameerlite) in [#​32315](https://github.com/BerriAI/litellm/pull/32315) - fix(guardrails): walk custom\_tool\_call\_output items in \_content\_utils by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​32969](https://github.com/BerriAI/litellm/pull/32969) - fix: show and allow editing team model aliases after team creation by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33047](https://github.com/BerriAI/litellm/pull/33047) - chore(deps): bump pillow to 12.3.0 to resolve osv-scan CVEs by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33093](https://github.com/BerriAI/litellm/pull/33093) - feat(mcp): mint gateway-bound envelope at the token endpoint for dcr\_bridge oauth\_delegate by [@​tin-berri](https://github.com/tin-berri) in [#​32828](https://github.com/BerriAI/litellm/pull/32828) - fix(mcp): surface rejected delegate-auth upstream tokens as connect-time 401 by [@​tin-berri](https://github.com/tin-berri) in [#​32741](https://github.com/BerriAI/litellm/pull/32741) - fix(proxy): track unauthenticated pass-through requests in spend logs by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​32410](https://github.com/BerriAI/litellm/pull/32410) - feat(lasso): send source.type for Used By attribution by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33090](https://github.com/BerriAI/litellm/pull/33090) - fix(responses): continue MCP gateway tool turns from the final response and surface failures by [@​thibault-linktree](https://github.com/thibault-linktree) in [#​33025](https://github.com/BerriAI/litellm/pull/33025) - fix(responses): continue MCP gateway tool turns from the final response and surface failures by [@​tin-berri](https://github.com/tin-berri) in [#​33099](https://github.com/BerriAI/litellm/pull/33099) - fix(completion): forward aws credential kwargs into litellm\_params so the responses bridge keeps WIF auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​32956](https://github.com/BerriAI/litellm/pull/32956) - fix(ui): respect litellm\_key\_header\_name in BYOK credential save and workflow runs fetches by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33103](https://github.com/BerriAI/litellm/pull/33103) - refactor(ui): standardize debounce waits behind shared DEBOUNCE\_WAIT\_MS constant by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33040](https://github.com/BerriAI/litellm/pull/33040) - feat(ui): rebuild the Virtual Keys table on the shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​32991](https://github.com/BerriAI/litellm/pull/32991) - fix: redact async complete streaming response for custom callbacks by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33106](https://github.com/BerriAI/litellm/pull/33106) - build(ui): bump [@​tanstack/react-pacer](https://github.com/tanstack/react-pacer) from 0.2.0 to 0.22.1 by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33041](https://github.com/BerriAI/litellm/pull/33041) - fix(ui): address Virtual Keys redesign review nits by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33112](https://github.com/BerriAI/litellm/pull/33112) - fix(openai/responses): clamp max\_output\_tokens below API minimum by [@​devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#​33098](https://github.com/BerriAI/litellm/pull/33098) - fix(prometheus): read v3 rate limiter remaining values for per-key model gauges by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​33119](https://github.com/BerriAI/litellm/pull/33119) - fix(ui): drop w-full from page-content wrappers to remove 32px horizontal overflow by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33118](https://github.com/BerriAI/litellm/pull/33118) - refactor(ui): migrate straightforward value debounces to react-pacer by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33042](https://github.com/BerriAI/litellm/pull/33042) - feat(mcp): interactive SSO sign-in for dcr\_bridge oauth\_delegate DCR clients by [@​tin-berri](https://github.com/tin-berri) in [#​32946](https://github.com/BerriAI/litellm/pull/32946) - test(proxy): add regression tests for management\_endpoints edge cases by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​32976](https://github.com/BerriAI/litellm/pull/32976) - fix(auto-router): correct Responses API tool\_choice shape and propagate alias litellm\_params by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​32974](https://github.com/BerriAI/litellm/pull/32974) - fix(ui): render the sidebar scrollbar with shadcn ScrollArea by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33124](https://github.com/BerriAI/litellm/pull/33124) - refactor(ui): migrate callback debounce sites to react-pacer with regression tests by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​33043](https://github.com/BerriAI/litellm/pull/33043) - chore: add CODEOWNERS for ui and proxy UI build artifacts by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33131](https://github.com/BerriAI/litellm/pull/33131) - feat(mcp): client-held refresh envelope for the dcr\_bridge oauth\_delegate flow by [@​tin-berri](https://github.com/tin-berri) in [#​32980](https://github.com/BerriAI/litellm/pull/32980) - feat(ui): rebuild the Teams table on the shared DataTable by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​33128](https://github.com/BerriAI/litellm/pull/33128) - fix(mcp): relay upstream OAuth token and DCR rejections instead of a generic 500 by [@​tin-berri](https://github.com/tin-berri) in [#​33113](https://github.com/BerriAI/litellm/pull/33113) - fix(keys): persist key\_type so the UI shows correct key scope instead of "All Proxy…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes
QA runbook
Final Attestation