chore: sync upstream v1.93.0 - #14
Merged
Merged
Conversation
…ed refs (BerriAI#32401) * fix(ui): forward refs through ui primitives and fail tests on swallowed refs Under React 18 a ref passed to a plain function component is dropped with only a dev console warning, so Base UI render-prop triggers composed over our shadcn-style primitives silently stop working (the tooltip just never opens; ui/badge.tsx hit exactly this on the shared DataTable branch). Label, Separator, Skeleton, UiLoadingSpinner and the Table family now use React.forwardRef like Button and Input already did, a contract test pins ref delivery for each, and setupTests turns React's ref warning into a test failure so the next primitive that swallows a ref fails CI instead of shipping a dead tooltip * fix(ui): include captured ref warnings in the tripwire error The afterEach tripwire threw a fixed message and discarded the collected React warnings, so a failure never said which component swallowed the ref. Append the captured warnings (component name + stack) to the thrown error.
…erriAI#32576) The App Router migration is complete: every page is a path route and the legacy `?page=` switch is gone from the index. This closes it out. The `/ui/` index (page.tsx) kept its own duplicate copy of teams state, a teams fetch, and keys/addKey plumbing solely to feed a second `UserDashboard` render for the `invitation_id` case. That was redundant: `ApiKeysDashboard` already renders `UserDashboard` sourcing its own data, so the index is thinned to just render `<ApiKeysDashboard />`. The login redirect, the legacy `?page=` deep-link redirect for old bookmarks, and the post-login return-URL handling stay on the index. The invitation entry point now resolves in one place. Modern invitation links already point at the dedicated `/onboarding` route; the dashboard layout now redirects legacy `/ui/?invitation_id=` links there too (via `migratedHref`, the same base-aware redirect the index uses for `?page=`), instead of re-rendering that route's page component inline. This removes an import of one route's `page.tsx` into another module, and lets the now-unreachable `if (invitation_id) return <Onboarding/>` branch in the shared `user_dashboard.tsx` be deleted along with its dead `Onboarding` import and `searchParams` read. A layout test asserts the redirect and fails if it regresses. `legacyPageHref` and the sidebar's migrated-vs-legacy href fallback are left in place; they are still live for the parent-category nav nodes (agentic, tools, experimental, settings) that are not page routes. eslint-metrics.json is resynced: -2 no-explicit-any from the removed `any` casts, plus pre-existing drift the gate requires the snapshot to match.
…ModifyResponseException (BerriAI#32289) * fix(guardrails/bedrock): honor disable_exception_on_block by raising ModifyResponseException The Bedrock-specific GuardrailInterventionNormalStringError predates the unified guardrails refactor and no proxy code path handles it, so a block with the flag set surfaced as an uncaught Exception -> HTTP 500 in pre_call mode and was silently discarded in during_call mode (model call proceeded in the parallel asyncio.gather; the block hook's data["mock_response"] mutation happened after route_request had already unpacked kwargs). Convert the block to ModifyResponseException at the raise site inside make_bedrock_api_request. That exception is the industry-standard proxy contract already caught in proxy_server, anthropic_endpoints, response_api _endpoints, and pass_through_endpoints; it turns into a 200 response with finish_reason=content_filter and the block message as content, which is exactly what the flag was documented to yield. Post-call blocks attach the LLM response to original_response so the synthetic reply reports the upstream call's real token usage instead of zero. Deletes the now-orphaned GuardrailInterventionNormalStringError class and the dead create_guardrail_blocked_response / mock_response plumbing in the Bedrock hooks; updates the existing tests that had locked in the buggy contract. Resolves LIT-4186 * chore(guardrails/bedrock): drop dead str branch in _update_messages_with_updated_bedrock_guardrail_response Follow-up to the disable_exception_on_block fix. That method used to receive either a BedrockGuardrailResponse or a plain string (the block message, when the flag was set). Now that a block always raises ModifyResponseException before this method runs, the string branch is unreachable; tighten the type to BedrockGuardrailResponse and delete the guard. * fix(guardrails/bedrock): streaming post_call block yields synthetic stream instead of surfacing as SSE 500 Regression from the LIT-4186 refactor: pre-refactor, the streaming post_call iterator caught GuardrailInterventionNormalStringError locally and replaced the assembled response with a synthetic content-filter message, then re-emitted it as chunks via MockResponseIterator. After the refactor the exception was re-raised as ModifyResponseException, which async_streaming_data_generator serializes as a proxy 500 error frame because the SSE response headers are already flushed by the time the block fires. Non-streaming paths still let ModifyResponseException propagate to the endpoint handler (which converts it into a 200). Streaming can't do that, so keep the local synthesis: on the exception, rebind the assembled response to a ModelResponse whose single choice carries the block message as content and finish_reason=content_filter, and let the downstream MockResponseIterator emit it as chunks. Same shape a non-streaming block produces. Adds a mapped-file regression test that mutation-kills the raise behavior and locks in the synthetic-stream contract. * fix(guardrails/bedrock): preserve upstream usage on streaming post_call block Non-streaming post_call blocks report the upstream LLM call's real token usage via ModifyResponseException.original_response, which the endpoint handler unwraps through _blocked_response_usage. Streaming post_call synthesizes its own ModelResponse locally (the exception can't escape the SSE generator), and previously left .usage unset, so the client saw accurate billing on non-streaming blocks and zero on streaming blocks -- silent revenue leak. Copy the assembled response's .usage onto the synthetic block response before yielding. Pre-refactor code had the same gap (create_guardrail_blocked_response never set usage); this is a net improvement, not a regression fix.
…ctive auth type The edit form decided auth mode from the saved mcpServer.auth_type in fetchTools while the authorize flow used the current form value, so a token authorized after switching the form to a client-forwarded mode was never forwarded as the x-mcp header until the server was saved. A shared getEffectiveAuthType (form value falling back to the saved record) is now the single decision point for token receipt and tool loading The save path classified the staged token with getMcpOAuthMode, which returns null for true_passthrough and oauth_delegate, so the staged token was dropped on save instead of being committed to sessionStorage the way the create form's submit path does. The passthrough branch now also covers the client-forwarded modes; the token still never enters the server row
BerriAI#32670) create_model now waits until the new deployment is servable on the data plane (polls /v1/models) before returning, instead of assuming /model/new makes it instantly callable. On a split control/data-plane proxy the gateway only sees a model after its next DB reload, so an immediate call raced the reload and 400'd with "Invalid model name passed" (embeddings, responses, messages, ocr, ...). It also stops pinning model_info.id to the model_name, letting the proxy assign a unique model_id. Re-registering a fixed-name deployment (the batch suite's openai-batch et al.) after a failed teardown no longer collides on the model_id unique constraint (prisma UniqueViolationError surfaced as the generic 500 "Failed to add model to db", erroring every batch_lifecycle case at setup)
… in ModifyResponseException streaming path (BerriAI#32665) * fix(guardrails/bedrock): honor disable_exception_on_block by raising ModifyResponseException The Bedrock-specific GuardrailInterventionNormalStringError predates the unified guardrails refactor and no proxy code path handles it, so a block with the flag set surfaced as an uncaught Exception -> HTTP 500 in pre_call mode and was silently discarded in during_call mode (model call proceeded in the parallel asyncio.gather; the block hook's data["mock_response"] mutation happened after route_request had already unpacked kwargs). Convert the block to ModifyResponseException at the raise site inside make_bedrock_api_request. That exception is the industry-standard proxy contract already caught in proxy_server, anthropic_endpoints, response_api _endpoints, and pass_through_endpoints; it turns into a 200 response with finish_reason=content_filter and the block message as content, which is exactly what the flag was documented to yield. Post-call blocks attach the LLM response to original_response so the synthetic reply reports the upstream call's real token usage instead of zero. Deletes the now-orphaned GuardrailInterventionNormalStringError class and the dead create_guardrail_blocked_response / mock_response plumbing in the Bedrock hooks; updates the existing tests that had locked in the buggy contract. Resolves LIT-4186 * chore(guardrails/bedrock): drop dead str branch in _update_messages_with_updated_bedrock_guardrail_response Follow-up to the disable_exception_on_block fix. That method used to receive either a BedrockGuardrailResponse or a plain string (the block message, when the flag was set). Now that a block always raises ModifyResponseException before this method runs, the string branch is unreachable; tighten the type to BedrockGuardrailResponse and delete the guard. * fix(guardrails/bedrock): streaming post_call block yields synthetic stream instead of surfacing as SSE 500 Regression from the LIT-4186 refactor: pre-refactor, the streaming post_call iterator caught GuardrailInterventionNormalStringError locally and replaced the assembled response with a synthetic content-filter message, then re-emitted it as chunks via MockResponseIterator. After the refactor the exception was re-raised as ModifyResponseException, which async_streaming_data_generator serializes as a proxy 500 error frame because the SSE response headers are already flushed by the time the block fires. Non-streaming paths still let ModifyResponseException propagate to the endpoint handler (which converts it into a 200). Streaming can't do that, so keep the local synthesis: on the exception, rebind the assembled response to a ModelResponse whose single choice carries the block message as content and finish_reason=content_filter, and let the downstream MockResponseIterator emit it as chunks. Same shape a non-streaming block produces. Adds a mapped-file regression test that mutation-kills the raise behavior and locks in the synthetic-stream contract. * fix(guardrails/bedrock): preserve upstream usage on streaming post_call block Non-streaming post_call blocks report the upstream LLM call's real token usage via ModifyResponseException.original_response, which the endpoint handler unwraps through _blocked_response_usage. Streaming post_call synthesizes its own ModelResponse locally (the exception can't escape the SSE generator), and previously left .usage unset, so the client saw accurate billing on non-streaming blocks and zero on streaming blocks -- silent revenue leak. Copy the assembled response's .usage onto the synthetic block response before yielding. Pre-refactor code had the same gap (create_guardrail_blocked_response never set usage); this is a net improvement, not a regression fix. * fix(proxy): capture logging_obj before post_call_failure_hook pops it in ModifyResponseException streaming path post_call_failure_hook removes litellm_logging_obj from request_data before iterating callbacks (it's not serialisable). The streaming branch of the ModifyResponseException handler read it from _data after that call, so it always received None and CustomStreamWrapper.__init__ crashed with AttributeError: NoneType has no attribute model_call_details. Capture it before the hook runs so the streaming path gets a valid object. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(proxy): add regression for streaming ModifyResponseException logging_obj capture Covers the bug where logging_obj was read from request_data after post_call_failure_hook had already popped it, causing CustomStreamWrapper to crash with AttributeError. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(proxy): drive real chat_completion in ModifyResponseException streaming logging_obj regression The original test inlined the fix pattern (capture before pop) in its own body rather than calling the actual chat_completion handler in proxy_server.py, so a revert of the fix left the test passing. Confirmed via mutation check: reverting the two-line source fix and re-running left the test green. Rewrite the test to drive chat_completion directly: - patch _read_request_body so chat_completion sees the seeded dict - patch ProxyBaseLLMRequestProcessing.base_process_llm_request to raise ModifyResponseException with the same request_data - patch proxy_logging_obj so post_call_failure_hook mutates the dict the way production does (pops litellm_logging_obj) - intercept CustomStreamWrapper.__init__ and assert logging_obj is the non-None object seeded in request_data Mutation-verified: reverting the source fix now surfaces the exact production crash inside CustomStreamWrapper's __init__ (AttributeError: NoneType has no attribute model_call_details) rather than a silently-passing test. Addresses Greptile P1 on PR BerriAI#32665. --------- Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
…okenMode helper The helper extraction missed this call site, leaving an inline duplicate of the two-mode check that could drift from the shared definition
…h_workflow ci: add OSS daily branch workflow
* fix: rust ocr tests finally pass * fix: move realtime dir * fix(realtime): normalize azure realtime api_base to host for Foundry endpoints The azure realtime handler appended the realtime path to api_base verbatim, so a Foundry base carrying a project path (.../api/projects/<name>) produced an invalid realtime URL and the websocket handshake hung. Normalize api_base to scheme and host before building the realtime path so both Azure OpenAI and Foundry bases connect Point the e2e realtime azure deployment at the GA gpt-realtime model and stop passing the os.environ refs the realtime path never unwraps, resolving them from the gateway env by name instead. Drop the local docker-compose scaffolding from the tree * test(e2e): add Gateway.list_files and list_fine_tuning_jobs for the discovery suite The discovery endpoints suite calls client.gateway.list_files and list_fine_tuning_jobs, which did not exist on Gateway, so both tests errored with AttributeError before reaching the proxy. Add the two GET wrappers using the existing FileListResponse / FineTuningJobsResponse models * revert(realtime): drop azure realtime api_base host-normalization The azure realtime handshake failure was a config issue, not a litellm bug: the realtime base was set to the Azure AI Foundry project endpoint (.../api/projects/<p>), but the OpenAI-compatible realtime route lives at the resource root. litellm correctly appends the realtime path to whatever base it is given, so pointing the realtime deployment at the resource root is the fix and no core change is needed * fix(ocr): route azure_ai doc-intelligence to its own endpoint at the source get_llm_provider inherits AZURE_AI_API_BASE into api_base for every azure_ai/* OCR model, but Azure Document Intelligence is a separate resource reached via AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT, so doc-intelligence requests went to the wrong host. Stop inheriting the azure_ai base for doc-intelligence models so api_base stays unset and both the rust bridge and the python get_complete_url fall back to the document-intelligence endpoint. This drops the earlier _rust_bridge_api_base reorder, which only covered the rust path and let the env silently override an explicit api_base * refactor(ocr): consolidate azure doc-intelligence detection; keep explicit api_base Extract is_azure_document_intelligence_model as the single source of truth for the azure_ai doc-intelligence sub-route so the check is no longer duplicated across _prepare_ocr_request and _rust_bridge_api_base, and gate the dynamic_api_base suppression on the caller not supplying an api_base so an explicit endpoint is always honoured. Restore xai to the realtime PROVIDERS as a documented disabled entry instead of dropping it silently, and add a regression test pinning doc-intelligence api_base resolution. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Mubashir Osmani <mubashir@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…nt edits and emit context-management-2025-06-27 beta (LIT-3393) (BerriAI#32658) * fix(bedrock-invoke): retain clear_tool_uses_20250919 context_management edits and emit context-management-2025-06-27 beta (LIT-3393) Copy of BerriAI#29206 by oss-agent-shin, rebased onto litellm_internal_staging so CircleCI can run. Bedrock InvokeModel supports automatic tool-call clearing (clear_tool_uses_20250919) under the context-management-2025-06-27 beta, but LiteLLM stripped the edit and dropped the beta header, causing a Bedrock 400. This maps bedrock.context-management-2025-06-27 to itself in anthropic_beta_headers_config.json (bedrock_converse stays null) and rewrites _filter_context_management_for_bedrock_invoke around an allowlist of supported edit types that keeps each supported edit and adds its matching beta. * test(bedrock-invoke): restore beta-headers config cache with a shared fixture in LIT-3393 tests Greptile flagged that three of the four new tests reloaded the module-level beta-headers config into local mode without restoring it on teardown, leaking state into later tests in the same process. Move setup/teardown into a local_beta_headers_config fixture used by all four tests. --------- Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai>
…reaking DB e2e tests (BerriAI#32653)
…ion on the tools preview Authorization doubles as the admission fallback when x-litellm-api-key is absent, so a caller who authenticated the preview request that way had their LiteLLM key forwarded to the upstream as the oauth2/client-forwarded token. The preview now forwards Authorization only when the primary admission header is present, which is how the dashboard has always sent it; with no primary header there is no upstream token on the request at all. Applies to oauth2 and both client-forwarded modes; parametrized regression test plus the admission header added to the existing extraction tests to mirror the real UI request shape
Phase 0 of the dashboard table-standardization effort: one composable DataTable built on TanStack react-table and the shadcn-style primitives in components/ui/table.tsx (Base UI, Tailwind v4), plus its behavioral test suite. No existing tables are migrated in this change. The component owns the TanStack instance and a shadcn shell, and exposes composable slots (toolbar, pagination, footer) plus DataTableToolbar, DataTablePagination, DataTableViewOptions, and DataTableSortHeader. Sorting and pagination each use a single mode enum (none/client/server) so server modes only surface state via callbacks and never reorder or slice locally. columnMeta.ts defines the canonical ColumnMeta augmentation. The rendering shell imports only components/ui/table primitives; no tremor or antd.
…itellm_/elegant-edison-bf44a2
Proof-of-concept consumer for the shared DataTable added in the previous commit. Swaps the antd Table in the Workflow Runs page for DataTable in client-pagination mode, keeping the existing cell renderers, row-click drawer, and empty state. Adds a focused test that the rows render through DataTable, a row click routes the detail fetch to the correct run, and the empty state shows.
…tadata The redacted resource kept the path, but hosted MCP servers routinely embed the credential in the path (for example /mcp/s/<token>/mcp), and mcp_tool_call_metadata is readable by a caller who can invoke the tool, so the path leaked the upstream credential into spend logs. Only scheme, host, and port are logged now
…y gate and tools preview The gateway authorize/token/register gate and the preview header extraction each carried their own inline copy of the oauth2 + client-forwarded mode set, which could drift from the discovery constant the registry builders use; all three surfaces mean the same thing (modes that run the upstream OAuth browser flow), so they now read the one constant
Second proof-of-concept consumer for the shared DataTable. Replaces the hand-rolled tremor table in the Team Info Virtual Keys tab with DataTable in server-sort and server-pagination mode plus column resizing; the file drops about 150 lines. Sortable headers now use DataTableSortHeader, pagination is a detached DataTablePagination driven by the page state, the id-cell still opens the key drawer, and the body scrolls under a sticky header via maxBodyHeight. Two behavior changes: the pagination control is the standardized bar (row range plus page-size select) rather than the old Previous/Next buttons, and a sort header cycles ascending/descending without a third unsorted state, which also removes a latent case where clearing the sort left the server sorted. Updates the TeamVirtualKeysTable and TeamInfo tests to the new pagination, adds a test that a sort-header click routes to useKeys as a server sort, and lowers the no-large-inline-object-arg metric by one and the file's no-nested-ternary suppression from two to one to match the leaner code.
…_ui_enum feat(mcp/ui): expose true_passthrough and oauth_delegate auth types with a no-auth warning
…relevant field changes An admin who ran Authorize & Fetch and then changed a field that determines which upstream OAuth token gets minted kept using the stale token for tool preview, sessionStorage, and (on the backend) the stored per-user credential and its cache. Grounded in RFC 8707/8693 and the MCP auth spec, a token is bound to one tuple: resource/audience (url), OAuth mode/grant (auth_type, oauth_flow_type), the authorization-server endpoints, and the OAuth client + scopes. A shared getOAuthAuthorizationIdentity captures exactly those fields; transport (http/sse on the same url is the same audience) and delegate_auth_to_upstream (a downstream-usage toggle never sent to the authorize request) are excluded. UI: both the create and edit forms now discard the held token (React state / sessionStorage / hook, plus the fetched token + DCR client in form.credentials) whenever the identity diverges from the one it was authorized against, re-applying the admin's in-flight edit so it is never wiped. The check lives in one shared helper so the two forms cannot drift. Backend: editing an MCP server now compares the pre/post identity and, on a mint-relevant change, purges every stored per-user OAuth credential for the server (DB row + per-user token cache) so no user forwards a token minted for a resource/AS/client that no longer matches. Best-effort; a purge failure never fails the update.
…er-user token store Review follow-ups on the stale-token invalidation. The backend identity now decrypts client_id and client_secret before comparing: the stored values are NaCl-encrypted with a fresh nonce on every write, so comparing ciphertext flagged every routine save as a mint-relevant change and purged per-user tokens that were still valid. The identity also gains spec_path, the audience for OpenAPI servers, and parses credentials stored as a JSON string The purge now routes each (user, server) through the manager's invalidate_user_oauth_token_cache, which becomes the single invalidation point covering both the legacy per-user token cache and the v2 per-user OAuth token store; previously the purge evicted only the legacy cache while the revoke path evicted only the v2 store, so each path left the other cache serving a replaced token until its TTL. A credential row racing in between the find and the delete is now detected via the delete_many count and logged; its cache entry expires by TTL On the dashboard, CLEARED_ON_INVALIDATION and the staleness check move to types.tsx as the single shared implementation for both forms. The edit form's transport handler now rechecks the identity after its programmatic setFieldsValue calls, which antd does not report through onValuesChange, so a token no longer survives a transport switch that clears the mint target. The create form rebuilds formValues from the post-reset form state after an invalidation instead of publishing the pre-reset snapshot, so the tool preview can no longer refetch with the discarded DCR client. Both transport handlers now share the recheck, which also stops the create form from over-invalidating on an http to sse swap that keeps the same url and therefore the same audience
…discipline The purge takes an injectable invalidate_token_cache callable defaulting to the manager's shared invalidation, and MCPServerManager takes an injectable per_user_token_cache alongside the existing per_user_oauth_token_store, so tests inject fakes instead of monkeypatching the global manager and the module-level cache. The new identity helpers drop Any for object throughout
…ilure cannot fail the edit The snapshot read only feeds the stale-token purge decision; leaving it unguarded meant a failed read would 500 an edit whose update would have succeeded, and it broke test_edit_mcp_server_redacts_credentials, whose mocked prisma is not awaitable on the un-patched get_mcp_server path. A failure now logs and skips the purge, consistent with the purge half already being best-effort. Adds the first endpoint-level coverage of the edit purge wiring: purge on a mint-relevant change, no purge when the identity is unchanged, and edit success with purge skipped when the snapshot read raises
…validation test The staged access token never reaches formValues (it is not a registered form field), so the assertion could not fail; the DCR client pair is the leak the test actually pins, proven by the mutation run
…ate caches on server delete LiteLLM_MCPUserCredentials stores BYOK API keys in the same column as per-user OAuth tokens, so the purge on a mint-relevant config change now deletes only rows whose payload decodes as an OAuth2 credential, each by its (user_id, server_id) pair, instead of every row for the server. An api_key server whose url changes purges nothing. delete_mcp_server now also invalidates each enumerated user's cached token so a re-created server reusing the id cannot serve tokens minted for the deleted one, and both cache drops are best-effort
…zation identity The identity used to pick the audience from spec_path only when values.transport was OPENAPI, but the create form keeps transport in component state rather than form values, so spec_path edits on OpenAPI servers never invalidated a held token. Comparing url and spec_path independently mirrors the backend's mcp_oauth_token_identity and fires regardless of whether transport is present. Invalidation now also wipes only credentials; the admin-typed endpoint fields are kept
…dit form For authorization_code the edit preview listed tools by server_id only, relying on the stored per-user DB credential, so a token authorized in the edit session gave an empty preview until the admin saved; the create form previews the identical state through the config-based preview endpoint, which takes the token explicitly. The edit fetch now routes through that same endpoint when a staged interactive token is held, built from the form values with the saved record as fallback, and keeps the by-server_id listing for every other case
The create and edit submit paths for true_passthrough and oauth_delegate persist only the tool configuration: the parametrized create test authorizes, disables the allowlist, and asserts nothing is persisted before submit, then that the create payload carries allowed_tools but no credentials and no occurrence of the token anywhere in the serialized payload, no per-user DB credential is written, and the token is committed to sessionStorage only, keyed to the created server. The edit save test gains the same serialized-payload assertion
…ction The field doc still said scheme + host + path while the redactor now strips the path along with userinfo, query, and fragment, since hosted MCP servers routinely embed the credential in the path
…recharts (BerriAI#32725) * refactor(ui): convert user agent and per-user usage charts to shadcn/recharts Swap the tremor BarChart import for the shared shadcn/recharts wrapper in user_agent_activity.tsx (DAU/WAU/MAU charts) and per_user_usage.tsx (usage distribution histogram). All chart props are unchanged; the wrapper exposes the same tremor prop surface with matching defaults. Extend user_agent_activity.test.tsx and add per_user_usage.test.tsx with parity assertions on the real recharts SVG output: bar series per category, stacked x positions, resolved fill colors, axis bucket labels, legend text, and value formatter output on axis ticks. Remove the dead ResizeObserver polyfill in user_agent_activity.test.tsx now that the scoped global mock in tests/setupTests.ts renders charts, which also lowers the no-explicit-any metric by one. * test(ui): harden bar x-position parsing against recharts path format
…_comment docs(anthropic): clarify the Opus 4.5 branch in adaptive-effort translation
…_issues fix(proxy): enforce budget and cost tracking for Dashscope tiered pricing
…nfirmation (BerriAI#32945) * fix(ui): show sidebar copy confirmation only on a successful write The sidebar account menu's copy button switched to the checkmark synchronously, before the clipboard write settled, so it confirmed a copy that never happened when navigator.clipboard was undefined on non-secure origins or when writeText rejected. The handler now guards navigator.clipboard, awaits the write, and flips to the checkmark only on success Also updates the header accent emoji in the same menu * refactor(ui): extract a shared CopyButton for the sidebar account menu The copy-icon-to-checkmark pattern was hand-rolled in several places, including the sidebar account menu whose private copy button held the false-confirmation bug. Extract a single canonical CopyButton into components/shared, built on the Button primitive with a guarded and awaited clipboard write so the checkmark appears only on a real success, and have SidebarAccountMenu consume it The success and failure-mode coverage now lives in the shared component's own test; the sidebar test keeps one case asserting the email row is wired to it
…charts (BerriAI#32729) * refactor(ui): convert entity usage and usage page charts to shadcn/recharts Swap the tremor BarChart/DonutChart render sites in EntityUsage, SpendByProvider, TopKeyView, TopModelView, KeyModelUsageView and UsagePageView to the shared shadcn/recharts wrappers. Convert the two sole-chart Daily Spend cards and the KeyModelUsageView card to the shadcn Card primitives. Close the donut parity gap with strictly additive optional DonutChart props: showLabel/label render a center total (tremor showed valueFormatter(sum) by default) and startAngle/endAngle forward to the Pie so both provider donuts keep tremor's clockwise-from-12 layout. Defaults preserve the previous wrapper behavior. DailyData and two site-local row types move from interface to type alias so they satisfy the wrappers' Record<string, unknown> constraint; interfaces lack implicit index signatures. Tests now assert on real recharts output: bar/sector counts, cyan fills, axis labels, donut center totals, and the TopKeyView bar-click drill-down into the key info modal. The dead tremor chart mocks in UsagePageView.test.tsx are removed and lint metrics/suppressions are regenerated for the dropped tremor imports. * fix(ui): compute donut center label only when shown and assert Model Usage renders as a card title
…#32936) * feat(guardrails): add pre_mcp_call support to Content Filter * test(guardrails): cover canonical MCP key gate under pre_mcp_call mode * fix(guardrails): scan MCP arguments per value and gate mixed-mode scans by call type * fix(guardrails): cap MCP argument scan depth and register the walker with the recursion detector * test(guardrails): update LIT-4226 UI settings tests for content filter pre_mcp_call support * fix(guardrails): use builtin generics in MCP scan annotations to satisfy strict-rule budget Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…t_registry refactor(e2e): bucket rate limits, budgets, and spend tracking under quota_management
* feat(proxy): add expires filter to GET /key/list Add an opt-in expires query param to GET /key/list so callers can fetch only expired or only active keys without paginating every page and filtering client-side. 'expired' matches keys whose expires is in the past (NULL expires excluded); 'active' matches keys that never expire or expire in the future. Omitting the param preserves existing behavior for every caller. An unrecognized value returns HTTP 400 rather than silently returning all keys. The filter is pushed to the database via the existing Prisma where builder so callers avoid pulling the full key table into application memory. Resolves LIT-3387 * refactor(proxy): declare VALID_EXPIRES_FILTER_VALUES before its first use
…nents (BerriAI#32952) Split for the usage (UsagePage) segment. Most of the folder is the usage page's own view, but four pieces are reused elsewhere and stay in @/components/UsagePage: TopKeyView (old-usage), KeyModelUsageView and value_formatters (activity_metrics), and the shared types (activity_metrics, chartUtils). The other 21 files move into usage/_components, preserving the folder structure. The external consumers import only the retained files, so they are untouched. The moved files' imports of the retained files become @/components/UsagePage paths, other escaping relative imports are absolutized, and lint suppressions are re-keyed for moved files only. No behavior change.
chore(ci): promote internal staging to main
…_bridge fix(completion): forward aws credential kwargs into litellm_params so the responses bridge keeps WIF auth (cherry picked from commit c75fccf)
chore(release): backport BerriAI#32956 onto patch-1.93.0rc1 for the 1.93.0 rc2 cut
…1.93.0 stable cut (BerriAI#33847) * fix(ci): bump pillow to 12.3.0 to resolve osv-scan CVEs (BerriAI#33093) (cherry picked from commit 20e646c) * chore(deps): pin httplib2 and setuptools transitive floors (BerriAI#33233) Raise the constraint floors for two transitive dependencies so resolution moves them to their latest maintenance releases: httplib2 0.31.2 -> 0.32.0 and setuptools 82.0.1 -> 83.0.0. Both are pulled in only by optional integrations (Google API client, grpc tooling, lunary observability, the nvidia-riva extra), all lower-bound only, so the floors stay inside every requirer's allowed range and a default install is unaffected (cherry picked from commit 8b32320) * fix(anthropic/passthrough): drop incompatible temperature when downgrading adaptive thinking for pre-4.6 models (BerriAI#33244) * fix(anthropic/passthrough): drop temperature and cap thinking budget when downgrading adaptive thinking for pre-4.6 models * test(anthropic/passthrough): use sufficient max_tokens for reasoning_effort thinking mapping * fix(anthropic/passthrough): drop incompatible temperature when downgrading adaptive thinking for pre-4.6 models Narrow the fix to the temperature reconciliation; the reasoning_effort budget cap is reverted because the live translation grid relies on budget_tokens >= max_tokens to reject unsupported effort tiers (xhigh/max) on budget-mode models, so capping turned those 400s into 200s. --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 71dffc1) * build: raise requires-python cap to <3.15 so Python 3.14 installs current releases (BerriAI#33438) * build: drop requires-python upper cap so Python 3.14 resolves to current releases The <3.14 cap made pip on Python 3.14 fall back to litellm 1.83.7, a pre-April release whose old auth flow fails with 400s. The cap was added in d9a4602 because deps lacked 3.14 wheels and uv could not resolve the 3.14 split; both are fixed now via the existing python_version markers plus a ddtrace version split (2.x has no cp314 wheels, 3.16+ does). Verified on 3.14.5: uv sync --all-extras installs, litellm and proxy_server import (rust bridge falls back to pure python), real provider calls succeed sync/async/streaming, and the core-utils test suite passes. * build: cap requires-python at <3.15 and keep ddtrace on one major per python band Reviewer preference to bound the supported window at the newest tested minor rather than leaving it open-ended, and Greptile flagged the ddtrace 3.14+ range spanning two majors; every ddtrace 4.x ships cp314 wheels so the band is now >=4.0,<5.0, matching the single-major convention of the 2.x band. (cherry picked from commit c6d49a8) * build(deps): update ddtrace to the 4.x line A single ddtrace constraint now covers every supported Python version, so this collapses the version split introduced in BerriAI#33438. Also aligns the build_from_pip image pin and updates the type-only Tracer import to its current module path (cherry picked from commit edc38ea) * fix(docker): restore litellm-proxy-extras source dir in runtime images (BerriAI#33592) * fix(docker): restore litellm-proxy-extras source dir in runtime images BerriAI#30243 narrowed the runtime stage to an allowlist COPY, which dropped /app/litellm-proxy-extras from the published images. Downstream migration jobs point prisma migrate deploy at that path; with the schema gone (or a schema with no adjacent migrations dir, where prisma exits 0 without applying anything) those jobs went green while never migrating the database. Restore the folder in all three runtime stages and assert in image-scan that the schema and a non-empty migrations dir ship at the source path * chore(ci): drop image-scan migration-assets assertion (cherry picked from commit 111d447) * fix(model_armor): restore reference attachments via skip_unscannable_attachments and remove the attachment count cap (BerriAI#33554) * fix(model_armor): add skip_unscannable_attachments to allow reference-only attachments through * fix(model_armor): wire skip_unscannable_attachments through guardrail config * fix(model_armor): make max_file_attachments configurable and scan overflow instead of dropping * fix(model_armor): remove the per-request attachment count cap and scan all attachments --------- Co-authored-by: yucheng <yucheng@berri.ai> (cherry picked from commit 0d7b0f7) * build(rust): raise pyo3 to 0.29 so the native bridge compiles on Python 3.14 (BerriAI#33798) pyo3 0.23.5 hard-caps the interpreter at Python 3.13, so building the native bridge against a 3.14 interpreter aborts inside pyo3-ffi's build script before anything links. This raises pyo3 and pyo3-async-runtimes to 0.29 (currently the newest line, and the range starting at 0.26 that supports 3.14) and migrates the three call sites whose APIs were renamed across that range: Python::with_gil is now Python::attach and Python::allow_threads is now Python::detach. On a GIL-enabled interpreter those are pure renames with identical semantics, so behavior on 3.10 through 3.13 is unchanged Verified by compiling the native module for cp313 and cp314 and driving it directly on both interpreters: gil_stats reports exactly one GIL release per sync OCR call and the async path completes, matching the 0.23.5 baseline. cargo fmt, clippy, and the workspace tests pass on both 3.13 and 3.14 with the lockfile locked, and the lock churn is confined to the pyo3 crates Part of BerriAI#26343; addresses the pyo3 build failure reported in BerriAI#33116 (cherry picked from commit f3d2015) * build(deps): allow redisvl, pypdf, and openapi-core on Python 3.14 (BerriAI#33801) Remove the python_version < '3.14' environment markers from redisvl, pypdf, and openapi-core now that all three install and import cleanly on 3.14. The relock is marker-only: no package version changed for any Python branch, and the locked versions (redisvl 0.4.1, pypdf 6.13.3, openapi-core 0.22.0) now serve 3.14 as well. semantic-router and aurelio-sdk stay gated because every published release caps python_requires below 3.14 (cherry picked from commit 967d934) * build(deps): bump mcp lock to 1.28.1 to clear image-scan findings (BerriAI#33803) * build(deps): bump mcp lock to 1.28.1 to clear image-scan findings * build(deps): require mcp>=1.28.1 (cherry picked from commit 40e914c) * fix(proxy): source /v1/models token limits from the cost map instead of Router.get_model_group_info (BerriAI#33721) * fix(proxy): source /v1/models token limits from cost map instead of Router.get_model_group_info Resolves the per-model get_model_group_info fan-out on GET /v1/models (and /models) that pegged the event loop on wildcard listings (BerriAI#33636). create_model_info_response now reads max_input_tokens/max_output_tokens from litellm.get_model_info (the static cost map) rather than the router, which aggregated and deepcopied every deployment in a group per listed model. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): inject model-info lookup into create_model_info_response for deterministic coverage Inject the cost-map lookup (defaulting to litellm.get_model_info) so the except and max_output_tokens branches are exercised deterministically and the token-limit tests no longer hardcode mutable cost-map values. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(proxy): surface custom deployment token limits on /v1/models via cheap index lookup Add Router.get_configured_token_limits, an O(1) model-name index lookup that reads a concrete deployment's configured max_input_tokens/max_output_tokens without triggering pattern matching or deep copies. create_model_info_response layers this over the cost map so custom deployments absent from the cost map still surface their limits, and admin-configured limits override cost-map defaults, while wildcard-expanded names stay on the fast path. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 8536e3b) --------- Co-authored-by: yucheng-berri <yucheng@berri.ai> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
….93.0rc2 to complete the 1.93.0 stable cut (BerriAI#33869) * fix(docker): bake prisma CLI and engines at a fixed path so fresh-DB migrations work for any uid offline (BerriAI#33853) * fix(docker): bake prisma CLI and engines at a fixed path so fresh-DB migrations work for any uid offline The runtime image shipped the prisma CLI and engines under /root/.cache, the default HOME-derived prisma-python cache location. Any deployment whose runtime HOME is not /root (kubernetes runAsUser, docker --user, HOME overrides) missed that cache on a fresh database, fell back to a nodeenv Node download that crashes on Wolfi (libatomic.so.1), and started the proxy with zero tables while every DB-backed endpoint returned 500 The bake now lives at /opt/prisma, a path no HOME resolution or cache volume mount can shadow. The builder records the engine paths there at generate time, and the runtime stage pins PRISMA_BINARY_CACHE_DIR, PRISMA_CLI_PATH, PRISMA_CLI_QUERY_ENGINE_TYPE=binary and PRISMA_OFFLINE_MODE so both litellm-proxy-extras and prisma-python resolve the baked CLI and engines directly. prisma migrate deploy on a fresh database now needs no npm and no network access for any runtime uid, including readOnlyRootFilesystem deployments Verified against live containers: fresh and existing databases as root, uid 12345, HOME overridden, on an internal-only docker network, and with a read-only root filesystem all migrate and serve /team/new successfully Fixes BerriAI#33650, BerriAI#24554 * chore(docker): fail the image build if the baked prisma CLI layout drifts Asserts the baked CLI shim is executable and its entrypoint exists in the runtime stage after the COPY and chmod, so a layout change in a future prisma-python release breaks the image build loudly instead of silently degrading the migration path at container startup (cherry picked from commit 567ebcb) * fix(router): treat malformed configured token limits as absent on /v1/models (BerriAI#33864) A deployment whose model_info carried a non-numeric max_input_tokens or max_output_tokens (for example "128,000" or an empty string) made the bare int() in get_configured_token_limits raise inside the per-model /v1/models loop, so one misconfigured deployment turned the entire listing into a 500. Coerce each configured limit safely and treat malformed values as absent, matching the graceful degradation the listing had before the cost-map switch (cherry picked from commit ef7007c)
…dels create_model_info_response cast cost-map max_input_tokens / max_output_tokens with unguarded int(). The surrounding try/except covers only the get_model_info lookup, so a deployment whose model_info carries a non-numeric limit (e.g. "128,000" or an empty string) raised inside the per-model listing loop and failed the entire GET /v1/models and /models response with a 500, taking healthy deployments down with it. A deployment's model_info is registered into litellm.model_cost verbatim, so the malformed value reaches the cost map and not just the router index. Router.get_configured_token_limits already coerced this safely for the deployment path; the cost-map path was missed, so the two together still regressed. Both now share coerce_token_limit in litellm_core_utils, which returns None for a malformed value so the listing omits that one limit instead of failing, matching the graceful degradation the endpoint had before the cost-map switch. (cherry picked from commit ab02127)
…malformed_cost_map_limits_1930 fix(proxy): treat malformed cost-map token limits as absent on /v1/models
Mattchewone
added
upstream-sync
Resolve conflicts for the upstream v1.93.0 sync PR so it can merge in the GitHub UI. Keep Bitovi Mantle, VK/budget UI, Redis datetime cache codec, and deploy/workflows; take upstream for unrelated product and UI build noise. Co-authored-by: Cursor <cursoragent@cursor.com>
Mattchewone
changed the title
Mattchewone
removed
the
needs-conflict-resolution
Update model_info mocks for get_available_models_for_user and get_configured_token_limits, and expect model_max_budget_in_team on add_new_member so proxy-unit guardrails CI passes on the sync branch. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upstream stable sync
Stable tag:
v1.93.0052b5a2169d8d3082e1d66e69f200a72b0c1e274upstream/mainor BerriAI staging).github/workflowsretained (we do not adopt BerriAI CI)Conflicts with
litellm_internal_staginghave been resolved on this branch (merge commit). Ready to merge in the GitHub UI intolitellm_internal_stagingUpstream release notes
Full notes: https://github.com/BerriAI/litellm/releases/tag/v1.93.0
Verify using the release tag (convenience):
Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:
Expected output:
What's Changed