feat(ui): give each Models + Endpoints tab its own path by ryan-crabbe-berri · Pull Request #34327 · BerriAI/litellm

ryan-crabbe-berri · 2026-07-23T00:12:19Z

TLDR

Problem this solves:

Model tabs were not linkable, bookmarkable or shareable
One shared URL /ui/models-and-endpoints for every tab
A model's detail view lived in React state, so it had no URL

How it solves it:

Each tab has its own path, e.g. /ui/models-and-endpoints/add
A model or team detail view is now a shareable ?model=<id> / ?team=<id> URL
The 488-line monolith is dissolved into one small page per tab

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The Admin UI ships as a Next static export (output: "export"), so the load-bearing question is whether a hard load of a sub-path resolves to a real file without any change to a customer's nginx or the FastAPI StaticFiles mount. It does, because each tab path is known at build time and gets its own prerendered index.html. Build output:

$ npm run build   # (in ui/litellm-dashboard)
...
├ ○ /models-and-endpoints
├ ○ /models-and-endpoints/add
├ ○ /models-and-endpoints/health
├ ○ /models-and-endpoints/llm-credentials
├ ○ /models-and-endpoints/model-group-alias
├ ○ /models-and-endpoints/pass-through
├ ○ /models-and-endpoints/price-data
└ ○ /models-and-endpoints/retry-settings
○  (Static)  prerendered as static content

Model and team detail views are driven by query params on top of whatever tab you are on, so they ride the same already-served index.html and need no route file of their own.

Since this is a UI change, here is the manual pass to capture screenshots against a live proxy serving the built UI (log in as a Proxy Admin so every tab is visible):

Go to http://localhost:4000/ui/models-and-endpoints and confirm the All Models tab renders and the address bar stays on the base path
Click through each tab and confirm the URL changes to the matching path: Add Model -> /models-and-endpoints/add, LLM Credentials -> /llm-credentials, Pass-Through Endpoints -> /pass-through, Health Status -> /health, Model Retry Settings -> /retry-settings, Model Group Alias -> /model-group-alias, Price Data Reload -> /price-data
Hard-refresh (Cmd+R) while on http://localhost:4000/ui/models-and-endpoints/llm-credentials and confirm it reloads straight onto the LLM Credentials tab instead of 404ing or bouncing to the first tab
On the All Models tab, click a model row and confirm the URL gains ?model=<id> and the model detail view opens; copy that URL into a new tab and confirm it cold-loads straight into the same model's detail; press the browser Back button and confirm you land back on the tab you came from
As a non-admin user, hard-load http://localhost:4000/ui/models-and-endpoints/llm-credentials and confirm it redirects to the base models path since that tab is not permitted for the role

Type

🆕 New Feature
🧹 Refactoring

Changes

The tab surface used to be one 488-line ModelsAndEndpointsView that held every tab's state and rendered all panels through a Tremor TabGroup. Reading the panels showed they already fetch their own cached data, so most of that hoisted state was redundant plumbing. This splits the surface into one small page per tab under models-and-endpoints/, each owning only its own state, with a persistent layout.tsx owning the header, cost banner, tab bar and refresh. The base segment stays models-and-endpoints, so the sidebar, MIGRATED_PAGES and existing bookmarks are untouched. Because tabs live in a shared layout, switching them re-renders rather than remounts, and each tab's data is React Query cached, so navigation stays cheap.

The bigger change is the drill-in. It used to be setSelectedModelId / setSelectedTeamId React state doing a full-page takeover, so a model's detail view had no URL and could not be shared. It is now real navigation: opening a model pushes ?model=<id> onto the current path and the layout renders ModelInfoView from that param; teams use ?team=<id> and TeamInfoView. Closing removes the param and returns you to the tab you came from. This makes model and team detail views shareable, bookmarkable and back-button friendly, and it removes the last piece of cross-tab shared state. Query params are used rather than a path segment like /model/<id> on purpose, so a cold load rides the base route's already-served index.html and no customer nginx or StaticFiles config has to learn a new dynamic route.

The tab bar moves off the phased-out Tremor TabGroup onto antd Tabs, matching the direction the rest of the dashboard is heading. A shared useModelDashboardData hook supplies the few derived lists (model groups, access groups, all model names) the layout and a couple of pages need, and the latent uploadProps coupling (the Add Model form's upload handler was being shared into the Credentials panel) is untangled so each consumer owns its own.

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

greptile-apps · 2026-07-23T00:18:42Z

Greptile Summary

This PR dissolves a 488-line ModelsAndEndpointsView monolith into a layout.tsx that owns the tab bar, header, and detail overlay, with one small Next.js page per tab under models-and-endpoints/. Model and team detail views are now URL-driven (?model=<id> / ?team=<id>) via window.history.pushState, making them shareable and back-button friendly without requiring new static route files.

Tab routing: Each tab has its own prerendered path (/add, /health, /llm-credentials, etc.); role-based redirect guards prevent non-admin users from directly loading restricted tabs.
Detail navigation: useModelDetailRouting pushes query params via pushState (deliberately bypassing router.push, which is a no-op for same-path query-only changes in the static export); the layout reads ?model / ?team from useSearchParams and renders ModelInfoView or TeamInfoView accordingly.
Shared data hook: useModelDashboardData supplies derived lists (model groups, access groups, all models) to both the layout and page components, sharing a single React Query cache entry.

Confidence Score: 5/5

This is a well-scoped UI refactoring with no backend changes; the tab routing, redirect logic, and detail-overlay navigation are all covered by dedicated test files.

The 488-line monolith is replaced by clean, focused per-tab pages. Role-based redirect logic is exercised in tests, the new navigation helpers have their own test suite, and the static-export build constraint is verified by the tabRoutes tests. No auth, data, or API contracts are changed.

detailNavigation.ts — the pushState-over-router.push design assumption should be documented in source so it is not silently reverted in a future refactor.

Important Files Changed

Filename	Overview
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/layout.tsx	New layout owning tab bar, header, and detail overlay routing; clean role-based tab visibility and redirect logic
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/detailNavigation.ts	useModelDetailRouting hook uses window.history.pushState to set ?model/?team params, bypassing Next.js router intentionally for same-path navigation
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/tabRoutes.ts	Clean route helpers for tab slugs and hrefs; slugFromPathname correctly extracts segment from both /ui and dev paths
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/useModelDashboardData.ts	Shared hook for model groups/access groups/proxy model list; deduplication via Set, React Query cache ensures single fetch
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/health/page.tsx	Health tab now its own page; correctly passes deployment IDs (not model names) to HealthCheckComponent
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/retry-settings/page.tsx	Retry settings extracted as standalone page with cleanup-safe async fetch via active flag
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/layout.test.tsx	Good coverage of tab navigation, redirect, and detail overlay; mocks omit isLoading so the loading-guard branch of the redirect effect is not exercised
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/detailNavigation.test.ts	Tests validate pushState-based param manipulation correctly; mock matches the runtime assumption that pushState updates window.location.search
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/page.tsx	All-models page cleanly delegates to AllModelsTab, wiring openModel/openTeam from the detail routing hook
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/vertexCredentialsUpload.ts	Extracts the vertex credentials file-reader upload handler into its own module, untangling the previous cross-tab coupling
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/ModelsAndEndpointsView.tsx	Deleted: 488-line monolith replaced by layout + per-tab pages
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/model-group-alias/page.tsx	Standalone page fetching model group alias settings with cleanup-safe async effect
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/components/PriceDataManagementTab.tsx	Minor: uses a relative ../../hooks/... import path instead of the @/ alias used elsewhere in the module

_{Reviews (6): Last reviewed commit: "fix(ui): drive model/team drill-in with ..." | Re-trigger Greptile}

…L-driven detail Dissolve the 488-line ModelsAndEndpointsView monolith into one page per tab under the models-and-endpoints route, with a persistent layout owning the header, cost banner, tab bar and refresh. Each tab page owns only its own state; shared lists come from a small useModelDashboardData hook. Replace the stateful model/team drill-in (setSelectedModelId/setSelectedTeamId full-page takeover) with real URL navigation: ?model=<id> and ?team=<id> render ModelInfoView/TeamInfoView from the layout, so a model or team detail view is now shareable, bookmarkable and back-button friendly. Removes the empty placeholder pages from the first commit. Swap the tab bar off phased-out tremor onto antd Tabs.

ryan-crabbe-berri · 2026-07-23T01:02:48Z

@greptileai re review

codspeed-hq · 2026-07-23T01:25:55Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_models_tab_routes (1ee5bc1) with litellm_internal_staging (0c2b86e)¹}

No successful run was found on litellm_internal_staging (abf18f8) during the generation of this report, so 0c2b86e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

AllModelsTab, ModelRetrySettingsTab and PriceDataManagementTab rooted their render in a Tremor <TabPanel>, which only renders inside a Tremor <TabGroup>. After the decomposition these panels live under antd Tabs / as route pages with no such ancestor, so All Models (and the other two) rendered blank. Root them in a plain container instead. The existing component tests mocked @tremor/react (stubbing TabPanel to render children), which hid this; add a regression test that renders with real Tremor and asserts the content is visible standalone. Also type visibleSlugs/TAB_LABELS with the canonical ModelTabSlug so a tab added without a matching label is a compile error.

codecov · 2026-07-23T02:06:59Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…c mount The drill-in close (Back to Models) and open were no-ops: the dashboard is a static export served under /ui, a prefix the Next router (basePath "") does not know, so a router.push to the current pathname with only the query changed is deduped and never re-renders. Drive the ?model=/?team= overlay via real browser navigation (window.location) so open and close reliably work; verified live. Also address review feedback: gate the tab-permission redirect on teams/uiSettings having loaded so a team admin hard-loading /add is not bounced to the base before their membership resolves, and memoize getProviderFromModel on modelCostMapData so the health tab's provider labels refresh when the cost map loads.

ryan-crabbe-berri · 2026-07-23T02:22:07Z

@greptileai re review

ryan-crabbe-berri · 2026-07-23T02:22:42Z

@greptileai re review

router.replace is unreliable under the /ui static mount (same class of Next-router issue that broke the drill-in back button), so the forbidden-tab redirect could fail to fire. Use window.location.replace, which keeps the no-history semantics of a permission redirect and is deterministic. Redirect stays gated on teams/uiSettings having loaded.

ryan-crabbe-berri · 2026-07-23T02:29:32Z

@greptileai re review

…side nav Switch the ?model=/?team= overlay navigation from window.location.assign to window.history.pushState, which Next's App Router observes. This keeps navigation client-side (no full page reload, React Query cache preserved) while still working for the same-path query-only change that router.push cannot do under the /ui static mount. Open, close (Back to Models) and browser Back are all verified in the built UI. Adds unit coverage for the open/close/read behavior.

ryan-crabbe-berri · 2026-07-23T02:50:53Z

@greptileai good catch on the full reload; that was a real downside. I couldn't take the exact suggestion of router.push though. The drill-in open/close is a same-path, query-only change (?model= / ?team= on the current tab path), and under the /ui static mount the Next router dedupes a router.push to the current pathname and never re-renders. That is actually the same failure that made the "Back to Models" button do nothing. Tab switching works only because it changes the path, not just the query.

I switched the drill-in to window.history.pushState, which Next's App Router observes, so it stays client-side (no full reload, React Query cache preserved) while still updating useSearchParams for the query-only change. Verified against the built UI: opening a model, "Back to Models", and the browser Back button all work with no full navigation, and shareable cold-loads of ?model= still resolve. Added unit coverage for the open/close/read behavior in detailNavigation.test.ts. Fixed in 1ee5bc1

greptile-apps · 2026-07-23T02:52:07Z

The window.history.pushState approach is the right call here — that's a real quirk of Next.js App Router under a static mount where same-pathname router.push calls are deduped. The implementation in detailNavigation.ts looks clean.

One observation about the test coverage: the mock —

vi.mock("next/navigation", () => ({ useSearchParams: () => new URLSearchParams(window.location.search) }));

— reads window.location.search at call time, so it always returns a fresh snapshot. That makes the spy-based tests (verifying pushState was called with the right URL) accurate, but the reactive read path — that useSearchParams actually re-renders the component after openModel / close — isn't exercised. The "reads modelId and teamId" test sets the URL before rendering the hook, so it doesn't catch a hypothetical case where the hook returns stale params after a pushState call.

Since you've verified the real behavior in the built UI, this isn't a blocker, but it's worth knowing the unit tests cover call-site correctness, not the re-render loop. If you ever want to fill that gap, you'd need either a more realistic useSearchParams stub that re-renders on a popstate-like event, or an integration test against the real Next.js router.

Aside from that, the fix is sound — the approach is correct and the tradeoff (direct pushState vs. router.push) is well-reasoned and documented in the commit message.

ryan-crabbe-berri · 2026-07-23T02:52:22Z

@greptileai re review

* fix(responses): preserve reasoning through prompt hooks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(responses): minimize regression test diff Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses): handle non-message-only prompt input Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): raise dashboard session budget default to $1 and make it configurable in config and Admin UI Every dashboard login mints a 24h session key whose max_budget comes from litellm.max_ui_session_budget, and all dashboard LLM traffic (playground, auto router per-tier Test Connection probes) spends against and is gated by that one key. The $0.25 default locked sessions out mid-testing with "Budget has been exceeded ... Max budget: 0.25" and the setting appeared in no docs, no UI, and no error text, so it read as a hardcoded cap. Raise the default to $1. Give the setting an explicit typed arm in the config loader (float coercion for env-var strings, null disables the cap). Surface it on the Admin UI General settings tab through the existing litellm_settings bridge as a new Dollar field type (positive USD, unbounded above; the existing Float type is validated to (0, 1] for fractions), with a spec-level default so clearing the field restores $1 instead of silently removing the cap, and enroll it in LITELLM_SETTINGS_SAFE_DB_OVERRIDES so UI edits propagate to peer workers. * fix(ui): resolve General settings rows by field name, not filtered index The General tab renders generalSettings with TypedDictionary and prompt-caching rows filtered out, but the Update and Reset handlers indexed into the unfiltered array, so any row rendered after a filtered-out entry read another field's value. max_ui_session_budget is the first General-tab row positioned after the prompt-caching entries, so its Update sent that row's boolean and failed Dollar validation. Reset also cleared the local input to null, which reads as unset or unlimited while the backend had restored the default. Handlers now resolve the row by field name and drop the index parameter, and reset displays the row's field_default_value. Component tests drive the real /config/list ordering through the actual clicks and fail under either original behavior. * feat(guardrails): add only_scan_new_messages for per-session incremental scanning (#33278) * feat(guardrails): add only_scan_new_messages for per-session incremental scanning Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(guardrails): use fixed TTL constant and revert unrelated test formatting Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(guardrails): run only_scan_new_messages in the unified apply_guardrail path The initial wiring lived in BedrockGuardrail.async_pre_call_hook, but the proxy routes Bedrock through the unified apply_guardrail interface, so the flag had no effect live. Move incremental selection into apply_guardrail: filter the flat texts list against per-session scanned hashes, skip the Bedrock call when nothing is new, and mark hashes only after a successful (non-blocked) scan. Full-context fallback is preserved when there is no session id, the cache is unavailable, or a masking guardrail is configured. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(guardrails): cover session-id fallbacks and mark_texts_scanned guards Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(guardrails): fall back to full scan when incremental guardrail masks content, use shared cache Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(guardrails): cover generic agent multi-turn incremental scan Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(guardrails): cover incremental scan cache resolver fallbacks Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(guardrails): cover flag interactions and /v1/messages incremental scan semantics * feat(guardrails): make GUARDRAIL_SCANNED_MESSAGES_CACHE_TTL_SECONDS env configurable * test(guardrails): prove skip_system/skip_tool are enforced upstream of incremental scan --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> Co-authored-by: Yucheng Zhu <yucheng@berri.ai> * fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache (#33261) * fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache * fix(proxy): make CLI SSO flow state redis-authoritative across workers The CLI SSO flow is stored in a DualCache whose get_cache is memory-first, so the worker that served /sso/cli/start keeps serving its stale in-memory flow and never observes the sso_complete/session_data update another worker writes during the OAuth callback. Attaching Redis alone is not enough; poll on the original worker returns pending forever. Read and write the flow directly through the attached Redis backend when present so every worker sees the same authoritative state, falling back to the in-memory DualCache only when no Redis is configured. * fix(proxy): serialize CLI SSO flow as JSON for the redis round trip RedisCache stores values via str(value) and parses reads with json.loads then ast.literal_eval. The completed flow contains a LitellmUserRoles enum in session_data.user_role, whose repr is not a parseable literal, so any worker reading the completed flow from redis raised SyntaxError and returned 400 "CLI login session not found". Writing the flow as json.dumps makes the round trip lossless (the enum is a str subclass) and fails loudly at write time if a non-serializable value is ever added to the flow. * fix(proxy): point CLI SSO session-not-found hint at configuring Redis The error message and warning still told users to set enable_redis_auth_cache, but the CLI SSO session cache now gets Redis unconditionally whenever one is configured, so that flag no longer affects CLI login --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> * test(e2e): move Admin UI Playwright suite to tests/e2e/ui (#34196) Relocates ui/litellm-dashboard/e2e_tests to tests/e2e/ui so all end to end suites live under tests/e2e. The suite stays in TypeScript and becomes a self-contained npm package with its own package.json, lockfile and tsconfig instead of leaning on the dashboard's toolchain; the dashboard drops its @playwright/test dependency, e2e scripts and knip/vitest/tsconfig carve-outs. CI paths follow the move: both CircleCI jobs (main e2e and the SERVER_ROOT_PATH migration smoke) and the test_server_root_path workflow now install and run Playwright from tests/e2e/ui, with the node cache keyed on both lockfiles. classify_changes.sh treats tests/e2e/ui as client so spec edits keep skipping backend jobs. The suite's mock LLM fixture is excluded from the e2e basedpyright zero-error gate in pyrightconfig.json since it belongs to the TS suite, not the typed Python harness. * fix(scim): use members_with_roles as the source of truth for group membership (#34162) * fix(scim): use members_with_roles as the source of truth for group membership SCIM group provisioning tracked membership inconsistently. Team creation and the real team endpoints persist membership in members_with_roles (and each member's user.teams), but the SCIM group PATCH handler and the GET /Groups listing read the legacy team.members String[] column, which team creation never populates. Seeding a PATCH result from that empty column made an Okta "add member" operation recompute the member set from scratch and silently drop everyone already in the team, so users ended up missing from the groups they were provisioned into. Reading the same empty column on GET /Groups reported an empty member list back to the IdP, which drove repeated re-provisioning. Separately, add_new_member appended the team id to user.teams with an unconditional array push. Under the concurrent group PATCHes an IdP sends during a reconcile, each request passed the members_with_roles duplicate check and pushed, so user.teams accumulated duplicate ids for the same team. A duplicate also breaks auth logic that keys off the number of teams a user belongs to. Read current membership from members_with_roles in the SCIM group PATCH seed and the GET /Groups listing, and make the user.teams append idempotent via a filtered update that no-ops once the team is present. Resolves LIT-4283 * fix(scim): address review; atomic user-creation and stop writing legacy members Keep the concurrent-safe team append but create the user via an atomic upsert (create-or-update) instead of a check-then-create, so provisioning the same new user concurrently cannot race into a duplicate-key failure; the team is still appended idempotently by a filtered update so an existing user gets no duplicate team id. Stop writing the legacy team.members column in the group PATCH apply so the only membership record is the source of truth (members_with_roles plus each member's user.teams), reconciled by team_member_add/team_member_delete. Tests: existing add_new_member and team-creation mocks updated to the upsert plus filtered-append shape, and new tests cover atomic creation and that the PATCH apply does not write the legacy members column. * fix(scim): sync team roster and dedup teams for existing-user email upsert (#34183) * fix(scim): sync team roster and dedup teams for existing-user email upsert When POST /scim/v2/Users matched an already-existing user by email, handle_existing_user_by_email raw-wrote the user's teams array but never touched the team roster, so the user appeared in the group on their profile yet was absent from the team directly (members_with_roles stayed empty). It also did not dedup the teams built from repeated SCIM groups. Route the existing-user team assignment through the same _handle_team_membership_changes / team_member_add path the PUT update_user handler uses, so members_with_roles, LiteLLM_TeamMembership, and the user's teams array stay in sync, and dedup the teams derived from user.groups. The user_id rewrite to the new userName is preserved and sequenced before the roster sync so the roster never references a stale primary key. * fix(scim): surface roster add failures on existing-user email upsert Route the existing-email upsert's roster sync through patch_team_membership with a new opt-in raise_on_error flag so a genuine team_member_add failure propagates instead of being swallowed, and the deduped teams array is only persisted after the roster sync succeeds. Without this, a failed add left the endpoint reporting success while user.teams listed a team members_with_roles never received. The benign already-a-member case stays a no-op even under the strict path, and the flag defaults to False so the PUT update_user, PATCH patch_user, and group callers keep their existing best-effort behavior. SCIM POST is idempotent, so surfacing the error lets the IdP retry and converge. * fix(scim): surface roster removal failures symmetrically with adds Make team_member_delete failures fail loud under the strict roster sync used by the existing-email upsert, mirroring the add path, so a swallowed removal can no longer let the user's teams array drop a team the roster still holds. The idempotent case where the user is already absent from the team stays a no-op, matching how an add treats the user already being in the team. Best-effort behavior is preserved for the default raise_on_error=False callers. * fix(scim): prune deleted user from teams' members_with_roles (#34180) SCIM delete_user removed the user from the legacy team.members column and deleted their team membership rows, but never pruned members_with_roles, which is the source of truth ScimTransformations reads for GET /Groups/{id}. A deleted user therefore lingered as a dangling member reference on every team they belonged to Prune each of the user's teams directly via team_member_delete before deleting the user row, and only for teams whose members_with_roles actually contain the user, so a real DB failure surfaces (the endpoint fails loudly and the user is kept; SCIM DELETE is idempotent, so the IdP retries) while a user who was never in a team's members_with_roles stays a no-op. patch_team_membership is left unchanged for its other callers * test(e2e): fail the run when a Rust gateway silently serves /messages through Python (#34208) A gateway whose native extension is unavailable falls back to the Python implementation without raising, so it answers /v1/messages normally and the only difference on the wire is the absent x-litellm-rust header. Nothing in the suite read that header, so a Rust deployment that had stopped running Rust produced a fully green e2e run. Assert the marker on the streamed Messages assertions when E2E_EXPECT_RUST is set. It stays opt-in because the same suite image also runs against the standard gateway, which has no Rust path and must keep passing; the two deployments are already separate Applications, so this is one value on the Rust instance rather than branching inside the tests. * fix(team): make team member add atomic to prevent concurrent-add member loss (#34185) _add_team_members_to_team reconciled membership by reading the complete_team_data snapshot captured at the start of team_member_add, appending in memory, and writing the whole members_with_roles array back. Two concurrent /team/member_add calls for the same team read the same snapshot, so the last write wins and one member is silently lost. This affects every concurrent team member add, including the SCIM group PATCH op:add path that routes through team_member_add Reconcile members_with_roles inside a transaction that locks the team row with SELECT ... FOR UPDATE before re-reading the current membership, so concurrent writers serialize on the row lock and each appends onto the other's committed result. The interactive transaction is exposed through a thin PrismaClient.tx() passthrough and the locked read is encapsulated in TeamRepository.get_members_with_roles_locked, and the SCIM group PATCH applies membership as deltas so concurrent adds are not clobbered * fix(ui): resolve SSO and SMTP settings from a typed config object (#33576) The SSO and Email Server settings pages read only stored config, so a gateway configured entirely through environment variables rendered every field blank even though both features were live. Rather than add per-endpoint env fallback, resolve each setting through one typed config object. A FieldDescriptor names, for one setting, where it lives in the stored row (db_key), which process env var carries it (env_var), whether it is a secret, and its effective default. A pure resolve_fields reconciles a descriptor table against the stored row and the process environment with a fixed precedence and reports per-field provenance (db, env, default, or unset). The SSO descriptor table single-sources the field-to-env mapping that the read and write paths previously duplicated, so they can no longer drift. get_sso_settings and the /get/config/callbacks alerting block read through the resolver instead of their own inline fallbacks. get_sso_settings no longer decrypts stored values into os.environ; decryption happens once inside the resolver via the pure helper, so a GET stops mutating the process environment. The SSO response carries provenance so the UI can distinguish an env-sourced value from a stored one, and secrets are masked at the endpoint (the resolver returns them unmasked so the login path could consume them). os.environ remains the runtime carrier; the SSO login and mail-send paths are unchanged. The settings pages also submit only fields an admin actually edited, so a rendered mask or env-sourced value is never written back over a working secret, and generic_scope is a real SSO form field. Omitting a field from /update/sso_settings clears it, which provider switching relies on; the deeper write-path concern that behaviour points at is tracked in LIT-4498. * fix(scim): parse membership id from filtered PATCH path when value omitted (#34181) Okta commonly sends SCIM membership removals as a filtered path with no request body value, e.g. Groups PATCH members[value eq "uid"] and Users PATCH groups[value eq "tid"]. The patch handlers pulled ids only from op.value, so these removes were a silent no-op and the member or team was never dropped Add a linear-time filter parser reused by both the Groups members path and the Users groups path so the id is taken from the [value eq "..."] filter when op.value is absent, for add and remove ops. The eq operator is matched case-insensitively, both quote styles are accepted, and the quoted value is unescaped. The path-filter fallback only fires when the request body value is omitted, so an explicit empty value no longer resurrects the filter id, and the compared value must be quoted per the SCIM filter grammar * fix(ui): remove misleading os.environ tooltip from logging settings (#34305) Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic): strip uniqueItems + other unsupported array/object constraints from output_format schema (#33981) (#34313) * fix(anthropic): strip uniqueItems + other unsupported array/object constraints from output_format schema Anthropic's structured outputs (`output_format`) validate the JSON schema against a strict subset and reject cross-element / count constraints that a constrained-decoding grammar cannot enforce, returning a 400 `invalid_request_error`. `filter_anthropic_output_schema` already stripped the numeric / string / item-count constraints (minimum, maximum, exclusiveMinimum/Maximum, minLength, maxLength, minItems, maxItems) but still let these through: - uniqueItems - contains / minContains / maxContains - minProperties / maxProperties so a request using them fails with e.g. "output_format.schema: For 'array' type, property 'uniqueItems' is not supported". This is provider-visible: newer Claude models on the native `output_format` path (e.g. `azure_ai`) 400, while `vertex_ai` is unaffected because it is forced onto the permissive tool-use path (#18625 / #19201). Add the missing keywords to the unsupported-field set and the description map, and skip the advisory description note for a disabled boolean constraint (`uniqueItems: false`) so it isn't misdescribed as required. * fix(anthropic): serialize contains sub-schema in output_format advisory note Address Greptile review: the `contains` advisory note previously discarded the sub-schema, so the description only said an item must match "a schema" without saying which. It now serializes the sub-schema as JSON (e.g. "array must contain an item matching: {\"type\": \"integer\", \"const\": 1}"), matching the other stripped constraints which carry their value. Sub-schema (dict/list) values are json.dumps'd; scalar constraints are unchanged. * style(anthropic): apply ruff format to output_format filter change * style(test): ruff format anthropic schema filter tests * test(anthropic): cover output_format array/object constraint filtering in test_litellm tree Mirrors the schema-filter tests under tests/test_litellm/ so the coverage job exercises the new uniqueItems/contains/min-maxProperties handling and the uniqueItems: false branch. --------- Co-authored-by: Darien Kindlund <darien@kindlund.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(e2e): satisfy basedpyright in test_session_anomaly (typed Success construction) (#34288) Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic): strip all remaining output_format schema keywords rejected by Anthropic * refactor(proxy): type the PATCH /team/{team_id} request body (#34195) * feat(ui): add react-hook-form + zod form infrastructure Introduce the shared form layer the dashboard's antd forms will migrate onto, with no user-visible change yet. - pin react-hook-form, @hookform/resolvers, and zod (kept on 3.25.76 and imported via the zod/v4 entrypoint so openai's optional zod ^3 peer still resolves and npm ci stays clean) - vendor the base-vega Field family into components/shared/form as forwardRef components on the repo's cva.config, since base-vega ships no form primitive and its field source imports class-variance-authority and is React 19 style - add a FormField bridge that binds a react-hook-form Controller to the Field layer and wires label, description, and error ids into aria attributes - add pickDirty, which narrows a submitted body to the top-level keys the user actually touched so a partial update stops re-sending untouched fields pickDirty reads dirtiness at the top level because react-hook-form tracks it per leaf, so an edited array arrives as [true, false] and a cleared list as an empty array that still carries its default-length dirty markers; the falsy clear tokens (null, [], {}, 0, false) all survive. Tests cover the Field primitives, the FormField aria wiring against a live zod resolver, and pickDirty both as a unit and driven through a real react-hook-form instance. * test(ui): lock pickDirty behavior on a pure field-array reorder react-hook-form compares each array element to its default positionally by value, so useFieldArray move/swap and a reordered scalar array all mark the moved indices dirty and pickDirty sends the whole array; a swap of two equal elements is a value-level no-op and is correctly omitted. Covers the reorder case a review flagged as untested. * feat(proxy): publish a typed request body for PATCH /team/{team_id} The route validated its body into UpdateTeamRequest but read it off the raw request, so the OpenAPI spec carried no requestBody and the dashboard's generated client could not type the call at all. - add PatchTeamRequest, UpdateTeamRequest with an optional team_id, since PATCH takes the id from the path; a body team_id is still accepted when it matches - validate the body through PatchTeamRequest before delegating to update_team - declare the request body on the route and regenerate schema.d.ts The handler keeps reading the raw body rather than declaring a typed parameter. FastAPI validates a declared body before the handler runs, which would replace the 400 for a non-object body with a 422 and move absent-vs-null out of reach of the RFC 7386 metadata merge; those are pinned by existing tests, so the schema is declared on the route instead and every error path is unchanged. Validation is shape-preserving: the body is dumped with exclude_unset so an omitted field never reaches the write, an explicit null still clears, and a partial object_permission does not gain sibling sub-keys, which would wipe them given the column merges rather than replaces. Tests extend the existing patch harness rather than replacing it. * refactor(proxy): declare the PATCH /team/{team_id} body as a typed parameter Replaces the hand-written OpenAPI declaration added earlier in this branch. The route now takes data: PatchTeamRequest, so FastAPI generates the request body itself and emits a $ref to the model instead of an inlined copy that would go stale as fields are added. The earlier approach was a workaround built on a wrong premise. Declaring the body does not cost absent-vs-null: model_fields_set preserves it, which is how POST /team/update already gets its tri-state, and a nested null inside metadata survives validation untouched, so the RFC 7386 merge is unaffected. The one real change is the status code for a malformed body. The route answered 400 for a non-object body and 500 for a wrongly typed field, reporting a caller mistake as a server fault; both are now 422, matching POST /team/update and the other typed management endpoints. The two tests that pinned the old parse-level errors are replaced by one that pins the 422 through the ASGI stack, and the handler drops its manual parsing entirely. * feat(ui): edit fallback chains from router settings (#32841) * feat(ui): edit fallback chains from router settings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ui): address review nits on edit fallbacks modal Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ui): fetch models via react-query in edit fallbacks modal Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Mubashir Osmani <mubashir@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): use official Google Drive streamable HTTP MCP server (#34322) Co-authored-by: Mubashir Osmani <mubashir@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ui): migrate api-reference to shadcn (#34263) * test(ui): characterise the API reference page before the shadcn migration Pins the behaviour the migration must preserve: the three SDK tabs and their accessible names, the default selection, that selecting a tab surfaces that SDK's snippet wired to the resolved base url, and the title, blurb and docs link. Written against the current Tremor markup with role and text queries so it carries over unedited. * refactor(ui): migrate the API reference page to shadcn Replaces the Tremor Grid, Text and Tab primitives on the API reference route with the shadcn Tabs primitive and token utilities, and prunes the file's now stale no-restricted-imports suppression. Markup only; the characterisation test added in the previous commit passes unedited. The wrapper keeps an explicit grid-cols-1 because Tremor's Grid defaults to numItems=1 and emitted it; the implicit auto column that replaces it sizes to the widest child and made the code block overflow the viewport. * test(ui): scope the API reference snippet assertion to the selected tab panel Asserts against the rendered tabpanel instead of searching every mounted code block, so the check keeps proving the selected tab drives the snippet even if the panels are ever kept mounted. * refactor(ui): migrate prompts list page to shadcn (#34289) * test(ui): pin prompts panel toolbar and delete behaviour before migration * refactor(ui): migrate prompts list panel to shadcn * fix(ui): resolve prompts environment label and hold the delete dialog while deleting * refactor(ui): migrate old-usage to shadcn (#34304) * test(ui): characterise the old usage page before migrating it Role- and text-based coverage of the route as it behaves on Tremor, so the shadcn migration has a regression net it did not get to write. Pins the DISABLE_EXPENSIVE_DB_QUERIES branch (warning copy, the docs link and its target, and that every expensive query is skipped), the admin vs non-admin tab set, the cost cards, and the provider and customer tables * refactor(ui): migrate old-usage to shadcn Replaces Tremor with the installed shadcn primitives and the shared recharts wrappers on the only file the route owns. Tabs, cards, tables, the key select and the tag multi-select come from src/components/ui; the bar, area and donut charts come from src/components/shared/charts. Tremor BarList has no shared equivalent, so Total Spend Per Team is composed from ui/meter, which also means the per-team totals stay numbers in state instead of pre-formatted strings; a team total of 1,000 or more used to make the bar widths NaN. The Database Query Limit Reached warning moves with it: same copy, same docs link, still short-circuiting every expensive query. Drops the file's no-restricted-imports suppression and the dead customTooltip, getTopKeys, DataDict and UserData symbols. The characterisation test from the previous commit is unchanged and green on both sides * refactor(ui): migrate transform-request to shadcn (#34303) * test(ui): characterise transform-request panel behaviour before migration * refactor(ui): migrate transform-request to shadcn * fix(ui): keep transform-request panels within the fixed-height content fold * fix(ui): let transform-request flow naturally so the shell scrolls instead of clipping * test(ui): select the copy button by its accessible name * chore(ui): bump next to 16.2.11 (#34329) Moves the dashboard's next pin from 16.2.6 to the latest 16.2.x patch and bumps eslint-config-next to match. Regenerating the lock also healed in explicit bundled-dependency records under @tailwindcss/oxide-wasm32-wasi * test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895) * test(e2e): cover key max_budget blocks on personal, team, and team-member keys * refactor(e2e): convert budget enforcement cases to the resources-fixture pattern The E2ECase class pattern existed only in this file; every other suite uses plain pytest tests with the resources fixture. Rewrites the nine cases as two spec classes and removes the now-dead E2ECase protocol and run_case driver from lifecycle.py * fix(mcp): mint an ephemeral OAuth client when passthrough authorize has no client_id Resolves LIT-4581 A true_passthrough MCP server created without the at-creation auth step has no stored client_id, and the tools-page browser flow supplies none, so GET /v1/mcp/server/oauth/{id}/authorize dead-ended on a 400 missing_client_id. The client-forwarded-token modes forbid the gateway from persisting an OAuth client, so client acquisition moves into the one chokepoint every caller crosses: the authorize endpoint. resolve_ephemeral_dcr_client owns the whole mint policy (mode gate, authorization-url precondition, required S256 PKCE, redirect trust, then a TTL-deduped, per-server single-flighted RFC 7591 mint). The minted client rides the encrypted OAuth state; /callback seals it with the upstream code and server_id into an llm_ptcode_ gateway code, and redeem_passthrough_authorization_code recovers it at the token endpoint (server binding plus required code_verifier) to authenticate the upstream exchange. Nothing is persisted; every value rides the encrypted blobs, so it works across replicas. Client acquisition is one predicate applied across the whole auth-mode matrix: the gateway mints for a clientless authorize iff true_passthrough (any dcr_bridge) or oauth_delegate-and-not-dcr_bridge, and the UI gatewayMintsClientFor mirrors that set exactly so the browser pre-registers a client through the dcr_bridge front door only for the cells the gateway does not mint (the interactive oauth_delegate dcr_bridge sign-in and the legacy oauth2 passthrough). A minted flow runs the bridge short-circuit arm; the relay front door stays for external clients that present their own client_id. Both sides are pinned against the same truth table (test_resolve_ephemeral_dcr_client_mint_set_is_exact and the gatewayMintsClientFor matrix test) so no mode can silently diverge. The authorization_code hook and M2M/token-exchange modes are unchanged. * fix(tests): remove importlib.reload of http_handler that breaks client injection in later tests (#34336) The huggingface embedding test fixture reloaded litellm.llms.custom_httpx.http_handler, creating a new HTTPHandler class object. llm_http_handler keeps the class captured at import time, so any test running later in the same process that injects a client built from the reloaded class fails the isinstance check and the mock is silently discarded, causing a real network call. Under pytest-xdist loadscope this surfaced as a deterministic failure of test_accept_header_in_completion_request_jwt whenever an unrelated PR shifted worker distribution. Also removes the same reload pattern from the vertex rerank integration test (both were previously removed in a6df01caec and resurrected by a merge conflict resolution) and hardens the agentcore victim test by dropping the bare except that swallowed the real error * fix(sagemaker_chat): forward stream events as they arrive to cut TTFT Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(e2e): require a cache-hit row instead of skipping when absent (#34283) Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci: run UI unit tests on a 16-core runner (#34330) The UI vitest suite is CPU-bound; move it to a 16-core larger runner and raise vitest fork concurrency from 4 to 14 (leaving headroom for the coordinator, jsdom, and the OS) so the full suite and PR-scoped runs finish faster. * test(e2e): induce spend failure row deterministically instead of skipping (#34282) Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(sagemaker_chat): drop redundant first-delta guard that could mask failure Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(bedrock): include type in tool_choice disable_parallel_tool_use config for Converse * chore(ui): add filename, size, JSX-handler, prefer-const, and antd lint rules (#34341) * chore(ui): add filename, size, JSX-handler, prefer-const, and antd lint rules Wires up five error-level ESLint rules on the dashboard, grandfathering every current offender into eslint-suppressions.json so the gate only bites new code and ratchets down as files are fixed - local/filename-pascal-case: new local rule requiring PascalCase .tsx names, exempting Next.js reserved files (page, layout, route, ...) and test/spec files (239 grandfathered) - max-lines: 800 lines over src/**, excluding tests, src/data, and generated schema.d.ts (20 grandfathered) - local/no-complex-jsx-arrow: new local rule flagging inline JSX arrow handlers with block bodies over two statements; each failure is a small extract-to-named -handler refactor (65 grandfathered) - prefer-const: flipped from off to error (103 grandfathered) - no-restricted-imports: added antd to the phase-out ban alongside tremor, and pointed both messages at shadcn/ui primitives (405 antd import sites grandfathered) Both new local rules ship with RuleTester coverage * fix(ui): preserve secondary extensions in filename-pascal-case suggestion The suggestion text built the rename from only the head segment, so a multi-dot file like my-component.utils.tsx was told to become MyComponent.tsx instead of MyComponent.utils.tsx. Rebuild it from the PascalCased head plus the untouched remaining segments, and add tests covering multi-dot filenames and the hyphenated Next.js reserved names (global-error, apple-icon, opengraph-image, twitter-image) * fix(bedrock): let parallel_tool_calls-derived disable flag win over raw tool_choice value * fix(budget): resolve word-form budget_duration so it no longer silently resets daily (#34250) * fix(budget): resolve word-form budget_duration so it no longer silently resets daily Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(budget): normalize legacy word-form budget_duration on key edit load so untouched saves stay canonical Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(budget): preserve canonical budget_duration in key update submit handler Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: milan <milan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(autoroute): discover models via /v1/models so an AI-API-only key works (#34259) Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(sagemaker): forward native streaming events as they arrive to cut TTFT Mirror the sagemaker_chat fix on the native sagemaker/ streaming path: the sync and async completion handlers read the invocations-response-stream body with iter_bytes(chunk_size=1024) / aiter_bytes(chunk_size=1024), so httpx withholds bytes until 1024 accumulate and tokens arrive in gap-then-burst waves. Drop the fixed chunk size so each decoded event is forwarded as its bytes arrive. Also add a boundary-agnostic decoder test proving frames reassemble correctly regardless of where transport reads split the stream. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(sagemaker): cover sync native streaming path via injectable make_sync_call Extract the inline sync streaming post/decode into make_sync_call so it can be exercised with an injected client, mirroring make_async_call, and add a sync regression test that each token is forwarded after exactly one pulled frame. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(sagemaker): assert make_sync_call maps non-200 to SagemakerError Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(ui): give each Models + Endpoints tab its own path (#34327) * feat(ui): give each Models + Endpoints tab its own path * refactor(ui): decompose Models + Endpoints into per-tab pages with URL-driven detail Dissolve the 488-line ModelsAndEndpointsView monolith into one page per tab under the models-and-endpoints route, with a persistent layout owning the header, cost banner, tab bar and refresh. Each tab page owns only its own state; shared lists come from a small useModelDashboardData hook. Replace the stateful model/team drill-in (setSelectedModelId/setSelectedTeamId full-page takeover) with real URL navigation: ?model=<id> and ?team=<id> render ModelInfoView/TeamInfoView from the layout, so a model or team detail view is now shareable, bookmarkable and back-button friendly. Removes the empty placeholder pages from the first commit. Swap the tab bar off phased-out tremor onto antd Tabs. * fix(ui): render model tab panels standalone instead of Tremor TabPanel AllModelsTab, ModelRetrySettingsTab and PriceDataManagementTab rooted their render in a Tremor <TabPanel>, which only renders inside a Tremor <TabGroup>. After the decomposition these panels live under antd Tabs / as route pages with no such ancestor, so All Models (and the other two) rendered blank. Root them in a plain container instead. The existing component tests mocked @tremor/react (stubbing TabPanel to render children), which hid this; add a regression test that renders with real Tremor and asserts the content is visible standalone. Also type visibleSlugs/TAB_LABELS with the canonical ModelTabSlug so a tab added without a matching label is a compile error. * fix(ui): make model/team drill-in navigation work under the /ui static mount The drill-in close (Back to Models) and open were no-ops: the dashboard is a static export served under /ui, a prefix the Next router (basePath "") does not know, so a router.push to the current pathname with only the query changed is deduped and never re-renders. Drive the ?model=/?team= overlay via real browser navigation (window.location) so open and close reliably work; verified live. Also address review feedback: gate the tab-permission redirect on teams/uiSettings having loaded so a team admin hard-loading /add is not bounced to the base before their membership resolves, and memoize getProviderFromModel on modelCostMapData so the health tab's provider labels refresh when the cost map loads. * fix(ui): use window.location.replace for the tab-permission redirect router.replace is unreliable under the /ui static mount (same class of Next-router issue that broke the drill-in back button), so the forbidden-tab redirect could fail to fire. Use window.location.replace, which keeps the no-history semantics of a permission redirect and is deterministic. Redirect stays gated on teams/uiSettings having loaded. * fix(ui): drive model/team drill-in with history.pushState for client-side nav Switch the ?model=/?team= overlay navigation from window.location.assign to window.history.pushState, which Next's App Router observes. This keeps navigation client-side (no full page reload, React Query cache preserved) while still working for the same-path query-only change that router.push cannot do under the /ui static mount. Open, close (Back to Models) and browser Back are all verified in the built UI. Adds unit coverage for the open/close/read behavior. * feat(organization): add RESTful PATCH /v2/organization/{organization_id} (#32350) * fix(organization): persist cleared fields on /organization/update Clearing an org field (the Metadata box or a TPM/RPM/max_budget limit) via PATCH /organization/update looked like it saved but reverted on refresh; the partial-update merge could not tell a cleared field from an untouched one and dropped every clear The endpoint now decides SET vs CLEAR vs UNTOUCHED purely from which keys the raw request body carried, via a pure build_organization_update_plan. Budget nulls flow to update_budget (null clears via exclude_unset), metadata is replace-when-sent (written as {} for the non-nullable Json column), and a budget write on an org with no budget_id creates and links a budget row. This removes the exclude_none dump, both "if v is not None" filters, and the additive _update_dictionary merge Resolves LIT-3664 * feat(organization): add RESTful PATCH /v2/organization/{organization_id} Adds a v2 organization-update endpoint with a deterministic partial-update contract, and reverts the v1 /organization/update changes so its public behavior stays untouched On v2 a field present in the request body is written (null/[]/{} clears, a value sets) and an omitted field is left untouched; presence is read from model_fields_set. Clearing a TPM/RPM/max_budget limit or the metadata now persists instead of being dropped as if it were never sent. Metadata is replace-when-sent and written as {} when cleared, since the org metadata Json column is non-nullable. Budget nulls flow to update_budget, and an org with no budget row gets one created and linked. The endpoint is hidden from the public Swagger docs via include_in_schema=False, and stays typed in the generated dashboard schema Resolves LIT-3664 * test(organization): cover v2 auth guard, negative budget, and object_permission Adds v2 endpoint tests that were missing: the real _verify_org_access path rejects a non-admin caller with 403 and writes nothing, a negative max_budget is rejected with 400 before any DB access, and a sent object_permission is passed to the upsert helper with its id linked onto the org write Refs LIT-3664 * fix(organization): 400 on null-clear of required org fields; drop dead budget upsert organization_alias and models are non-nullable columns, so a v2 request clearing them with null hit a 500 (NOT NULL violation) and could partially apply the budget half of the request first; the endpoint now returns a 400 with a clear message. Also removes the unreachable "create a budget when the org has none" branch from _apply_organization_budget_updates, since budget_id is a non-nullable FK and every org already has one, so the endpoint no longer needs to link a newly-created budget id Refs LIT-3664 * fix(organization): let v2 clear object permissions when sent as null Sending object_permission: null now detaches the org's permission by setting the nullable object_permission_id to null, instead of being a silent no-op, so the endpoint honors its documented "null clears" contract and an admin can actually revoke vector-store/MCP access. Sending a value still merges as before Refs LIT-3664 * fix(organization): make v2 PATCH atomic, strict, and 422-consistent Tighten the PATCH /v2/organization/{id} endpoint against standard HTTP PATCH (RFC 5789 / RFC 7396 JSON Merge Patch) semantics: - Apply the budget-row and org-row writes in one prisma transaction so a failure between them can no longer half-apply the patch (RFC 5789 requires a PATCH to apply atomically). The budget write is inlined as a tx-aware call mirroring the team-member budget path rather than the standalone update_budget route handler - Set extra="forbid" on OrganizationUpdateRequestV2 so an unknown or misspelled key is a 422 instead of a silently dropped no-op; the contract is presence-driven, so swallowing unknown keys is unsafe - Return 422 (not 400) for the hand-rolled field validations (negative budgets, null-clear of required organization_alias/models, invalid model_max_budget) so every validation failure matches the 422 that pydantic already returns for bad values - Document the per-field clear tokens accurately: null clears budget limits and metadata, [] clears models, and organization_alias cannot be cleared Tests cover the single-transaction write path, unknown-field rejection, the 422 status changes, and the budget_reset_at recompute. * fix(organization): reject empty object_permission on v2 PATCH instead of silently keeping grants object_permission is a nested merge field on PATCH /v2/organization/{id}: a sent object merges into the existing permission row (updating one grant list without touching the others), and null detaches it. An empty {} therefore merged nothing and left every existing vector-store/MCP grant in place, so an admin who sent {"object_permission": {}} to strip access silently kept it. Reject a present-but-empty object_permission with a 422 that points the caller at null, mirroring how the endpoint already rejects a null clear of the required organization_alias/models. This keeps merge semantics for non-empty payloads and does not affect the Admin UI, which only ever sends a fully populated object or omits the field. * fix(organization): JSON-serialize model_max_budget on the v2 budget write model_max_budget is a Json column on the budget table. Route the budget-row write through jsonify_object so a dict value is serialized the same way new_budget and the org-row metadata write already do it, keeping every Json column on this endpoint written consistently. Raw dicts already round-trip (update_budget writes them unserialized), so this is not a correctness fix so much as making the one Json column on the budget path follow the same serialization as the rest of the file. Added a test that a patched model_max_budget reaches the budget write JSON-serialized. * refactor(organization): trim v2 docstrings and consolidate planner tests Trim the verbose docstrings on the v2 endpoint, request model, and the two pure helpers to the essential contract, and drop a stale line that still referenced update_budget's exclude_unset (the budget write is inlined now). Collapse the nine per-case planner tests into one parametrized test asserting exact budget/org split per body, and fold the two model-validation rejection cases into one parametrized test. Same 36 test cases run; the planner assertions get stronger (exact-equality instead of presence/absence) and the test additions shrink by ~85 lines. * refactor(organization): inline the v2 update planner into the endpoint Fold the OrganizationUpdatePlan dataclass and build_organization_update_plan helper into update_organization_v2. The budget-vs-org split is a few dict comprehensions built in one shot, so the extra type plus builder was more ceremony than the job needed. Drops the now-unused dataclass/AbstractSet imports and the isolated planner unit tests; the split is exercised end-to-end by the endpoint tests. * fix(organization): run v2 object permission upsert inside the update transaction prepare_object_permission_upsert splits the shared helper's read-and-merge step from its write so the v2 endpoint can upsert the permission row on the same prisma transaction as the budget and org writes. Previously the upsert ran before the transaction, so a rolled-back org write left merged grants live on the permission row the org still pointed at. The upsert record now pins object_permission_id, since the column's @default(uuid()) would otherwise mint a fresh-create id different from the one linked on the org. v1 and the team/key callers of handle_update_object_permission_common keep their existing behavior * fix(lint): keep the v2 org PR within the strict-rule budget The strict gate flagged the PR's new code after the base merge: 11 UP045 Optional fields and a typing.List on OrganizationUpdateRequestV2, Dict annotations in the new upsert helper and the TypeAdapter, and a B008 from the v2 endpoint's Depends default. The model and helper now use pipe unions and builtin generics, and the endpoint takes its auth dependency via Annotated, which avoids the call-in-default pattern B008 targets * fix(routes): expose /v2/organization on the backend component allowlist The component-split coverage test requires every app route on a component; the new v2 org PATCH belongs with the other management endpoints on the backend, alongside the existing /v2/key and /v2/team prefixes * fix(organization): clear budget_reset_at when budget_duration is cleared via v2 PATCH * refactor(ui): migrate search-tools info view to shadcn (#34323) * test(ui): pin search-tools info view behavior before shadcn migration Rewrite the markup-coupled copy-button assertions in SearchToolView to role queries plus lucide icon-state, and add a role/text characterization suite for SearchConnectionTest, which had none. Both are green against the current antd/Tremor components so they can act as an unedited regression net across the migration. * refactor(ui): migrate search-tools info view to shadcn Port the search-tools detail view and its two helpers off antd and Tremor onto the installed shadcn primitives plus token utilities: - SearchToolView (the info page reached by clicking a tool) now uses ui/button, ui/card and a plain CSS-grid header instead of Tremor Card/Grid/Title/Text and antd Button - SearchToolTester swaps antd Input/Button/Spin and Tremor Card/Title for ui/input, ui/button and UiLoadingSpinner, with no inline styles - SearchConnectionTest swaps antd Button/Divider/Typography and the inline keyframe spinner for ui/button, ui/separator and UiLoadingSpinner Markup only; no behavior change. The list page (SearchTools) and its create and edit forms stay on antd because they are Form-bearing and blocked until the forms migration. Icons move from antd and heroicons to lucide. The retired antd no-restricted-imports suppressions are pruned from the baseline. * fix(ui): drop redundant vertical padding in SearchToolTester card The shadcn Card already applies py-6 and gap-6 to its flex children, so the pt-6/pb-6/mb-6 added during the migration stacked on top of it and roughly doubled the vertical whitespace. Keep only px-6 (Card has no horizontal padding) and let the Card own the vertical rhythm, which restores the original 24px spacing. * style(ui): format SearchConnectionTest test file with prettier * fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid (#34325) * fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid The non_root image baked the prisma CLI and engines under /app/.cache and used the CLI's default (library) engine mode. Prisma stopped baking the library engine, so `prisma migrate deploy` fell back to downloading it at startup, which needs network egress and a writable cache. Under an arbitrary non-root uid (OpenShift restricted-v2), an air-gapped network, or a readOnlyRootFilesystem, that download fails and the proxy starts on an empty schema while every DB endpoint returns 500. The migration entrypoint exits 0 on that failure, so a default-uid `docker run` with network never surfaced it Bake to /opt/prisma, a fixed world-readable path no cache mount shadows, and pin PRISMA_CLI_PATH plus PRISMA_CLI_QUERY_ENGINE_TYPE=binary so the baked binary engine is used directly, matching Dockerfile and Dockerfile.database. A build-time guard asserts the binary query engine is present, so a future prisma change that stops baking it fails the image build instead of silently degrading migrations Adds docker/test_offline_migration.sh, run from image-scan, which migrates a fresh Postgres with no egress as a non-root uid and asserts the schema was created, the case a default-uid `docker run` with network cannot catch * test(docker): move the offline migration check into a gated pytest and stop pinning XDG_CACHE_HOME at the read-only bake The offline migration check lived in docker/ as a shell script. It now lives in tests/proxy_migration_tests/ as a pytest gated on LITELLM_IMAGE, matching the sibling schema-migration test gated on DATABASE_URL, and image-scan invokes it with pytest instead of bash. It also asserts the migration entrypoint's exit code alongside the table count, so a crash or a container-startup failure fails loudly rather than only surfacing as a low table count Runtime XDG_CACHE_HOME pointed at /opt/prisma/.cache, which is baked a+rX with no write, so any XDG-aware library writing a cache at runtime would be denied for every uid. Leave it unset so it falls back to $HOME/.cache (/app/.cache, created here and owned by the runtime uid), matching Dockerfile and Dockerfile.database which never pin XDG at runtime. A second test guards against a future edit pointing a cache or home var back at the read-only bake * test(e2e): cover key budget_duration resets on personal, team, and team-member keys (#33896) * Added config.yaml as bind --------- Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Tin Chi Lo <tin@berri.ai> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> Co-authored-by: Yucheng Zhu <yucheng@berri.ai> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Darien Kindlund <darien@kindlund.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Mubashir Osmani <mubashir@berri.ai> Co-authored-by: shivam <shivam@berri.ai> Co-authored-by: milan <milan@berri.ai> Co-authored-by: mateo <mateo@berri.ai>

…tellm_bulk_team_member_update Resolves the modify/delete conflict on ModelsAndEndpointsView.test.tsx by taking the deletion: #34327 dissolved the ModelsAndEndpointsView monolith into per-tab pages, so both the component and its test are gone. The only change this branch had made there was mocking getProxyBaseUrl/getGlobalLitellmHeaderName so the transitive networking import from the bulk-update helper resolved; the replacement page.test.tsx mocks @/components/team/TeamInfo outright, so that import never loads and no equivalent mock is needed.

* fix(sagemaker): forward native streaming events as they arrive to cut TTFT Mirror the sagemaker_chat fix on the native sagemaker/ streaming path: the sync and async completion handlers read the invocations-response-stream body with iter_bytes(chunk_size=1024) / aiter_bytes(chunk_size=1024), so httpx withholds bytes until 1024 accumulate and tokens arrive in gap-then-burst waves. Drop the fixed chunk size so each decoded event is forwarded as its bytes arrive. Also add a boundary-agnostic decoder test proving frames reassemble correctly regardless of where transport reads split the stream. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(sagemaker): cover sync native streaming path via injectable make_sync_call Extract the inline sync streaming post/decode into make_sync_call so it can be exercised with an injected client, mirroring make_async_call, and add a sync regression test that each token is forwarded after exactly one pulled frame. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(sagemaker): assert make_sync_call maps non-200 to SagemakerError Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(ui): give each Models + Endpoints tab its own path (#34327) * feat(ui): give each Models + Endpoints tab its own path * refactor(ui): decompose Models + Endpoints into per-tab pages with URL-driven detail Dissolve the 488-line ModelsAndEndpointsView monolith into one page per tab under the models-and-endpoints route, with a persistent layout owning the header, cost banner, tab bar and refresh. Each tab page owns only its own state; shared lists come from a small useModelDashboardData hook. Replace the stateful model/team drill-in (setSelectedModelId/setSelectedTeamId full-page takeover) with real URL navigation: ?model=<id> and ?team=<id> render ModelInfoView/TeamInfoView from the layout, so a model or team detail view is now shareable, bookmarkable and back-button friendly. Removes the empty placeholder pages from the first commit. Swap the tab bar off phased-out tremor onto antd Tabs. * fix(ui): render model tab panels standalone instead of Tremor TabPanel AllModelsTab, ModelRetrySettingsTab and PriceDataManagementTab rooted their render in a Tremor <TabPanel>, which only renders inside a Tremor <TabGroup>. After the decomposition these panels live under antd Tabs / as route pages with no such ancestor, so All Models (and the other two) rendered blank. Root them in a plain container instead. The existing component tests mocked @tremor/react (stubbing TabPanel to render children), which hid this; add a regression test that renders with real Tremor and asserts the content is visible standalone. Also type visibleSlugs/TAB_LABELS with the canonical ModelTabSlug so a tab added without a matching label is a compile error. * fix(ui): make model/team drill-in navigation work under the /ui static mount The drill-in close (Back to Models) and open were no-ops: the dashboard is a static export served under /ui, a prefix the Next router (basePath "") does not know, so a router.push to the current pathname with only the query changed is deduped and never re-renders. Drive the ?model=/?team= overlay via real browser navigation (window.location) so open and close reliably work; verified live. Also address review feedback: gate the tab-permission redirect on teams/uiSettings having loaded so a team admin hard-loading /add is not bounced to the base before their membership resolves, and memoize getProviderFromModel on modelCostMapData so the health tab's provider labels refresh when the cost map loads. * fix(ui): use window.location.replace for the tab-permission redirect router.replace is unreliable under the /ui static mount (same class of Next-router issue that broke the drill-in back button), so the forbidden-tab redirect could fail to fire. Use window.location.replace, which keeps the no-history semantics of a permission redirect and is deterministic. Redirect stays gated on teams/uiSettings having loaded. * fix(ui): drive model/team drill-in with history.pushState for client-side nav Switch the ?model=/?team= overlay navigation from window.location.assign to window.history.pushState, which Next's App Router observes. This keeps navigation client-side (no full page reload, React Query cache preserved) while still working for the same-path query-only change that router.push cannot do under the /ui static mount. Open, close (Back to Models) and browser Back are all verified in the built UI. Adds unit coverage for the open/close/read behavior. * feat(organization): add RESTful PATCH /v2/organization/{organization_id} (#32350) * fix(organization): persist cleared fields on /organization/update Clearing an org field (the Metadata box or a TPM/RPM/max_budget limit) via PATCH /organization/update looked like it saved but reverted on refresh; the partial-update merge could not tell a cleared field from an untouched one and dropped every clear The endpoint now decides SET vs CLEAR vs UNTOUCHED purely from which keys the raw request body carried, via a pure build_organization_update_plan. Budget nulls flow to update_budget (null clears via exclude_unset), metadata is replace-when-sent (written as {} for the non-nullable Json column), and a budget write on an org with no budget_id creates and links a budget row. This removes the exclude_none dump, both "if v is not None" filters, and the additive _update_dictionary merge Resolves LIT-3664 * feat(organization): add RESTful PATCH /v2/organization/{organization_id} Adds a v2 organization-update endpoint with a deterministic partial-update contract, and reverts the v1 /organization/update changes so its public behavior stays untouched On v2 a field present in the request body is written (null/[]/{} clears, a value sets) and an omitted field is left untouched; presence is read from model_fields_set. Clearing a TPM/RPM/max_budget limit or the metadata now persists instead of being dropped as if it were never sent. Metadata is replace-when-sent and written as {} when cleared, since the org metadata Json column is non-nullable. Budget nulls flow to update_budget, and an org with no budget row gets one created and linked. The endpoint is hidden from the public Swagger docs via include_in_schema=False, and stays typed in the generated dashboard schema Resolves LIT-3664 * test(organization): cover v2 auth guard, negative budget, and object_permission Adds v2 endpoint tests that were missing: the real _verify_org_access path rejects a non-admin caller with 403 and writes nothing, a negative max_budget is rejected with 400 before any DB access, and a sent object_permission is passed to the upsert helper with its id linked onto the org write Refs LIT-3664 * fix(organization): 400 on null-clear of required org fields; drop dead budget upsert organization_alias and models are non-nullable columns, so a v2 request clearing them with null hit a 500 (NOT NULL violation) and could partially apply the budget half of the request first; the endpoint now returns a 400 with a clear message. Also removes the unreachable "create a budget when the org has none" branch from _apply_organization_budget_updates, since budget_id is a non-nullable FK and every org already has one, so the endpoint no longer needs to link a newly-created budget id Refs LIT-3664 * fix(organization): let v2 clear object permissions when sent as null Sending object_permission: null now detaches the org's permission by setting the nullable object_permission_id to null, instead of being a silent no-op, so the endpoint honors its documented "null clears" contract and an admin can actually revoke vector-store/MCP access. Sending a value still merges as before Refs LIT-3664 * fix(organization): make v2 PATCH atomic, strict, and 422-consistent Tighten the PATCH /v2/organization/{id} endpoint against standard HTTP PATCH (RFC 5789 / RFC 7396 JSON Merge Patch) semantics: - Apply the budget-row and org-row writes in one prisma transaction so a failure between them can no longer half-apply the patch (RFC 5789 requires a PATCH to apply atomically). The budget write is inlined as a tx-aware call mirroring the team-member budget path rather than the standalone update_budget route handler - Set extra="forbid" on OrganizationUpdateRequestV2 so an unknown or misspelled key is a 422 instead of a silently dropped no-op; the contract is presence-driven, so swallowing unknown keys is unsafe - Return 422 (not 400) for the hand-rolled field validations (negative budgets, null-clear of required organization_alias/models, invalid model_max_budget) so every validation failure matches the 422 that pydantic already returns for bad values - Document the per-field clear tokens accurately: null clears budget limits and metadata, [] clears models, and organization_alias cannot be cleared Tests cover the single-transaction write path, unknown-field rejection, the 422 status changes, and the budget_reset_at recompute. * fix(organization): reject empty object_permission on v2 PATCH instead of silently keeping grants object_permission is a nested merge field on PATCH /v2/organization/{id}: a sent object merges into the existing permission row (updating one grant list without touching the others), and null detaches it. An empty {} therefore merged nothing and left every existing vector-store/MCP grant in place, so an admin who sent {"object_permission": {}} to strip access silently kept it. Reject a present-but-empty object_permission with a 422 that points the caller at null, mirroring how the endpoint already rejects a null clear of the required organization_alias/models. This keeps merge semantics for non-empty payloads and does not affect the Admin UI, which only ever sends a fully populated object or omits the field. * fix(organization): JSON-serialize model_max_budget on the v2 budget write model_max_budget is a Json column on the budget table. Route the budget-row write through jsonify_object so a dict value is serialized the same way new_budget and the org-row metadata write already do it, keeping every Json column on this endpoint written consistently. Raw dicts already round-trip (update_budget writes them unserialized), so this is not a correctness fix so much as making the one Json column on the budget path follow the same serialization as the rest of the file. Added a test that a patched model_max_budget reaches the budget write JSON-serialized. * refactor(organization): trim v2 docstrings and consolidate planner tests Trim the verbose docstrings on the v2 endpoint, request model, and the two pure helpers to the essential contract, and drop a stale line that still referenced update_budget's exclude_unset (the budget write is inlined now). Collapse the nine per-case planner tests into one parametrized test asserting exact budget/org split per body, and fold the two model-validation rejection cases into one parametrized test. Same 36 test cases run; the planner assertions get stronger (exact-equality instead of presence/absence) and the test additions shrink by ~85 lines. * refactor(organization): inline the v2 update planner into the endpoint Fold the OrganizationUpdatePlan dataclass and build_organization_update_plan helper into update_organization_v2. The budget-vs-org split is a few dict comprehensions built in one shot, so the extra type plus builder was more ceremony than the job needed. Drops the now-unused dataclass/AbstractSet imports and the isolated planner unit tests; the split is exercised end-to-end by the endpoint tests. * fix(organization): run v2 object permission upsert inside the update transaction prepare_object_permission_upsert splits the shared helper's read-and-merge step from its write so the v2 endpoint can upsert the permission row on the same prisma transaction as the budget and org writes. Previously the upsert ran before the transaction, so a rolled-back org write left merged grants live on the permission row the org still pointed at. The upsert record now pins object_permission_id, since the column's @default(uuid()) would otherwise mint a fresh-create id different from the one linked on the org. v1 and the team/key callers of handle_update_object_permission_common keep their existing behavior * fix(lint): keep the v2 org PR within the strict-rule budget The strict gate flagged the PR's new code after the base merge: 11 UP045 Optional fields and a typing.List on OrganizationUpdateRequestV2, Dict annotations in the new upsert helper and the TypeAdapter, and a B008 from the v2 endpoint's Depends default. The model and helper now use pipe unions and builtin generics, and the endpoint takes its auth dependency via Annotated, which avoids the call-in-default pattern B008 targets * fix(routes): expose /v2/organization on the backend component allowlist The component-split coverage test requires every app route on a component; the new v2 org PATCH belongs with the other management endpoints on the backend, alongside the existing /v2/key and /v2/team prefixes * fix(organization): clear budget_reset_at when budget_duration is cleared via v2 PATCH * refactor(ui): migrate search-tools info view to shadcn (#34323) * test(ui): pin search-tools info view behavior before shadcn migration Rewrite the markup-coupled copy-button assertions in SearchToolView to role queries plus lucide icon-state, and add a role/text characterization suite for SearchConnectionTest, which had none. Both are green against the current antd/Tremor components so they can act as an unedited regression net across the migration. * refactor(ui): migrate search-tools info view to shadcn Port the search-tools detail view and its two helpers off antd and Tremor onto the installed shadcn primitives plus token utilities: - SearchToolView (the info page reached by clicking a tool) now uses ui/button, ui/card and a plain CSS-grid header instead of Tremor Card/Grid/Title/Text and antd Button - SearchToolTester swaps antd Input/Button/Spin and Tremor Card/Title for ui/input, ui/button and UiLoadingSpinner, with no inline styles - SearchConnectionTest swaps antd Button/Divider/Typography and the inline keyframe spinner for ui/button, ui/separator and UiLoadingSpinner Markup only; no behavior change. The list page (SearchTools) and its create and edit forms stay on antd because they are Form-bearing and blocked until the forms migration. Icons move from antd and heroicons to lucide. The retired antd no-restricted-imports suppressions are pruned from the baseline. * fix(ui): drop redundant vertical padding in SearchToolTester card The shadcn Card already applies py-6 and gap-6 to its flex children, so the pt-6/pb-6/mb-6 added during the migration stacked on top of it and roughly doubled the vertical whitespace. Keep only px-6 (Card has no horizontal padding) and let the Card own the vertical rhythm, which restores the original 24px spacing. * style(ui): format SearchConnectionTest test file with prettier * test(ui): characterise the memory page's drawer and header before migrating Adds role/text-based coverage for MemoryDetailDrawer (which had none) and extends MemoryView's test past the mocked table to the header, the create modal trigger and the detail drawer round trip. Both are green against the current antd components, so they act as an unedited regression net for the shadcn migration that follows. * refactor(ui): migrate memory page to shadcn Replaces the antd Drawer with ui/sheet and the antd Button, Typography and Space usage with ui/button plus token utilities, and swaps the @ant-design PlusOutlined icon for lucide's Plus. Toasts now go through the shared MessageManager so the route no longer imports antd directly. The route's tests were written against the antd components in the previous commit and are unchanged here, so they pass on both implementations. MemoryEditModal is left alone because it is built on antd Form; the table already sits on the shared DataTable. * fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid (#34325) * fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid The non_root image baked the prisma CLI and engines under /app/.cache and used the CLI's default (library) engine mode. Prisma stopped baking the library engine, so `prisma migrate deploy` fell back to downloading it at startup, which needs network egress and a writable cache. Under an arbitrary non-root uid (OpenShift restricted-v2), an air-gapped network, or a readOnlyRootFilesystem, that download fails and the proxy starts on an empty schema while every DB endpoint returns 500. The migration entrypoint exits 0 on that failure, so a default-uid `docker run` with network never surfaced it Bake to /opt/prisma, a fixed world-readable path no cache mount shadows, and pin PRISMA_CLI_PATH plus PRISMA_CLI_QUERY_ENGINE_TYPE=binary so the baked binary engine is used directly, matching Dockerfile and Dockerfile.database. A build-time guard asserts the binary query engine is present, so a future prisma change that stops baking it fails the image build instead of silently degrading migrations Adds docker/test_offline_migration.sh, run from image-scan, which migrates a fresh Postgres with no egress as a non-root uid and asserts the schema was created, the case a default-uid `docker run` with network cannot catch * test(docker): move the offline migration check into a gated pytest and stop pinning XDG_CACHE_HOME at the read-only bake The offline migration check lived in docker/ as a shell script. It now lives in tests/proxy_migration_tests/ as a pytest gated on LITELLM_IMAGE, matching the sibling schema-migration test gated on DATABASE_URL, and image-scan invokes it with pytest instead of bash. It also asserts the migration entrypoint's exit code alongside the table count, so a crash or a container-startup failure fails loudly rather than only surfacing as a low table count Runtime XDG_CACHE_HOME pointed at /opt/prisma/.cache, which is baked a+rX with no write, so any XDG-aware library writing a cache at runtime would be denied for every uid. Leave it unset so it falls back to $HOME/.cache (/app/.cache, created here and owned by the runtime uid), matching Dockerfile and Dockerfile.database which never pin XDG at runtime. A second test guards against a future edit pointing a cache or home var back at the read-only bake * test(e2e): drive a real Linear OAuth MCP through chat completions under both ingress headers * fix(ui): keep the memory detail sheet inside the viewport on narrow screens The migrated sheet asked for a flat 720px width while its only max-width came from the primitive's sm:-scoped rule, so below the sm breakpoint no cap applied at all: on a 375px viewport the sheet rendered 720px wide with its left edge at -305px, and because it is position:fixed there was no scroll to reach the hidden content. The primitive's own w-3/4 default did not have this problem; the fixed pixel width is what removed the guard. Caps the width to the viewport at every breakpoint and only asks for 720px from sm up. Verified in a browser at 375px, 700px and 1280px: the sheet is now 375, 700 and 720 wide respectively, always at left 0. * feat(mcp): gateway DCR session admission at the aggregate /mcp endpoint (LIT-3637) Admits a keyless SSO user (no virtual key) at the aggregate /mcp endpoint from a gateway DCR session bearer, resolving team/org/SCIM/budget authorization fresh on every call. - Aggregate DCR front door: stateless /register (sealed llm_dcrc_ client ids), SSO-backed /authorize + /authorize/complete, and /token minting identity-only session tokens with PKCE, single-use codes/flows, and rotating refresh tokens. - Admission: a session-shaped Authorization at the aggregate scope opens via _admit_gateway_session, reloads the live user, and runs the centralized policy gate; failures return the RFC 9728 invalid_token challenge. Gated on the un-forgeable, server-only mcp_admitted_user_subject marker, so virtual-key and JWT auth are unchanged. - Authorization model: an admitted subject is resolved as one plain UserAPIKeyAuth per grant source (its own grants, plus each team it is a live roster member of), each answered by the SAME resolver virtual keys use, then unioned. That branch is the FIRST statement of BOTH public resolvers, so no single-credential prelude runs for it and a fault in a lookup it never uses cannot deny its grants. A source team counts only while it is a live grantor: roster membership, not blocked, and neither the team nor its owning org over budget (enforced through the SAME _team_max_budget_check / _organization_max_budget_check owners common_checks uses for keys). Each team source carries that team's own org, so the existing org ceiling caps it; for a keyless source the org list only ever intersects (a ceiling must not become a grant) and an unresolvable ceiling denies rather than silently uncapping, on both the server and tool axes. _roster_team_object is the single owner of "which teams count": a team whose roster no longer lists the user neither grants servers nor throttles, in one place. - Rate limits: the subject is bounded by its user rpm/tpm AND by the per-server mcp_rpm_limit of the team a call is ATTRIBUTED to — the same single source billing charges, from the same owner. A key charges its one pinned team's bucket; a keyless subject has no team_id, so admission stamps each granting team's limit map onto the auth (server-only field, stripped from validated input like the marker) and the limiter emits that team's mcp_per_team descriptor. Charging every granting team instead would let one cross-team user drain several teams' SHARED buckets on a single call and block their other members; and a server the user's OWN grant reaches charges no team bucket at all, because no team provided it. Per-KEY MCP limits do not apply because there is no key. - Wrapper channels: the manager-level union treats the admitted subject by the same grant model. The admin-role short-circuit and the absolute no_mcp_servers early-return are key-credential rules and never apply to it (a session bearer is a third-party client credential, not the dashboard, and the subject's opt-out silences only its own source). Operator-open channels (allow_all_keys, the user's own BYOM submissions) are owned by one operator_open_server_ids helper that BOTH the server union and the admitted tool resolution consult (suppress-BYOM-when- explicitly-scoped is a key-credential rule and never applies to the subject, whose user row carries the DB-default empty mcp_servers), so an open-channel server is default-open for tools instead of listable but uninvokable. - Redirect URIs: one owner, validate_redirect_uri_shape, decides redirect-URI hygiene (bad scheme, fragment, missing host, userinfo, backslash host) and resolves allowlisted native callbacks, shared by DCR registration and the OAuth endpoints. Registration keeps a deliberately wider trust policy than validate_trusted_redirect_uri: public dynamic registration accepts any https client, and its controls are mandatory S256 PKCE plus the consent screen. - Egress leak-defense: a gateway admission credential (session bearer / bridge envelope) is scrubbed from EVERY egress header context, anchored to the credential shape, so it can never be forwarded upstream and replayed. - Single-use guard: auth-code, refresh and connect-flow claims resolve the proxy's cross-worker redis cache themselves rather than trusting the cache passed in, and fail CLOSED on a Redis fault instead of falling back to a per-worker count that a captured id could replay through another worker. - Sign-in return_to: one shared, never-raising helper persists a safe return_to for every sign-in branch (SSO/Okta/generic and username/password), and every branch RESUMES through the same _sso_return_to_redirect the SSO callback uses, so however a deployment signs in the stored value is honored identically (same-origin path directly; control_plane_url via the one-time login-code handoff). A stale cookie is ignored rather than failing a completed sign-in. - Budgets, both halves: ENFORCEMENT (an already over-budget team or its owning org stops being a grantor, in the source gate) and ACCOUNTING (a team-derived tool call is billed to the granting team and ITS org, so that budget accumulates and the right organization is charged). A server the user's own grant reaches bills the user; when several teams grant one server the pick is the lowest team_id, stable and auditable. Billing rides a COPY, so authorization still sees the full union, and it is inert when the target server cannot be resolved from the tool name. Deferred (tracked): client-selected server scoping of the session token (LIT-4680). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(mcp): trim redundant comments and dedupe admission-arm tests Compress the security rationale in the gateway-session admission path of user_api_key_auth_mcp.py, keeping the load-bearing "why" and dropping the restatement, and remove a garbled dead comment in get_allowed_tools_for_server In the tests, hoist the duplicated _team / _admitted_subject fixtures to module-level factories and parametrize the four fail-closed session-bearer variants into one case. No behavior change; the 294 tests in the file still pass * test(e2e): cover key budget_duration resets on personal, team, and team-member keys (#33896) * fix(ui): gate the keyless connect redirect on internal-user roles isAdminRole compares against a list that mixes raw and formatted role strings: it holds raw org_admin but not the "Org Admin" that formatUserRole produces, and AuthContext stores the formatted form. A keyless org admin therefore read as a non-admin and was redirected to the connect page. Gate positively on internalUserRoles instead, which carries both representations, so the redirect targets the persona it is meant for and any role that is not unambiguously an internal user is left on the dashboard. The shared admin list is left alone: completing it would change org-admin access across every isAdminRole caller, which is a roles-policy decision of its own. * refactor(ui): migrate workflow runs to shadcn (#34370) * test(ui): characterise the workflow runs detail drawer before migrating it Pins the drawer's behaviour against the current antd implementation: the metadata fields it surfaces, the timeline ordered by sequence number, the empty-events copy, the messages section staying collapsed until opened, the in-drawer refresh refetching, and the close control dismissing it. Every assertion is role/text based so the same file can stay green once the component moves off antd, without being edited. * refactor(ui): migrate workflow runs to shadcn Replaces the antd Drawer, Collapse, Button, Spin, Tooltip and Empty on the Workflow Runs page with the installed Base UI primitives (Sheet, Collapsible, Button, UiLoadingSpinner, Tooltip) and lucide icons, and moves the page's hardcoded hex colours, fonts and geometry onto design tokens and utility classes so the page can be themed. The only inline styles left are the gantt bars' computed left/width, which are runtime values. Behaviour is unchanged: the drawer's characterisation tests were written against the antd version in the previous commit and pass here without being edited. Retires the file's now-unused antd no-restricted-imports suppression. * refactor(ui): migrate models and endpoints table onto the shared DataTable (#34363) * refactor(ui): migrate models and endpoints table onto the shared DataTable Rebuilds the All Models table on the shared DataTable, following the 2a treatment from the Models + Endpoints design: one card holding search, the Team and View selectors, refresh, columns and filters, with the active filters on a chip row and the pagination footer at the bottom. Retires the last hand-rolled tremor renderer (all_models_table.tsx) and the antd/tremor column defs in molecules/models/columns.tsx, replacing them with a thin AllModelsTable consumer plus AllModelsTableColumns built from the shared cell library. Behavior is preserved end to end. The server sort field mapping now lives next to the column ids so the two cannot drift. Status keeps its column and its sort, hidden by default behind the Columns menu because the design shows nine columns. Access groups collapse into a "+N more" tooltip instead of a per-row expand toggle, and the full reset moves into the filter drawer footer where the design puts it. Adds the shadcn hover-card primitive (Base UI PreviewCard in the base-vega style) for the model information hover, which needs an interactive surface a tooltip cannot provide. * fix(ui): stop the models tab re-querying on mount The mount-time effect fires the debounced search with the initial empty value, and its callback rebuilt the pagination object unconditionally. That produced a second render (and a second query) roughly 300ms after mount with no user input, which on a slow CI machine swapped the table's row nodes mid-interaction and made a click land on a detached node. resetToFirstPage now returns the existing state when already on the first page, so React bails out instead of re-rendering. Pinned with a test that asserts no additional query after the debounce settles; it fails without the fix. * refactor(ui): migrate request logs table onto the shared DataTable (#34343) * refactor(ui): migrate request logs table onto the shared DataTable Moves the Request Logs tab off the local view_logs/table.tsx clone and onto the shared DataTable in server sort, pagination, and filter mode. The container is split into RequestLogsPanel (data owner: the spend-logs query, the session dedup and composition pipeline, and the detail drawer), a thin RequestLogsTable, and RequestLogsTableColumns. The clone itself stays for now because TopModelView and TopKeyView still consume it The advanced filter bar moves into the shared DataTableFilterDrawer, so filters commit on Apply and render as removable chips. That makes the per-keystroke debounce in the query hook redundant, and the hook now takes ColumnFiltersState, PaginationState, and SortingState directly instead of carrying its own filter shape. Reset still restores the default 24 hour window alongside the filters Adds shared/PaginatedSearchSelect, a Base UI combobox with server-side search and infinite scroll, and uses it for the Key Alias and Model filters. That retires the three logs-only antd pickers (PaginatedKeyAliasSelect, PaginatedModelSelect, FilterTeamDropdown) and the FilterComponent molecule they plugged into. The shared TeamDropdown is deliberately untouched: six other surfaces still render it, five of them as a bare child of an antd Form.Item that injects value/onChange implicitly * test(ui): pin team-scoped key alias filtering in the logs filter drawer The Key Alias filter narrows its options to the team selected in the same drawer, a cross-filter dependency carried over from the antd picker it replaced. Nothing covered it: the live QA pass explicitly did not exercise it either, so it was the one behaviour in this migration that could regress silently Asserts the selected team id reaches useInfiniteKeyAliases, that the lookup stays unscoped when no team is picked, and that the scope does not leak into the Model lookup, which shares the same combobox but takes no team * refactor(mcp): delete unreachable v1 OBO handler and gate REST OAuth on v2 resolver The v2 credential resolver owns oauth2_token_exchange end to end: any server with a token-exchange config maps to a non-None TokenExchangeConfig spec, and that config is in _create_mcp_client's override-exclusion set, so a caller x-mcp-* override cannot force it back to v1 either. The v1 handler resolve_mcp_auth reached at spec is None was therefore dead for OBO, including its warn-then-proceed-unauthenticated fall-through. Delete auth/token_exchange.py and the exchange branch, dropping the subject_token parameter that only fed it. Separately, the REST listing and call paths still ran the v1 per-user OAuth lookup for servers the v2 resolver owns. Unlike the two protocol-path call sites they gated on auth_type == oauth2 only, with no to_server_spec check, so a migrated authorization_code server did a DB round-trip whose Authorization header _resolve_v2_auth then discards. Add the same guard via _is_v1_resolved_oauth2_server, shared by the per-server lookup and the prefetch preflight. Also collapses MCPOAuth2TokenCache.async_get_token's now single-caller require_client_credentials_flow kwarg and removes the dead _get_bulk_user_oauth_headers helper (zero callers). * feat(ui): rebuild Organization Settings on react-hook-form + zod with a dirty-field PATCH (#34324) * feat(ui): rebuild Organization Settings on react-hook-form + zod with a dirty-field PATCH Replaces the antd Settings form in organization_view.tsx with OrgSettingsForm, the first consumer of the shared RHF + zod form kit. The form derives a minimal payload from RHF dirty tracking via pickDirty and sends it to the typed PATCH /v2/organization/{organization_id}, so untouched fields are omitted, emptied widgets clear with null ([] for lists), and the old full-send builder with its length > 0 clear-dropping guards is deleted. Adds src/lib/forms/useZodForm.ts so every form gets the z.input/z.output generics and zodResolver wiring from one place, and forwardRefs ui/textarea so RHF can register it under React 18 * fix(ui): forwardRef InputGroupTextarea to match the forwardRef'd Textarea * test(ui): pin that an mcp server edit preserves existing org toolsets * docs(ui): explain the useZodForm generics * chore(ui): re-prune eslint suppressions after rebase onto staging * feat(cost-optimization): add spend-by-tool and cache leakage views Adds GET /v1/tool/spend returning per-tool and daily tool spend with a deduplicated request total, and a cache leakage breakdown on the Prompt Caching tab of the Cost Optimization page. Tool-spend rows are validated at the boundary with pydantic, the endpoint is scoped to proxy admins, date params are cast to timestamptz for real-Postgres query_raw, and the leakage math treats litellm-normalized prompt_tokens as cache-inclusive (uncached = max(0, prompt - cache_read - cache_creation)). * fix(ui): hold the landing until the role hydrates before deciding the redirect AuthContext sets token and clears authLoading in one effect, then a second token-keyed effect populates userRole, so there is a render where the user is signed in but userRole is still the initial empty string. The positive internalUserRoles check reads that interim role as non-internal, which let the api-keys dashboard paint for a frame before the role arrived and the keyless redirect ran. Treat "signed in on the post-login landing with an unhydrated role" as a resolving state that holds the loading screen, so the dashboard never flashes. Every login=success token carries a required user_role claim, so the role always hydrates within a tick and this cannot hang; it is scoped to the landing, so ordinary dashboard visits are unaffected. * refactor(e2e): drop require_env, read os.environ where a cred is used (#34413) require_env hard-failed a test (and, for the shared litellm-ops secret, drove piling every provider credential into one blob) whenever an optional cred was absent. Most call sites either read a value the test actually uses or just gated on the runner's env for a key the gateway consumes. Read os.environ directly where the test uses the value; drop the presence-only gates so those cases run against the proxy instead of pre-failing on the runner's environment. Removes the require_env helper from e2e_config. * feat(cost-optimization): add by-model view to cache leakage table with plain-language columns Adds a By virtual key / By model toggle to the cache leakage table. The model view aggregates the daily activity model breakdown and is scoped to Anthropic (Claude) models, which support prompt caching. Renames the columns to plain language: Uncached input, Cache hit rate, and Potential savings (replacing Realized caching savings and Est. savings left), with a tooltip on Potential savings that spells out how it is calculated * feat(cost-optimization): sortable cache leakage columns and clearer token column name Makes the three metric columns on the cache leakage table sortable, each with a sensible first-click direction: most uncached tokens and biggest potential savings first, worst cache hit rate first. Repeat clicks toggle the direction. Renames Uncached input to Uncached input tokens, since the column is a token count * fix(ui): keep cache leakage time range picker inline at narrow widths The card header used flex-wrap, so the date picker was the element that gave way when the row ran out of room; at higher browser zoom it dropped onto its own line under the description. Pin the picker with shrink-0 and let the title/description block shrink instead (min-w-0), so the copy wraps to a second line and the picker stays on the right. Below md the header stacks, since a 300px input plus its nowrap label leaves nothing usable beside it. * refactor(ui): migrate agents to shadcn (#34365) * test(ui): make the agents route's tests markup-agnostic before migration Rewrites the two assertions that were coupled to antd's DOM and adds the missing characterisation test for agent_cost_view, so the suite describes behaviour rather than antd markup and can stay untouched across the shadcn migration. The skill selection test reached the checkbox with a querySelector on input[type=checkbox]; antd renders an input while Base UI renders a span[role=checkbox], so it now queries by role and accessible name, which both libraries derive from the wrapping label. The delete confirmation test queried role=dialog; antd Modal is a dialog while Base UI AlertDialog is an alertdialog, so it now anchors on the confirmation text and accepts either role. agent_cost_view had no test at all; it gets one covering the null render, the dollar-prefixed values, the omitted rows, and a zero cost that must not be mistaken for unset. All 55 tests pass against the current antd components. * refactor(ui): migrate agents to shadcn Replaces antd and Tremor with shadcn (base-vega) primitives across the five files the agents route exclusively owns. Markup only; no behaviour, data fetching or route structure changes. Modal becomes AlertDialog, with a plain destructive Button in the footer rather than AlertDialogAction, because that action is AlertDialog.Close and would dismiss the dialog before the delete request settles, losing the in-flight state. Alert, Tag, Spin, Space, Collapse, Descriptions, Typography and the antd icons map onto alert, badge, ui-loading-spinner, flex/grid utilities, collapsible, a definition list, semantic headings and lucide. The shadcn CLI emits alert.tsx importing cva from class-variance-authority, which this project does not depend on; it uses the cva object syntax from lib/cva.config. The generated file fails to typecheck, so the adapted copy lives in components/shared instead, per the convention that ui/ stays CLI-managed. Colour comes from tokens throughout, so the info callout is now the neutral card style rather than antd's blue, and nothing hardcodes a colour in the way of a later theme change. The 55 tests in the route pass unchanged from the previous commit. The visual gate re-baselined agents and all 34 other routes stayed pixel-identical. * refactor(ui): extract shared tab-routing helpers and adopt them in Models + Endpoints (#34435) * refactor(ui): extract shared tab-routing helpers Every per-tab-routed page copy-pastes the same URL<->slug logic and the same active-tab/redirect engine. Extract two reusable pieces: - createTabRoutes(baseSegment, slugs) in utils/tabRoutes.ts returns { baseSegment, slugs, tabHref, slugFromPathname }, the trailing-slash href builder (via migratedHref) and the pathname->slug reader. - useTabRouting({ routes, baseTabKey, visibleKeys, ready }) derives the active tab from the pathname, redirects an unknown/forbidden slug to base once ready, and returns an onTabChange navigator. visibleKeys + ready exist so a role-gated page can pass its filtered tab set and defer the redirect until permissions resolve, rather than bouncing a user off a still-loading valid tab. Both are pure/unit-tested. No page consumes them yet. * refactor(ui): migrate Models + Endpoints onto the shared tab-routing helpers Replace the page's hand-rolled tabRoutes.ts (base segment + slug tuple + href builder + slugFromPathname) with createTabRoutes, keeping the existing named exports as thin re-exports so callers and tests are unchanged. The layout drops its local activeSlug/isKnownSlug/activeKey derivation, its redirect useEffect and its router.push onChange in favor of useTabRouting, passing the role-filtered visibleKeys and a ready flag (!teamsLoading && !uiSettingsLoading) so the permission-gated redirect behavior is preserved exactly. The antd tab bar, role-gated tab set, the refresh button and the ?model=/?team= drill-in overlay are untouched; the file's pre-existing antd import is now recorded in the suppressions baseline since editing it makes it a linted-as-changed file. The existing models-and-endpoints layout.test.tsx and tabRoutes.test.ts pass unchanged, which is the regression guarantee. * fix(bedrock-mantle): backfill usage on non-streaming /v1/messages responses Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(proxy): add overwrite_user_with_key_hash to stamp outgoing user param with key hash (#34417) * feat(proxy): add overwrite_user_with_key_hash to stamp outgoing user param with key hash Adds a litellm_settings flag that forces the outgoing user param to the authenticated key's hashed token before the request is forwarded to the provider. The value overrides any caller-supplied user, so providers see a stable, tamper-proof identifier they can rate-limit or ban on, and the hash matches user_api_key_hash in spend logs for easy mapping back to the key owner. Off by default * fix(proxy): hash non-sk credentials before stamping user param UserAPIKeyAuth only hashes sk-prefixed keys and JWTs; custom-auth credentials stay raw on api_key, so stamping them directly would forward auth material to the provider. Pass through the two known hashed forms (sha256 hex, hashed-jwt-*) and hash anything else * refactor(proxy): stamp only standard virtual keys, skip jwt and custom auth A hashed JWT rotates on every token re-issue so it is useless as a stable ban id, and custom-auth credentials arrive raw on api_key. Instead of hashing whatever we hold, the stamp now applies only when api_key is the sha256 hex digest of a standard virtual key; other auth methods are explicitly out of scope until the stamped identifier is configurable * fix(proxy): gate user stamping on server-set virtual key provenance Shape alone cannot distinguish a key hash from a raw custom-auth credential that happens to be 64 hex chars. Adds via_virtual_key, a server-only marker on UserAPIKeyAuth following the mcp_admitted_user_subject pattern: stripped from all validated input so handlers and claims cannot forge it, set by post-construction assignment only at the DB virtual-key auth return. Stamping now requires the marker and the hash shape * test(proxy): prove db auth path sets via_virtual_key marker The stamping unit tests set the marker manually, so deleting the assignment in _user_api_key_auth_builder would pass every existing test; this exercises the real builder path with a mocked identity store and fails if the marker is not set * fix(proxy): stamp master-key requests with the master key alias Master-key auth substitutes LITELLM_PROXY_MASTER_KEY_ALIAS for api_key so the key and its hash never propagate; that made master-key traffic bypass stamping and pass the caller-supplied user through. The master path now sets via_virtual_key and the stamp gate accepts the alias alongside the sha256 shape, so admin traffic gets the same tamper-proof id that spend logs already record for it * fix(proxy): restore via_virtual_key marker on key-cache hits Cached PROXY_ADMIN auth objects early-return before the marked DB and master-key returns, and cache serialization drops the exclude=True marker, so cached admin traffic bypassed stamping. Key-cache entries are written only after the proxy validated a virtual key or the master key, so the cache-hit boundary restores the marker; the UI-login JWT fallback constructs its token from a decrypted blob, not this cache, and stays unmarked * fix(ui): find logs by request id across pages and dates (LIT-3981) (#31743) * fix(spend): resolve spend logs by request_id across all dates (LIT-3981) The /spend/logs/ui search only filtered the page already loaded, so a log id copied from another page or from outside the active date window could not be found. request_id is the primary key of LiteLLM_SpendLogs, so when it is supplied on the internal UI route the mandatory date window is dropped and the lookup resolves across all time. The date window stays required when no request_id is given, and the public /spend/logs/v2 contract is unchanged. A non-admin id lookup is gated by the same ownership check the detail endpoint uses, so the relaxed window cannot be used to read another tenant's log by id * fix(ui): send the logs request_id search to the server (LIT-3981) The "Search by Request ID" box filtered only the rows already on the current page, so an id from another page never matched. It now feeds the existing server-side request_id filter via handleFilterChange, which debounces, resets to page one, and rides the existing react-query key. The dead client-side filter and its searchTerm state are removed; the session composition and dedup logic is unchanged. The box is now an exact request_id lookup, matching its label; the incidental client-side model and user substring matching it used to do is dropped in favor of the dedicated filters * refactor(spend): model the request_id spend-log lookup as an explicit point lookup (LIT-3981) The date-window relaxation for request_id lookups rode an apply_date_window flag threaded through the date validation and parsing. Model the two intents directly instead. A UI request_id query is a point lookup on the @id primary key that drops the time window and authorizes by row ownership; every other query, including the public /spend/logs/v2 route, takes the range-scan path that still requires a window Because the ownership check fully authorizes the single row, the general user/team scoping is now skipped for id lookups rather than layered on top redundantly. The confusing `is_v2 or request_id is None` guard is gone, and moving the date requirement into the range-scan branch lets the type checker narrow the dates it parses Behavior is preserved: the v2 contract still requires dates even when a request_id is supplied, and a non-owner is still rejected with 403. A regression test covers the non-admin owner id lookup, which resolves across all time and filters by the primary key alone * fix(proxy): reject failed atomic budget reservations under fail_closed_budget_enforcement (#34429) * fix(proxy): reject request when budget reservation write fails under fail_closed_budget_enforcement With general_settings.fail_closed_budget_enforcement set to true, the read-time spend check already returns 503 when spend cannot be verified, but the atomic pre-call reservation still failed open: reserve_budget_for_request swallowed _CounterReservationUnavailable per counter and degraded to read-time-only enforcement, so concurrent requests could all pass the same under-budget read during a Redis outage and overspend past the configured budget. Now the strict flag is threaded into reserve_budget_for_request and a failed reservation write raises 503, releasing any counters that already reserved. Default behavior with the flag absent or false is unchanged. Fixes #33923 * fix(proxy): pass 503 budget-enforcement detail as plain string * ci: only run CodSpeed on backend changes (#34345) Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): keep a key's MCP toolsets when saving an edit The key edit form seeded mcp_servers_and_groups from the key with only servers and accessGroups, but handleKeyUpdate writes mcp_toolsets from that same value, so every save posted an empty list and the backend merge applied it literally. A key granted a toolset lost the grant on any edit, including a budget change, and then got a 403 from /toolset/<name>/mcp Read toolsets in both places the form initializes from keyData, declare mcp_toolsets on KeyResponse.object_permission so a write-without-read is a type error, and carry toolsets through the create flow, which only looked at servers and accessGroups * fix(ui): allow null for mcp_toolsets in the dashboard key response type The generated schema declares object_permission.mcp_toolsets as string[] | null; the handwritten KeyResponse shape omitted the null. ObjectPermissionsView consumes the same value, so its prop type widens with it * fix(cost_tracking): map OpenAI cache_write_tokens for prompt cache creation billing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost_tracking): map cache_write_tokens on Responses API usage path The Responses API (/v1/responses) usage transform rebuilt prompt token details and dropped OpenAI's input_tokens_details.cache_write_tokens, so gpt-5.6 cache-creation tokens were never logged or billed via that route. Map it in the transform, and make PromptTokensDetailsWrapper keep cache_write_tokens and cache_creation_tokens in sync on assignment. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(spend_tracking): populate cache_creation_input_tokens for Responses API logs On the /v1/responses path the response usage is not chat-Usage-shaped, so additional_usage_values could not derive cache tokens from response_obj.usage and the Admin UI Logs cache-creation token row stayed empty. Fall back to the normalized standard_logging usage_object's prompt_tokens_details for both the cache-read and cache-creation counts. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(cost_tracking): cover OpenAI Responses API cache cost breakdown itemization (#34309) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost_calculator): sum mirrored cache token fields once in combine_usage_objects combine_usage_objects iterates prompt_tokens_details model_fields and sums each; with cache_write_tokens and cache_creation_tokens now mirroring each other via __setattr__, the pair was summed twice, doubling cache creation counts for Anthropic batch cost calc, mid-stream fallback usage merges, and realtime usage. Collapse the mirrored pair to one representative before summing. * fix(proxy): restore atomic user upsert when adding team members (#34457) * fix(proxy): restore atomic user upsert when adding team members Parallel /team/new calls naming the same not-yet-existing member were returning 500 "Unique constraint failed on the fields: (`user_id`)". The upsert in add_new_member passed an empty update branch. Prisma only compiles an upsert down to a single INSERT ... ON CONFLICT when that branch writes something; with an empty one it emits SELECT-then-INSERT instead, so concurrent requests all read "no such user" and all insert. Postgres statement logs confirm it: the empty form logs BEGIN/SELECT/INSERT/COMMIT, the non-empty form logs INSERT ... ON CONFLICT ("user_id") DO UPDATE SET. Re-state user_id in the update branch as a no-op so the native upsert path comes back. The teams append stays in the filtered update below it, so an already-existing member still cannot pick up a duplicate team id. tests/test_team.py::test_team_new failed 9 of 15 runs against a live proxy before this and 0 of 15 after. The existing unit test asserted only that upsert had been called on a mock, so it passed either way; it now pins the shape of both branches and fails when the update branch goes back to empty. * test: point the live codex tests at gpt-5.3-codex OpenAI deprecated gpt-5.2-codex, so test_openai_codex and test_openai_codex_stream started failing against the live API with model_not_found. gpt-5.3-codex is the current codex model; both tests pass on it. The remaining gpt-5.2-codex references in the suite are mocked transformation tests and are unaffected. * test(e2e): update models page specs for the shared DataTable The DataTable migration in #34363 changed three things the models page specs were pinned to, and five tests went red. Row click no longer opens the detail view; the Model ID cell owns that now, so both specs click its `model-id-<id>` test id instead of the row. The search box placeholder switched from an ASCII "..." to a real ellipsis, so the specs use getByPlaceholder with a substring instead of an exact attribute match that punctuation can break again. The results count moved from `models-results-count` ("Showing 1 - 50 of 137 results") to the shared pagination's `pagination-range` ("Showing 1-50 of 137"). The Team-BYOK test also filtered rows on the team alias, which the Team ID column has never rendered in either the old or the new table; it filters on the team id now, which is what the column actually shows and what the assertion's own comment intends. Verified against a local proxy serving a fresh build with the seeded e2e postgres and mock upstream: all five failing tests pass, and the full suite is 82 passed / 4 skipped at CI parity (workers=1). * feat(ui): show in the log drawer and session sidebar when an auto-router served a request The dashboard already receives the requested model name as model_group on every spend-log row, but LogEntry dropped the field, so nothing distinguished an auto-routed request from a direct one. Surface it precisely rather than by comparing requested against resolved: model_group differs from model for plain aliases and wildcard deployments too, so a bare mismatch tags almost every row and identifies nothing. The indication is driven instead by which deployments are auto-routers, resolved from every page of /v2/model/info and shared through context. The request drawer header names the router in a badge next to the provider; the session sidebar swaps the entry's leading icon. Rows that no auto-router served render exactly as before. * test(ui): characterise budgets, skills and ui-theme panels before migration Adds a role/text-based characterisation test for UIThemeSettings, which had none, and extends the skills panel test to cover the delete confirmation. Both are green against the current antd/Tremor components so they can prove the shadcn migration keeps behaviour identical without being edited. * test(ui): decouple access-groups, vector-stores and organizations tests from antd markup Prepares the shadcn migration of these three routes by removing every assertion that depends on the current component library, so the same tests can gate the migration without being edited. FiltersButton and its OrganizationFilters consumer both asserted on the ".ant-badge" wrapper class; they now assert the active-filter indicator element itself, and FiltersButton additionally asserts that it is absent when there are no active filters. TestVectorStoreTab drove the antd Select with fireEvent.mouseDown and picked options by node; it now clicks through the combobox role and the option text, which works against any listbox implementation. The vector-stores index test relied on Tremor mounting every TabPanel at once, so it read the Manage tab's table without ever opening that tab. It now clicks the tab first, which is what a user does and what any tabs implementation supports. VectorStoreTester had no test at all, so this adds a characterisation suite covering the empty state, the blank-query guard, the search call and its rendered result, result expansion, Enter versus Shift+Enter, the failure path and clearing history. All of these pass against the current antd and Tremor components * refactor(ui): migrate budgets, skills and ui-theme to shadcn Replaces antd and Tremor with the installed shadcn primitives on the three route-exclusive panels: Tremor tabs, buttons and text on budgets; the antd delete Modal and Tremor button on skills; the Tremor card, inputs and buttons on ui-theme. Markup only, no behaviour change. The characterisation tests added in the previous commit are untouched and stay green, and the ui-theme inputs now carry real label associations. Shared components stay on antd; they are reached by other routes and are migrated separately. The form-bearing files on these routes are left alone. * test(ui): pin mcp-servers, tag-management and tool-policies behaviour before the shadcn migration Rewrite the two markup-coupled assertions off antd class selectors and onto role/text queries, and add characterisation tests for the nine route-owned components that had none. Both rewritten tests and all nine new ones are green against the current antd and Tremor components, so the migration that follows can be judged by tests it never touched. * refactor(ui): migrate access-groups, vector-stores, organizations to shadcn Moves the nine files these three routes exclusively own off antd and Tremor onto the shadcn primitives in src/components/ui. Scope came from the migration analyzer's import closure, so nothing reached by a second route is touched and every file carrying an antd Form is left alone until #34195 lands. access-groups gets the page header, search box and the whole detail view; vector-stores gets the tab shell, the store picker and the tester panel; organizations gets the organization detail view and the three filter controls. Two changes are behavioural rather than cosmetic. The vector-stores tab strip moves from Tremor, which mounts every panel at once, to Base UI, which mounts only the active panel; that is the correct behaviour and the reworked test now opens the tab it asserts on. The antd Select on the Test Vector Store tab becomes a combobox rather than a plain select so its showSearch type-ahead survives. organization_view keeps one antd import, the ColumnsType used to build the extra columns it hands to the shared MemberTable; that is dictated by the shared component's API and goes away when MemberTable migrates. eslint-suppressions.json ratchets down accordingly: eight files lose their no-restricted-imports entry and organization_view drops from three to one. Every test passes unedited across the migration, and the visual gate reports the three migrated routes changed with the other 32 pixel-identical * test(ui): pin logging-and-alerts, caching and policies behaviour before the shadcn migration Establishes the regression net for the upcoming markup migration of these three routes. Every assertion here is written against the current antd and Tremor components and passes against them, so it carries no knowledge of the markup that replaces them and stays meaningful afterwards. Adds characterisation tests for the seven components that had none, and rewrites cache_dashboard's chart-card lookup to anchor on each chart's own title instead of asserting a global count of card nodes, which would break the moment another card appears on the page. No component is touched in this commit. * fix(ui): keep tab panel state across tab switches on the migrated routes Greptile caught a real regression in the shadcn migration: starting to edit organization settings and switching to another tab silently discarded the unsaved input. antd Tabs and Tremor TabGroup mount a panel lazily and then keep it mounted, so a half-filled form or a search history survives leaving the tab and coming back. Base UI unmounts inactive panels instead. Its keepMounted escape hatch is not equivalent either: it mounts every panel eagerly, which renders work the user may never ask for and, on the organization view, put the organization name on screen twice. useVisitedTabs reproduces the original semantics by tracking which tabs have been opened and keeping only those mounted. It is applied to the two tab strips whose panels wrap stateful children: organization Settings, and the vector-stores Create and Test tabs, where an in-progress upload or a search history was equally exposed. The access-group detail tabs render lists derived from props, so they stay lazy. The added regression test fails without the fix and passes with it, and it also passes against the pre-migration antd component, so it pins parity rather than the new markup. * refactor(ui): migrate logging-and-alerts, caching and policies to shadcn Markup-only migration of the 17 files these three routes exclusively own, replacing antd and Tremor with the installed shadcn (base-vega) primitives and lucide icons. No route behaviour changes; the tests written in the previous commit are untouched here and pass against both the old and the new markup. Colour now comes from tokens rather than from hardcoded utilities, so the health-check button, the alerts and the badges no longer pin their own palette. email_settings also loses an invalid DOM nesting (a table cell inside a div, and a div inside a paragraph) that React had been warning about. Two modals on the policies page moved from the Policies panel up to the panel root. Base UI Tabs mounts only the active panel, unlike Tremor, and both are opened from the Templates tab, so leaving them nested would have made "Use Template" do nothing. Retires 53 antd import suppressions from the eslint baseline. * refactor(ui): migrate mcp-servers, tag-management and tool-policies to shadcn Replaces antd and Tremor with shadcn primitives across the 18 files these three routes exclusively own. Markup only: no behaviour, data flow or copy changed, and no shared or form-bearing component is touched, so the blast radius stops at these pages. The 12 tests covering these components are unchanged from the previous commit and still pass, which is the evidence that the rewrite preserved behaviour. Also prunes the six antd no-restricted-imports suppressions these files no longer need. * fix(ui): render each policy template parameter once A template with no LLM enrichment rendered every parameter field twice: the shared list already covers them, because nonEnrichmentParams is the full parameter list when there is no enrichment, and a second no-enrichment branch mapped the same list again. Predates the shadcn migration and was carried forward by it. The test now asserts exactly one field per parameter, and fails if the duplicate branch comes back. * fix(ui): make the suggested MCP network range keyboard operable The suggested CIDR chip was a click-only span both before and after the shadcn migration, so keyboard users could not reach or activate it. Render it as a Button, which brings focus and Enter/Space activation with it, and cover the keyboard path with a test that fails against the old span. * test: remove tests that mutation analysis proved assert nothing 25 test functions across three files pass unchanged when every function they execute is mutated; the owning file killed zero of their scored mutants. Four zero-kill tests tied to the fix in #31288 are kept for rewrite instead of removal. * test: replace deprecated gpt-5-codex with gpt-5.3-codex * feat(anthropic): add Claude Opus 5 Registers claude-opus-5 across the cost maps and provider lists so the model prices, reports its real 1M/128K limits, and advertises its capabilities instead of falling through the generalization patterns at zero cost. Adds the first-party entry plus the Bedrock (base, global, us, eu, au, jp), Vertex AI, and Azure AI variants. Pricing matches Opus 4.8 at $5/$25 per MTok with the usual 1.1x regional premium on the cross-region inference profiles, and fast mode is priced at 2x through provider_specific_entry on the first-party entry only. Two fields deliberately differ from Opus 4.8: prompt_cache_min_tokens drops to 512, and bedrock_output_config_effort_ceiling is omitted because Bedrock accepts output_config.effort="max" for Opus 5. * test(e2e): pin the a2a bridge agent's Anthropic key so message/send works (#34512) The a2a completion-bridge tests registered the agent with only custom_llm_provider and model, so the bridge's litellm.acompletion had no api_key and relied on the gateway resolving ANTHROPIC_API_KEY from its ambient env. When that env var is absent, POST /a2a/{id} returns 500 with "Missing Anthropic API Key" and every message/send test (completion bridge, pinned v0.3/v1.0 message shapes, semver serves) fails while the register/discovery/rejection tests still pass. Give A2ABridgeParams an optional api_key and reg…

feat(ui): give each Models + Endpoints tab its own path

38b6cda

greptile-apps Bot reviewed Jul 23, 2026

View reviewed changes

Comment thread ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/ModelsAndEndpointsView.tsx Outdated

yuneng-berri approved these changes Jul 23, 2026

View reviewed changes

yuneng-berri merged commit 0b0d59d into litellm_internal_staging Jul 23, 2026
77 of 78 checks passed

yuneng-berri deleted the litellm_models_tab_routes branch July 23, 2026 04:39

mateo-berri mentioned this pull request Jul 23, 2026

test(e2e): replace custom endpoints_client with provider SDK clients #34358

Open

42 tasks

cursor Bot mentioned this pull request Jul 24, 2026

fix(lint): baseline antd suppressions for new models-and-endpoints route files #34460

Draft

4 tasks

ryan-crabbe-berri mentioned this pull request Jul 25, 2026

revert(ui): return Models + Endpoints tabs to in-memory routing, keep the ?model drill-in #34629

Merged

5 tasks

Navigation Menu

Uh oh!

Uh oh!

feat(ui): give each Models + Endpoints tab its own path - #34327

feat(ui): give each Models + Endpoints tab its own path#34327
yuneng-berri merged 6 commits into
litellm_internal_stagingfrom
litellm_models_tab_routes

ryan-crabbe-berri commented Jul 23, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Jul 23, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

codspeed-hq Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 23, 2026

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026 •

edited

Loading

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

greptile-apps Bot commented Jul 23, 2026

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

ryan-crabbe-berri commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Type

Changes

Final Attestation

Uh oh!

greptile-apps Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

codspeed-hq Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

codecov Bot commented Jul 23, 2026

Codecov Report

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

greptile-apps Bot commented Jul 23, 2026

Uh oh!

ryan-crabbe-berri commented Jul 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ryan-crabbe-berri commented Jul 23, 2026 •

edited

Loading

greptile-apps Bot commented Jul 23, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 23, 2026 •

edited

Loading

ryan-crabbe-berri commented Jul 23, 2026 •

edited

Loading