Skip to content

fix(anthropic): strip uniqueItems + other unsupported array/object constraints from output_format schema - #33981

Merged
mateo-berri merged 5 commits into
BerriAI:litellm_oss_daily_2026_07_17from
dkindlund:fix/anthropic-output-format-array-object-constraints
Jul 22, 2026
Merged

fix(anthropic): strip uniqueItems + other unsupported array/object constraints from output_format schema#33981
mateo-berri merged 5 commits into
BerriAI:litellm_oss_daily_2026_07_17from
dkindlund:fix/anthropic-output-format-array-object-constraints

Conversation

@dkindlund

@dkindlund dkindlund commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

No existing issue tracks the uniqueItems gap specifically. Related prior work on this same filter: #19444 (the filter's own tracking issue), #21045, #22447, #22500, #29203 and #29624 (stripping numeric/count/length constraints for Anthropic and Bedrock structured outputs), and #18625 / #19201 (why vertex_ai Claude avoids this path; it is forced onto tool-use, which ignores these keywords)

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Anthropic's structured outputs compile the output_format JSON schema into a constrained-decoding grammar and reject keywords the grammar cannot enforce with a 400 invalid_request_error, e.g. output_format.schema: For 'array' type, property 'uniqueItems' is not supported. filter_anthropic_output_schema already strips minimum/maximum/exclusiveMinimum/exclusiveMaximum/minLength/maxLength/minItems/maxItems into description notes, but not uniqueItems, contains/minContains/maxContains, or minProperties/maxProperties, so those reached the API and 400'd. vertex_ai Claude is unaffected because it is routed through tool-use emulation (input_schema), which silently ignores these keywords; that asymmetry is what made this easy to miss

All runs below are fully e2e with zero mocks: a live LiteLLM proxy in front of a real Azure AI Foundry Claude deployment (azure_ai/claude-sonnet-4-6), costing real $. The before run is at fcd236097e (current litellm_internal_staging tip, pre-fix); the after run is at this PR's head dba83b71ba. Same five requests both times, each a response_format: json_schema call whose schema carries one group of unsupported keywords:

Schema keyword(s) Before fcd236097e After dba83b71ba
none (control) 200 200
uniqueItems 400 200
contains+minContains+maxContains 400 200
minProperties+maxProperties 400 200
multipleOf (not in this PR) 400 400

Example request (the uniqueItems case; the other payloads differ only in which keyword they set):

curl -sS "http://localhost:$PORT/v1/chat/completions" \
  -H "Authorization: Bearer sk-1234" -H 'Content-Type: application/json' -d '{
    "model": "claude-sonnet-4-6",
    "messages": [{"role": "user", "content": "List three distinct colors."}],
    "response_format": {"type": "json_schema", "json_schema": {"name": "colors", "schema": {
      "type": "object",
      "properties": {"colors": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}},
      "required": ["colors"], "additionalProperties": false}}}
  }'

Raw results, before (fcd236097e):

clean            | HTTP 200 | {"colors":["red","blue","green"]}
uniqueItems      | HTTP 400 | output_format.schema: For 'array' type, property 'uniqueItems' is not supported
contains         | HTTP 400 | output_format.schema: For 'array' type, property 'contains' is not supported
minMaxProperties | HTTP 400 | output_format.schema: For 'object' type, property 'minProperties' is not supported
multipleOf       | HTTP 400 | output_format.schema: For 'integer' type, property 'multipleOf' is not supported

Raw results, after (dba83b71ba):

clean            | HTTP 200 | {"colors":["red","blue","green"]}
uniqueItems      | HTTP 200 | {"colors":["red","green","blue"]}
contains         | HTTP 200 | {"nums":[1,2,3]}
minMaxProperties | HTTP 200 | {"city":"Paris"}
multipleOf       | HTTP 400 | output_format.schema: For 'integer' type, property 'multipleOf' is not supported

The multipleOf row shows the strip list is still not exhaustive after this PR; it fails identically before and after and is left for a follow-up, since this PR only targets the array/object keywords that were reported failing. These keywords are also confirmed rejected by the Anthropic API directly (not an Azure quirk): the same schemas 400 against api.anthropic.com with the structured-outputs-2025-11-13 beta header

Type

🐛 Bug Fix

Changes

litellm/llms/anthropic/chat/transformation.py: add uniqueItems, contains, minContains, maxContains, minProperties, maxProperties to filter_anthropic_output_schema's unsupported_fields set and constraint_labels map, so they are stripped from the output_format schema and their intent is moved into the field description, exactly as already done for the numeric/length constraints and as the Anthropic SDK does. A disabled boolean constraint (uniqueItems: false) is stripped without adding a misleading advisory note, and sub-schema constraints (contains) are JSON-serialized into the note so the requirement is preserved

Tests: tests/litellm/llms/anthropic/test_anthropic_schema_filter.py covers uniqueItems, contains/minContains/maxContains, minProperties/maxProperties, and the uniqueItems: false edge case; tests/test_litellm/llms/anthropic/test_anthropic_output_format_filter.py mirrors them so the coverage-uploading CI job exercises the new branches

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@greptile-apps

greptile-apps Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR extends filter_anthropic_output_schema to strip the remaining JSON Schema keywords that Anthropic's output_format rejects with a 400 — uniqueItems, contains/minContains/maxContains, and minProperties/maxProperties — and moves their intent into the field description, matching the existing behaviour for numeric/string/array count constraints.

  • transformation.py: Adds six new keywords to unsupported_fields and constraint_labels; adds logic to skip advisory notes for falsy boolean values (uniqueItems: false) and to JSON-serialize sub-schema values (contains) so the description carries the full constraint detail.
  • Test files: Four new unit tests covering each new keyword group and the uniqueItems: false edge case; the new tests/test_litellm/ mirror file ensures the CI coverage job also runs these cases. Existing test changes are purely cosmetic reformatting.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to a schema-filtering utility that removes unsupported keywords before they reach Anthropic's API, with no effect on other providers or code paths.

The fix is additive (expanding an existing set/map), handles edge cases correctly (falsy booleans, sub-schema serialization), and is covered by both existing and new unit tests. No production logic outside the filter function is touched.

No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/anthropic/chat/transformation.py Adds uniqueItems, contains/minContains/maxContains, and minProperties/maxProperties to filter_anthropic_output_schema's unsupported-field set; correctly skips advisory notes for falsy boolean constraints and JSON-serializes sub-schema values.
tests/litellm/llms/anthropic/test_anthropic_schema_filter.py Adds four new tests covering uniqueItems, contains/minContains/maxContains, minProperties/maxProperties, and the uniqueItems: false edge case; existing test reformatting is cosmetic-only with no assertion changes.
tests/test_litellm/llms/anthropic/test_anthropic_output_format_filter.py New file mirroring the schema-filter tests under tests/test_litellm/ (mock-only, no network calls) to ensure the CI coverage job exercises the new keyword handling.

Reviews (3): Last reviewed commit: "test(anthropic): cover output_format arr..." | Re-trigger Greptile

Comment thread litellm/llms/anthropic/chat/transformation.py Outdated
@dkindlund

Copy link
Copy Markdown
Contributor Author

@greptileai I've addressed the contains advisory-note feedback in ce142f8 (sub-schema now serialized as JSON). Please re-review.

@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

dkindlund and others added 2 commits July 20, 2026 11:40
…nstraints from output_format schema

Anthropic's structured outputs (`output_format`) validate the JSON schema
against a strict subset and reject cross-element / count constraints that a
constrained-decoding grammar cannot enforce, returning a 400
`invalid_request_error`.

`filter_anthropic_output_schema` already stripped the numeric / string /
item-count constraints (minimum, maximum, exclusiveMinimum/Maximum, minLength,
maxLength, minItems, maxItems) but still let these through:

- uniqueItems
- contains / minContains / maxContains
- minProperties / maxProperties

so a request using them fails with e.g. "output_format.schema: For 'array'
type, property 'uniqueItems' is not supported".

This is provider-visible: newer Claude models on the native `output_format`
path (e.g. `azure_ai`) 400, while `vertex_ai` is unaffected because it is
forced onto the permissive tool-use path (BerriAI#18625 / BerriAI#19201).

Add the missing keywords to the unsupported-field set and the description map,
and skip the advisory description note for a disabled boolean constraint
(`uniqueItems: false`) so it isn't misdescribed as required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ry note

Address Greptile review: the `contains` advisory note previously discarded the
sub-schema, so the description only said an item must match "a schema" without
saying which. It now serializes the sub-schema as JSON (e.g. "array must
contain an item matching: {\"type\": \"integer\", \"const\": 1}"), matching the
other stripped constraints which carry their value. Sub-schema (dict/list)
values are json.dumps'd; scalar constraints are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dkindlund
dkindlund force-pushed the fix/anthropic-output-format-array-object-constraints branch from ce142f8 to 1ea0ef1 Compare July 20, 2026 15:40
@dkindlund
dkindlund changed the base branch from main to litellm_oss_daily_2026_07_17 July 20, 2026 15:40
dkindlund and others added 3 commits July 20, 2026 11:49
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g in test_litellm tree

Mirrors the schema-filter tests under tests/test_litellm/ so the coverage
job exercises the new uniqueItems/contains/min-maxProperties handling and the
uniqueItems: false branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dkindlund
dkindlund force-pushed the fix/anthropic-output-format-array-object-constraints branch from 7174484 to dba83b7 Compare July 20, 2026 15:59
@codspeed-hq

codspeed-hq Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing dkindlund:fix/anthropic-output-format-array-object-constraints (1ea0ef1) with litellm_oss_daily_2026_07_17 (1ebf2a7)

Open in CodSpeed

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri
mateo-berri self-requested a review July 22, 2026 22:49

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks!

@mateo-berri
mateo-berri merged commit f05f079 into BerriAI:litellm_oss_daily_2026_07_17 Jul 22, 2026
77 of 78 checks passed
mateo-berri added a commit that referenced this pull request Jul 22, 2026
…nstraints from output_format schema (#33981) (#34313)

* fix(anthropic): strip uniqueItems + other unsupported array/object constraints from output_format schema

Anthropic's structured outputs (`output_format`) validate the JSON schema
against a strict subset and reject cross-element / count constraints that a
constrained-decoding grammar cannot enforce, returning a 400
`invalid_request_error`.

`filter_anthropic_output_schema` already stripped the numeric / string /
item-count constraints (minimum, maximum, exclusiveMinimum/Maximum, minLength,
maxLength, minItems, maxItems) but still let these through:

- uniqueItems
- contains / minContains / maxContains
- minProperties / maxProperties

so a request using them fails with e.g. "output_format.schema: For 'array'
type, property 'uniqueItems' is not supported".

This is provider-visible: newer Claude models on the native `output_format`
path (e.g. `azure_ai`) 400, while `vertex_ai` is unaffected because it is
forced onto the permissive tool-use path (#18625 / #19201).

Add the missing keywords to the unsupported-field set and the description map,
and skip the advisory description note for a disabled boolean constraint
(`uniqueItems: false`) so it isn't misdescribed as required.



* fix(anthropic): serialize contains sub-schema in output_format advisory note

Address Greptile review: the `contains` advisory note previously discarded the
sub-schema, so the description only said an item must match "a schema" without
saying which. It now serializes the sub-schema as JSON (e.g. "array must
contain an item matching: {\"type\": \"integer\", \"const\": 1}"), matching the
other stripped constraints which carry their value. Sub-schema (dict/list)
values are json.dumps'd; scalar constraints are unchanged.



* style(anthropic): apply ruff format to output_format filter change



* style(test): ruff format anthropic schema filter tests



* test(anthropic): cover output_format array/object constraint filtering in test_litellm tree

Mirrors the schema-filter tests under tests/test_litellm/ so the coverage
job exercises the new uniqueItems/contains/min-maxProperties handling and the
uniqueItems: false branch.



---------

Co-authored-by: Darien Kindlund <darien@kindlund.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
chakarai1234 added a commit to chakarai1234/litellm that referenced this pull request Jul 23, 2026
* fix(responses): preserve reasoning through prompt hooks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(responses): minimize regression test diff

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(responses): handle non-message-only prompt input

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): raise dashboard session budget default to $1 and make it configurable in config and Admin UI

Every dashboard login mints a 24h session key whose max_budget comes from
litellm.max_ui_session_budget, and all dashboard LLM traffic (playground,
auto router per-tier Test Connection probes) spends against and is gated
by that one key. The $0.25 default locked sessions out mid-testing with
"Budget has been exceeded ... Max budget: 0.25" and the setting appeared
in no docs, no UI, and no error text, so it read as a hardcoded cap.

Raise the default to $1. Give the setting an explicit typed arm in the
config loader (float coercion for env-var strings, null disables the
cap). Surface it on the Admin UI General settings tab through the
existing litellm_settings bridge as a new Dollar field type (positive
USD, unbounded above; the existing Float type is validated to (0, 1] for
fractions), with a spec-level default so clearing the field restores $1
instead of silently removing the cap, and enroll it in
LITELLM_SETTINGS_SAFE_DB_OVERRIDES so UI edits propagate to peer workers.

* fix(ui): resolve General settings rows by field name, not filtered index

The General tab renders generalSettings with TypedDictionary and
prompt-caching rows filtered out, but the Update and Reset handlers
indexed into the unfiltered array, so any row rendered after a
filtered-out entry read another field's value. max_ui_session_budget is
the first General-tab row positioned after the prompt-caching entries,
so its Update sent that row's boolean and failed Dollar validation.
Reset also cleared the local input to null, which reads as unset or
unlimited while the backend had restored the default.

Handlers now resolve the row by field name and drop the index parameter,
and reset displays the row's field_default_value. Component tests drive
the real /config/list ordering through the actual clicks and fail under
either original behavior.

* feat(guardrails): add only_scan_new_messages for per-session incremental scanning (#33278)

* feat(guardrails): add only_scan_new_messages for per-session incremental scanning

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* fix(guardrails): use fixed TTL constant and revert unrelated test formatting

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* fix(guardrails): run only_scan_new_messages in the unified apply_guardrail path

The initial wiring lived in BedrockGuardrail.async_pre_call_hook, but the proxy
routes Bedrock through the unified apply_guardrail interface, so the flag had no
effect live. Move incremental selection into apply_guardrail: filter the flat
texts list against per-session scanned hashes, skip the Bedrock call when nothing
is new, and mark hashes only after a successful (non-blocked) scan. Full-context
fallback is preserved when there is no session id, the cache is unavailable, or a
masking guardrail is configured.

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* test(guardrails): cover session-id fallbacks and mark_texts_scanned guards

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* fix(guardrails): fall back to full scan when incremental guardrail masks content, use shared cache

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* test(guardrails): cover generic agent multi-turn incremental scan

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* test(guardrails): cover incremental scan cache resolver fallbacks

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* test(guardrails): cover flag interactions and /v1/messages incremental scan semantics

* feat(guardrails): make GUARDRAIL_SCANNED_MESSAGES_CACHE_TTL_SECONDS env configurable

* test(guardrails): prove skip_system/skip_tool are enforced upstream of incremental scan

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>

* fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache (#33261)

* fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache

* fix(proxy): make CLI SSO flow state redis-authoritative across workers

The CLI SSO flow is stored in a DualCache whose get_cache is memory-first, so
the worker that served /sso/cli/start keeps serving its stale in-memory flow and
never observes the sso_complete/session_data update another worker writes during
the OAuth callback. Attaching Redis alone is not enough; poll on the original
worker returns pending forever.

Read and write the flow directly through the attached Redis backend when present
so every worker sees the same authoritative state, falling back to the in-memory
DualCache only when no Redis is configured.

* fix(proxy): serialize CLI SSO flow as JSON for the redis round trip

RedisCache stores values via str(value) and parses reads with
json.loads then ast.literal_eval. The completed flow contains a
LitellmUserRoles enum in session_data.user_role, whose repr is not a
parseable literal, so any worker reading the completed flow from redis
raised SyntaxError and returned 400 "CLI login session not found".
Writing the flow as json.dumps makes the round trip lossless (the enum
is a str subclass) and fails loudly at write time if a non-serializable
value is ever added to the flow.

* fix(proxy): point CLI SSO session-not-found hint at configuring Redis

The error message and warning still told users to set enable_redis_auth_cache,
but the CLI SSO session cache now gets Redis unconditionally whenever one is
configured, so that flag no longer affects CLI login

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>

* test(e2e): move Admin UI Playwright suite to tests/e2e/ui (#34196)

Relocates ui/litellm-dashboard/e2e_tests to tests/e2e/ui so all end to end
suites live under tests/e2e. The suite stays in TypeScript and becomes a
self-contained npm package with its own package.json, lockfile and tsconfig
instead of leaning on the dashboard's toolchain; the dashboard drops its
@playwright/test dependency, e2e scripts and knip/vitest/tsconfig carve-outs.

CI paths follow the move: both CircleCI jobs (main e2e and the
SERVER_ROOT_PATH migration smoke) and the test_server_root_path workflow now
install and run Playwright from tests/e2e/ui, with the node cache keyed on
both lockfiles. classify_changes.sh treats tests/e2e/ui as client so spec
edits keep skipping backend jobs. The suite's mock LLM fixture is excluded
from the e2e basedpyright zero-error gate in pyrightconfig.json since it
belongs to the TS suite, not the typed Python harness.

* fix(scim): use members_with_roles as the source of truth for group membership (#34162)

* fix(scim): use members_with_roles as the source of truth for group membership

SCIM group provisioning tracked membership inconsistently. Team creation and
the real team endpoints persist membership in members_with_roles (and each
member's user.teams), but the SCIM group PATCH handler and the GET /Groups
listing read the legacy team.members String[] column, which team creation never
populates. Seeding a PATCH result from that empty column made an Okta "add
member" operation recompute the member set from scratch and silently drop
everyone already in the team, so users ended up missing from the groups they
were provisioned into. Reading the same empty column on GET /Groups reported an
empty member list back to the IdP, which drove repeated re-provisioning.

Separately, add_new_member appended the team id to user.teams with an
unconditional array push. Under the concurrent group PATCHes an IdP sends during
a reconcile, each request passed the members_with_roles duplicate check and
pushed, so user.teams accumulated duplicate ids for the same team. A duplicate
also breaks auth logic that keys off the number of teams a user belongs to.

Read current membership from members_with_roles in the SCIM group PATCH seed and
the GET /Groups listing, and make the user.teams append idempotent via a
filtered update that no-ops once the team is present.

Resolves LIT-4283

* fix(scim): address review; atomic user-creation and stop writing legacy members

Keep the concurrent-safe team append but create the user via an atomic upsert
(create-or-update) instead of a check-then-create, so provisioning the same new
user concurrently cannot race into a duplicate-key failure; the team is still
appended idempotently by a filtered update so an existing user gets no duplicate
team id. Stop writing the legacy team.members column in the group PATCH apply so
the only membership record is the source of truth (members_with_roles plus each
member's user.teams), reconciled by team_member_add/team_member_delete.

Tests: existing add_new_member and team-creation mocks updated to the upsert
plus filtered-append shape, and new tests cover atomic creation and that the
PATCH apply does not write the legacy members column.

* fix(scim): sync team roster and dedup teams for existing-user email upsert (#34183)

* fix(scim): sync team roster and dedup teams for existing-user email upsert

When POST /scim/v2/Users matched an already-existing user by email,
handle_existing_user_by_email raw-wrote the user's teams array but never
touched the team roster, so the user appeared in the group on their profile
yet was absent from the team directly (members_with_roles stayed empty). It
also did not dedup the teams built from repeated SCIM groups.

Route the existing-user team assignment through the same
_handle_team_membership_changes / team_member_add path the PUT update_user
handler uses, so members_with_roles, LiteLLM_TeamMembership, and the user's
teams array stay in sync, and dedup the teams derived from user.groups. The
user_id rewrite to the new userName is preserved and sequenced before the
roster sync so the roster never references a stale primary key.

* fix(scim): surface roster add failures on existing-user email upsert

Route the existing-email upsert's roster sync through patch_team_membership
with a new opt-in raise_on_error flag so a genuine team_member_add failure
propagates instead of being swallowed, and the deduped teams array is only
persisted after the roster sync succeeds. Without this, a failed add left the
endpoint reporting success while user.teams listed a team members_with_roles
never received.

The benign already-a-member case stays a no-op even under the strict path, and
the flag defaults to False so the PUT update_user, PATCH patch_user, and group
callers keep their existing best-effort behavior. SCIM POST is idempotent, so
surfacing the error lets the IdP retry and converge.

* fix(scim): surface roster removal failures symmetrically with adds

Make team_member_delete failures fail loud under the strict roster sync used by
the existing-email upsert, mirroring the add path, so a swallowed removal can no
longer let the user's teams array drop a team the roster still holds. The
idempotent case where the user is already absent from the team stays a no-op,
matching how an add treats the user already being in the team. Best-effort
behavior is preserved for the default raise_on_error=False callers.

* fix(scim): prune deleted user from teams' members_with_roles (#34180)

SCIM delete_user removed the user from the legacy team.members column and deleted
their team membership rows, but never pruned members_with_roles, which is the
source of truth ScimTransformations reads for GET /Groups/{id}. A deleted user
therefore lingered as a dangling member reference on every team they belonged to

Prune each of the user's teams directly via team_member_delete before deleting
the user row, and only for teams whose members_with_roles actually contain the
user, so a real DB failure surfaces (the endpoint fails loudly and the user is
kept; SCIM DELETE is idempotent, so the IdP retries) while a user who was never
in a team's members_with_roles stays a no-op. patch_team_membership is left
unchanged for its other callers

* test(e2e): fail the run when a Rust gateway silently serves /messages through Python (#34208)

A gateway whose native extension is unavailable falls back to the Python
implementation without raising, so it answers /v1/messages normally and the
only difference on the wire is the absent x-litellm-rust header. Nothing in
the suite read that header, so a Rust deployment that had stopped running
Rust produced a fully green e2e run.

Assert the marker on the streamed Messages assertions when E2E_EXPECT_RUST is
set. It stays opt-in because the same suite image also runs against the
standard gateway, which has no Rust path and must keep passing; the two
deployments are already separate Applications, so this is one value on the
Rust instance rather than branching inside the tests.

* fix(team): make team member add atomic to prevent concurrent-add member loss (#34185)

_add_team_members_to_team reconciled membership by reading the complete_team_data
snapshot captured at the start of team_member_add, appending in memory, and
writing the whole members_with_roles array back. Two concurrent /team/member_add
calls for the same team read the same snapshot, so the last write wins and one
member is silently lost. This affects every concurrent team member add, including
the SCIM group PATCH op:add path that routes through team_member_add

Reconcile members_with_roles inside a transaction that locks the team row with
SELECT ... FOR UPDATE before re-reading the current membership, so concurrent
writers serialize on the row lock and each appends onto the other's committed
result. The interactive transaction is exposed through a thin PrismaClient.tx()
passthrough and the locked read is encapsulated in
TeamRepository.get_members_with_roles_locked, and the SCIM group PATCH applies
membership as deltas so concurrent adds are not clobbered

* fix(ui): resolve SSO and SMTP settings from a typed config object (#33576)

The SSO and Email Server settings pages read only stored config, so a gateway
configured entirely through environment variables rendered every field blank
even though both features were live. Rather than add per-endpoint env fallback,
resolve each setting through one typed config object.

A FieldDescriptor names, for one setting, where it lives in the stored row
(db_key), which process env var carries it (env_var), whether it is a secret,
and its effective default. A pure resolve_fields reconciles a descriptor table
against the stored row and the process environment with a fixed precedence and
reports per-field provenance (db, env, default, or unset). The SSO descriptor
table single-sources the field-to-env mapping that the read and write paths
previously duplicated, so they can no longer drift.

get_sso_settings and the /get/config/callbacks alerting block read through the
resolver instead of their own inline fallbacks. get_sso_settings no longer
decrypts stored values into os.environ; decryption happens once inside the
resolver via the pure helper, so a GET stops mutating the process environment.
The SSO response carries provenance so the UI can distinguish an env-sourced
value from a stored one, and secrets are masked at the endpoint (the resolver
returns them unmasked so the login path could consume them). os.environ remains
the runtime carrier; the SSO login and mail-send paths are unchanged.

The settings pages also submit only fields an admin actually edited, so a
rendered mask or env-sourced value is never written back over a working
secret, and generic_scope is a real SSO form field. Omitting a field from
/update/sso_settings clears it, which provider switching relies on; the deeper
write-path concern that behaviour points at is tracked in LIT-4498.

* fix(scim): parse membership id from filtered PATCH path when value omitted (#34181)

Okta commonly sends SCIM membership removals as a filtered path with no request
body value, e.g. Groups PATCH members[value eq "uid"] and Users PATCH
groups[value eq "tid"]. The patch handlers pulled ids only from op.value, so
these removes were a silent no-op and the member or team was never dropped

Add a linear-time filter parser reused by both the Groups members path and the
Users groups path so the id is taken from the [value eq "..."] filter when
op.value is absent, for add and remove ops. The eq operator is matched
case-insensitively, both quote styles are accepted, and the quoted value is
unescaped. The path-filter fallback only fires when the request body value is
omitted, so an explicit empty value no longer resurrects the filter id, and the
compared value must be quoted per the SCIM filter grammar

* fix(ui): remove misleading os.environ tooltip from logging settings (#34305)

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(anthropic): strip uniqueItems + other unsupported array/object constraints from output_format schema (#33981) (#34313)

* fix(anthropic): strip uniqueItems + other unsupported array/object constraints from output_format schema

Anthropic's structured outputs (`output_format`) validate the JSON schema
against a strict subset and reject cross-element / count constraints that a
constrained-decoding grammar cannot enforce, returning a 400
`invalid_request_error`.

`filter_anthropic_output_schema` already stripped the numeric / string /
item-count constraints (minimum, maximum, exclusiveMinimum/Maximum, minLength,
maxLength, minItems, maxItems) but still let these through:

- uniqueItems
- contains / minContains / maxContains
- minProperties / maxProperties

so a request using them fails with e.g. "output_format.schema: For 'array'
type, property 'uniqueItems' is not supported".

This is provider-visible: newer Claude models on the native `output_format`
path (e.g. `azure_ai`) 400, while `vertex_ai` is unaffected because it is
forced onto the permissive tool-use path (#18625 / #19201).

Add the missing keywords to the unsupported-field set and the description map,
and skip the advisory description note for a disabled boolean constraint
(`uniqueItems: false`) so it isn't misdescribed as required.



* fix(anthropic): serialize contains sub-schema in output_format advisory note

Address Greptile review: the `contains` advisory note previously discarded the
sub-schema, so the description only said an item must match "a schema" without
saying which. It now serializes the sub-schema as JSON (e.g. "array must
contain an item matching: {\"type\": \"integer\", \"const\": 1}"), matching the
other stripped constraints which carry their value. Sub-schema (dict/list)
values are json.dumps'd; scalar constraints are unchanged.



* style(anthropic): apply ruff format to output_format filter change



* style(test): ruff format anthropic schema filter tests



* test(anthropic): cover output_format array/object constraint filtering in test_litellm tree

Mirrors the schema-filter tests under tests/test_litellm/ so the coverage
job exercises the new uniqueItems/contains/min-maxProperties handling and the
uniqueItems: false branch.



---------

Co-authored-by: Darien Kindlund <darien@kindlund.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(e2e): satisfy basedpyright in test_session_anomaly (typed Success construction) (#34288)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(anthropic): strip all remaining output_format schema keywords rejected by Anthropic

* refactor(proxy): type the PATCH /team/{team_id} request body (#34195)

* feat(ui): add react-hook-form + zod form infrastructure

Introduce the shared form layer the dashboard's antd forms will migrate onto,
with no user-visible change yet.

- pin react-hook-form, @hookform/resolvers, and zod (kept on 3.25.76 and
  imported via the zod/v4 entrypoint so openai's optional zod ^3 peer still
  resolves and npm ci stays clean)
- vendor the base-vega Field family into components/shared/form as forwardRef
  components on the repo's cva.config, since base-vega ships no form primitive
  and its field source imports class-variance-authority and is React 19 style
- add a FormField bridge that binds a react-hook-form Controller to the Field
  layer and wires label, description, and error ids into aria attributes
- add pickDirty, which narrows a submitted body to the top-level keys the user
  actually touched so a partial update stops re-sending untouched fields

pickDirty reads dirtiness at the top level because react-hook-form tracks it
per leaf, so an edited array arrives as [true, false] and a cleared list as an
empty array that still carries its default-length dirty markers; the falsy
clear tokens (null, [], {}, 0, false) all survive.

Tests cover the Field primitives, the FormField aria wiring against a live
zod resolver, and pickDirty both as a unit and driven through a real
react-hook-form instance.

* test(ui): lock pickDirty behavior on a pure field-array reorder

react-hook-form compares each array element to its default positionally by
value, so useFieldArray move/swap and a reordered scalar array all mark the
moved indices dirty and pickDirty sends the whole array; a swap of two equal
elements is a value-level no-op and is correctly omitted. Covers the reorder
case a review flagged as untested.

* feat(proxy): publish a typed request body for PATCH /team/{team_id}

The route validated its body into UpdateTeamRequest but read it off the raw
request, so the OpenAPI spec carried no requestBody and the dashboard's
generated client could not type the call at all.

- add PatchTeamRequest, UpdateTeamRequest with an optional team_id, since PATCH
  takes the id from the path; a body team_id is still accepted when it matches
- validate the body through PatchTeamRequest before delegating to update_team
- declare the request body on the route and regenerate schema.d.ts

The handler keeps reading the raw body rather than declaring a typed parameter.
FastAPI validates a declared body before the handler runs, which would replace
the 400 for a non-object body with a 422 and move absent-vs-null out of reach of
the RFC 7386 metadata merge; those are pinned by existing tests, so the schema is
declared on the route instead and every error path is unchanged.

Validation is shape-preserving: the body is dumped with exclude_unset so an
omitted field never reaches the write, an explicit null still clears, and a
partial object_permission does not gain sibling sub-keys, which would wipe them
given the column merges rather than replaces.

Tests extend the existing patch harness rather than replacing it.

* refactor(proxy): declare the PATCH /team/{team_id} body as a typed parameter

Replaces the hand-written OpenAPI declaration added earlier in this branch. The
route now takes data: PatchTeamRequest, so FastAPI generates the request body
itself and emits a $ref to the model instead of an inlined copy that would go
stale as fields are added.

The earlier approach was a workaround built on a wrong premise. Declaring the
body does not cost absent-vs-null: model_fields_set preserves it, which is how
POST /team/update already gets its tri-state, and a nested null inside metadata
survives validation untouched, so the RFC 7386 merge is unaffected.

The one real change is the status code for a malformed body. The route answered
400 for a non-object body and 500 for a wrongly typed field, reporting a caller
mistake as a server fault; both are now 422, matching POST /team/update and the
other typed management endpoints. The two tests that pinned the old parse-level
errors are replaced by one that pins the 422 through the ASGI stack, and the
handler drops its manual parsing entirely.

* feat(ui): edit fallback chains from router settings (#32841)

* feat(ui): edit fallback chains from router settings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): address review nits on edit fallbacks modal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): fetch models via react-query in edit fallbacks modal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): use official Google Drive streamable HTTP MCP server (#34322)

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): migrate api-reference to shadcn (#34263)

* test(ui): characterise the API reference page before the shadcn migration

Pins the behaviour the migration must preserve: the three SDK tabs and their
accessible names, the default selection, that selecting a tab surfaces that
SDK's snippet wired to the resolved base url, and the title, blurb and docs
link. Written against the current Tremor markup with role and text queries so
it carries over unedited.

* refactor(ui): migrate the API reference page to shadcn

Replaces the Tremor Grid, Text and Tab primitives on the API reference route
with the shadcn Tabs primitive and token utilities, and prunes the file's now
stale no-restricted-imports suppression. Markup only; the characterisation
test added in the previous commit passes unedited.

The wrapper keeps an explicit grid-cols-1 because Tremor's Grid defaults to
numItems=1 and emitted it; the implicit auto column that replaces it sizes to
the widest child and made the code block overflow the viewport.

* test(ui): scope the API reference snippet assertion to the selected tab panel

Asserts against the rendered tabpanel instead of searching every mounted code
block, so the check keeps proving the selected tab drives the snippet even if
the panels are ever kept mounted.

* refactor(ui): migrate prompts list page to shadcn (#34289)

* test(ui): pin prompts panel toolbar and delete behaviour before migration

* refactor(ui): migrate prompts list panel to shadcn

* fix(ui): resolve prompts environment label and hold the delete dialog while deleting

* refactor(ui): migrate old-usage to shadcn (#34304)

* test(ui): characterise the old usage page before migrating it

Role- and text-based coverage of the route as it behaves on Tremor, so the
shadcn migration has a regression net it did not get to write. Pins the
DISABLE_EXPENSIVE_DB_QUERIES branch (warning copy, the docs link and its
target, and that every expensive query is skipped), the admin vs non-admin
tab set, the cost cards, and the provider and customer tables

* refactor(ui): migrate old-usage to shadcn

Replaces Tremor with the installed shadcn primitives and the shared recharts
wrappers on the only file the route owns. Tabs, cards, tables, the key select
and the tag multi-select come from src/components/ui; the bar, area and donut
charts come from src/components/shared/charts. Tremor BarList has no shared
equivalent, so Total Spend Per Team is composed from ui/meter, which also means
the per-team totals stay numbers in state instead of pre-formatted strings; a
team total of 1,000 or more used to make the bar widths NaN.

The Database Query Limit Reached warning moves with it: same copy, same docs
link, still short-circuiting every expensive query.

Drops the file's no-restricted-imports suppression and the dead customTooltip,
getTopKeys, DataDict and UserData symbols. The characterisation test from the
previous commit is unchanged and green on both sides

* refactor(ui): migrate transform-request to shadcn (#34303)

* test(ui): characterise transform-request panel behaviour before migration

* refactor(ui): migrate transform-request to shadcn

* fix(ui): keep transform-request panels within the fixed-height content fold

* fix(ui): let transform-request flow naturally so the shell scrolls instead of clipping

* test(ui): select the copy button by its accessible name

* chore(ui): bump next to 16.2.11 (#34329)

Moves the dashboard's next pin from 16.2.6 to the latest 16.2.x patch and bumps eslint-config-next to match. Regenerating the lock also healed in explicit bundled-dependency records under @tailwindcss/oxide-wasm32-wasi

* test(e2e): cover key max_budget blocks on personal, team, and team-member keys (#33895)

* test(e2e): cover key max_budget blocks on personal, team, and team-member keys

* refactor(e2e): convert budget enforcement cases to the resources-fixture pattern

The E2ECase class pattern existed only in this file; every other suite uses
plain pytest tests with the resources fixture. Rewrites the nine cases as two
spec classes and removes the now-dead E2ECase protocol and run_case driver
from lifecycle.py

* fix(mcp): mint an ephemeral OAuth client when passthrough authorize has no client_id

Resolves LIT-4581

A true_passthrough MCP server created without the at-creation auth step
has no stored client_id, and the tools-page browser flow supplies none,
so GET /v1/mcp/server/oauth/{id}/authorize dead-ended on a 400
missing_client_id. The client-forwarded-token modes forbid the gateway
from persisting an OAuth client, so client acquisition moves into the one
chokepoint every caller crosses: the authorize endpoint.

resolve_ephemeral_dcr_client owns the whole mint policy (mode gate,
authorization-url precondition, required S256 PKCE, redirect trust, then
a TTL-deduped, per-server single-flighted RFC 7591 mint). The minted
client rides the encrypted OAuth state; /callback seals it with the
upstream code and server_id into an llm_ptcode_ gateway code, and
redeem_passthrough_authorization_code recovers it at the token endpoint
(server binding plus required code_verifier) to authenticate the upstream
exchange. Nothing is persisted; every value rides the encrypted blobs, so
it works across replicas.

Client acquisition is one predicate applied across the whole auth-mode
matrix: the gateway mints for a clientless authorize iff true_passthrough
(any dcr_bridge) or oauth_delegate-and-not-dcr_bridge, and the UI
gatewayMintsClientFor mirrors that set exactly so the browser pre-registers
a client through the dcr_bridge front door only for the cells the gateway
does not mint (the interactive oauth_delegate dcr_bridge sign-in and the
legacy oauth2 passthrough). A minted flow runs the bridge short-circuit
arm; the relay front door stays for external clients that present their
own client_id. Both sides are pinned against the same truth table
(test_resolve_ephemeral_dcr_client_mint_set_is_exact and the
gatewayMintsClientFor matrix test) so no mode can silently diverge. The
authorization_code hook and M2M/token-exchange modes are unchanged.

* fix(tests): remove importlib.reload of http_handler that breaks client injection in later tests (#34336)

The huggingface embedding test fixture reloaded
litellm.llms.custom_httpx.http_handler, creating a new HTTPHandler class
object. llm_http_handler keeps the class captured at import time, so any
test running later in the same process that injects a client built from
the reloaded class fails the isinstance check and the mock is silently
discarded, causing a real network call. Under pytest-xdist loadscope this
surfaced as a deterministic failure of
test_accept_header_in_completion_request_jwt whenever an unrelated PR
shifted worker distribution.

Also removes the same reload pattern from the vertex rerank integration
test (both were previously removed in a6df01caec and resurrected by a
merge conflict resolution) and hardens the agentcore victim test by
dropping the bare except that swallowed the real error

* fix(sagemaker_chat): forward stream events as they arrive to cut TTFT

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): require a cache-hit row instead of skipping when absent (#34283)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run UI unit tests on a 16-core runner (#34330)

The UI vitest suite is CPU-bound; move it to a 16-core larger runner and raise vitest fork concurrency from 4 to 14 (leaving headroom for the coordinator, jsdom, and the OS) so the full suite and PR-scoped runs finish faster.

* test(e2e): induce spend failure row deterministically instead of skipping (#34282)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(sagemaker_chat): drop redundant first-delta guard that could mask failure

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): include type in tool_choice disable_parallel_tool_use config for Converse

* chore(ui): add filename, size, JSX-handler, prefer-const, and antd lint rules (#34341)

* chore(ui): add filename, size, JSX-handler, prefer-const, and antd lint rules

Wires up five error-level ESLint rules on the dashboard, grandfathering every
current offender into eslint-suppressions.json so the gate only bites new code
and ratchets down as files are fixed

- local/filename-pascal-case: new local rule requiring PascalCase .tsx names,
  exempting Next.js reserved files (page, layout, route, ...) and test/spec files
  (239 grandfathered)
- max-lines: 800 lines over src/**, excluding tests, src/data, and generated
  schema.d.ts (20 grandfathered)
- local/no-complex-jsx-arrow: new local rule flagging inline JSX arrow handlers
  with block bodies over two statements; each failure is a small extract-to-named
  -handler refactor (65 grandfathered)
- prefer-const: flipped from off to error (103 grandfathered)
- no-restricted-imports: added antd to the phase-out ban alongside tremor, and
  pointed both messages at shadcn/ui primitives (405 antd import sites grandfathered)

Both new local rules ship with RuleTester coverage

* fix(ui): preserve secondary extensions in filename-pascal-case suggestion

The suggestion text built the rename from only the head segment, so a
multi-dot file like my-component.utils.tsx was told to become
MyComponent.tsx instead of MyComponent.utils.tsx. Rebuild it from the
PascalCased head plus the untouched remaining segments, and add tests
covering multi-dot filenames and the hyphenated Next.js reserved names
(global-error, apple-icon, opengraph-image, twitter-image)

* fix(bedrock): let parallel_tool_calls-derived disable flag win over raw tool_choice value

* fix(budget): resolve word-form budget_duration so it no longer silently resets daily (#34250)

* fix(budget): resolve word-form budget_duration so it no longer silently resets daily

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(budget): normalize legacy word-form budget_duration on key edit load so untouched saves stay canonical

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(budget): preserve canonical budget_duration in key update submit handler

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(autoroute): discover models via /v1/models so an AI-API-only key works (#34259)

Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(sagemaker): forward native streaming events as they arrive to cut TTFT

Mirror the sagemaker_chat fix on the native sagemaker/ streaming path: the
sync and async completion handlers read the invocations-response-stream body
with iter_bytes(chunk_size=1024) / aiter_bytes(chunk_size=1024), so httpx
withholds bytes until 1024 accumulate and tokens arrive in gap-then-burst
waves. Drop the fixed chunk size so each decoded event is forwarded as its
bytes arrive.

Also add a boundary-agnostic decoder test proving frames reassemble correctly
regardless of where transport reads split the stream.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(sagemaker): cover sync native streaming path via injectable make_sync_call

Extract the inline sync streaming post/decode into make_sync_call so it can be
exercised with an injected client, mirroring make_async_call, and add a sync
regression test that each token is forwarded after exactly one pulled frame.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(sagemaker): assert make_sync_call maps non-200 to SagemakerError

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): give each Models + Endpoints tab its own path (#34327)

* feat(ui): give each Models + Endpoints tab its own path

* refactor(ui): decompose Models + Endpoints into per-tab pages with URL-driven detail

Dissolve the 488-line ModelsAndEndpointsView monolith into one page per tab
under the models-and-endpoints route, with a persistent layout owning the
header, cost banner, tab bar and refresh. Each tab page owns only its own
state; shared lists come from a small useModelDashboardData hook.

Replace the stateful model/team drill-in (setSelectedModelId/setSelectedTeamId
full-page takeover) with real URL navigation: ?model=<id> and ?team=<id> render
ModelInfoView/TeamInfoView from the layout, so a model or team detail view is
now shareable, bookmarkable and back-button friendly. Removes the empty
placeholder pages from the first commit.

Swap the tab bar off phased-out tremor onto antd Tabs.

* fix(ui): render model tab panels standalone instead of Tremor TabPanel

AllModelsTab, ModelRetrySettingsTab and PriceDataManagementTab rooted their
render in a Tremor <TabPanel>, which only renders inside a Tremor <TabGroup>.
After the decomposition these panels live under antd Tabs / as route pages with
no such ancestor, so All Models (and the other two) rendered blank. Root them in
a plain container instead.

The existing component tests mocked @tremor/react (stubbing TabPanel to render
children), which hid this; add a regression test that renders with real Tremor
and asserts the content is visible standalone.

Also type visibleSlugs/TAB_LABELS with the canonical ModelTabSlug so a tab added
without a matching label is a compile error.

* fix(ui): make model/team drill-in navigation work under the /ui static mount

The drill-in close (Back to Models) and open were no-ops: the dashboard is a
static export served under /ui, a prefix the Next router (basePath "") does not
know, so a router.push to the current pathname with only the query changed is
deduped and never re-renders. Drive the ?model=/?team= overlay via real browser
navigation (window.location) so open and close reliably work; verified live.

Also address review feedback: gate the tab-permission redirect on teams/uiSettings
having loaded so a team admin hard-loading /add is not bounced to the base before
their membership resolves, and memoize getProviderFromModel on modelCostMapData so
the health tab's provider labels refresh when the cost map loads.

* fix(ui): use window.location.replace for the tab-permission redirect

router.replace is unreliable under the /ui static mount (same class of Next-router
issue that broke the drill-in back button), so the forbidden-tab redirect could
fail to fire. Use window.location.replace, which keeps the no-history semantics of
a permission redirect and is deterministic. Redirect stays gated on teams/uiSettings
having loaded.

* fix(ui): drive model/team drill-in with history.pushState for client-side nav

Switch the ?model=/?team= overlay navigation from window.location.assign to
window.history.pushState, which Next's App Router observes. This keeps navigation
client-side (no full page reload, React Query cache preserved) while still working
for the same-path query-only change that router.push cannot do under the /ui static
mount. Open, close (Back to Models) and browser Back are all verified in the built
UI. Adds unit coverage for the open/close/read behavior.

* feat(organization): add RESTful PATCH /v2/organization/{organization_id} (#32350)

* fix(organization): persist cleared fields on /organization/update

Clearing an org field (the Metadata box or a TPM/RPM/max_budget limit) via PATCH /organization/update looked like it saved but reverted on refresh; the partial-update merge could not tell a cleared field from an untouched one and dropped every clear

The endpoint now decides SET vs CLEAR vs UNTOUCHED purely from which keys the raw request body carried, via a pure build_organization_update_plan. Budget nulls flow to update_budget (null clears via exclude_unset), metadata is replace-when-sent (written as {} for the non-nullable Json column), and a budget write on an org with no budget_id creates and links a budget row. This removes the exclude_none dump, both "if v is not None" filters, and the additive _update_dictionary merge

Resolves LIT-3664

* feat(organization): add RESTful PATCH /v2/organization/{organization_id}

Adds a v2 organization-update endpoint with a deterministic partial-update contract, and reverts the v1 /organization/update changes so its public behavior stays untouched

On v2 a field present in the request body is written (null/[]/{} clears, a value sets) and an omitted field is left untouched; presence is read from model_fields_set. Clearing a TPM/RPM/max_budget limit or the metadata now persists instead of being dropped as if it were never sent. Metadata is replace-when-sent and written as {} when cleared, since the org metadata Json column is non-nullable. Budget nulls flow to update_budget, and an org with no budget row gets one created and linked. The endpoint is hidden from the public Swagger docs via include_in_schema=False, and stays typed in the generated dashboard schema

Resolves LIT-3664

* test(organization): cover v2 auth guard, negative budget, and object_permission

Adds v2 endpoint tests that were missing: the real _verify_org_access path rejects a non-admin caller with 403 and writes nothing, a negative max_budget is rejected with 400 before any DB access, and a sent object_permission is passed to the upsert helper with its id linked onto the org write

Refs LIT-3664

* fix(organization): 400 on null-clear of required org fields; drop dead budget upsert

organization_alias and models are non-nullable columns, so a v2 request clearing them with null hit a 500 (NOT NULL violation) and could partially apply the budget half of the request first; the endpoint now returns a 400 with a clear message. Also removes the unreachable "create a budget when the org has none" branch from _apply_organization_budget_updates, since budget_id is a non-nullable FK and every org already has one, so the endpoint no longer needs to link a newly-created budget id

Refs LIT-3664

* fix(organization): let v2 clear object permissions when sent as null

Sending object_permission: null now detaches the org's permission by setting the nullable object_permission_id to null, instead of being a silent no-op, so the endpoint honors its documented "null clears" contract and an admin can actually revoke vector-store/MCP access. Sending a value still merges as before

Refs LIT-3664

* fix(organization): make v2 PATCH atomic, strict, and 422-consistent

Tighten the PATCH /v2/organization/{id} endpoint against standard HTTP
PATCH (RFC 5789 / RFC 7396 JSON Merge Patch) semantics:

- Apply the budget-row and org-row writes in one prisma transaction so a
  failure between them can no longer half-apply the patch (RFC 5789 requires
  a PATCH to apply atomically). The budget write is inlined as a tx-aware
  call mirroring the team-member budget path rather than the standalone
  update_budget route handler
- Set extra="forbid" on OrganizationUpdateRequestV2 so an unknown or
  misspelled key is a 422 instead of a silently dropped no-op; the contract
  is presence-driven, so swallowing unknown keys is unsafe
- Return 422 (not 400) for the hand-rolled field validations (negative
  budgets, null-clear of required organization_alias/models, invalid
  model_max_budget) so every validation failure matches the 422 that
  pydantic already returns for bad values
- Document the per-field clear tokens accurately: null clears budget limits
  and metadata, [] clears models, and organization_alias cannot be cleared

Tests cover the single-transaction write path, unknown-field rejection, the
422 status changes, and the budget_reset_at recompute.

* fix(organization): reject empty object_permission on v2 PATCH instead of silently keeping grants

object_permission is a nested merge field on PATCH /v2/organization/{id}: a
sent object merges into the existing permission row (updating one grant list
without touching the others), and null detaches it. An empty {} therefore
merged nothing and left every existing vector-store/MCP grant in place, so an
admin who sent {"object_permission": {}} to strip access silently kept it.

Reject a present-but-empty object_permission with a 422 that points the caller
at null, mirroring how the endpoint already rejects a null clear of the
required organization_alias/models. This keeps merge semantics for non-empty
payloads and does not affect the Admin UI, which only ever sends a fully
populated object or omits the field.

* fix(organization): JSON-serialize model_max_budget on the v2 budget write

model_max_budget is a Json column on the budget table. Route the budget-row
write through jsonify_object so a dict value is serialized the same way
new_budget and the org-row metadata write already do it, keeping every Json
column on this endpoint written consistently.

Raw dicts already round-trip (update_budget writes them unserialized), so this
is not a correctness fix so much as making the one Json column on the budget
path follow the same serialization as the rest of the file. Added a test that
a patched model_max_budget reaches the budget write JSON-serialized.

* refactor(organization): trim v2 docstrings and consolidate planner tests

Trim the verbose docstrings on the v2 endpoint, request model, and the two
pure helpers to the essential contract, and drop a stale line that still
referenced update_budget's exclude_unset (the budget write is inlined now).

Collapse the nine per-case planner tests into one parametrized test asserting
exact budget/org split per body, and fold the two model-validation rejection
cases into one parametrized test. Same 36 test cases run; the planner
assertions get stronger (exact-equality instead of presence/absence) and the
test additions shrink by ~85 lines.

* refactor(organization): inline the v2 update planner into the endpoint

Fold the OrganizationUpdatePlan dataclass and build_organization_update_plan
helper into update_organization_v2. The budget-vs-org split is a few dict
comprehensions built in one shot, so the extra type plus builder was more
ceremony than the job needed. Drops the now-unused dataclass/AbstractSet
imports and the isolated planner unit tests; the split is exercised end-to-end
by the endpoint tests.

* fix(organization): run v2 object permission upsert inside the update transaction

prepare_object_permission_upsert splits the shared helper's read-and-merge
step from its write so the v2 endpoint can upsert the permission row on the
same prisma transaction as the budget and org writes. Previously the upsert
ran before the transaction, so a rolled-back org write left merged grants
live on the permission row the org still pointed at. The upsert record now
pins object_permission_id, since the column's @default(uuid()) would
otherwise mint a fresh-create id different from the one linked on the org.
v1 and the team/key callers of handle_update_object_permission_common keep
their existing behavior

* fix(lint): keep the v2 org PR within the strict-rule budget

The strict gate flagged the PR's new code after the base merge: 11 UP045
Optional fields and a typing.List on OrganizationUpdateRequestV2, Dict
annotations in the new upsert helper and the TypeAdapter, and a B008 from
the v2 endpoint's Depends default. The model and helper now use pipe
unions and builtin generics, and the endpoint takes its auth dependency
via Annotated, which avoids the call-in-default pattern B008 targets

* fix(routes): expose /v2/organization on the backend component allowlist

The component-split coverage test requires every app route on a component;
the new v2 org PATCH belongs with the other management endpoints on the
backend, alongside the existing /v2/key and /v2/team prefixes

* fix(organization): clear budget_reset_at when budget_duration is cleared via v2 PATCH

* refactor(ui): migrate search-tools info view to shadcn (#34323)

* test(ui): pin search-tools info view behavior before shadcn migration

Rewrite the markup-coupled copy-button assertions in SearchToolView to
role queries plus lucide icon-state, and add a role/text characterization
suite for SearchConnectionTest, which had none. Both are green against the
current antd/Tremor components so they can act as an unedited regression
net across the migration.

* refactor(ui): migrate search-tools info view to shadcn

Port the search-tools detail view and its two helpers off antd and Tremor
onto the installed shadcn primitives plus token utilities:

- SearchToolView (the info page reached by clicking a tool) now uses
  ui/button, ui/card and a plain CSS-grid header instead of Tremor
  Card/Grid/Title/Text and antd Button
- SearchToolTester swaps antd Input/Button/Spin and Tremor Card/Title for
  ui/input, ui/button and UiLoadingSpinner, with no inline styles
- SearchConnectionTest swaps antd Button/Divider/Typography and the inline
  keyframe spinner for ui/button, ui/separator and UiLoadingSpinner

Markup only; no behavior change. The list page (SearchTools) and its create
and edit forms stay on antd because they are Form-bearing and blocked until
the forms migration. Icons move from antd and heroicons to lucide. The
retired antd no-restricted-imports suppressions are pruned from the
baseline.

* fix(ui): drop redundant vertical padding in SearchToolTester card

The shadcn Card already applies py-6 and gap-6 to its flex children, so
the pt-6/pb-6/mb-6 added during the migration stacked on top of it and
roughly doubled the vertical whitespace. Keep only px-6 (Card has no
horizontal padding) and let the Card own the vertical rhythm, which
restores the original 24px spacing.

* style(ui): format SearchConnectionTest test file with prettier

* fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid (#34325)

* fix(docker): bake non_root prisma engines at /opt/prisma so migrations run offline for any uid

The non_root image baked the prisma CLI and engines under /app/.cache and used
the CLI's default (library) engine mode. Prisma stopped baking the library
engine, so `prisma migrate deploy` fell back to downloading it at startup,
which needs network egress and a writable cache. Under an arbitrary non-root
uid (OpenShift restricted-v2), an air-gapped network, or a readOnlyRootFilesystem,
that download fails and the proxy starts on an empty schema while every DB
endpoint returns 500. The migration entrypoint exits 0 on that failure, so a
default-uid `docker run` with network never surfaced it

Bake to /opt/prisma, a fixed world-readable path no cache mount shadows, and
pin PRISMA_CLI_PATH plus PRISMA_CLI_QUERY_ENGINE_TYPE=binary so the baked binary
engine is used directly, matching Dockerfile and Dockerfile.database. A
build-time guard asserts the binary query engine is present, so a future prisma
change that stops baking it fails the image build instead of silently degrading
migrations

Adds docker/test_offline_migration.sh, run from image-scan, which migrates a
fresh Postgres with no egress as a non-root uid and asserts the schema was
created, the case a default-uid `docker run` with network cannot catch

* test(docker): move the offline migration check into a gated pytest and stop pinning XDG_CACHE_HOME at the read-only bake

The offline migration check lived in docker/ as a shell script. It now lives in
tests/proxy_migration_tests/ as a pytest gated on LITELLM_IMAGE, matching the
sibling schema-migration test gated on DATABASE_URL, and image-scan invokes it
with pytest instead of bash. It also asserts the migration entrypoint's exit
code alongside the table count, so a crash or a container-startup failure fails
loudly rather than only surfacing as a low table count

Runtime XDG_CACHE_HOME pointed at /opt/prisma/.cache, which is baked a+rX with
no write, so any XDG-aware library writing a cache at runtime would be denied
for every uid. Leave it unset so it falls back to $HOME/.cache (/app/.cache,
created here and owned by the runtime uid), matching Dockerfile and
Dockerfile.database which never pin XDG at runtime. A second test guards against
a future edit pointing a cache or home var back at the read-only bake

* test(e2e): cover key budget_duration resets on personal, team, and team-member keys (#33896)

* Added config.yaml as bind

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Tin Chi Lo <tin@berri.ai>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Darien Kindlund <darien@kindlund.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: mateo <mateo@berri.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants