chore(ci): promote internal staging to main - #35501
Merged
Merged
Conversation
Gemini returns each thoughtSignature on exactly one part. LiteLLM stores a function-call signature both message-level (thought_signatures) and on the tool call itself, then re-attached it to BOTH the text part and the function-call part when serializing history. gemini-3 and newer models bill every replayed copy as the previous turn's full reasoning token count, so long agentic sessions doubled their context growth and hit the 1,048,576-token limit Only attach a message-level signature to the text part when the same signature is not already carried by a tool-call part: - compare signature values instead of boolean presence so a distinct text-part signature is never dropped - ignore the gemini-3 dummy-signature fallback during detection so replaying gemini-2.5 history to a newer model keeps the real text signature - count signatures carried by server-side tool invocations so they are not re-attached to the text part gemini-2.5 responses (signature on the text part, function call unsigned) are unaffected: the text signature is preserved as before
…okup
/tag/list returned HTTP 500 for every internal user with
"LiteLLM_VerificationTokenActions.find_many() got an unexpected keyword
argument 'select'". The non-admin branch scopes the tag list to keys owned
by the caller, and that lookup passed select={"token": True}; prisma-client-py
0.11.0 has no select kwarg on find_many, so the call raised TypeError and the
handler's except block turned it into a 500. Since the Admin UI calls
/tag/list on load, Tags was broken for every non-admin user.
/tag/daily/activity shares the same helper and was failing the same way
The kwarg is dropped rather than replaced; the generated client has no
projection API, and a user's key set is small enough that selecting all
columns is not worth working around
The reason this shipped green is that the existing test asserted the call was
made with select={"token": True} against an AsyncMock, which accepts any
keyword. The verification-token table double now binds each call against the
real find_many signature, so an unsupported kwarg raises the same TypeError
production does
…itellm_/jovial-heisenberg-68775c
Move the budgets table onto the paged management list route so sorting, filtering and search happen server-side instead of over whichever rows happened to be in memory. Adds useResourceList, a generic hook that owns page, page_size, sort, q and filters for a server-driven table, folds them into one JSON:API query and returns exactly the props DataTable's server modes want. Budgets is its first consumer. The budget id column now renders in full with a copy button instead of a fixed-width cell, and the table gains Reset and Created columns.
Paging, sorting, filtering and search for an entity collection, declared once as a ListSpec and served by handle_list. The route injects a ListExecutor that owns its table, so this module never imports Prisma. The caller's scope is derived from the caller alone and ANDed with whatever they filtered on, so a query parameter can only narrow what they may read. This is the shared half of the budgets list; it lands here so the endpoint has something to register against, and drops out when the framework arrives on its own branch.
The Budgets page reads /budget/list, which returns the whole table as a bare array with no way to page, sort or filter it. A customer with enough budgets to fill the page has no way to find one. Registers LiteLLM_BudgetTable against the management/v1 list contract: sortable on budget_id, max_budget, tpm_limit, rpm_limit and created_at, default order newest-first with budget_id breaking ties, search on budget_id, and filters for budget_duration, max_budget and created_at. budget_duration is deliberately not sortable; the column holds "7d"/"30d" strings, so a lexicographic ORDER BY puts "30d" ahead of "7d". tpm_limit and rpm_limit are BigInt? in Prisma, so rows validate through a pydantic model on the way out and serialize as JSON numbers. A caller without admin view is refused 403 as a problem document rather than served an empty page. /budget/list is untouched.
…t/MappingProxyType from LIT002
The gateway publishes a tool as `<server prefix><separator><tool name>` and has to recover that boundary on the way back in, to compare a called name against a toolset or allow/deny list and to rebuild the native name sent upstream. Several sites recovered it by cutting at the FIRST separator and others reconstructed it by hand from `MCPServer.name` with a literal `-`, so both disagreed with the prefix the server actually publishes `get_server_prefix` publishes short_prefix, then alias, then server_name, then server_id; it never reads `name`. A server with no alias therefore publishes its hyphen-filled UUID `server_id` as the prefix, and cutting at the first separator leaves most of the UUID glued to the tool name. Every comparison against the stored `(server_id, tool_name)` toolset row then misses: an allowlist denies a tool the list endpoint just advertised, and a disallowed entry stops blocking, which fails open Recover the boundary in one place instead. `match_known_server_prefix` matches a name against the server's registered prefixes, longest first so a prefix that itself contains the separator beats a shorter prefix that is merely its leading segment, and returns None when the name carries none of them. `strip_known_server_prefix` and `is_tool_name_prefixed` both delegate to it, and the sites that receive a wire name call the owner rather than re-deriving the boundary. `split_server_prefix_from_name` stays for the routing pair it was written for, with a docstring saying so The server-level permission checks are the other half. They run after the boundary is already resolved, so their input is bare and the correction there is to derive the wire form rather than strip it back out; stripping a stored entry a second time cuts a boundary the caller already consumed, which breaks a native name that itself opens with the server prefix. Deriving from `get_server_prefix` alone is not enough either, because routing resolves an inbound name against every prefix from `iter_known_server_prefixes`, so enforcement keyed to the published spelling answers for fewer names than are reachable. Turning `LITELLM_USE_SHORT_MCP_TOOL_PREFIX` on republishes every tool under the short ID while an entry stored under the alias stays routable and silently stops being enforced, which is a fail-open on a config nobody edited. `iter_known_tool_name_spellings` yields the bare name plus the wire form under each accepted prefix, and the allow list, the deny list, `allowed_params` and the routing map that `_create_prefixed_tools` builds now all key off that one function, so the set of names enforcement honors and the set routing accepts cannot drift apart `_tool_name_matches` takes the server as a required argument, so a future caller cannot silently fall back to guessing, and it matches against that same spelling set, so `tools/list` hides exactly what dispatch refuses. Answering for fewer spellings in the filter than enforcement honors leaves a blocked tool advertised, which is how the alias-form entry above stayed listed even once the call was refused. The OpenAPI registry lookup builds its key the same way registration does, via `add_server_prefix_to_name` and `get_server_prefix`, because registration used exactly one key; a server whose `name` differs from its published prefix stops missing its own tools
The REST listing filter matched key/team grants through _tool_name_matches, which after the prefix-boundary change answers for every spelling routing accepts. Key-level entries in mcp_tool_permissions and toolset rows name a tool on one server and dispatch compares them bare, so a wire-form entry advertised a tool that tools/call then refused. REST listing now goes through filter_tools_by_key_team_permissions, the same function the MCP list path uses. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… matcher The allow list, the deny list, allowed_params and the discovery filter all ask the same question, "which configured entry names this tool on this server", and each answered it in its own idiom: any() over a spelling tuple, all() over the same tuple negated, a next() that pulled a value out of a dict, and a lowercased set membership. Two review findings on this PR were symptoms of that duplication. Deriving the operands differently at one site produced the over-strip; needing a value rather than a boolean at another produced a truthiness test that read an explicitly empty allowed_params list as "nothing configured" and allowed every parameter. match_known_tool_name returns the matching entry or None, and all four sites read it, so no site can test a container's values to decide membership and the empty-list fail-open is no longer representable. Matching is case-insensitive everywhere, which closes the last divergence between discovery and dispatch: a case-variant disallowed_tools entry used to hide a tool from tools/list while tools/call still executed it. Executable lines over the merge-base drop from +9 to +4, all of it the new owner; mcp_server_manager.py loses 12 lines and the discovery filter loses 17.
Bugbot flagged REST listing advertising key/team grants that tools/call then refuses. The listing side was fixed by routing through filter_tools_by_key_team_permissions, but the two paths still answered the question with separate implementations that only happened to agree: listing stripped the known prefix and compared bare, dispatch compared whatever name it was handed, and each carried its own reading of None and of an empty list. Changing either side silently diverges from the other, which is how this defect appeared in the first place. MCPRequestHandler.tool_is_granted owns the whole decision, and both is_tool_allowed_for_server and filter_tools_by_key_team_permissions read it. None still means no tool-level restriction and an empty list still grants nothing, now stated once. Grants are stored bare by every writer, so matching stays exact against the bare name, deliberately unlike the server-level lists, which honor every spelling routing accepts. test_key_team_listing_and_dispatch_agree drives both production paths over one matrix. It asserts the expected verdict as well as the agreement, because two paths reading one predicate makes equality alone tautological: a wrong predicate keeps them consistent and the agreement assertion alone survived two mutants that the verdict assertion kills.
… does Greptile flagged that match_known_tool_name case-folded both the configured entry and the derived spellings. Routing keeps two tools whose names differ only in case as two tools, so folding merged identities the dispatcher separates: on a server exposing getPet and getpet, an allowlist naming getPet also granted getpet, and a blocklist naming getPet also denied getpet. That is unauthorized execution on one arm and the wrong tool denied on the other. Matching is now exact, which is what identity means here. The case leniency it replaces was never typo tolerance; _register_openapi_tools rewrites every operationId through sanitize_openapi_tool_name, so an allowed_tools entry holding the spec's own spelling never equals the registered name. That link is recovered by replaying the same rewrite, and only on servers that carry a spec_path, which is how the rest of the manager already recognizes an OpenAPI server. Every name that rewrite produces is lowercased, so no two tools on such a server can differ only in case and the fold cannot merge anything. Native servers get no folding at all. test_case_folding_applies_to_openapi_ servers_and_not_to_native_ones pins both halves, and two tests pin that a policy naming one tool leaves its case-variant sibling alone. Dropping the spec_path guard, dropping the fold, and forcing the fold path are all killed. The two pre-existing case-insensitivity tests describe OpenAPI servers in their own docstrings but built fixtures without a spec_path, a shape production never produces for one; they now set it.
The collector read @pytest.mark.covers off every collected item, and collection does not evaluate skips, so a test carrying both a skip and a covers marker reported its cell as covered while asserting nothing. 17 files under tests/e2e do exactly that, which inflated the headline from 290/434 to 311/434. A cell now counts as covered only when at least one test pytest would actually run declares it; a cell claimed by both a live and a skipped test stays covered. Skip state comes from pytest's own evaluator, so skip, skipif (bool and string conditions), and module-level pytestmark resolve exactly as they do in the e2e run. Cells left uncovered this way are listed under the headline and exported as skipped_markers (JSON) and litellm_e2e_coverage_skipped_markers (Prometheus) so the gap surfaces instead of disappearing; the Loki line contract is unchanged. A marker on a skipped test that points outside the registry is still an orphan, so --strict keeps its reach. Because skipif resolves against the environment the collector runs in, the number now depends on that environment; run it where the e2e suite runs. A pytest.skip() call inside a test body remains invisible to a static pass, which the module docstring and README both state.
…itellm_/coverage-collector-skip-markers-d3f666
Greptile found that the OpenAPI fallback added a commit ago collapsed operation
IDs that registration keeps apart: foo/bar and foo.bar register as two tools but
sanitize_openapi_tool_name rewrites both to foo_bar, so a policy naming either
also decided the other.
The cause was two owners for one map, and picking the wrong one. Registration
names an operationId inline at _register_openapi_tools with
operation_id.replace(" ", "_").lower(), which keeps / and . ; the separate
sanitize_openapi_tool_name replaces every character outside [a-zA-Z0-9_-] and
belongs to register_tools_from_openapi, which has no production caller. Nothing
made the matcher use the one that actually registers, so it used the lookalike.
That inline expression is now openapi_tool_name in utils, and both registration
and the matcher call it. Replaying the registering function is the whole safety
argument, and it is structural rather than a claim: two operationIds that
register as two tools normalize to two names here by construction, because this
is the map that registered them. A coarser lookalike cannot be substituted
without a test failing.
The matcher also loses its exact-then-fallback split. The transform is identity
on native servers and idempotent on already-registered names, so normalizing
both sides is exact matching where no OpenAPI spec is involved. Executable lines
drop by three this round; the branch is +5 over the merge-base for four shared
owners that removed duplication at six call sites.
…page (#34867) * fix(mcp): annotate connected-app reachability on the gateway connect page The MCP connect page resolved its server grid through the dashboard identity (admin shortcut or view_all returns the whole registry) while the gateway DCR session it sets up resolves servers as an admitted subject through grant sources only, so the page showed servers and tool counts the session is never served. GET /v1/mcp/server now accepts connected_app_view=true and stamps each returned server with connected_app_reachable, computed by the same _reload_admitted_user + get_allowed_mcp_servers pair the live session uses. The connect page requests the flag in connect mode and renders unreachable servers dimmed with a label, excluded from the Connected count and tool-count fetches. Failure to build the admitted set marks everything unreachable, which matches what such a session would actually be served. Default behavior without the param is unchanged for every existing consumer. * fix(mcp): block connecting unavailable servers from the connect-mode detail view A server the connect page marks unavailable could still be added through its detail view Connect action, so the selection could contain servers the connected-app session is never served. The unavailability decision now lives in one predicate, connectUnavailabilityLabel, consumed by the card indicator, the detail view action area, the toggle-on path, the oauth auto-select effect, and the Connected count, so no interaction path can disagree with the label. This also closes the same pre-existing hole for servers marked not supported on this connection, whose detail view likewise offered Connect, and removes a grandfathered nested ternary, ratcheting the eslint suppressions baseline down * fix(mcp): hide unreachable servers on the connect page instead of dimming them Product decision: the connect page should only show what a connected-app session will actually be served, so annotated-unreachable servers are now filtered out of the connect-mode list at fetch time rather than rendered dimmed. Unsupported auth types keep their existing dimmed label since they are a property of the server, not the caller. A user with zero reachable servers gets an explanatory empty state pointing at grants. The list filter is the single source: counts, tabs, auto-select, detail view, and tool-count fetches all derive from the already-filtered state * fix(mcp): guarantee the connect view lists every session-reachable server The connect view's membership came from the dashboard resolver with the admitted-subject answer only annotated on top, so a server reachable by the session but missing from the dashboard list would be invisible on the page; an under-report, the mirror of the bug this PR fixes. The connect view now unions in any session-reachable server the dashboard resolver did not list, built from the registry and redacted through the same ladder, so page membership equals the admitted set by construction in both directions * fix(mcp): honor connected_app_view only for the dashboard UI session credential The reachability view resolves through the owning user's admitted identity, so a caller-passed virtual key could use the param to enumerate servers beyond its own scope (ids, names, descriptions of the owner's wider grants). The view is now gated on is_ui_session_credential, a predicate factored out of resolve_ui_session_team_ids so the two user-identity widening sites share one trust boundary: the SSO-minted dashboard session token acting as its user. Any other credential gets the param as a no-op and the admitted resolver is never consulted for it * fix(mcp): resolve UI sessions with the admitted-user context everywhere, not per endpoint The list endpoint unioned in session-reachable servers itself while tool counts, Connect actions, and credential endpoints still authorized through build_effective_auth_contexts, whose contexts carry team grants but never the user row's own object permission; a user-granted server could render on the connect page while every interaction on it failed. The admitted-user context (the same auth a gateway session resolves with) is now appended inside build_effective_auth_contexts for UI session credentials, so the page list and every per-server action endpoint answer identically, and the list endpoint's one-off union is deleted. Caller-passed keys are still never widened (is_ui_session_credential gate inside the context builder) and a reload failure falls back to team contexts only * fix(mcp): resolve non-admin dashboard sessions as the admitted subject on tool routes Server reachability on the REST tool routes came from the widened context union while tool permission checks ran on the bare session key, which carries no object permission, so a dashboard user could invoke tools their user-level grant excludes. Rather than bookkeeping which context granted which server, the routes now choose one principal at the boundary: acting_user_auth swaps a non-admin UI session for the admitted-subject auth, the same identity a gateway session resolves with, so reachability, per-source fail-closed tool ceilings, rate limits, and billing attribution all bind through the admitted arms that already exist downstream. Admin sessions keep their operator view and caller-passed credentials are never widened. One swap point per route, no per-server principal picking, no parallel permission logic * fix(mcp): derive the connect page's detail view from the reachable server list The detail view held its own copy of the server object, so it outlived the list it came from. When a refetch dropped that server as unreachable, the open detail view kept rendering it and its Connect action still ran: the guard looked the server back up by id or name in the current list, found nothing, and fell through, because a missing target read as "nothing to block" rather than "no longer connectable" Store the selected server's id and derive the row from the list instead. A server the list no longer carries cannot be the detail view's subject, so the stale render, the stale tools query and the guard bypass stop being reachable states rather than being blocked one at a time. handleToggle now takes the server it is toggling, which deletes the lookup that could miss at all * refactor(mcp): one owner for the identity a dashboard session acts as Three call sites reloaded the admitted subject independently, and the management endpoint carried its own copy of the reload, the HTTPException swallow and the logging. admitted_user_context is now the only place that answers "what user identity does this dashboard session act as", and the connected-app reachability helper reads it, which also drops its dead empty-user_id branch That owner now carries the request's tracing span onto the admitted principal. _reload_admitted_user builds a fresh auth from the user row and has no span of its own, so swapping it in on the REST tool routes silently detached every downstream lookup and the tool-call logging from the request's trace Toolset scoping and the acting-as-user swap are mutually exclusive, so they now share one owner on the tools list route. The admitted subject resolves per grant source and a team source deliberately carries none of the caller's object_permission, so a toolset narrowing layered on top would evaporate on every team-granted server: the request would be admitted through the toolset grant and then served tools from servers the toolset never named. A request carrying a toolset name stays on the caller's own credential, exactly as it did before the swap * fix(mcp): commit every async connect-page write against the list as it stands Three continuations in the panel decided against state captured before their await and committed after it, so a reachability refetch landing in between could not be seen handleToggle validated the server at click time and then, once listMCPTools resolved, wrote its name into the selection whatever the list had since become; a server the refresh had dropped was selected anyway. It now re-asks connectableNow at the commit, and that predicate resolves the id against the current list, so absence fails closed instead of reading as nothing to block The load pipeline was worse, because its cancel flag was shared across runs: the successor's effect body reset it to false before the predecessor's fetch resolved, so a superseded load could still run setServers and put the dropped server back on the page outright. The flag is now a per-effect local that only that run's cleanup can clear, which is also what makes unmount stop the chunked tool-count loop again. The load passes its own liveness check down to the tool-count and oauth-status writes rather than having them consult a flag they share with every other run * fix(mcp): write the connect-page server list to its ref as it is committed connectableNow resolves a server id against serversRef, but that ref was a mirror kept in step by a passive effect, so it lagged the state it mirrored by however long React took to render and flush. A continuation resolving inside that window read the previous list: the commit-time reachability check would find a server the refetch had already dropped, call it connectable, and select it, which is the mismatch the check exists to prevent The lag was the whole defect, so the mirror is gone. commitServers writes the ref and the state together, at the one point the list is ever replaced, and the ref is now never older than the last committed list. Readers that want the newest answer (connectableNow, the oauth auto-select effect) get it; rendering still derives from state, so what is on screen is unchanged Pinned by a test that resolves the refetch and the in-flight Connect in the same tick, with no render flushed between them, which is the interleaving the earlier regression could not reach. The two prop mirrors are deliberately untouched: their staleness is inherent to appending to a parent-owned list from an async callback rather than caused by the mirror, and no reachability decision reads them
fix(mcp): recover the tool-name prefix boundary from registered prefixes
…ip-markers-d3f666 fix(e2e): exclude skipped tests from coverage-registry numerator
…ens (LIT-5036) (#35315) PR #35185 added classifier_context_window_size and classifier_context_per_turn_chars to ComplexityRouterConfig; they worked via config.yaml and the API but had no UI control on the Add Model or Edit Auto-Router screens. Wires the two fields into both, shown only when the LLM classifier is selected.
…imeouts aiohttp 3.14.0 and 3.14.1 re-arm the sock_read timer on a keep-alive connection after it has already been returned to the idle pool. The stray timer stamps a SocketTimeoutError on the pooled connection without closing it, so the pool keeps handing it out and the next request to pick it up fails instantly on an error left behind by an earlier, unrelated request. Because a single pool is shared across providers, the failures appear simultaneously across Vertex AI, Bedrock, Anthropic and OpenAI-compatible deployments as sub-millisecond "Connection timed out" errors. uv.lock resolved aiohttp 3.14.1 and the published images install via `uv sync --frozen`, so every image built from that lock shipped the regression. The wheel's own metadata declared `aiohttp>=3.10,<4.0`, which also left pip consumers free to resolve into the same broken window, so both the runtime floor and the uv constraint move to >=3.14.2. Upstream fixed this in aio-libs/aiohttp#12954, released in aiohttp 3.14.2; the lock now resolves 3.14.3. Raising the floor rather than capping below 3.14 keeps the advisories that the existing 3.14.1 floor cleared, so no osv-scanner ignores are needed. litellm requires Python >=3.10 and aiohttp 3.14.2 requires >=3.10, so no supported interpreter loses support. Both new tests fail on the previous pins and pass on these.
…5160) An MCP permission level answers which servers and tools it permits, and a level that answers nothing places no restriction. Key auth was reading a lookup FAULT as that same answer, so the end user, agent and org ceilings quietly disappeared for as long as one lasted, while the keyless gateway-admitted path failed closed on the very same fault. Those levels now separate the two fault classes the user level already did. A principal row that NAMES an object_permission_id whose contents cannot be read is a known entitlement with unknown contents, so it denies. A lookup that fails before we can tell whether the principal is entitled at all still places no ceiling, that being the state which existed before the level did; denying there would refuse MCP to the majority of callers, who have no such entitlement configured. The keyless path is unchanged. Resolves LIT-4960
* feat(proxy): add generic list handler for /management/v1 Adds the ListSpec/QueryPlan machinery the control-plane list endpoints are meant to share, so a resource declares what it exposes instead of hand-rolling its own paging, sorting and filter parsing. build_query_plan is pure: it turns query parameters into a QueryPlan or an RFC 9457 problem without any I/O, which is what lets the plan be asserted as a value. The database half is a ListExecutor protocol injected by the caller, so this module has no Prisma dependency at all. Four things the framework guarantees rather than leaving to each resource: the spec's unique tiebreaker is always the final sort key, so pages cannot repeat rows when the leading column is all nulls; ordering is NULLS LAST in both directions, since Postgres otherwise floats empty values to the top the moment the sort direction flips; the scope predicate is a separate conjunct ahead of every caller filter, so a filter on a scoped column cannot widen it; and a denied scope is a 403 problem rather than a 200 with an empty list. No route and no consumer yet; budgets registers against it next. The facet endpoint's has_more shapes are untouched, and a test pins them so page mode cannot quietly absorb them. * fix(proxy): accept the bare filter[field] form in the list framework Section 5 of the design doc spells equality without an operator bracket (`?filter[status]=active`, and `/management/v1/keys?filter[team_id]=` in the sub-resource paragraph); only the other operators carry a second bracket. The parser only understood `filter[field][op]`, so the canonical spelling came back as an unknown query parameter. `filter[field]` now resolves to the field's `eq` operator, which means it still goes through the declared operator set rather than around it: a field that does not offer `eq` rejects the shorthand. The allowed-parameter list advertises the bare spelling for `eq` and the bracketed one for everything else. Drops two guards from the key parser that could not fire. Operator validation already rejects every malformed operator, and `field in spec.filters` already rejects every field nobody declared, so a well-formedness check on top of them was unreachable; the tests cover the malformed keys directly instead. * fix(proxy): validate list specs at construction and reject repeated params Two gaps a review flagged on the list framework. The page-size cap was only enforced against a supplied page_size, so a spec whose default_page_size exceeded its max_page_size served more rows than the resource allows on exactly the request that omits the parameter. A default of zero was worse: it reached the total_pages division and made the resource 500 on every request. ListSpec now validates 1 <= default_page_size <= max_page_size when it is built, so a misconfigured resource fails as it is registered rather than per request. default_sort is checked against sortable for the same reason; caller-supplied sort was already validated, but the default never passed through that path and a typo there reached the ORDER BY clause untouched. Raising is right here despite the usual model-failures-as-values rule: there is no request in flight and no caller to answer. Repeated query parameters silently collapsed to their last value, so ?page=1&page=999 paged from 999 and a repeated sort key quietly won, which is the same silently-altered-semantics failure the surface already rejects unknown parameters to avoid. They are now a 400. The check lives in handle_list rather than build_query_plan because a Mapping[str, str] cannot represent a repeat at all; the boundary that can see one is the boundary that rejects it. A denied scope still outranks it, matching every other rejection here. Also corrects the order_by_sql docstring, which claimed every field reaching it had been validated against sortable. That held for caller-supplied sort only. * refactor(proxy): model list predicates as frozen values instead of dicts The LIT002 budget rejected the framework: building a where-fragment meant a dict literal per operator, and a dict keyed by a column name chosen at runtime cannot be frozen into a TypedDict or a dataclass field, so there was no spelling of the old shape the rule would accept. Replacing the fragments with a tagged union removes the construction entirely. A plan's where is now a tuple of frozen Compare / Within / IsNull / AnyOf, matched exhaustively, and the field name is a value rather than a key. That also retires the Mapping[str, object] the plan used to carry, which said nothing about what was inside it and left the fragment shape as a convention two sides had to keep agreeing on. Scope predicates take the same type, so a resource declares its row filter in the same vocabulary rather than hand-rolling a backend dict. where_sql renders a plan for a raw-SQL executor, binding every caller-supplied value to a numbered placeholder and writing only spec-declared column names into the statement. It is the counterpart to order_by_sql, which already existed for the same reason: nulls ordering forces the executor onto raw SQL, so the escaping and placeholder arithmetic belong in one reviewed place rather than in each consumer. Also folds the two remaining mutable builds out of the module (set comprehensions and Counter to frozenset/tuple, the serialized page to a tuple pydantic coerces), and lifts the LIKE escaper into common.py so the facet endpoint and the framework share one copy instead of two that can drift. No behavioural change to the facet endpoint; its tests, including the one pinning the escaping, pass untouched.
…itellm_/management-v1-budgets # Conflicts: # litellm/proxy/management_endpoints/management_v1/list_framework.py # litellm/types/proxy/management_endpoints/management_v1.py
) * fix(aiohttp): dispose recycled client sessions deterministically LiteLLMAiohttpTransport replaced its cached aiohttp.ClientSession on loop-mismatch, loop-inspection failure, and "Session is closed" retry without reliably closing the previous session: - the close task from asyncio.create_task() was never referenced, so it could be garbage-collected before running; - the (RuntimeError, AttributeError) fallback branch replaced the session without closing it at all; - sessions bound to a closed event loop were abandoned to the GC ("rely on GC"), and sessions bound to a loop running in another thread were closed from the wrong loop. Replaced sessions surfaced as intermittent "Unclosed client session" / "Unclosed connector" errors from the event-loop exception handler at GC time. _close_recycled_session() now covers the three lifecycles a recycled session can be in: same-loop closes keep a strong task reference until completion; sessions owned by a loop running elsewhere are closed on their own loop via run_coroutine_threadsafe; sessions whose loop is gone are disposed synchronously through the connector teardown that aiohttp's own finalizer uses, which releases pooled connections and silences the finalizer warnings. Fixes #24230 * fix(aiohttp): guard threadsafe close callback against cancelled futures --------- Co-authored-by: Anmol Jaiswal <68013660+anmolg1997@users.noreply.github.com>
Replaces the double's Any annotations and List/Dict aliases with concrete types, matching the equivalent double in the tool policy tests: kwargs are object, records are Sequence[Mock] held as a tuple, and the call log is list[dict[str, object]]. Behaviour is unchanged; the double still binds every call against the real generated prisma action signature, verified by reintroducing the select kwarg and watching the regression tests fail
#35380) All three send a `telemetry` object in the arguments to Datadog's search_datadog_logs tool. Datadog tightened that tool's input schema to reject unknown properties, so every call now fails validation with 'unexpected additional properties ["telemetry"]' before the behavior each test exists to prove is reached. `telemetry` was never a documented Datadog parameter; the tests relied on the server ignoring extra properties. The proxy transmitted exactly what the tests supplied and surfaced the upstream error faithfully, so this is test-side. The covers markers and registry rows stay put: the collector counts a cell as covered only when a test pytest would actually run declares it, so skipping hands all four cells back to the gap list where they belong.
PR #35308 landed a different shape than this branch was written against: `where` is a tuple of frozen predicates rather than a Prisma-shaped mapping, `ListSpec` carries both the row and the wire type, and `where_sql` / `order_by_sql` render for a raw-SQL executor. The budgets executor now queries through `query_raw` the way the spend logs facet does, selecting only the columns it serves. Also casts datetime binds in `where_sql`. They cross into the query engine as JSON, so an uncast placeholder arrives as text and Postgres refuses `timestamp >= text` outright; every `filter[created_at][gte|lte]` was answering 500. The cast reads the bind as an instant and drops it to naive UTC to match Prisma's TIMESTAMP(3) column, the same one /spend/logs/ui applies.
…tion (#35147) The ID-JAG egress arm could only assert a caller that presented its own IdP identity token on the request, so an agent holding a brokered LiteLLM credential got a 412 and never reached the upstream. The assertion captured at SSO login was already persisted per user for exactly this purpose, but nothing read it back. The arm now falls back to that stored assertion, keyed on the authenticated principal's user_id. The identity is always taken from the credential the gateway authenticated, never from a caller-supplied field, so no caller can select whose identity is asserted upstream. A missing, expired, or unidentified subject stays a 412; ID-JAG exists to assert a specific user and a missing subject has no safe substitute. A store outage is the one exception: it is surfaced as a typed AssertionStoreUnavailable and mapped to 503, so a database blip cannot 500 the egress or the upstream-401 retry, and does not tell the user to sign in again over something they cannot fix. Sourcing a subject from the store rather than the request changed what invalidation can rely on, so the exchanged-token cache changed with it. The entry is now addressed by a slot key derived from the principal, plus the caller's own token when it presented one, with a fingerprint of the subject token and config stored beside the bearer and compared on every read. A mismatch reads as a miss and re-mints, so a rotated assertion or an edited server config cannot be served a bearer authorized under the old inputs, and two callers cannot receive each other's. Invalidation is a single delete of a key it can always compute, needing no store lookup on the recovery path. The upstream-401 invalidate-and-retry path was also gated on a truthy inbound subject token, which skipped recovery entirely for store-sourced calls. The gate is now mode-aware: token_exchange still requires an inbound token because it has nothing else to mint from, id_jag does not. oauth2_id_jag is also now selectable in the admin dashboard with its own field set, instead of being reachable only from config.yaml or the REST API. The auth-type selects drop antd list virtualization: at eleven options the last one no longer mounts, which is a scroll in a browser but makes the option unreachable to anything reading the rendered list. Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
Contributor
`import litellm` reaches litellm/integrations/otel/model/config.py via litellm_core_utils/litellm_logging.py, so pydantic-settings is needed at import time. It was declared only in the `proxy` extra, which left a plain `pip install litellm` unimportable on every platform. Adds tests/base_sdk_tests/check_base_sdk_install.py and a base_sdk_install CircleCI job that builds the wheel, installs it into a clean venv with no extras, and smoke-checks the import, a mock completion, a mock embedding, the bundled pricing metadata and the token counter. The check is stdlib-only on purpose; installing pytest into that venv would add packaging, pluggy and iniconfig and could mask the class of undeclared dependency it exists to catch. Previously the Windows job was the only one installing without extras, so this class of break was caught by accident rather than by design.
…-tests-e7dc24 fix(ci): let the E2E proxy accept the mock testing params its suite sends
…-tests-3948e1 test(proxy): separate the member_add permission gate from the provisioning gate
…ata-4685a9 test(logging): pin routing_decision and internal_call_origin in the gcs pubsub spend log fixture
…-Router screens (#35500) PR #35471 added classifier_context_include_assistant_turns to ComplexityRouterConfig. It worked through config.yaml and the model API but had no control on the Add Model or Edit Auto-Router screens, so an operator working from the dashboard could not reach it. Wires it into the create and edit forms, shown only when the LLM classifier is selected, matching what #35315 did for the two context-window fields The create and edit stacks share the rendered control but keep their own serializer, their own hydration, and their own managed-key set, so the field is added in five places rather than one. A field wired into only one stack fails in a way neither serializer unit test can see, since those are handed a form value assembled by hand, so the edit-modal test drives the real component through open, edit and save The switch is emitted even when false, because there the operator turning it off is a choice that has to overwrite a stored true rather than an absent value a truthiness gate would drop
…745045 test(proxy): assert _delete_deployment's still-desired id set instead of a delete count
… rubric on the window it was given (#35504) Two changes to the classifier's system role, both narrowing it rather than adding to it classifier_tier_rubric let an operator replace the tier definitions. It shipped in #35471 alongside the assistant-turn context window, but the two answer different halves of the same report and only the context window was asked for. The override carried a composed prompt, an overridable and a non-overridable half, a blank-is-unset rule, a length-warning validator and a pair of dashboard controls. All of it goes The rubric then closes on one of two lines, chosen by classifier_context_window_size. At 0 no conversation is quoted, so the line is the original one, byte for byte: a deployment that sends no context is told to classify the current message and nothing else, which is what it could see all along. Above 0 the turns are quoted, and the original line told the model to disregard them, which is how a request whose difficulty was established in an earlier turn came back SIMPLE on the word "yes". There the line instead says to classify the current message using the quoted turns as context, and to rate what a short reply approves rather than the reply The choice keys on the window and not on classifier_context_include_assistant_turns. Whether the quoted turns are the user's alone or include the assistant's replies does not change what the model needs told, and whose turn is whose is already on the turns. Keying it on the assistant toggle would put the default deployment back on the original line, which is the configuration the report was raised against Folds in #35508, which built the window-dependent framing on top of the override this removes; that PR is closed in favour of this one
… budgets About 35,000 fixes ruff marks safe across 32 rules (UP006/UP045/UP007 modern annotations, UP032 f-strings, SIM114/SIM118, RET501, and friends), removal of the 1,296 typing imports the rewrite orphaned, and hand fixes for what the fixers could not see: five star-import freeloaders of typing names, two F823 late-import annotations, the /get/config/list introspection crash on types.UnionType, redundant function-local RoleMappings imports in ui_sso.py that shadowed the module-level name once the annotation lost its quotes, and one FURB168 tautology. B009/B010/PIE804/RUF019 are excluded on purpose: their safe fixes rewrite getattr/setattr/**-splat/key-in-dict escape hatches into forms basedpyright then rejects (283 new errors measured), so their budgets stay at base values. ruff-strict-budget.json drops by 39,579 this commit (39,968 across the branch) with 28 rules at an actual 0 and 9 more sharply down. type-discipline-budget.json ratchets LIT002/LIT006/LIT009 down; LIT001 moves to the now-honest total: the checker matches the spelling `set` but not the alias `Set`, so the 160 typing.Set annotations rewritten to set[...] were always mutable-set annotations and only now count.
…t-fail-4acb08 fix(deps): move pydantic-settings into the base dependencies
…o /ui/connect A keyless internal user signing in to the Admin UI was redirected off the post-login landing to /ui/connect, which renders nothing but the MCP apps panel, so a plain gateway sign-in ended on an MCP OAuth surface the user never asked for. The landing now renders the keys dashboard for every role. The key lookup that existed only to make that routing decision goes with it, along with the useKeys enabled flag it was the sole caller of and the role-hydration hold that guarded its one-frame dashboard flash The gateway DCR consent flow moves the other way. Its /authorize handed the browser to /ui/chat/integrations, whose layout hard-blocks when enable_chat_ui is off, which is the default, and client-side redirects to /ui/ without the query string; that destroys the connect_flow handle and strands the MCP client until the 600s flow cookie expires. It now lands on /ui/connect, which reads connect_flow and connect_client, mounts the consent banner and puts the apps panel in connect mode. /ui/chat/integrations keeps its connect-mode handling this release so flows sealed before the deploy still finish Resolves LIT-5104 Resolves LIT-4911
…itellm_up035_abc_imports # Conflicts: # litellm/llms/bedrock/base_aws_llm.py # litellm/proxy/proxy_cli.py # litellm/proxy/proxy_server.py # litellm/repositories/model_repository.py # litellm/router.py
refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets
build(makefile): swap npm ci for npm install so bootstrap no-ops on unchanged ui deps
…team_id}/callback (#35512) * fix(team-callbacks): report API-registered callbacks from GET /team/{team_id}/callback POST /team/{team_id}/callback writes metadata["logging"] while the GET read metadata["callback_settings"], so every team configured through the API or the Admin UI got back an empty list. c620d76 migrated the writer to the new key and left this reader on the old one. Resolve the read the same way request-time resolution does in _get_dynamic_logging_metadata: a logging slot that is present wins outright and callback_settings stays as the deprecated fallback, so the endpoint reports what a request would really do rather than the union of both shapes. An empty logging list therefore reports no callbacks, matching a request that fires none. Decrypt callback_vars for the response and mask the credential keys. Ciphertext would be unusable to the caller, and a value encrypted under a key that is no longer classified as sensitive would otherwise come back as a raw blob. Resolves LIT-5093 * Update litellm/proxy/management_endpoints/team_callback_endpoints.py Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(team-callbacks): mask callback vars that fail to decrypt decrypt_callback_vars passes a value through untouched when it cannot be decrypted, which happens to existing rows after a salt-key rotation. Under a key that is not classified as sensitive that blob reached the caller as opaque ciphertext it could not use or tell apart from a real value, so mask anything still carrying the encrypted prefix. Raised by Greptile on the first commit. --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
fix(ui): land general login on the keys dashboard, send MCP consent to /ui/connect
build(makefile): run bootstrap before pre-commit lint
chore: rebuild Admin UI bundle
A single read of key_info.spend races the batched spend writer: deltas earned before a reset flush to the DB up to ~60s later (proxy_batch_write_at) and land on the row after the reset zeroed it. The stage runs on Jul 30 and Aug 2 failed test_key_budget_reset_at_advances_after_window exactly this way, with spend back at the driven total while budget_reset_at had advanced and calls flowed again. Replace the single reads in rung 3 (spend zeroed after reset) and rung 4 (roomy window keeps spend) with _poll_key_spend, which re-reads to a 90s deadline covering one full flush-plus-reset cycle. A reset that never zeroes the row keeps spend pinned and still times out, so the regression guard keeps its teeth.
…LIT-5118 / LIT-5119 The strict-priority e2e (added with the zero-increment limiter fix) can never pass on stage: the proxy there does not run the dynamic_rate_limiter_v3 callbacks + priority_reservation settings the module requires, confirmed by zero limiter log lines across every gateway and backend pod during the 2026-08-02 run. Config lives in the infra repo; LIT-5118 tracks adding it. The throughput SLO test failed the same run with 65.9% of requests dying at the ELB as 502/503 before reaching a pod. The per-replica SLO rework fixed the RPS-floor assertion but cannot help when stage idles at one warm gateway replica; LIT-5119 tracks pre-scaling the fleet for the load phase. Both skips name their ticket, and the coverage registry returns the two cells to the gap list while they are in place.
…and_load_tests test(e2e): skip the strict-priority and throughput SLO tests pending LIT-5118 / LIT-5119
…to_deadline test(e2e): poll key spend to a deadline in budget reset advances tests
tin-berri
approved these changes
Aug 3, 2026
shin-berri
approved these changes
Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
How it solves it:
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes
QA runbook
Final Attestation