[Infra] Promote internal staging to main - #26375
Merged
Merged
Conversation
…transformation logic and tests
…ertex AI credentials When aimage_edit or image_edit was called with Vertex AI Gemini/Imagen models via YAML-style config (vertex_project / vertex_credentials in proxy YAML), the credentials were dropped during handler-to-config plumbing, causing fallback to Application Default Credentials and DefaultCredentialsError. Root cause: image_edit_handler and async_image_edit_handler did not pass litellm_params to validate_environment, unlike image_generation_handler. Fixes: 1. Widen BaseImageEditConfig.validate_environment signature to accept litellm_params and api_base (optional kwargs). 2. Forward dict(litellm_params) and litellm_params.api_base from both sync and async image_edit handlers to validate_environment. 3. Update VertexAIImagenImageEditConfig.validate_environment to read vertex_ai_project/vertex_ai_credentials from litellm_params first, matching Gemini config pattern (secondary latent bug fix). 4. Widen all image-edit config override signatures to match base. Made-with: Cursor
Adds three test cases to prevent regression of the Vertex AI image_edit credentials bug: 1. test_validate_environment_signature_includes_litellm_params: ensures all image-edit configs accept litellm_params (contract for the handler) 2. test_vertex_gemini_image_edit_reads_credentials_from_litellm_params: verifies Gemini config reads from litellm_params first 3. test_vertex_imagen_image_edit_reads_credentials_from_litellm_params: verifies Imagen config reads from litellm_params first These tests catch if the fix is accidentally reverted or if new image-edit configs are added without the litellm_params parameter. Made-with: Cursor
…Imagen get_complete_url VertexAIImagenImageEditConfig.get_complete_url was resolving vertex_project and vertex_location only from env vars and global settings, ignoring litellm_params. Users supplying project/location exclusively via YAML config would get a ValueError or wrong URL even after auth headers were fixed. Mirrors the pattern already used by VertexAIGeminiImageEditConfig and image_generation counterpart (safe_get_vertex_ai_project/location). Also fixes api_key type hint in MockImageEditConfig (str -> Optional[str]) and adds a test covering get_complete_url credential resolution. Made-with: Cursor
[Fix] Align image URL fetch with validated HTTP client in Bedrock and token counter paths
…strictions [Fix] Extend request body parameter restrictions to cloud provider auth fields
Adds a "Total Spend (USD)" column backed by the new membership.total_spend field. Cumulative across budget cycles; tracking began 2026-04-21.
Consolidate 6 distinct cache-key prefixes (v2-dependencies-,
v1-router-testing-deps-, v1-router-unit-deps-, v1-llm-translation-deps-,
v1-llm-responses-deps-, v3-litellm-uv-deps-, ui-e2e-py-deps-v2-) onto a
single v1-uv-cache-<uv.lock checksum> key shared across all Python jobs.
Cache only ~/.cache/uv (the content-addressed uv download cache,
hash-verified against uv.lock at install time). Drop ./.venv,
~/.local/{bin,lib}, and /home/circleci/.{pyenv,local} from cache paths.
~/.cache/uv is the only path uv sync needs to avoid re-downloading from
PyPI; everything else is rebuilt each run from that verified cache.
Remove partial-prefix restore-keys fallbacks — cache either hits exactly
on the uv.lock hash or rebuilds cleanly.
First run after merge will cold-miss on the new key; subsequent runs
hit the unified cache.
Brings the Snowflake, S3 Vectors, Vertex AI, and Bedrock URL construction paths in line with the existing pattern of validating interpolated values before use.
Adds a formatBudgetReset helper (dayjs-based, with validity guard) that renders the next reset as "today" / "in N days" / "on MMM D, YYYY". The team budget card now shows the team's reset timestamp and the member- default reset (when a shared team_member_budget is configured), and the Members tab gains a Budget Reset column per member.
…ation [Fix] Enforce format constraints on provider URL parameters
Adds back the per-cycle spend column that was replaced by Total Spend in 331e3f2. Current Cycle Spend reads membership.spend (zeroed on budget_reset_at) — this is the value enforced against the member's budget, so admins need it to see whether a member is approaching their cap for the active window. Total Spend remains for lifetime analytics.
Bring string input handling for image/mask parameters in line with the multipart-only contract expected by the image edit endpoint.
…itellm_team_member_total_spend_frontend
…dgets Two independent bugs both masked budget_reset_at from consumers that needed it: 1. /team/info.team_member_budget_table was typed as LiteLLM_BudgetTable (the user-settable allowlist), which dropped server-managed fields. Switched to LiteLLM_BudgetTableFull so budget_reset_at and created_at are serialized. 2. _clone_team_default_budget_for_member copied the pool's numeric fields but never set budget_reset_at on the cloned row. With budget_duration present but no reset timestamp, the reset job never fires on the member's budget (its query is reset_at <= now, which never matches NULL). Now computes budget_reset_at from the cloned budget_duration via get_budget_reset_time so each member's cycle starts at clone time rather than inheriting the pool's stale reset.
…itellm_team_member_total_spend_frontend
… prefix (#26078) (#26117) * fix(mcp_semantic_tool_filter): match canonical tools that arrive with a client-side namespace prefix. `SemanticMCPToolFilter._get_tools_by_names` matched by exact equality between the canonical name stored in the router (`<server><MCP_TOOL_PREFIX_SEPARATOR><tool>`) and the name in the incoming `tools[]` list. MCP clients such as opencode wrap every tool name with their own additive alias prefix (`<client_alias>_<canonical>`), so the two never matched, the filter dropped every tool to zero, and the proxy forwarded `tools: []` with `tool_choice: auto` — which strict upstream providers reject with a 400. The fix adds anchored suffix matching with a separator check: the canonical must form the complete tail of the incoming name and be preceded by `_` or `-`. Exact matches still win over suffix matches, incoming tools are returned at most once, and the original tool object is passed through unchanged so the client-facing name survives for tool-call round-trips. Seven unit tests in a new TestGetToolsByNames class cover exact match, underscore- and dash-prefixed variants, non-separator-anchored suffixes (which must not match), exact-wins-over-prefixed precedence, deduplication when two canonicals suffix-match the same incoming tool, and ordering-follows-router-output. Fixes #26078 * review: strengthen the suffix-fallback tie-breaker and the deduplication regression test (Greptile comments on #26117) - test_same_tool_not_returned_twice now passes two distinct canonicals ("read_file" and "file") that both suffix-match the same incoming tool, rather than the same canonical twice, so the assertion actually exercises the used_ids dedup path instead of the duplicate-input-list path. - The suffix fallback in _get_tools_by_names now prefers the shortest incoming name that still qualifies under the separator-anchored match. In the one-prefix-per-client opencode scenario this is a no-op, but in multi-namespace configurations the shortest qualifying name is the least-wrapped one and is the most defensible deterministic choice, replacing the dict-insertion-order fallback. - Adds test_suffix_fallback_prefers_shortest_candidate covering the new tie-breaker directly. Still 15 tests passing locally (was 14). * review(#26117): gate suffix-matching on canonical containing MCP_TOOL_PREFIX_SEPARATOR @krrish-berri-2 flagged a possible collision in the suffix fallback: a local user function whose name happens to end in a bare canonical substring (e.g. my_firecrawl_scrape vs canonical firecrawl_scrape) would be spuriously selected. Server-registered MCP tools are always emitted as <server_name><MCP_TOOL_PREFIX_SEPARATOR><tool_name> via add_server_prefix_to_name, so a canonical without the separator is not a namespaced MCP tool and does not warrant suffix matching. Added that guard to _name_matches_canonical with a regression test (test_does_not_collide_with_local_function_on_unprefixed_canonical) that reproduces the collision before the fix and is pinned after. Pre-existing TestGetToolsByNames fixtures that relied on bare canonicals (get_weather, search, read_file, write/delete/read) were switched to realistic server-prefixed ones so they continue to exercise the suffix-fallback path under the new guard. The opencode scenario (client prefix on already-server-prefixed canonical) is unchanged. --------- Co-authored-by: sakenuGOD <sakenuGOD@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
…26111) * fix(model-info): include reasoning effort support fields in get_model_info _get_model_info_helper constructs ModelInfoBase explicitly but never reads supports_xhigh/minimal/none_reasoning_effort from the cost map JSON. Add the three fields so get_model_info() returns them correctly. Also add supports_minimal_reasoning_effort to the ModelInfo TypedDict (xhigh and none were already declared, minimal was missing). * fix(model-registry): add missing reasoning effort fields for claude 4.6/4.7 Claude Opus 4.7 supports max reasoning effort (above xhigh). The field was present for Opus 4.6 but missing for all Opus 4.7 entries (base, dated, Bedrock, Vertex AI, Azure AI). All Claude 4.6/4.7 models (Opus 4.6, Sonnet 4.6, Opus 4.7) support minimal reasoning effort via adaptive thinking. Add the field to all provider variants. * fix(adapter): map output_config.effort to reasoning_effort (#25079) Anthropic's adaptive thinking (thinking.type="adaptive") and output_config.effort were silently dropped when translating to OpenAI format, resulting in no reasoning_effort on the outgoing request. Adapter changes (format translation): - adapters/transformation.py: add "adaptive" branch to translate_anthropic_thinking_to_reasoning_effort(); pass through output_config.effort as-is in _translate_thinking_to_openai(); add "output_config" to translatable_anthropic_params - adapters/handler.py: extract output_config from extra_kwargs into request_data so it reaches the translation layer - responses_adapters/transformation.py: add "adaptive" branch and output_config param to translate_thinking_to_reasoning() Handler changes (model-aware normalization): - utils.py: add normalize_reasoning_effort_value() that uses get_model_info() to map "max" → "xhigh"/"high" and "minimal" → "minimal"/"low" based on model capabilities - adapters/handler.py: call normalization before responses routing - responses_adapters/handler.py: call normalization after translation Relates to #25079 * test(reasoning-effort): add tests for effort capability fields and normalize logic Test coverage for: - get_model_info returning supports_minimal/max_reasoning_effort fields - JSON registry entries for claude 4.6/4.7 across all providers - normalize_reasoning_effort_value degradation chains and exception fallback - Adapter translation of adaptive thinking + output_config.effort * fix: forward custom_llm_provider to normalize_reasoning_effort_value in responses adapter
…scovery (#26228) `get_file_ids_from_messages` and `update_messages_with_model_file_ids` assume every content block with `type: "file"` has a nested `file` dict in the OpenAI Chat Completions shape. That assumption is too strong: `type: "file"` is a public content-block discriminator and several real producers emit blocks that use it without the OpenAI `file` sub-dict. For example, LangChain v1's `_normalize_messages` rewrites OpenAI file blocks into `{"type":"file","id":"...","base64":"...","mime_type":"...","extras":{}}` before they reach LiteLLM. `AnthropicConfig.validate_environment` calls both helpers unconditionally on every Anthropic (and Anthropic-via-Vertex) request, so any such block raises `KeyError: 'file'` which the Vertex partner layer then wraps as a `500 InternalServerError` before the LLM is even contacted. This patch switches both helpers from `c["file"]` to a defensive `c.get("file")` + dict check. When the block does not match the OpenAI shape there is no file_id to extract or remap, so we skip it and leave the block untouched for the downstream provider transformer to handle. Adds 5 regression tests covering the LangChain v1 shape, the OpenAI happy path, mixed shapes in one message, `file` set to a non-dict value, and the remap path for non-OpenAI blocks. Related to #24503, which proposed raising `BadRequestError` in the same spots. For these two discovery functions specifically, the skip semantics is strictly more permissive: well-formed OpenAI blocks still yield their file_id, and legitimate non-OpenAI blocks stop crashing the request.
…tive /v1/messages (#25883) When reasoning_auto_summary is enabled (via litellm_settings or env var), automatically set thinking.display="summarized" on native /v1/messages requests. This ensures thinking content is returned in the response instead of being omitted (the default on Claude 4.7+). Only applies when thinking is enabled (type != "disabled"). The existing reasoning_auto_summary flag already handles the /v1/responses path (summary="detailed") and the chat/completions adapter path — this extends coverage to the native messages handler.
* fix(ovhcloud): fix tool calling * fix import order
* fix(anthropic): handle tool_choice type 'none' in messages API * test(anthropic): add regression test for tool_choice type 'none' --------- Co-authored-by: BillionClaw <267901332+BillionClaw@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
…5788) When backend filters (e.g. Key Alias) are active on the Request Logs page, the manual Fetch button called logs.refetch() which re-runs the main TanStack Query. That query does not carry backend-only filter params such as key_alias, so the button had two problems: 1. It fired a redundant API request without the active filters. 2. It did not refresh the filtered result set — backendFilteredLogs stayed frozen at the last debounce-triggered fetch. Fix: expose refetchWithFilters() from useLogFilterLogic and route the Fetch button through it when hasBackendFilters is true. This cancels any in-flight debounce and calls performSearch with the current filter state, keeping all active filters intact. Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com> Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
… Logs (#25789) The useEffect that re-fetches logs on sort/page/time changes: useEffect(() => { if (hasBackendFilters && accessToken) { performSearch(filters, currentPage); } }, [sortBy, sortOrder, currentPage, startTime, endTime, isCustomDate]); intentionally omits `filters` and `hasBackendFilters` from its dep array to avoid double-fetches when a filter is applied. The side-effect is a stale-closure bug: the effect captures `filters` and `hasBackendFilters` from the render where its deps last changed, not from the render where the user selected, e.g., a Key Alias. Reproduce: set Key Alias → results appear correctly → change page or sort → the effect fires with the OLD `filters` snapshot (no key_alias) → API request is sent without the filter → table shows unfiltered data. Fix: store the latest `filters` and `hasBackendFilters` in refs that are kept in sync on every render. The sort/page/time effect reads from the refs instead of the closure so it always uses the current filter state without altering the dep array. Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com> Co-authored-by: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
The mocked async_increment_cache_pipeline is invoked from Router's deployment_callback_on_success, registered as an async success callback. Those callbacks are enqueued to GLOBAL_LOGGING_WORKER and run on a background task, so the mock may not have been called yet when the test asserts on it. Flush the worker before asserting.
Fix bugs that bypasses per-team member budget limit
The previous `while not self._queue.empty(): await self._queue.join()` pattern skipped the join entirely when the worker had already dequeued a task but not yet called task_done(). asyncio.Queue.join() tracks _unfinished_tasks (incremented by put, decremented by task_done), not queue depth, so it already handles that case on its own.
The create-branch job in create-release.yml calls the reusable
create-release-branch.yml workflow, which requires contents: write.
The top-level permissions: {} blocks the inherited default, and only
the release job overrode it, so the nested call failed with:
The nested job 'create-branch' is requesting 'contents: write',
but is only allowed 'contents: none'.
Add the permission at the calling job level so the reusable
workflow is granted what it needs.
[Fix] Tests - drain logging worker in test_router_caching_ttl to fix flakiness
Relative labels ("today", "in 2 days", "on May 12, 2026") mixed three
shapes in one column, breaking scannability. Always render MMM D, YYYY
for consistency and easier at-a-glance comparison across members.
…ion locations (#26281) Vertex multi-region endpoints (e.g. us, eu) use the rep host pattern, not {geo}-aiplatform.googleapis.com. Regional IDs still contain a hyphen. common_utils.get_vertex_base_url centralizes the rule for SDK/API URL building. Proxy pass-through duplicates the same branching in a local get_vertex_base_url (with trailing slashes) to avoid importing from common_utils there; live WebSocket passthrough uses the same multi-region host logic for wss://. Tests cover us/eu for the common_utils helper. Made-with: Cursor
[Fix] Infra: grant contents:write to create-release-branch caller job
[Fix] Deflake spend tracking tests
…is (#26162) (#26318) Temporary MCP OAuth sessions were kept in process-local memory, so on multi-instance/LB proxy deployments a session created on instance A could not be found when the follow-up /server/oauth/{server_id}/... request landed on instance B. Persist temporary session records to Redis (encrypted with the existing proxy encryption helpers) as a best-effort L2 cache alongside the current in-memory L1. Convert get_cached_temporary_mcp_server to async and await it from the authorize/token/register OAuth endpoints. Made-with: Cursor
[Fix] Reset budget windows failing due to Prisma Json? null filter
…itellm_team_member_total_spend_frontend
Out of scope for the members-tab feature and regressed legacy teams whose budget_reset_at is null (duration was previously shown as a fallback).
Members tab column reads this field; dropping it from the type in the previous revert broke the type check without affecting the reverted render logic.
…d_frontend Surface per-member budget cycle in Teams > Members tab
[Infra] Bump version 1.83.12 → 1.83.13
|
Michael Riad Zaky seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Contributor
Contributor
Low: Security improvements in a release promotionThis PR promotes internal staging to main. It contains numerous positive security changes: expanding the banned request-body parameter list (adding Status: 0 open Posted by Veria AI · 2026-04-24T00:49:07.615Z |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Contributor
Greptile SummaryThis is a staging-to-main promotion covering ~88 files across several features: team-member default budget fallback enforcement, Redis-backed temporary MCP OAuth session sharing, DashScope image generation provider, multi-region Vertex AI URL support, Gemini video-metadata fixes for pre-3 models, reasoning-effort normalization for Anthropic pass-through, GPT-5-chat model routing fixes, security hardening of banned request-body params, and a Confidence Score: 4/5Safe to merge after reviewing the OVHCloud param-forwarding behaviour change and the unguarded ValueError in the WebSocket passthrough. All findings are P2: the OVHCloud litellm/proxy/proxy_server.py (_reseed_spend_from_db pattern), litellm/llms/ovhcloud/chat/transformation.py (param forwarding change), litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py (WebSocket ValueError guard)
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/auth_checks.py | Adds team-level default member budget fallback: new get_team_member_default_budget helper with caching, and updated _check_team_member_budget to consult team.metadata["team_member_budget_id"] when no per-member budget exists. |
| litellm/proxy/proxy_server.py | Adds _reseed_spend_from_db which makes direct DB queries on spend-counter cold-start; bypasses established get_* helper functions, potentially violating project DB-access rules. |
| litellm/proxy/auth/auth_utils.py | Security improvement: adds aws_sts_endpoint, aws_web_identity_token, aws_role_name, and vertex_credentials to banned request-body params to prevent credential injection attacks. |
| litellm/llms/ovhcloud/chat/transformation.py | Removes get_supported_openai_params override; all OVHCloud models now pass tools/tool_choice/function_call to the provider regardless of model support — intentional but a backwards-incompatible behaviour change. |
| litellm/proxy/common_utils/reset_budget_job.py | Switches key/team budget-limits queries to raw SQL (IS NOT NULL) to work around prisma-client-python's inability to filter on Json? nullable columns; writes still use the ORM. |
| litellm/proxy/management_endpoints/mcp_management_endpoints.py | Adds Redis write-through cache for temporary MCP OAuth sessions (encrypted) so sessions are shared across proxy instances; get_cached_temporary_mcp_server made async to support Redis lookups. |
| litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py | Adds multi-region Vertex AI URL support and input validation; get_vertex_base_url now raises ValueError on None location, which can surface as an unhandled 500 in the WebSocket passthrough. |
| litellm/litellm_core_utils/logging_worker.py | Fixes flush() to use await self._queue.join() directly instead of the race-prone while not self._queue.empty(): await self._queue.join() pattern. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Incoming Request] --> B{Auth / Budget Check}
B --> C{Team member has budget table?}
C -- Yes --> D[Use member max_budget]
C -- No --> E{team.metadata has team_member_budget_id?}
E -- Yes --> F[get_team_member_default_budget DB lookup + cache]
F --> G[Use default max_budget]
E -- No --> H[No budget limit applied]
D --> I{budget exceeded?}
G --> I
I -- Yes --> J[Raise BudgetExceededError]
I -- No --> K[Route Request]
K --> L[Increment spend counters]
L --> M{Counter in cache?}
M -- No --> N[_reseed_spend_from_db direct DB cold-start read]
N --> O[Seed counter + increment]
M -- Yes --> P[Atomic increment in-memory + Redis]
Comments Outside Diff (2)
-
litellm/llms/ovhcloud/chat/transformation.py, line 21-28 (link)Removing tool-param filter is a backwards-incompatible behaviour change
Deleting
get_supported_openai_paramsmeanstools,tool_choice,function_call, andresponse_formatare now forwarded to OVHCloud for every model, including ones that don't support function calling. Previously, litellm surfaced a clear parameter-not-supported error; now those params silently pass through and the OVHCloud API returns the rejection. Callers that relied on litellm's up-front validation will now see provider-side errors instead, and any model routing logic that checkedget_supported_openai_paramsto decide whether to include tools will now behave differently. Ruleb48b7341advises guarding such changes with a feature flag.Rule Used: What: avoid backwards-incompatible changes without... (source)
-
litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py, line 2285-2292 (link)Nonevertex_location now raises instead of silently producing a bad URLget_vertex_base_urlnow raisesValueErrorwhenvertex_location is None. Invertex_ai_live_websocket_passthroughthe call is:host_location = resolved_location or vertex_llm_base.get_default_vertex_location() host = get_vertex_base_url(host_location).removeprefix("https://").rstrip("/")
If
get_default_vertex_location()returnsNone(no default configured) andresolved_locationis alsoNone, this becomes an unhandledValueErrorthat surfaces as a 500 to the WebSocket client instead of a clean 400/422. Consider guarding:if host_location is None: raise HTTPException(422, "vertex_location is required").
Reviews (1): Last reviewed commit: "Merge pull request #26370 from BerriAI/l..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+1911
to
+1973
| async def _reseed_spend_from_db(counter_key: str) -> float: | ||
| """ | ||
| Read the authoritative spend for a missing counter from the DB. The | ||
| counter_key prefix encodes the table to query: | ||
|
|
||
| spend:key:{token} -> LiteLLM_VerificationToken.spend | ||
| spend:team:{team_id} -> LiteLLM_TeamTable.spend | ||
| spend:team_member:{uid}:{tid} -> LiteLLM_TeamMembership.spend | ||
| spend:user:{user_id} -> LiteLLM_UserTable.spend | ||
| spend:org:{org_id} -> LiteLLM_OrganizationTable.spend | ||
|
|
||
| Returns 0.0 if prisma is unavailable, the row is missing, or the | ||
| key format is unrecognized. On failure, logs and returns 0.0 rather | ||
| than raising so the caller can still record the current increment. | ||
| """ | ||
| if prisma_client is None: | ||
| return 0.0 | ||
| # Per-window counters (spend:*:window:{duration}) share prefixes with | ||
| # primary counters but don't correspond to a DB row; their ambiguity | ||
| # would otherwise be silently parsed as a regular counter and miss. | ||
| if ":window:" in counter_key: | ||
| return 0.0 | ||
| try: | ||
| if counter_key.startswith("spend:key:"): | ||
| token = counter_key[len("spend:key:") :] | ||
| row = await prisma_client.db.litellm_verificationtoken.find_unique( | ||
| where={"token": token} | ||
| ) | ||
| elif counter_key.startswith("spend:team_member:"): | ||
| suffix = counter_key[len("spend:team_member:") :] | ||
| if ":" not in suffix: | ||
| return 0.0 | ||
| user_id, team_id = suffix.rsplit(":", 1) | ||
| row = await prisma_client.db.litellm_teammembership.find_unique( | ||
| where={"user_id_team_id": {"user_id": user_id, "team_id": team_id}} | ||
| ) | ||
| elif counter_key.startswith("spend:team:"): | ||
| team_id = counter_key[len("spend:team:") :] | ||
| row = await prisma_client.db.litellm_teamtable.find_unique( | ||
| where={"team_id": team_id} | ||
| ) | ||
| elif counter_key.startswith("spend:user:"): | ||
| user_id = counter_key[len("spend:user:") :] | ||
| row = await prisma_client.db.litellm_usertable.find_unique( | ||
| where={"user_id": user_id} | ||
| ) | ||
| elif counter_key.startswith("spend:org:"): | ||
| org_id = counter_key[len("spend:org:") :] | ||
| row = await prisma_client.db.litellm_organizationtable.find_unique( | ||
| where={"organization_id": org_id} | ||
| ) | ||
| else: | ||
| return 0.0 | ||
| except Exception: | ||
| verbose_proxy_logger.exception( | ||
| "Failed to reseed spend counter %s from DB", counter_key | ||
| ) | ||
| return 0.0 | ||
| if row is None: | ||
| return 0.0 | ||
| return float(getattr(row, "spend", 0.0) or 0.0) | ||
|
|
||
|
|
Contributor
There was a problem hiding this comment.
_reseed_spend_from_db calls prisma_client.db.<table>.find_unique(...) directly rather than through the established get_team/get_user/get_key helper functions. While this only fires on a cold-start (first access per counter key), it still introduces ad-hoc DB queries that bypass the shared caching/helper layer, which rules d7156c05 and 0c2a17ad exist to prevent. Using the existing cache-aware helpers (e.g. get_key_object, get_team_object) would give the same spend base value while keeping DB access patterns consistent.
Rule Used: What: In critical path of request, there should be... (source)
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| import httpx | ||
| from litellm.utils import ModelResponseStream, _get_model_info_helper | ||
| from litellm.utils import ModelResponseStream |
Comment on lines
+55
to
+58
| from litellm.proxy.common_utils.encrypt_decrypt_utils import ( | ||
| decrypt_value_helper, | ||
| encrypt_value_helper, | ||
| ) |
shin-berri
approved these changes
Apr 24, 2026
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
[Infra] Promote internal staging to main
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
18 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🚄 Infrastructure
Changes