merge mian - #27644
Conversation
…onds fields OVHCloud is deprecating two response fields on 2026-05-11: - reasoning_content replaced by reasoning (LLM reasoning models) - duration replaced by seconds (Speech-to-Text models) Adds backward-compatible support for both field names during the transition window, preferring the new field when present and falling back to the legacy field. Fixes #26586
Replaces falsy or with explicit is not None check so that a valid seconds=0.0 value is not silently dropped during field migration. Addresses Greptile review feedback on #26595
…ions for Responses API Fixes #25840 The OTel integration's set_attributes() method never populates gen_ai.output.messages, gen_ai.system_instructions, or gen_ai.response.finish_reasons for /v1/responses calls because ResponsesAPIResponse uses 'output' instead of 'choices' and the system prompt arrives as 'instructions' instead of 'system_instructions'. Changes: - Add elif branch for response_obj.get('output') to extract response text from Responses API output items (type='message'/output_text) and tool calls (type='function_call') - Coalesce system_instructions/instructions/system kwargs so the system prompt is captured for Responses API, Anthropic Messages API, and Vertex AI Gemini paths - Handle plain-string system prompts without unnecessary wrapping - Extract response_obj.get('status') as finish reason for Responses API - Add _transform_responses_api_output_to_otel() method
…uctions, and finish reasons Add 21 tests covering the new Responses API OTel attribute handling: TestOpenTelemetryResponsesAPI (13 tests): - gen_ai.output.messages from output items (text, function_call, mixed, multi-part) - gen_ai.response.finish_reasons from ResponsesAPIResponse.status - gen_ai.system_instructions from instructions/system/system_instructions kwargs - Precedence and absence edge cases - Regression test for existing choices-based responses TestTransformResponsesAPIOutput (8 tests): - Message with output_text, function_call items, unknown types - Edge cases: empty output, empty text, missing call_id, default role, non-dict items
Build the tool_call part dict separately with an explicit type annotation so mypy can track the type, avoiding the 'Unsupported target for indexed assignment' error on tool_call["parts"][0]["id"].
…r-tool-call attrs - Replace isinstance(item, dict) with hasattr(item, 'get') so Pydantic model instances (ResponseOutputMessage, ResponseFunctionToolCall) are accepted alongside plain dicts (P1) - Use 'is not None' guards instead of or-chain for system_instructions coalescing to prevent falsy values (e.g. []) falling through to the wrong kwarg (P2) - Emit per-tool-call span attributes (gen_ai.completion.N.function_call.*) for Responses API function_call items, matching the choices branch parity with _tool_calls_kv_pair (P2) - Add 4 new tests: Pydantic-like objects, falsy fallthrough guard, per-tool-call attribute emission, multiple tool call indexing
…onses Adds transform_response to OVHCloudChatConfig to normalise the new easoning field to easoning_content in non-streaming responses, matching the existing streaming fix in chunk_parser. Addresses maintainer feedback on #26595
The parent OpenAIGPTConfig already handles reasoning->reasoning_content for non-streaming via _extract_reasoning_content. The override was dead code giving false confidence. Streaming fix in chunk_parser is the only change needed for chat completions. Addresses Agent Shin review feedback on #26595
…y handles non-streaming via _extract_reasoning_content
…mation The openai SDK returns ResponseOutputMessage and ResponseOutputText as raw Pydantic v2 models that lack .get() (unlike LiteLLM's own wrapper objects). Add a _to_dict() helper that normalizes plain dicts, BaseLiteLLMOpenAIResponseObject (has .get()), and raw Pydantic models (has .model_dump()) into a consistent dict interface.
Iterate user/key/team/team_member/org/end_user/tag spend dicts in sorted order inside each Prisma transaction so concurrent pods acquire row locks in the same order, avoiding PostgreSQL deadlocks under load.
…nto OpenAPI schema
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
Squash-merged by litellm-agent from oss-pr-review-agent-shin[bot]'s PR.
…26325) Squash-merged by litellm-agent from milan-berri's PR.
Squash-merged by litellm-agent from Michael-RZ-Berri's PR.
Squash-merged by litellm-agent from noahnistler's PR.
…n /v1/messages (#27534) Squash-merged by litellm-agent from Anai-Guo's PR.
…27531) Squash-merged by litellm-agent from krisxia0506's PR.
…27521) Squash-merged by litellm-agent from Bytechoreographer's PR.
Squash-merged by litellm-agent from shivamrawat1's PR.
Squash-merged by litellm-agent from oss-agent-shin's PR.
…6_2026 [litellm-agent] Staging → litellm_internal_staging (5/6/2026)
Reject fnmatch wildcards on non-scope claims when the claim string contains whitespace so malformed iss values cannot match patterns like trusted.*. Merge every entry when team_id_jwt_field resolves to a list instead of keeping only the first element. Co-authored-by: Cursor <cursoragent@cursor.com>
[litellm-agent] Staging → litellm_internal_staging (5/7/2026)
[litellm-agent] Staging → litellm_internal_staging (5/9/2026)
cb7f60e
into
litellm_anthropic_dummy_tool_default
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Replace removes all occurrences of "responses/" not just prefix
- Changed both router alias registrations to only strip a leading responses/ prefix with startswith and removeprefix.
- ✅ Fixed: Duplicate AZURE_REDIS_SCOPE constant defined in two modules
- Reused AZURE_REDIS_SCOPE from the credential provider in _redis.py and removed the duplicate definition.
Preview (aa587bd9d3)
diff --git a/litellm/_redis.py b/litellm/_redis.py
--- a/litellm/_redis.py
+++ b/litellm/_redis.py
@@ -19,6 +19,7 @@
from litellm import get_secret, get_secret_str
from litellm._redis_credential_provider import (
+ AZURE_REDIS_SCOPE,
AzureADCredentialProvider,
GCPIAMCredentialProvider,
_generate_gcp_iam_access_token,
@@ -28,9 +29,7 @@
from ._logging import verbose_logger
-AZURE_REDIS_SCOPE = "https://redis.azure.com/.default"
-
def _get_redis_kwargs():
arg_spec = inspect.getfullargspec(redis.Redis)
diff --git a/litellm/router.py b/litellm/router.py
--- a/litellm/router.py
+++ b/litellm/router.py
@@ -7077,8 +7077,8 @@
k: v for k, v in _model_info.items() if k not in _custom_pricing_fields
}
_backend_alias_cost = {_model_name: _shared_model_info}
- if "responses/" in _model_name:
- _stripped_model_name = _model_name.replace("responses/", "")
+ if _model_name.startswith("responses/"):
+ _stripped_model_name = _model_name.removeprefix("responses/")
_backend_alias_cost[_stripped_model_name] = _shared_model_info
litellm.register_model(model_cost=_backend_alias_cost)
@@ -7785,8 +7785,8 @@
k: v for k, v in _model_info_dict.items() if k not in _custom_pricing_fields
}
_backend_alias_cost = {_model_name: _shared_model_info}
- if "responses/" in _model_name:
- _stripped_model_name = _model_name.replace("responses/", "")
+ if _model_name.startswith("responses/"):
+ _stripped_model_name = _model_name.removeprefix("responses/")
_backend_alias_cost[_stripped_model_name] = _shared_model_info
litellm.register_model(model_cost=_backend_alias_cost)You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit aa587bd. Configure here.
| ) | ||
| _backend_alias_cost = {_model_name: _shared_model_info} | ||
| if "responses/" in _model_name: | ||
| _stripped_model_name = _model_name.replace("responses/", "") |
There was a problem hiding this comment.
Replace removes all occurrences of "responses/" not just prefix
Low Severity
str.replace("responses/", "") removes all occurrences of "responses/" anywhere in the model name, not just a leading prefix. A model name like "responses/responses/model" would collapse to "model" instead of "responses/model". Additionally, the "responses/" in _model_name check triggers even when it appears mid-string (e.g. hypothetical "some/responses/gpt-4o"), which would produce a malformed stripped name. Using removeprefix("responses/") (or checking startswith) would be safer and match the apparent intent.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit aa587bd. Configure here.
|
|
||
| from ._logging import verbose_logger | ||
|
|
||
| AZURE_REDIS_SCOPE = "https://redis.azure.com/.default" |
There was a problem hiding this comment.
Duplicate AZURE_REDIS_SCOPE constant defined in two modules
Low Severity
AZURE_REDIS_SCOPE is defined identically in both litellm/_redis.py and litellm/_redis_credential_provider.py. If the scope string ever needs updating, one definition could easily be missed, leading to inconsistent authentication behavior. One module could import the constant from the other.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit aa587bd. Configure here.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis is a large multi-feature merge bringing Azure AD Redis auth, a new
Confidence Score: 4/5The PR is safe to merge; the changes are well-tested and address real issues with no functional regressions identified. The deadlock-prevention sort, guardrail reconciliation, and Azure AD Redis auth all look correct. The one inline finding (duplicate constant) is a maintenance concern with no runtime impact. The
|
| Filename | Overview |
|---|---|
| litellm/_redis.py | Adds Azure AD Redis auth via credential provider and connection factory; AZURE_REDIS_SCOPE is duplicated from _redis_credential_provider.py. |
| litellm/_redis_credential_provider.py | New AzureADCredentialProvider mirrors GCPIAMCredentialProvider; get_credentials_async correctly delegates sync get_token to a thread. |
| litellm/proxy/management_endpoints/key_management_endpoints.py | New bulk_update_team_keys endpoint with proper auth anchoring, batch-size cap, and KeyUpdateFields allowlist; _process_single_key_update refactored to accept a pre-fetched key row. |
| litellm/proxy/auth/user_api_key_auth.py | Routing override matcher gains scope/wildcard support; whitespace anti-injection guard prevents multi-value iss claims from matching wildcards. |
| litellm/proxy/auth/handle_jwt.py | New get_all_jwt_team_ids reads both plural and singular team-id claims and deduplicates; used by SSO callback to fix silent single-team drops on Okta/Auth0. |
| litellm/proxy/guardrails/guardrail_registry.py | Adds provenance tracking (_sources) and reconcile_db_guardrails to purge stale DB-backed guardrails; all CRUD paths update _sources correctly. |
| litellm/proxy/db/db_spend_update_writer.py | All spend-update loops now sort by entity ID before batch processing to enforce consistent lock ordering across pods and prevent deadlocks. |
| litellm/proxy/_experimental/mcp_server/oauth_utils.py | get_request_base_url moved here from discoverable_endpoints.py; new validate_trusted_redirect_uri accepts same-origin OR loopback, with safe fallback to loopback-only when proxy origin can't be determined. |
| litellm/integrations/opentelemetry.py | Responses API output items now emit GenAI OTEL spans; system prompt coalesced from system_instructions/instructions/system kwargs. |
| litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py | New _filter_context_management_for_bedrock_invoke strips unsupported context_management edit types before the final field allowlist, fixing Bedrock 400s from Claude Code's clear_thinking edits. |
| litellm/proxy/_types.py | Adds TEAM_KEY_BULK_UPDATE route enum, scope field to JWTRoutingOverride, and extends internal_user_view_only_routes with compliance routes. |
| litellm/types/proxy/management_endpoints/key_management_endpoints.py | New KeyUpdateFields (with extra="forbid" allowlist) and BulkUpdateTeamKeysRequest types with proper mutual-exclusivity validation. |
Reviews (1): Last reviewed commit: "Merge pull request #27549 from BerriAI/s..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| from ._logging import verbose_logger | ||
|
|
||
| AZURE_REDIS_SCOPE = "https://redis.azure.com/.default" |
There was a problem hiding this comment.
AZURE_REDIS_SCOPE is defined independently here and also in _redis_credential_provider.py with the same value. Since _redis.py already imports from _redis_credential_provider, re-exporting the constant from there keeps the two files from drifting apart if the scope URL ever changes.
| AZURE_REDIS_SCOPE = "https://redis.azure.com/.default" | |
| from litellm._redis_credential_provider import AZURE_REDIS_SCOPE as AZURE_REDIS_SCOPE # re-export |
|
Michael Riad Zaky seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
16 participants
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes
Note
High Risk
High risk because it modifies authentication/authorization flows (JWT routing selectors, MCP OAuth redirect validation, new key-management endpoint) and changes Redis credential handling/token refresh paths which could impact connectivity and security.
Overview
Adds Azure AD authentication for Redis alongside existing GCP IAM support, including sync
redis_connect_funcand asynccredential_providerpaths to avoid token-expiry issues in pools/clusters.Enhances OpenTelemetry to support OpenAI Responses API (
output/status) and broader system-prompt kwarg handling (system_instructions/instructions/system), with new normalization helpers and extensive tests.Extends proxy management/auth with a new
POST /team/key/bulk_updateendpoint (strict allowlist of updatable fields), improved JWT team-claim reconciliation (get_all_jwt_team_ids), JWT routing overrides supportingscopewith wildcard matching, and MCP OAuth redirect validation widened to same-origin or loopback (logic moved tooauth_utils).Includes several operational fixes: Bedrock Invoke now filters/preserves supported
context_managementcompact edits (and injects beta), OVHCloud response field migrations are normalized, health-check caching waits on Redis locks instead of duplicating work, spend-update writers sort batch updates to reduce deadlocks, guardrails track provenance (dbvsconfig) and reconcile stale DB entries, and proxy forwarded headers are coerced to validstr/bytesvalues.Reviewed by Cursor Bugbot for commit aa587bd. Bugbot is set up for automated code reviews on this repo. Configure here.