[litellm-agent] Staging → litellm_internal_staging (5/6/2026) - #27256
Conversation
…onds fields OVHCloud is deprecating two response fields on 2026-05-11: - reasoning_content replaced by reasoning (LLM reasoning models) - duration replaced by seconds (Speech-to-Text models) Adds backward-compatible support for both field names during the transition window, preferring the new field when present and falling back to the legacy field. Fixes #26586
Replaces falsy or with explicit is not None check so that a valid seconds=0.0 value is not silently dropped during field migration. Addresses Greptile review feedback on #26595
…ions for Responses API Fixes #25840 The OTel integration's set_attributes() method never populates gen_ai.output.messages, gen_ai.system_instructions, or gen_ai.response.finish_reasons for /v1/responses calls because ResponsesAPIResponse uses 'output' instead of 'choices' and the system prompt arrives as 'instructions' instead of 'system_instructions'. Changes: - Add elif branch for response_obj.get('output') to extract response text from Responses API output items (type='message'/output_text) and tool calls (type='function_call') - Coalesce system_instructions/instructions/system kwargs so the system prompt is captured for Responses API, Anthropic Messages API, and Vertex AI Gemini paths - Handle plain-string system prompts without unnecessary wrapping - Extract response_obj.get('status') as finish reason for Responses API - Add _transform_responses_api_output_to_otel() method
…uctions, and finish reasons Add 21 tests covering the new Responses API OTel attribute handling: TestOpenTelemetryResponsesAPI (13 tests): - gen_ai.output.messages from output items (text, function_call, mixed, multi-part) - gen_ai.response.finish_reasons from ResponsesAPIResponse.status - gen_ai.system_instructions from instructions/system/system_instructions kwargs - Precedence and absence edge cases - Regression test for existing choices-based responses TestTransformResponsesAPIOutput (8 tests): - Message with output_text, function_call items, unknown types - Edge cases: empty output, empty text, missing call_id, default role, non-dict items
Build the tool_call part dict separately with an explicit type annotation so mypy can track the type, avoiding the 'Unsupported target for indexed assignment' error on tool_call["parts"][0]["id"].
…r-tool-call attrs - Replace isinstance(item, dict) with hasattr(item, 'get') so Pydantic model instances (ResponseOutputMessage, ResponseFunctionToolCall) are accepted alongside plain dicts (P1) - Use 'is not None' guards instead of or-chain for system_instructions coalescing to prevent falsy values (e.g. []) falling through to the wrong kwarg (P2) - Emit per-tool-call span attributes (gen_ai.completion.N.function_call.*) for Responses API function_call items, matching the choices branch parity with _tool_calls_kv_pair (P2) - Add 4 new tests: Pydantic-like objects, falsy fallthrough guard, per-tool-call attribute emission, multiple tool call indexing
…onses Adds transform_response to OVHCloudChatConfig to normalise the new easoning field to easoning_content in non-streaming responses, matching the existing streaming fix in chunk_parser. Addresses maintainer feedback on #26595
The parent OpenAIGPTConfig already handles reasoning->reasoning_content for non-streaming via _extract_reasoning_content. The override was dead code giving false confidence. Streaming fix in chunk_parser is the only change needed for chat completions. Addresses Agent Shin review feedback on #26595
…y handles non-streaming via _extract_reasoning_content
…mation The openai SDK returns ResponseOutputMessage and ResponseOutputText as raw Pydantic v2 models that lack .get() (unlike LiteLLM's own wrapper objects). Add a _to_dict() helper that normalizes plain dicts, BaseLiteLLMOpenAIResponseObject (has .get()), and raw Pydantic models (has .model_dump()) into a consistent dict interface.
Iterate user/key/team/team_member/org/end_user/tag spend dicts in sorted order inside each Prisma transaction so concurrent pods acquire row locks in the same order, avoiding PostgreSQL deadlocks under load.
…nto OpenAPI schema
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
|
Michael Riad Zaky seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Greptile SummaryThis automated staging PR bundles five distinct improvements: deadlock prevention in DB spend updates (sorted lock ordering across pods), an OpenAPI schema fix so WebSocket stubs no longer overwrite HTTP operations on shared paths, OTel Responses API support (output messages, tool calls, status-based finish_reasons, system_instructions coalescing), OVHCloud field migration shims for the 2026-05-11 API transition, and compliance route access for non-admin roles.
Confidence Score: 5/5Safe to merge — all changes are additive or narrowly scoped bug fixes with targeted test coverage. Every change is either a clear correctness fix (deadlock ordering, OpenAPI stub injection) or a non-breaking addition (OTel Responses API, OVHCloud migration shims, compliance route access). The deadlock fix is mechanical and consistent across all six spend buckets. The OpenAPI refactor adds a guard that only skips the stub when a real operation already exists, which is strictly safer than before. No auth logic is altered, no DB schema is changed, and no backwards-incompatible interfaces are introduced. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/integrations/opentelemetry.py | Adds OTel Responses API support: system_instructions coalescing (system_instructions/instructions/system kwargs), output message transformation, tool-call span attributes, and status-based finish_reasons. Logic is well-guarded and new helper methods are cleanly separated. |
| litellm/proxy/db/db_spend_update_writer.py | Adds sorted() around all spend-bucket iteration loops to enforce deterministic lock-acquisition order across pods and prevent PostgreSQL deadlocks. Change is consistent across all six transaction buckets (user, key, team, team_member, org, entity). |
| litellm/proxy/proxy_server.py | Extracts WebSocket stub injection into _inject_websocket_stubs_into_openapi_schema(); now uses setdefault + 'get' not in path_entry guard so HTTP operations on shared paths (e.g. POST /v1/responses) are never overwritten by a synthetic WebSocket GET stub. |
| litellm/proxy/_types.py | Adds compliance_check_routes (['/compliance/eu-ai-act', '/compliance/gdpr']) to internal_user_routes and internal_user_view_only_routes so non-admin roles can reach stateless compliance validators. |
| litellm/proxy/utils.py | Applies the same sorted() deadlock-prevention fix to ProxyUpdateSpend.update_end_user_spend, consistent with the db_spend_update_writer.py changes. |
| litellm/llms/ovhcloud/audio_transcription/transformation.py | Handles OVHCloud field migration: prefers seconds over duration (using is not None to correctly pass through 0.0), normalises to duration key for downstream consumers through the transition deadline (2026-05-11). |
| litellm/llms/ovhcloud/chat/transformation.py | Handles OVHCloud reasoning field migration: maps new reasoning field to reasoning_content only when reasoning_content is absent, preserving legacy behavior during transition window. |
Reviews (2): Last reviewed commit: "Merge PR #26670 into agent staging branc..." | Re-trigger Greptile
| # from __future__ import annotations must be the first non-comment statement | ||
| from __future__ import annotations |
There was a problem hiding this comment.
The comment is now inaccurate. The module docstring added by this PR is itself an expression statement (an
Expr node in Python's AST), so from __future__ import annotations is no longer the first non-comment statement. PEP 236 explicitly permits module docstrings before future imports, so behaviour is unchanged — but the inline explanation is misleading for anyone reading it in future.
| # from __future__ import annotations must be the first non-comment statement | |
| from __future__ import annotations | |
| # from __future__ import annotations must appear before any other imports/code | |
| # (module docstring above is permitted by PEP 236) | |
| from __future__ import annotations |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
@veria-labs please review |
|
@greptile please re-review |
…ng_05_06_2026 [litellm-agent] Staging → litellm_internal_staging (5/6/2026)
Automated staging PR created by litellm-agent.
This branch collects PRs approved by the agent on 5/6/2026.