Skip to content

[litellm-agent] Staging → litellm_internal_staging (5/6/2026) - #27256

Merged
Sameerlite merged 28 commits into
litellm_internal_stagingfrom
litellm_agent_oss_staging_05_06_2026
May 11, 2026
Merged

[litellm-agent] Staging → litellm_internal_staging (5/6/2026)#27256
Sameerlite merged 28 commits into
litellm_internal_stagingfrom
litellm_agent_oss_staging_05_06_2026

Conversation

@oss-pr-review-agent-shin

Copy link
Copy Markdown
Contributor

Automated staging PR created by litellm-agent.

This branch collects PRs approved by the agent on 5/6/2026.

KunalG67 and others added 21 commits April 27, 2026 17:15
…onds fields

OVHCloud is deprecating two response fields on 2026-05-11:
- reasoning_content replaced by reasoning (LLM reasoning models)
- duration replaced by seconds (Speech-to-Text models)

Adds backward-compatible support for both field names during the
transition window, preferring the new field when present and falling
back to the legacy field.

Fixes #26586
Replaces falsy or with explicit is not None check so that a valid
seconds=0.0 value is not silently dropped during field migration.

Addresses Greptile review feedback on #26595
…ions for Responses API

Fixes #25840

The OTel integration's set_attributes() method never populates
gen_ai.output.messages, gen_ai.system_instructions, or
gen_ai.response.finish_reasons for /v1/responses calls because
ResponsesAPIResponse uses 'output' instead of 'choices' and the
system prompt arrives as 'instructions' instead of 'system_instructions'.

Changes:
- Add elif branch for response_obj.get('output') to extract response
  text from Responses API output items (type='message'/output_text)
  and tool calls (type='function_call')
- Coalesce system_instructions/instructions/system kwargs so the
  system prompt is captured for Responses API, Anthropic Messages
  API, and Vertex AI Gemini paths
- Handle plain-string system prompts without unnecessary wrapping
- Extract response_obj.get('status') as finish reason for Responses API
- Add _transform_responses_api_output_to_otel() method
…uctions, and finish reasons

Add 21 tests covering the new Responses API OTel attribute handling:

TestOpenTelemetryResponsesAPI (13 tests):
- gen_ai.output.messages from output items (text, function_call, mixed, multi-part)
- gen_ai.response.finish_reasons from ResponsesAPIResponse.status
- gen_ai.system_instructions from instructions/system/system_instructions kwargs
- Precedence and absence edge cases
- Regression test for existing choices-based responses

TestTransformResponsesAPIOutput (8 tests):
- Message with output_text, function_call items, unknown types
- Edge cases: empty output, empty text, missing call_id, default role, non-dict items
Build the tool_call part dict separately with an explicit type
annotation so mypy can track the type, avoiding the
'Unsupported target for indexed assignment' error on
tool_call["parts"][0]["id"].
…r-tool-call attrs

- Replace isinstance(item, dict) with hasattr(item, 'get') so Pydantic
  model instances (ResponseOutputMessage, ResponseFunctionToolCall) are
  accepted alongside plain dicts (P1)
- Use 'is not None' guards instead of or-chain for system_instructions
  coalescing to prevent falsy values (e.g. []) falling through to the
  wrong kwarg (P2)
- Emit per-tool-call span attributes (gen_ai.completion.N.function_call.*)
  for Responses API function_call items, matching the choices branch
  parity with _tool_calls_kv_pair (P2)
- Add 4 new tests: Pydantic-like objects, falsy fallthrough guard,
  per-tool-call attribute emission, multiple tool call indexing
…onses

Adds transform_response to OVHCloudChatConfig to normalise the new

easoning field to 
easoning_content in non-streaming responses,
matching the existing streaming fix in chunk_parser.

Addresses maintainer feedback on #26595
The parent OpenAIGPTConfig already handles reasoning->reasoning_content
for non-streaming via _extract_reasoning_content. The override was dead
code giving false confidence. Streaming fix in chunk_parser is the only
change needed for chat completions.

Addresses Agent Shin review feedback on #26595
…y handles non-streaming via _extract_reasoning_content
…mation

The openai SDK returns ResponseOutputMessage and ResponseOutputText as
raw Pydantic v2 models that lack .get() (unlike LiteLLM's own wrapper
objects). Add a _to_dict() helper that normalizes plain dicts,
BaseLiteLLMOpenAIResponseObject (has .get()), and raw Pydantic models
(has .model_dump()) into a consistent dict interface.
Iterate user/key/team/team_member/org/end_user/tag spend dicts in sorted
order inside each Prisma transaction so concurrent pods acquire row locks
in the same order, avoiding PostgreSQL deadlocks under load.
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
@CLAassistant

CLAassistant commented May 6, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
4 out of 9 committers have signed the CLA.

✅ KunalG67
✅ pnookala-godaddy
✅ michelligabriele
✅ Sameerlite
❌ Aneesh-Fiddler
❌ Michael Riad Zaky
❌ cursoragent
❌ oss-pr-review-agent-shin[bot]
❌ oss-agent-shin


Michael Riad Zaky seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This automated staging PR bundles five distinct improvements: deadlock prevention in DB spend updates (sorted lock ordering across pods), an OpenAPI schema fix so WebSocket stubs no longer overwrite HTTP operations on shared paths, OTel Responses API support (output messages, tool calls, status-based finish_reasons, system_instructions coalescing), OVHCloud field migration shims for the 2026-05-11 API transition, and compliance route access for non-admin roles.

  • Deadlock prevention (db_spend_update_writer.py, proxy/utils.py): all six spend-bucket loops now iterate in sorted() order so concurrent pods acquire row-level locks in a consistent sequence. A comprehensive parametrized test suite verifies the ordering across all buckets.
  • OpenAPI schema fix (proxy_server.py): _inject_websocket_stubs_into_openapi_schema now uses setdefault + a \"get\" not in path_entry guard, preventing the synthetic WebSocket GET stub from silently dropping a real POST (e.g. /v1/responses) from Swagger UI.
  • OTel + OVHCloud (opentelemetry.py, ovhcloud/): Responses API output is now surfaced as OTEL GenAI 1.38 span attributes; OVHCloud secondsduration and reasoningreasoning_content normalisations handle falsy-zero edge cases correctly with is not None guards.

Confidence Score: 5/5

Safe to merge — all changes are additive or narrowly scoped bug fixes with targeted test coverage.

Every change is either a clear correctness fix (deadlock ordering, OpenAPI stub injection) or a non-breaking addition (OTel Responses API, OVHCloud migration shims, compliance route access). The deadlock fix is mechanical and consistent across all six spend buckets. The OpenAPI refactor adds a guard that only skips the stub when a real operation already exists, which is strictly safer than before. No auth logic is altered, no DB schema is changed, and no backwards-incompatible interfaces are introduced.

No files require special attention.

Important Files Changed

Filename Overview
litellm/integrations/opentelemetry.py Adds OTel Responses API support: system_instructions coalescing (system_instructions/instructions/system kwargs), output message transformation, tool-call span attributes, and status-based finish_reasons. Logic is well-guarded and new helper methods are cleanly separated.
litellm/proxy/db/db_spend_update_writer.py Adds sorted() around all spend-bucket iteration loops to enforce deterministic lock-acquisition order across pods and prevent PostgreSQL deadlocks. Change is consistent across all six transaction buckets (user, key, team, team_member, org, entity).
litellm/proxy/proxy_server.py Extracts WebSocket stub injection into _inject_websocket_stubs_into_openapi_schema(); now uses setdefault + 'get' not in path_entry guard so HTTP operations on shared paths (e.g. POST /v1/responses) are never overwritten by a synthetic WebSocket GET stub.
litellm/proxy/_types.py Adds compliance_check_routes (['/compliance/eu-ai-act', '/compliance/gdpr']) to internal_user_routes and internal_user_view_only_routes so non-admin roles can reach stateless compliance validators.
litellm/proxy/utils.py Applies the same sorted() deadlock-prevention fix to ProxyUpdateSpend.update_end_user_spend, consistent with the db_spend_update_writer.py changes.
litellm/llms/ovhcloud/audio_transcription/transformation.py Handles OVHCloud field migration: prefers seconds over duration (using is not None to correctly pass through 0.0), normalises to duration key for downstream consumers through the transition deadline (2026-05-11).
litellm/llms/ovhcloud/chat/transformation.py Handles OVHCloud reasoning field migration: maps new reasoning field to reasoning_content only when reasoning_content is absent, preserving legacy behavior during transition window.

Reviews (2): Last reviewed commit: "Merge PR #26670 into agent staging branc..." | Re-trigger Greptile

Comment thread litellm/utils.py
Comment on lines 3 to 4
# from __future__ import annotations must be the first non-comment statement
from __future__ import annotations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The comment is now inaccurate. The module docstring added by this PR is itself an expression statement (an Expr node in Python's AST), so from __future__ import annotations is no longer the first non-comment statement. PEP 236 explicitly permits module docstrings before future imports, so behaviour is unchanged — but the inline explanation is misleading for anyone reading it in future.

Suggested change
# from __future__ import annotations must be the first non-comment statement
from __future__ import annotations
# from __future__ import annotations must appear before any other imports/code
# (module docstring above is permitted by PEP 236)
from __future__ import annotations

@codecov

codecov Bot commented May 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.26027% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/integrations/opentelemetry.py 96.49% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@krrish-berri-2
krrish-berri-2 requested a review from Sameerlite May 6, 2026 01:57
@krrish-berri-2

krrish-berri-2 commented May 6, 2026

Copy link
Copy Markdown
Contributor

@veria-labs please review

@krrish-berri-2

Copy link
Copy Markdown
Contributor

@greptile please re-review

@Sameerlite
Sameerlite merged commit c7e6bcc into litellm_internal_staging May 11, 2026
114 checks passed
@Sameerlite
Sameerlite deleted the litellm_agent_oss_staging_05_06_2026 branch May 11, 2026 05:31
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…ng_05_06_2026

[litellm-agent] Staging → litellm_internal_staging (5/6/2026)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants