fix(otel): populate gen_ai.output.messages and gen_ai.system_instructions for Responses API

[aneeshsangvikar]

Summary

Fixes #25840

The OTel integration's set_attributes() method never populates gen_ai.output.messages, gen_ai.system_instructions, or gen_ai.response.finish_reasons for /v1/responses (Responses API) calls. Token counts, costs, and input messages all work correctly — only the output content, system prompt, and finish reasons are missing.

Root Cause

• gen_ai.output.messages: The code only checks response_obj.get("choices"), but ResponsesAPIResponse uses output (a list of output items with type="message" / type="output_text") instead of choices. So response_obj.get("choices") returns None and the entire block is skipped.
• gen_ai.system_instructions: The code only checks kwargs.get("system_instructions") (Vertex AI Gemini path), but the Responses API passes the system prompt as kwargs["instructions"], and the Anthropic Messages API uses kwargs["system"].
• gen_ai.response.finish_reasons: Derived from choices[].finish_reason, which doesn't exist for Responses API. The equivalent is response_obj.get("status") (e.g. "completed").

Changes

1. Output messages (set_attributes())

Added an elif response_obj.get("output") branch after the existing choices check. A new method _transform_responses_api_output_to_otel() converts the Responses API output structure to the same {"role": ..., "parts": [...]} format used by chat completions. Handles both type="message" items (text output) and type="function_call" items (tool calls).

2. System instructions (set_attributes())

Coalesced three kwarg names that carry the same semantic data:

Call path	Kwarg name	Previously captured?
Vertex AI Gemini chat-completion	system_instructions	Yes
OpenAI Responses API	instructions	No
Anthropic Messages API	system	No

Also handles plain strings (common for Responses API instructions) without wrapping them in a message array.

3. Finish reasons (set_attributes())

For Responses API, extracts response_obj.get("status") (e.g. "completed") as the finish reason, since there are no choices to pull finish_reason from.

Testing

21 new unit tests added in tests/test_litellm/integrations/test_opentelemetry.py:

TestOpenTelemetryResponsesAPI (13 tests):

• gen_ai.output.messages from output items (text, function_call, mixed, multi-part, empty text)
• gen_ai.response.finish_reasons from ResponsesAPIResponse.status
• gen_ai.system_instructions from instructions / system / system_instructions kwargs
• Precedence and absence edge cases
• Regression test confirming existing choices-based responses still work

TestTransformResponsesAPIOutput (8 tests):

• Message with output_text, function_call items, unknown types
• Edge cases: empty output, empty text, missing call_id, default role, non-dict items

All 122 tests pass (101 existing + 21 new).

Impact

Any downstream OTel consumer (Datadog, Honeycomb, Fiddler, Phoenix, custom pipelines) that relies on gen_ai.output.messages will now receive response content for all /v1/responses calls. Previously these appeared as empty output in observability dashboards even though the LLM call succeeded and tokens were counted correctly.

The gen_ai.system_instructions fix also benefits the Anthropic Messages API pass-through endpoint where the system prompt was silently dropped.

Existing /chat/completions paths are unaffected — the new code is entirely in elif branches.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ yuneng-berri
✅ mateo-berri
❌ Aneesh-Fiddler
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing aneeshsangvikar:fix/otel-responses-api-clean (9928618) with main (3d2b8fe)}

[codecov]

Codecov Report

❌ Patch coverage is 96.49123% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/opentelemetry.py	96.49%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR fixes missing OTel span attributes (gen_ai.output.messages, gen_ai.system_instructions, gen_ai.response.finish_reasons) for /v1/responses (Responses API) calls by adding an elif response_obj.get("output") branch alongside the existing choices path, and broadening the system-prompt coalescing to include instructions (Responses API) and system (Anthropic) kwargs. All four issues flagged in the previous review round (falsy system_instructions fallthrough, missing per-tool-call attributes, isinstance(dict) guard blocking Pydantic objects, duplicate finish-reasons block) appear to have been addressed in the final code.

Confidence Score: 5/5

Safe to merge — changes are additive elif branches and a new private method; existing choices path is untouched.

No P0 or P1 findings. All four issues flagged in the prior review round (falsy system_instructions fallthrough, missing per-tool-call attributes, isinstance(dict) Pydantic guard, duplicate finish-reasons block) are resolved in the final file. New code paths are guarded by the existing outer try/except and covered by 21 new mock-based unit tests.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/integrations/opentelemetry.py	Adds elif branch for Responses API output/status, broadens system-prompt kwarg coalescing to include instructions/system, and introduces _transform_responses_api_output_to_otel; all four issues from prior review round addressed in the final file.
tests/test_litellm/integrations/test_opentelemetry.py	21 new unit tests added covering output messages, finish reasons, system instructions, per-tool-call attributes, Pydantic duck-typing, and regression for existing choices path; all use MagicMock with no real network calls.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["set_attributes(span, kwargs, response_obj)"] --> B{response_obj is not None?}
    B -- No --> Z[return]
    B -- Yes --> C{response_obj.get choices ?}

    C -- Yes --> D["_transform_choices_to_otel_semantic_conventions()"]
    D --> E["set gen_ai.output.messages"]
    E --> F["extract finish_reasons from choices[].finish_reason"]
    F --> G["set gen_ai.response.finish_reasons"]
    G --> H["for each choice: _tool_calls_kv_pair()"]
    H --> I["set gen_ai.completion.N.function_call.* attributes"]

    C -- No --> J{response_obj.get output ?}
    J -- No --> Z
    J -- Yes --> K["_transform_responses_api_output_to_otel(output_items)"]
    K --> L["set gen_ai.output.messages"]
    L --> M["for each function_call item: build tool_calls list"]
    M --> N["_tool_calls_kv_pair(tool_calls)"]
    N --> O["set gen_ai.completion.N.function_call.* attributes"]
    O --> P["response_obj.get status"]
    P --> Q["set gen_ai.response.finish_reasons = [status]"]

    A --> R["coalesce system_instructions / instructions / system"]
    R --> S{isinstance str?}
    S -- Yes --> T["set gen_ai.system_instructions = raw string"]
    S -- No --> U["_transform_messages_to_otel_semantic_conventions()"]
    U --> V["set gen_ai.system_instructions = JSON"]

Loading

_{Reviews (5): Last reviewed commit: "fix: remove duplicate gen_ai.response.fi..." | Re-trigger Greptile}

[aneeshsangvikar]

Review Comments — All Resolved

All four Greptile review findings have been addressed:

Finding	Severity	Resolution	Commit
isinstance(dict) drops Pydantic objects	P1	Replaced with hasattr(item, "get") + added test with Pydantic-like objects	466b4dd
Duplicate gen_ai.response.finish_reasons block	P1	Removed duplicate	9928618
or-chain falsy fallthrough	P2	Changed to is not None guards + added falsy-fallthrough test	466b4dd
Missing per-tool-call span attributes	P2	Added _tool_calls_kv_pair call for Responses API function_call items + added indexing tests	466b4dd

[krrish-berri-2]

@aneeshsangvikar — could you add a screenshot or short video showing that this change works as expected? It really helps reviewers verify the fix quickly. Thanks!

[aneeshsangvikar]

Verification

Here's the output of calling set_attributes() with a Responses API response object, showing all three previously-missing attributes are now populated:

======================================================================
Responses API (/v1/responses) -- OTel Span Attributes (WITH fix)
======================================================================
  Request model             | gpt-4o-mini
  Provider                  | openai
  Operation                 | responses
  Input tokens              | 25
  Output tokens             | 10
  System instructions       | You are a helpful math tutor.
  Input messages            | [{"role": "user", "parts": [{"type": "text", "content": "What is 2+2?"}]}]
  Output messages           | [{"role": "assistant", "parts": [{"type": "text", "content": "2 + 2 = 4."}]}]
  Finish reasons            | ["completed"]

======================================================================
BEFORE this fix, the last 3 rows would all show MISSING.
======================================================================

Before the fix: gen_ai.system_instructions, gen_ai.output.messages, and gen_ai.response.finish_reasons were never set for /v1/responses calls because set_attributes() only checked response_obj.get("choices") (chat completion format) and kwargs.get("system_instructions") (Gemini path).

After the fix: All three are populated using the Responses API equivalents (output, instructions, status).

Reproduction script (click to expand)

from unittest.mock import MagicMock
from litellm.integrations.opentelemetry import OpenTelemetry

otel = OpenTelemetry()
mock_span = MagicMock()

kwargs = {
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "instructions": "You are a helpful math tutor.",
    "optional_params": {},
    "litellm_params": {"custom_llm_provider": "openai"},
    "standard_logging_object": {
        "id": "resp_abc123",
        "call_type": "responses",
        "metadata": {},
    },
}

response_obj = {
    "id": "resp_abc123",
    "model": "gpt-4o-mini-2024-07-18",
    "status": "completed",
    "output": [
        {
            "type": "message",
            "role": "assistant",
            "content": [
                {"type": "output_text", "text": "2 + 2 = 4."}
            ],
        }
    ],
    "usage": {"prompt_tokens": 25, "completion_tokens": 10, "total_tokens": 35},
}

otel.set_attributes(span=mock_span, kwargs=kwargs, response_obj=response_obj)

for call in mock_span.set_attribute.call_args_list:
    print(f"  {call[0][0]}: {call[0][1]}")

[aneeshsangvikar]

@krrish-berri-2 Thanks for reviewing! The verification is in this comment above — it shows the before/after span attributes for a /v1/responses call, along with a self-contained reproduction script you can copy-paste to verify locally.

In short, the three previously-missing attributes are now populated:

Attribute	Before	After
gen_ai.output.messages	MISSING	[{"role": "assistant", "parts": [{"type": "text", "content": "2 + 2 = 4."}]}]
gen_ai.system_instructions	MISSING	You are a helpful math tutor.
gen_ai.response.finish_reasons	MISSING	["completed"]

All 126 unit tests pass (101 existing + 25 new), and all 42 CI checks are green.

[aneeshsangvikar]

@aneeshsangvikar — could you add a screenshot or short video showing that this change works as expected? It really helps reviewers verify the fix quickly. Thanks!

[oss-pr-review-agent-shin]

🤖 litellm-agent: Merged into staging branch litellm_agent_oss_staging_05_06_2026. Staging PR: #27256

---

Triage Summary
Fixes OpenTelemetry span attribute population for the Responses API. Coalesces three different kwarg names that carry the system prompt (system_instructions for Vertex AI Gemini, instructions for OpenAI Responses API, system for Anthropic) into a single resolution path, and handles both plain-string and structured formats. Also adds gen_ai.output.messages support for Responses API output items. Adds 627 lines of new tests in test_opentelemetry.py covering the new code paths.

Merge Confidence: 5/5 ✅ READY
Ready to ship.

All checks green. Greptile 5/5, no blocking pattern findings, no CircleCI runs (OSS-typical).