test(e2e): otel trace completeness on streaming chat, messages, and responses (LIT-3787)

[yucheng-berri]

Relevant issues

Related: LIT-4428 (product gap found while writing the responses test)

Linear ticket

https://linear.app/litellm-ai/issue/LIT-3787 (streaming follow-up of the merged non-streaming scenarios)
https://linear.app/litellm-ai/issue/LIT-4428 (tracks the two assertions the responses test knowingly relaxes)

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

This PR carries the full streaming trace-completeness suite. It was reviewed as a 3-PR stack (this PR, #33247 messages, #33262 responses) and the two stacked PRs were merged down into this branch; content is byte-identical to the verified stack tip. Uses the jaeger destination and read-back infrastructure merged in #33132; no new infra. Verification run on 2026-07-14, proxies from source at exact refs (no docker buildx on this machine); outputs verbatim.

1. Pass on current code, DEFAULT models with no override; the whole file (three non-streaming + three streaming tests), including streamed responses on gpt-5.5:

   $ LITELLM_PROXY_URL=http://localhost:4000 uv run pytest logging/test_otel_trace_e2e.py -v
   ============================== 6 passed in 41.41s ==============================
   

2. Evidence-first probes shaped the assertions before they were written. Streamed chat and messages spans carry the correlation tag, streaming fingerprints, and appear exactly once:

   chat:     content_type=text/event-stream chunks=4  traces=1 genai_count=1
             {"litellm.request.streaming": true, "gen_ai.usage.total_tokens": 20, "litellm.call_id": "8970b464-...", ...}
   messages: content_type=text/event-stream chunks=16 traces=1 genai_count=1
             chat claude-haiku-4-5: streaming=True call_id_present=True kind=client
   

   The responses probe found a product gap, filed as LIT-4428: a successful streamed /v1/responses call records spend correctly but its trace never receives the cost-write span, and its gen-AI span carries no litellm.request.streaming tag:

   recent spend rows: 3 (call_type='aresponses', spend > 0)
   traces containing a 'batch_write_to_db _PROXY_track_cost_callback' span (30m window, responses-only traffic): 0
   

   The responses test therefore relaxes exactly those two assertions (documented in its docstring with the ticket id); the chat and messages tests keep the strict versions. Tighten when LIT-4428 lands. Historical note: the earliest responses runs happened during an OpenAI org quota outage and used the env-overridable gemini model, which is why some probe outputs above are gemini-labeled; the outage is also how the in-stream error guard below earned its place (a streamed call commits its HTTP 200 before the upstream fails).

3. In-stream failures cannot masquerade as success (added per Cursor review): StreamingResponse captures the first SSE error event (OpenAI and Anthropic formats) and every streamed test asserts none arrived. Kill-proof, run while the org key was still quota-dead:

   E  AssertionError: the stream carried an upstream error event despite the 200:
      data: {"type":"error","sequence_number":2,"error":{"type":"insufficient_quota",...}}
   

4. Fail-before-fix at 1bd603d (parent of fix(otel): one v2 logger owns the global provider; scope tenant OTLP creds per exporter #30590, the same foil the merged non-streaming stack was proven against):

   chat:      E AssertionError: span(s) ['chat claude-haiku-4-5'] reference a parent that never
                reached the destination (orphaned trace); spans present: ['chat claude-haiku-4-5']
   messages:  E AssertionError: span(s) ['chat claude-haiku-4-5'] reference a parent that never
                reached the destination (orphaned trace); spans present: ['chat claude-haiku-4-5']
   responses: E AssertionError: no trace for this call arrived at the destination within the deadline
                (nothing tagged with its call id was found)
   

On that ref a streamed responses call exports nothing tagged to the destination at all, an even earlier break than the orphan. Re-run 2026-07-15 with the restored OpenAI key on the DEFAULT models: all three fail again, this boot with the orphan on every route including gpt-5.5 (which failure mode appears per route depends on how the #30590 startup race lands on a given boot; the test fails on either):

E  AssertionError: span(s) ['chat claude-haiku-4-5'] reference a parent that never
   reached the destination (orphaned trace)   (chat and messages)
E  AssertionError: span(s) ['chat gpt-5.5'] reference a parent that never
   reached the destination (orphaned trace)   (responses, on the default model)
================= 3 failed in 364.95s (0:06:04) ==================

A second candidate foil was evaluated and rejected with cause: 8f25942 (parent of #27757, the duplicate-streaming-spans fix) predates the OTEL v2 architecture entirely, so under this suite's v2 destination config it fails trivially at the readiness check rather than meaningfully at the duplicate assertion; the duplicate-span regression class is instead pinned by the exactly-one-gen-AI-span assertion in all three tests.

5. Static gates: make lint-e2e-basedpyright -> 0 errors; coverage_registry.collector --strict passes (Logging & Guardrails 5/51 -> 6/52 with the new stream row).

Type

✅ Test

Changes

Streaming closes the gen-AI span from the stream-consumption path: the span can only end after the last chunk arrives and usage is aggregated, a lifecycle that historically produced duplicate spans (#27757) and is equally exposed to the orphaning #30590 fixed (verified at its parent commit). None of the merged non-streaming tests exercise this path.

• coverage_registry/logging.yaml: adds the logging.otel.stream.exports_metric P0 row (event: stream, surfaces chat_completions/messages/responses, matching how langfuse declares separate stream cells); each test declares its surface via exercised_on.
• logging/test_otel_trace_e2e.py: three streamed tests with the shared completeness contract (recorded state via /health/readiness/details, then exactly ONE trace for the call's litellm.call_id, single root SERVER span, auth/db children, gen-AI span whose CHILD_OF chain reaches the root) plus stream-specific assertions: the response actually streamed, no in-stream error event arrived, exactly ONE gen-AI span exists for the call, and on the chat/messages surfaces the span records litellm.request.streaming=true. _assert_complete_trace/_settled_names gain a require_cost_span flag so the responses surface can skip the cost-span requirement (LIT-4428) while the siblings keep it strict. A typed _tag helper reads span tags.
• e2e_http.py (shared harness): StreamingResponse gains a stream_error field capturing the first in-stream SSE error event, covering both OpenAI (unspaced JSON) and Anthropic (event: error control line, spaced JSON) formats.
• logging/logging_client.py: messages_raw and responses_raw gain a stream mode that consumes the SSE body and counts events, mirroring chat_raw; ResponsesRequestBody gains an optional stream field.
• models.py: AnthropicMessagesBody gains an optional stream field (mirrors ChatBody).

No new cluster infra: the same jaeger deployment #33132 requires covers all three tests.

Behavior changes

None; test-only changes under tests/e2e.

QA runbook

• tests/e2e/logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_chat_completions_stream_exports_complete_trace - one successful streamed chat call shows up at the OTEL destination as one connected trace tree with a single streaming-marked model-call span

  • GET /health/readiness/details with the master key and confirm OpenTelemetryV2 appears in success_callbacks
  • POST /key/generate with {"models":["claude-haiku-4-5"],"key_alias":"otel-stream-chat-<uniq>"} and save the returned key
  • curl -N POST /chat/completions with "stream": true, max_tokens: 16, and a unique phrase; retry the first call on 401 for a few seconds; confirm content-type: text/event-stream, that data: chunks actually arrive with no in-stream "type":"error" event, and note the x-litellm-call-id response header
  • In Jaeger (http://localhost:16686, service litellm), search by tag litellm.call_id=<that id> and wait for the trace (spans flush in batches; the cost write lands last)
  • Confirm exactly ONE trace holds the call-id
  • Confirm a single root span POST /chat/completions (kind server) and no span referencing a parent missing from the trace
  • Confirm the children: auth /chat/completions, a postgres ... span, and batch_write_to_db _PROXY_track_cost_callback
  • Confirm exactly ONE chat claude-haiku-4-5 span (kind client), that its parent chain reaches the root, and that its tags include litellm.request.streaming: true
  • POST /key/delete to clean up

• tests/e2e/logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_messages_stream_exports_complete_trace - one successful streamed Anthropic-native call shows up at the OTEL destination as one connected trace tree with a single streaming-marked model-call span

  • GET /health/readiness/details with the master key and confirm OpenTelemetryV2 appears in success_callbacks
  • POST /key/generate with {"models":["claude-haiku-4-5"],"key_alias":"otel-stream-messages-<uniq>"} and save the returned key
  • curl -N POST /v1/messages with "stream": true, max_tokens: 16, and a unique phrase; retry the first call on 401 for a few seconds; confirm content-type: text/event-stream, that data: events actually arrive with no in-stream error event, and note the x-litellm-call-id response header
  • In Jaeger, search by tag litellm.call_id=<that id> and wait for the trace
  • Confirm exactly ONE trace holds the call-id
  • Confirm a single root span POST /v1/messages (kind server) and no span referencing a parent missing from the trace
  • Confirm the children: auth /v1/messages, a postgres ... span, and batch_write_to_db _PROXY_track_cost_callback
  • Confirm exactly ONE chat claude-haiku-4-5 span (kind client), that its parent chain reaches the root, and that its tags include litellm.request.streaming: true
  • POST /key/delete to clean up

• tests/e2e/logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_responses_stream_exports_complete_trace - one successful streamed Responses API call shows up at the OTEL destination as one connected trace tree with a single model-call span

  • GET /health/readiness/details with the master key and confirm OpenTelemetryV2 appears in success_callbacks
  • POST /key/generate with {"models":["gpt-5.5"],"key_alias":"otel-stream-responses-<uniq>"} and save the returned key
  • curl -N POST /v1/responses with "stream": true, max_output_tokens: 64, and a unique phrase; retry the first call on 401 for a few seconds; confirm content-type: text/event-stream, that events ending in response.completed arrive (an in-stream "type":"error" event means the upstream failed despite the 200), and note the x-litellm-call-id header
  • In Jaeger, search by tag litellm.call_id=<that id> and wait for the trace
  • Confirm exactly ONE trace holds the call-id
  • Confirm a single root span POST /v1/responses (kind server) and no span referencing a parent missing from the trace
  • Confirm the children: auth /v1/responses and a postgres ... span (the cost-write span is knowingly absent on this surface; see LIT-4428)
  • Confirm exactly ONE chat gpt-5.5 span (kind client) and that its parent chain reaches the root
  • POST /key/delete to clean up

Final attestation: "I agree this test is testing what the writer wanted to test."

---

Note

Low Risk
Test-only changes under tests/e2e; no production runtime behavior is modified.

Overview
Adds streaming OTEL trace completeness coverage alongside the existing non-streaming tests, targeting the stream-consumption span lifecycle (duplicate/orphaned gen-AI spans).

Registers logging.otel.stream.exports_metric in the e2e coverage registry and adds three Jaeger-backed tests for streamed /chat/completions, /v1/messages, and /v1/responses. Shared helpers gain require_cost_span (skipped for streamed responses per LIT-4428) and a _tag helper for span assertions. Chat/messages streams assert litellm.request.streaming=true, a single gen-AI span, real SSE delivery, and no in-stream errors; the responses stream test documents relaxed cost-span and streaming-tag expectations until LIT-4428.

E2E harness: StreamingResponse.stream_error captures the first SSE error on HTTP 200 streams; messages_raw / responses_raw and request models gain optional stream flags for consumed SSE bodies.

^{Reviewed by Cursor Bugbot for commit ad61aae. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

Adds streaming OTEL trace completeness coverage alongside the existing non-streaming tests, targeting the stream-consumption span lifecycle (duplicate/orphaned gen-AI spans historically caused by #27757 and the #30590 startup race).

• Three new Jaeger-backed tests for streamed /chat/completions, /v1/messages, and /v1/responses; chat/messages assert litellm.request.streaming=true on the gen-AI span and a single cost-write span, while the responses test documents two relaxed assertions (no cost-write span, no streaming tag) tracking LIT-4428.
• StreamingResponse.stream_error captures the first SSE error event on HTTP-200 streams so in-stream upstream failures cannot masquerade as success; messages_raw/responses_raw gain an optional stream flag with backward-compatible None-exclusion via exclude_none=True.
• require_cost_span flag added to _assert_complete_trace and _settled_names to skip the cost-span requirement on the responses surface without relaxing polling for other spans.

Confidence Score: 5/5

Safe to merge; all changes are confined to tests/e2e and have no effect on production runtime behavior.

The changes are test-only and well-documented with evidence of passing runs against the current proxy and failing runs against the foil commit. The only noted imperfection is that Anthropic two-line SSE errors store only the event: error control line rather than the informative data: payload, making test failure messages less diagnostic for the messages surface — but this does not affect test correctness.

tests/e2e/e2e_http.py — the _streaming_outcome error-capture loop stores the Anthropic event: error control line rather than the following data: line with error details.

Important Files Changed

Filename	Overview
tests/e2e/e2e_http.py	Adds stream_error field to StreamingResponse and updates _streaming_outcome to capture the first SSE error event; Anthropic two-line errors lose detail because the control event: error line is captured before the data: line.
tests/e2e/logging/test_otel_trace_e2e.py	Adds three well-structured streaming OTEL trace completeness tests (chat, messages, responses); require_cost_span flag correctly relaxes the responses cost-span assertion per LIT-4428; _tag helper and _settled_names changes are clean.
tests/e2e/logging/logging_client.py	Adds stream parameter to messages_raw and responses_raw; stream=None for the non-streaming path is correctly excluded by exclude_none=True in the transport, preserving backward compatibility.
tests/e2e/models.py	Adds optional `stream: bool
tests/e2e/coverage_registry/logging.yaml	Registers new logging.otel.stream.exports_metric P0 coverage row correctly referencing all three streaming surfaces.

_{Reviews (6): Last reviewed commit: "test(e2e): assert streamed responses spe..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_e2e_otel_stream_chat_trace (a3b2841) with litellm_internal_staging (65ca095)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (56f4dbf) during the generation of this report, so 65ca095 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yucheng-berri]

@greptileai

[yucheng-berri]

@greptileai

[yucheng-berri]

bugbot run

[yucheng-berri]

@greptileai

[mateo-berri]

The one finding is a diagnostic quality issue in the stream-error detection pattern that doesn't affect whether the OTEL assertions themselves pass or fail correctly

Legit or no? If legit, worth addressing?

[yucheng-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit ad61aae. Configure here.}

[mateo-berri]

LGTM; thanks!

[yucheng-berri]

@greptileai