test(e2e): otel trace completeness on streaming /v1/responses (LIT-3787)

[yucheng-berri]

Relevant issues

Related: LIT-4428 (product gap found while writing this test)

Linear ticket

https://linear.app/litellm-ai/issue/LIT-3787 (streaming follow-up; top of the streaming stack)
https://linear.app/litellm-ai/issue/LIT-4428 (tracks the two assertions this test knowingly relaxes)

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Stacked on the streaming /v1/messages PR; merge that first. Same infra, no new services. Run on 2026-07-14, proxies from source at exact refs. Outputs verbatim.

Provider disclosure: the OpenAI org behind the .env key is currently out of quota (insufficient_quota arrives inside the stream after an HTTP 200, since streaming commits the status before upstream completes), so verification ran with E2E_CHEAP_OPENAI_MODEL=gemini-2.5-flash (the constant is env-overridable by design). The route under test is unchanged; the expected span name derives from the constant.

1. Evidence-first, and it found a product gap. A successful streamed /v1/responses call records spend correctly, but its trace never receives the cost-write span, and the gen-AI span carries no litellm.request.streaming tag (streamed chat/messages and non-streamed responses all have both):

   recent gemini spend rows: 3
     2026-07-14T19:24:16 spend=0.0001529 tokens=77 call_type='aresponses'
     2026-07-14T19:21:58 spend=0.0001523 tokens=75 call_type='aresponses'
     2026-07-14T19:21:25 spend=8.43e-05 tokens=39 call_type='aresponses'
   traces containing a 'batch_write_to_db _PROXY_track_cost_callback' span (30m window, responses-only traffic): 0
   

   Filed as LIT-4428. This test therefore relaxes exactly those two assertions on this surface (documented in the docstring with the ticket id); everything else in the completeness contract is asserted, and the sibling streamed chat/messages tests keep the strict versions. Tighten when LIT-4428 lands.

2. Pass on current code (compose stack):

   $ LITELLM_PROXY_URL=http://localhost:4000 E2E_CHEAP_OPENAI_MODEL=gemini-2.5-flash uv run pytest \
       "logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_responses_stream_exports_complete_trace" -v
   ============================== 1 passed in 6.21s ===============================
   

3. Fail-before-fix at 1bd603d (parent of fix(otel): one v2 logger owns the global provider; scope tenant OTLP creds per exporter #30590): on that ref a streamed responses call exports nothing tagged to the destination at all, so the test fails at its first trace assertion (an even earlier break than the orphan the non-streaming tests catch):

   $ LITELLM_PROXY_URL=http://localhost:4611 E2E_OTEL_QUERY_URL=http://localhost:16690 \
       E2E_CHEAP_OPENAI_MODEL=gemini-2.5-flash uv run pytest "...::test_responses_stream_exports_complete_trace"
   E  AssertionError: no trace for this call arrived at the destination within the deadline
      (nothing tagged with its call id was found)
   ======================== 1 failed in 121.51s (0:02:01) =========================
   

4. Static gates: make lint-e2e-basedpyright -> 0 errors; coverage_registry.collector --strict passes.

Type

✅ Test

Changes

Third surface of the logging.otel.stream.exports_metric row: streamed /v1/responses.

• logging/test_otel_trace_e2e.py: adds test_responses_stream_exports_complete_trace (shared completeness contract + streamed-response checks + exactly one gen-AI span). _assert_complete_trace/_settled_names gain a require_cost_span flag so this surface can skip the cost-span requirement while the siblings keep it strict; the docstring names LIT-4428 as the tracked reason.
• logging/logging_client.py: responses_raw gains a stream mode; ResponsesRequestBody gains an optional stream field.

Behavior changes

None; test-only changes under tests/e2e.

QA runbook

• tests/e2e/logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_responses_stream_exports_complete_trace - one successful streamed Responses API call shows up at the OTEL destination as one connected trace tree with a single model-call span
  • GET /health/readiness/details with the master key and confirm OpenTelemetryV2 appears in success_callbacks
  • POST /key/generate with {"models":["gemini-2.5-flash"],"key_alias":"otel-stream-responses-<uniq>"} and save the returned key (gemini-2.5-flash via E2E_CHEAP_OPENAI_MODEL while the OpenAI org quota is exhausted; default is gpt-5.5)
  • curl -N POST /v1/responses with "stream": true, max_output_tokens: 64, and a unique phrase; retry the first call on 401 for a few seconds; confirm content-type: text/event-stream, that events ending in response.completed arrive (an in-stream "type":"error" event means the upstream failed despite the 200), and note the x-litellm-call-id header
  • In Jaeger (http://localhost:16686, service litellm), search by tag litellm.call_id=<that id> and wait for the trace (spans flush in batches)
  • Confirm exactly ONE trace holds the call-id
  • Confirm a single root span POST /v1/responses (kind server) and no span referencing a parent missing from the trace
  • Confirm the children: auth /v1/responses and a postgres ... span (the cost-write span is knowingly absent on this surface; see LIT-4428)
  • Confirm exactly ONE chat <model> span (kind client) and that its parent chain reaches the root
  • POST /key/delete to clean up

Final attestation: "I agree this test is testing what the writer wanted to test."

[greptile-apps]

Greptile Summary

This PR adds the third surface of the streaming OTEL trace completeness contract: a new e2e test for streamed /v1/responses calls. It extends _assert_complete_trace and _settled_names with a backward-compatible require_cost_span flag and adds streaming support to responses_raw in logging_client.py.

• test_responses_stream_exports_complete_trace asserts that a streamed Responses API call lands at the OTEL destination as one connected trace (root server span, auth/db children, single gen-AI client span) while knowingly skipping the cost-write span and streaming-tag checks pending LIT-4428.
• _assert_complete_trace / _settled_names default to require_cost_span=True; all existing callers are unaffected.
• responses_raw now mirrors the messages_raw streaming idiom: stream=True if stream else None on the body, routing through transport.stream() when streaming.

Confidence Score: 5/5

Test-only change; no production code is touched and all existing test assertions are preserved with their original defaults.

Both modified helpers default to their original behavior, so no existing test is weakened. The new test deliberately relaxes two assertions with a tracked ticket (LIT-4428) and documents the rationale clearly. The responses_raw streaming path is a direct copy of the already-reviewed messages_raw pattern.

No files require special attention.

Important Files Changed

Filename	Overview
tests/e2e/logging/logging_client.py	Adds optional stream parameter to responses_raw and a matching `stream: bool
tests/e2e/logging/test_otel_trace_e2e.py	Adds test_responses_stream_exports_complete_trace; _assert_complete_trace / _settled_names gain a backward-compatible require_cost_span=True flag so the new test can skip the cost-span check (LIT-4428) while all existing callers are unaffected.

_{Reviews (3): Last reviewed commit: "test(e2e): parenthesize the settle-names..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yucheng-berri]

@greptileai

[yucheng-berri]

@greptileai