test(e2e): otel streaming spans record a real ttft below span duration

[yucheng-berri]

Relevant issues

Follow-up scenario of the merged OTEL trace-completeness suite (#33132, #33234).

Linear ticket

https://linear.app/litellm-ai/issue/LIT-3787 (OTEL e2e scenario program)

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Verification run on 2026-07-16, compose e2e stack (proxy + jaeger destination), all outputs verbatim.

1. Evidence-first probe before assertions were written. On all three streamed surfaces the gen-AI span carries gen_ai.response.time_to_first_chunk with a real positive value strictly under the span's own duration:

   chat:      span='chat claude-haiku-4-5' duration=0.8040s ttfc=0.698983907699585   ttfc>0: True  ttfc<duration: True
   messages:  span='chat claude-haiku-4-5' duration=0.7672s ttfc=0.635624885559082   ttfc>0: True  ttfc<duration: True
   responses: span='chat gpt-5.5'          duration=2.1237s ttfc=0.9683389663696289  ttfc>0: True  ttfc<duration: True
   

2. Pass on current code, plus a full-file regression sweep of the merged OTEL suite:

   test_chat_completions_stream_records_real_ttft   PASSED
   test_messages_stream_records_real_ttft           PASSED
   test_responses_stream_records_real_ttft          PASSED
   ======================= 3 passed, 7 deselected in 21.83s =======================
   ======================== 10 passed in 76.33s (0:01:16) =========================
   

3. Fail-before-fix at 765fd07, the parent of feat(otel): stamp gen_ai.response.time_to_first_chunk on streaming LLM spans #32236 (the commit that introduced TTFT stamping on streaming LLM spans). Proxy run from source at that ref against a dedicated jaeger; all three tests fail at exactly the intended assertion — the gen-AI span reaches the destination with its full tag set but no TTFT attribute:

   E  AssertionError: the gen-AI span must record gen_ai.response.time_to_first_chunk for a
      streamed call; tags present: ['gen_ai.operation.name', 'gen_ai.provider.name', ...,
      'litellm.request.streaming', 'llm.is_streaming', ..., 'span.kind']
   FAILED test_chat_completions_stream_records_real_ttft
   FAILED test_messages_stream_records_real_ttft
   FAILED test_responses_stream_records_real_ttft
   

4. Static gates: make lint-e2e-basedpyright -> 0 errors; coverage_registry.collector --strict passes with the new row.

Type

✅ Test

Changes

TTFT is the streaming latency SLI: dashboards and alerting are built on gen_ai.response.time_to_first_chunk. The failure modes this pins are silent — a dropped attribute (verified live at 765fd07, where the span exports fine without it), a zero from missing/backfilled timestamps, or a value computed from the wrong clock anchor that lands at/above the whole span duration. None of the merged trace-completeness tests look at this attribute.

• coverage_registry/logging.yaml: adds the logging.otel.stream.records_ttft P1 row (event: stream, surfaces chat_completions/messages/responses); each test declares its surface via exercised_on.
• logging/test_otel_trace_e2e.py: three streamed tests (one per surface) sharing _assert_real_ttft: the call actually streams, exactly ONE trace holds the call's litellm.call_id, exactly ONE gen-AI span exists, and that span's gen_ai.response.time_to_first_chunk is present, numeric, positive, and strictly less than the span's own duration. The strict upper bound is sound by construction: the span opens at pre_call (before the upstream request that anchors TTFT's start) and closes only after the stream is drained (after the first chunk that anchors TTFT's end).
• logging/otel_client.py: JaegerSpan gains the duration field (microseconds, straight from the Jaeger query API).

No new cluster infra: the same jaeger deployment the merged suite uses covers all three tests.

Behavior changes

None; test-only changes under tests/e2e.

QA runbook

• tests/e2e/logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_chat_completions_stream_records_real_ttft - one successful streamed chat call records a real time-to-first-token on its model-call span, strictly below the span's total duration

  • GET /health/readiness/details with the master key and confirm OpenTelemetryV2 appears in success_callbacks
  • POST /key/generate with {"models":["claude-haiku-4-5"],"key_alias":"otel-ttft-chat-<uniq>"} and save the returned key
  • curl -N POST /chat/completions with "stream": true, max_tokens: 16, and a unique phrase; retry the first call on 401 for a few seconds; confirm content-type: text/event-stream, that data: chunks actually arrive with no in-stream error event, and note the x-litellm-call-id response header
  • In Jaeger (http://localhost:16686, service litellm), search by tag litellm.call_id=<that id> and wait for the trace (spans flush in batches)
  • Confirm exactly ONE trace holds the call-id and exactly ONE chat claude-haiku-4-5 span
  • Confirm that span's tags include gen_ai.response.time_to_first_chunk as a number
  • Confirm the value is greater than 0 and strictly less than the span's own duration
  • POST /key/delete to clean up

• tests/e2e/logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_messages_stream_records_real_ttft - one successful streamed Anthropic-native call records a real time-to-first-token on its model-call span, strictly below the span's total duration

  • GET /health/readiness/details with the master key and confirm OpenTelemetryV2 appears in success_callbacks
  • POST /key/generate with {"models":["claude-haiku-4-5"],"key_alias":"otel-ttft-messages-<uniq>"} and save the returned key
  • curl -N POST /v1/messages with "stream": true, max_tokens: 16, and a unique phrase; retry the first call on 401 for a few seconds; confirm content-type: text/event-stream, that data: events actually arrive with no in-stream error event, and note the x-litellm-call-id response header
  • In Jaeger, search by tag litellm.call_id=<that id> and wait for the trace
  • Confirm exactly ONE trace holds the call-id and exactly ONE chat claude-haiku-4-5 span
  • Confirm that span's tags include gen_ai.response.time_to_first_chunk as a number
  • Confirm the value is greater than 0 and strictly less than the span's own duration
  • POST /key/delete to clean up

• tests/e2e/logging/test_otel_trace_e2e.py::TestOtelTraceCompleteness::test_responses_stream_records_real_ttft - one successful streamed Responses API call records a real time-to-first-token on its model-call span, strictly below the span's total duration

  • GET /health/readiness/details with the master key and confirm OpenTelemetryV2 appears in success_callbacks
  • POST /key/generate with {"models":["gpt-5.5"],"key_alias":"otel-ttft-responses-<uniq>"} and save the returned key
  • curl -N POST /v1/responses with "stream": true, max_output_tokens: 64, and a unique phrase; retry the first call on 401 for a few seconds; confirm content-type: text/event-stream, that events actually arrive with no in-stream error event, and note the x-litellm-call-id response header
  • In Jaeger, search by tag litellm.call_id=<that id> and wait for the trace
  • Confirm exactly ONE trace holds the call-id and exactly ONE chat gpt-5.5 span
  • Confirm that span's tags include gen_ai.response.time_to_first_chunk as a number
  • Confirm the value is greater than 0 and strictly less than the span's own duration
  • POST /key/delete to clean up

Final attestation: "I agree this test is testing what the writer wanted to test."

---

Note

Low Risk
Test-only changes under tests/e2e; no production or runtime behavior is modified.

Overview
Extends the OTEL Jaeger e2e suite so streaming latency SLI behavior is covered: gen-AI spans must expose gen_ai.response.time_to_first_chunk as a real measurement, not a silent regression.

A new logging.otel.stream.records_ttft row is registered in the e2e coverage registry. JaegerSpan now parses duration from Jaeger (microseconds) so assertions can compare TTFT against total span time. Shared helper _assert_real_ttft checks one trace, one gen-AI span, a numeric TTFT tag, value > 0, and TTFT strictly less than span duration.

Three new tests exercise streamed /chat/completions, /v1/messages, and /v1/responses, each polling Jaeger by litellm.call_id like the existing trace-completeness tests.

^{Reviewed by Cursor Bugbot for commit c68dd46. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR adds three end-to-end streaming tests that verify the gen_ai.response.time_to_first_chunk OTEL attribute is present, positive, and strictly below the enclosing gen-AI span's total duration — one test per surface (/chat/completions, /v1/messages, /v1/responses). It also adds the duration field to JaegerSpan (microseconds, read from the Jaeger query API) and registers the new logging.otel.stream.records_ttft P1 coverage row.

• otel_client.py: JaegerSpan.duration field added (int = 0); no alias needed since Jaeger serialises it as lowercase duration, consistent with how start_time uses Field(alias=\"startTime\") only where camelCase is needed.
• test_otel_trace_e2e.py: shared _assert_real_ttft helper enforces the full attribute contract (presence, numeric type, positivity, strict upper-bound against span duration) and is reused by all three new test methods.
• coverage_registry/logging.yaml: new P1 row covers all three streaming surfaces in the right tier.

Confidence Score: 5/5

Test-only changes with no production code touched; the new tests follow established patterns and the JaegerSpan.duration addition maps directly to an existing Jaeger API field.

All three changed files are confined to the tests/e2e tree and do not affect any production code path. The duration field is a straightforward addition that reads a standard Jaeger API field already present in every response. The three new test methods faithfully follow the existing suite structure, and the shared helper enforces a multi-condition attribute contract that is difficult to accidentally weaken. The only noteworthy gap is a missing max_tokens cap on the responses test.

No files require special attention; test_otel_trace_e2e.py has one minor inconsistency in the responses test (missing max_tokens).

Important Files Changed

Filename	Overview
tests/e2e/coverage_registry/logging.yaml	Adds a new P1 coverage row logging.otel.stream.records_ttft covering all three streaming surfaces (chat_completions, messages, responses); entry is correctly structured and slots into the existing ordering.
tests/e2e/logging/otel_client.py	Adds duration: int = 0 to JaegerSpan; no alias required since the Jaeger API returns the field as lowercase duration in microseconds, consistent with how start_time uses Field(alias="startTime") only where camelCase is needed.
tests/e2e/logging/test_otel_trace_e2e.py	Adds _assert_real_ttft helper and three streamed tests; test_responses_stream_records_real_ttft omits max_tokens that the other two tests pass, minor inconsistency but not a correctness issue.

_{Reviews (1): Last reviewed commit: "test(e2e): otel streaming spans record a..." | Re-trigger Greptile}

[greptile-apps]

Missing max_tokens cap on the responses test

The chat_raw and messages_raw calls both pass max_tokens=16, but responses_raw here has no token limit. Without a cap the model may stream a much longer reply than the prompt warrants, adding unnecessary latency and cost to every CI run of this test. Consider adding max_tokens=16 (or max_output_tokens=16 if that is the correct kwarg for responses_raw) to match the other two tests.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[yucheng-berri]

The cap is already there — it lives in the client helper rather than the call site: responses_raw defaults to max_output_tokens=64 (tests/e2e/logging/logging_client.py), so every call through it is token-capped. Two notes on the suggestion itself: max_tokens is not a Responses API kwarg (the correct one, max_output_tokens, is what the helper sends), and 64 rather than 16 is deliberate on this surface — gpt-5.5 is a reasoning model, and a 16-token budget can be consumed entirely by reasoning, yielding an empty stream and a flaky test. This matches every other /v1/responses test in the suite (the merged trace-completeness tests and the DataDog log-delivery tests use the same helper default).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_e2e_otel_stream_ttft (c68dd46) with litellm_internal_staging (2162da5)}

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c68dd46. Configure here.}

[mateo-berri]

LGTM; thanks!