feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans

[devin-ai-integration]

TLDR

Problem this solves:

• OTEL v2 LLM spans never carry gen_ai.conversation.id
• Callers cannot group a conversation's turns in their tracing backend
• The session id they already send to LiteLLM is dropped on the span

How it solves it:

• Resolve the caller's session id once per LLM call event
• Stamp it as gen_ai.conversation.id on the chat <model> span
• Ignore ids LiteLLM minted itself so the attribute stays caller-owned
• Ship a deterministic tests/integration/observability suite (23 cells) that proves it against a real proxy

User Flow

Before: a developer who sends a session id with every turn sees no conversation id on the span, so their tracing backend cannot group the turns

1. They send POST http://localhost:4000/v1/chat/completions with "litellm_session_id": "conv-lit8243-body" in the body (or the same id in a langfuse_session_id or x-litellm-session-id header)
2. They get HTTP 200 and a normal chat.completion object back
3. In their OTLP backend the chat gpt-5.4-mini span has gen_ai.request.model, gen_ai.usage.* and litellm.call_id, but no gen_ai.conversation.id, so each turn shows up as an unrelated span

After: the same request produces a span that carries the caller's session id as gen_ai.conversation.id

1. They send the same POST http://localhost:4000/v1/chat/completions with "litellm_session_id": "conv-lit8243-body" (or the header variants)
2. They get HTTP 200 and the same chat.completion object back
3. The chat gpt-5.4-mini span now has gen_ai.conversation.id: "conv-lit8243-body", and every turn that reuses the id lands in the same conversation view
4. A request that sends no session id still gets a span with no gen_ai.conversation.id at all, rather than a made-up one

Relevant issues

Pylon #8891 (customer request for gen_ai.conversation.id on OTEL v2 spans)

Affected release

Linear ticket

Resolves LIT-8381

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Deterministic integration audit (!audit), one run per leg, both legs on a real proxy with --num_workers 2, real Postgres, real Redis, the scripted upstream from tests/integration/_support/upstream.py and an owned OTLP/HTTP sink. No provider call, no credential, no mock inside the proxy. Every cell is a test in tests/integration/observability/test_otel_conversation_id.py, registered in tests/integration/contracts.json

Merge base (Before): b395bfefddde142dc650383d71e2612851284545
Tip (After): 9b4d3e69f9529b71fd1a56af3a837d7f5e3fc69a
Run ids: base/run7 (b395bfe), head/run12 and head/run13 (9b4d3e6), all with --hypothesis-seed=4106601 --integration-order-seed=0 -p no:pytest-retry -p no:rerunfailures --timeout=90. diff head/run12/nodes.txt head/run13/nodes.txt is empty: identical collected and passed selections, 0 skipped, 0 retries

Matrix (node ids are test_otel_conversation_id.py::<name>; base column quotes the failing assertion from base/run7, head column is the run12 and run13 result)

cell	endpoint, mode, client	scenario	node	base b395bfe	head 9b4d3e6
H1	chat, non-stream, openai sdk	body litellm_session_id	test_chat_completion_sdk_body_litellm_session_id_lands_as_conversation_id	FAIL None == 'conv-53d96769...'	PASS
H2	chat, stream, async openai sdk	header x-litellm-session-id	test_chat_stream_async_sdk_x_litellm_session_id_header_lands_as_conversation_id	FAIL None == 'conv-79e2ca26...'	PASS
H3	messages, non-stream, anthropic sdk	header x-litellm-session-id	test_messages_sdk_x_litellm_session_id_header_lands_as_conversation_id	FAIL None == 'conv-a3f038a7...'	PASS
H4	messages, stream, async anthropic sdk	header langfuse_session_id	test_messages_stream_async_sdk_langfuse_session_id_header_lands_as_conversation_id	FAIL None == 'conv-5aaf5ee3...'	PASS
H5	responses, non-stream, openai sdk	header x-litellm-session-id	test_responses_sdk_x_litellm_session_id_header_lands_as_conversation_id	FAIL None == 'conv-d84eb402...'	PASS
H6	responses, stream, httpx	body metadata.session_id	test_responses_stream_raw_metadata_session_id_lands_as_conversation_id	FAIL None == 'conv-e433ccf6...'	PASS
H7	chat, non-stream, httpx	body metadata.session_id	test_chat_raw_metadata_session_id_lands_as_conversation_id	FAIL None == 'conv-96ced9b9...'	PASS
S1	chat, non-stream, httpx	int and list session ids match the spend row	test_integer_and_list_litellm_session_id_match_the_spend_row_or_are_dropped_together	FAIL None == '123'	PASS
S2	chat, non-stream, httpx	empty string leaves no attribute	test_empty_string_litellm_session_id_leaves_the_span_without_a_conversation_id	PASS (unchanged)	PASS
S3	chat, non-stream, httpx	5 KB header round trips	test_five_kilobyte_session_header_round_trips_to_the_span_and_the_spend_row	FAIL None == 'sss...'	PASS
S4	chat, non-stream, httpx	header sent twice	test_duplicate_session_header_lands_once_and_unchanged	FAIL None == 'conv-a2c2d1e0...'	PASS
S5	chat, non-stream, httpx	no api key, 401, no span	test_unauthenticated_request_with_session_header_is_rejected_and_leaves_no_span	PASS (unchanged)	PASS
S6	chat, non-stream, httpx	sink answers 403, later spans still land	test_sink_rejecting_with_403_drops_those_spans_and_later_spans_still_land	FAIL None == 'conv-301cc328...'	PASS
E1	chat, non-stream, httpx	no session input, no attribute	test_request_without_any_session_input_has_no_conversation_id	PASS (unchanged)	PASS
E2	chat, non-stream, httpx	missing_session_id: generate, minted id stays off the span	test_generate_policy_minted_session_id_reaches_the_spend_row_but_not_the_span	PASS (unchanged)	PASS
E3	chat, non-stream, httpx	generate policy plus langfuse_session_id header	test_generate_policy_keeps_the_langfuse_session_header_as_conversation_id	FAIL None == 'conv-61debc8d...'	PASS
E4	chat, non-stream, httpx	header, body and metadata ids, header wins and matches the spend row	test_header_body_and_metadata_session_ids_resolve_to_the_same_id_as_the_spend_row	FAIL None == 'conv-header-9f37483e...'	PASS
E5	chat, non-stream, httpx	same request three times, one span each	test_three_identical_requests_produce_one_span_each_with_the_same_conversation_id	FAIL (None, None, None) == ('conv-e36bde7e...', ...)	PASS
E6	chat, non-stream, httpx	metadata.trace_id only fills the spend row, not the span	test_metadata_trace_id_alone_fills_the_spend_row_but_not_the_span	PASS (unchanged)	PASS
C1	all three endpoints, mixed stream, 30 concurrent	sink outage mid burst, /health/services?service=otel while down, exactly once after recovery	test_sink_outage_during_a_mixed_burst_lands_every_response_exactly_once_after_recovery	FAIL (attribute never lands)	PASS
C2	chat, 20 concurrent	slow sink, exactly once, no deadlock	test_slow_sink_during_a_burst_lands_every_response_exactly_once	FAIL (attribute never lands)	PASS
C3	chat, 20 concurrent, two workers	one worker killed mid burst, survivor keeps serving, exactly once	test_killing_one_of_two_workers_mid_burst_keeps_serving_and_never_duplicates_a_span	FAIL None == 'conv-after-kill'	PASS
C4	chat, 10 concurrent	SIGTERM right after the burst, every span flushed before exit	test_terminating_the_proxy_right_after_a_burst_flushes_every_span_before_exit	FAIL (attribute never lands)	PASS

Every cell asserts the complete caller response, the complete request the scripted upstream received, the LiteLLM_SpendLogs row and the OTLP span, all matched by response id. The only difference between the legs is the new attribute; the five "unchanged" cells prove the attribute stays absent where no caller id exists on both legs

Before (b395bfe)

Case 1: whole selection on the merge base

1. git -C /home/ubuntu/repos/litellm_otelconv_base2 rev-parse HEAD

   b395bfefddde142dc650383d71e2612851284545
   

2. LITELLM_DISABLE_NO_REDIS_WARNING=true litellm --config <leg cfg> --port 4320 --num_workers 2 --use_v2_migration_resolver then curl -s -o /dev/null -w '%{http_code}\n' http://localhost:4320/health/readiness

   200
   

3. python -m pytest tests/integration/observability/test_otel_conversation_id.py -vv --strict-markers -p no:pytest-retry -p no:rerunfailures --timeout=90 --hypothesis-seed=4106601 --integration-order-seed=0 (run id base/run7)

   FAILED ...::test_chat_completion_sdk_body_litellm_session_id_lands_as_conversation_id
   E       assert None == 'conv-e626817645c641a6828b0de4b814425c'
   FAILED ...::test_messages_sdk_x_litellm_session_id_header_lands_as_conversation_id
   E       assert None == 'conv-3faf9a80833741c992ee036ed6f4fde7'
   FAILED ...::test_responses_sdk_x_litellm_session_id_header_lands_as_conversation_id
   E       assert None == 'conv-85f0a416b7cf4bfb869ec19fa75f55be'
   FAILED ...::test_killing_one_of_two_workers_mid_burst_keeps_serving_and_never_duplicates_a_span
   E       assert None == 'conv-after-kill'
   (14 more FAILED rows of the same shape, listed in the matrix)
   =================== 18 failed, 5 passed in 323.99s (0:05:23) ===================
   

Case 2: determinism check

Not applicable on the merge base; the base leg runs once and its 18 failures are the expected shape

After (9b4d3e6)

Case 1: whole selection on the tip

1. git -C /home/ubuntu/repos/litellm_otelconv_head rev-parse HEAD

   9b4d3e69f9529b71fd1a56af3a837d7f5e3fc69a
   

2. LITELLM_DISABLE_NO_REDIS_WARNING=true litellm --config <leg cfg> --port 4310 --num_workers 2 --use_v2_migration_resolver then curl -s -o /dev/null -w '%{http_code}\n' http://localhost:4310/health/readiness

   200
   

3. same pytest command as Before step 3 (run id head/run12)

   ======================== 23 passed in 141.58s (0:02:21) ========================
   

Case 2: determinism check

1. same pytest command, same seeds, second time (run id head/run13)

   ======================== 23 passed in 152.96s (0:02:32) ========================
   

2. diff head/run12/nodes.txt head/run13/nodes.txt && echo IDENTICAL

   IDENTICAL
   

3. junit summary of both runs (tests, failures, errors, skipped attributes of the testsuite element)

   head/run12/junit.xml tests=23 failures=0 errors=0 skipped=0
   head/run13/junit.xml tests=23 failures=0 errors=0 skipped=0
   

Type

🆕 New Feature

Caveats (if any)

Medium

• The resolver order is the live litellm_session_id (unless it echoes metadata.trace_id), then the header or metadata session_id trace control. When the proxy minted the body id (litellm_session_id_generated), only that minted value is dropped: the header trace control or a caller session_id on the other metadata key (metadata vs litellm_metadata) still wins
• StandardLoggingPayload.session_id is never a source, so a payload replayed through /v1/rust_control_plane/logs gets no gen_ai.conversation.id even when the original request carried a caller session. Replay rebuilds litellm_params from key metadata only and drops the generated marker, so a minted id and a caller id are indistinguishable there, and hiding both is the side that cannot mislabel a conversation. This is not a change against the default branch, which emits no attribute on replay either, and the replay route emits no chat span today in any case (it never sets api_call_start_time)
• Only the OTEL v2 chat <model> LLM span gets the attribute. The Langfuse-compat trace root span keeps its own session.id (from the metadata session_id trace control only) and is untouched
• On the proxy a caller id on litellm_metadata.session_id is lost when missing_session_id: generate has already minted metadata.session_id, because the request preprocessing merges the two keys before any callback runs; the resolver handles both keys but this PR does not change that proxy merge
• metadata.session_id on a /v1/responses body is forwarded to the provider unchanged on both legs (cell H6 asserts the full upstream request); this PR only adds the span attribute and does not strip it

Low

• The proxy back-fills litellm_session_id from the OTel trace id when the caller sends none, and missing_session_id: generate mints one. Both are recognised (value equals metadata.trace_id, or the generated marker is set) and dropped, so a request without a caller id yields no attribute (cells E1, E2, E6). If a caller deliberately reuses their own trace id as their live session id it is treated as back-filled and dropped
• A header session id (langfuse_session_id, x-litellm-session-id) still wins when the body only carries a back-filled or generated id (cells E3, E4)
• An empty-string session id is treated as absent (cell S2). Non-string ids are stringified, so 123 and ['a', 'b'] become the attribute value verbatim and match the spend row (cell S1)
• /health/services?service=otel answers 200 while the OTLP sink is down on both legs; it does not probe the exporter. Documented gap, unchanged by this PR
• Not covered by the deterministic audit and left to tests/e2e/: a real OTLP vendor backend receiving the attribute, and a real provider A/B at this tip

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/ad86be4efa404cbf8373c9f5948d8bf2
Open in Devin Desktop: https://app.devin.ai/desktop/session/ad86be4efa404cbf8373c9f5948d8bf2?variant=devin
Requested by: @yucheng-berri

---

Note

Medium Risk
Observability-only, but wrong session resolution would mislabel conversations in customer tracing backends; logic is narrow to OTEL v2 span attributes and heavily tested.

Overview
OTEL v2 GenAI LLM spans now stamp gen_ai.conversation.id when the caller supplies a real session id (body litellm_session_id, headers like x-litellm-session-id / langfuse_session_id, or metadata session_id).

A new caller_session_id resolver on the LLM call event picks that id once per call and threads it through LLMCallSpanData into the GenAI mapper. It does not treat proxy-minted ids (missing_session_id: generate), OTel trace_id backfill, or StandardLoggingPayload.session_id on replay as caller conversations, so the attribute stays absent or caller-owned rather than auto-filled.

Coverage adds unit cases for precedence and replay, plus a 23-test integration suite against a real proxy, OTLP sink, and spend logs.

^{Reviewed by Cursor Bugbot for commit 1923c87. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[mateo-berri]

bugbot run

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with all previous findings resolved and no new actionable issue at the latest head

Summary

Adds caller-owned session IDs to OpenTelemetry v2 LLM spans as gen_ai.conversation.id, while excluding generated and trace-backfilled IDs

• Resolves session identity once per LLM call and carries it through typed span data to the GenAI mapper
• Covers body, metadata, and header inputs across chat, Messages, and Responses APIs
• Adds integration coverage for streaming, concurrency, exporter failures, worker termination, and shutdown
• Fixes the integration collector race by locking both deque mutation and snapshot creation

_{Reviews (9) · Last reviewed commit: "test(otel): guard the accepted batch deq..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cursor]

Stale Bugbot comment from a previous run.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_otel_v2_gen_ai_conversation_id (1923c87) with main (40ec84c)}

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

bugbot run

[devin-ai-integration]

@greptileai

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 1923c87. Configure here.}