fix(otel v2): map the caller's Langfuse user, session and tags onto the root and generation spans

[devin-ai-integration]

TLDR

Problem this solves:

• langfuse_otel (OTel v2) dropped the caller's trace_user_id, session_id and tags
• Langfuse traces showed no user, no session and no tags, only the trace name from fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name #40793
• Same for the langfuse_trace_user_id / langfuse_session_id proxy headers and for direct SDK calls

How it solves it:

• The caller's trace controls are read once into a typed TraceControls (name, user_id, session_id, tags)
• LangfuseMapper.trace_attributes maps them to user.id, session.id, langfuse.trace.tags
• Stamped on the root observation (pre-call) and the generation span (close), so Langfuse fills the trace fields
• Nothing is emitted when the caller did not send a control; team_id / team_alias stay proxy-set

User Flow

Before: a developer sends their Langfuse user, session and tags with each request and Langfuse shows none of them

1. They send POST http://localhost:4000/v1/chat/completions with "metadata": {"trace_user_id": "u-42", "session_id": "s-7", "tags": ["prod", "eval"]} (or the langfuse_trace_user_id / langfuse_session_id headers)
2. The call returns 200 with the usual completion body
3. They open the trace in Langfuse: User and Session read "-", Tags is empty, only Name and the team metadata are filled
4. They cannot filter or group the traces by user, session or tag in Langfuse

After: the same request fills User, Session and Tags on the Langfuse trace

1. They send the same POST http://localhost:4000/v1/chat/completions with the same metadata (or headers)
2. The call returns 200 with the usual completion body
3. They open the trace in Langfuse: User is u-42, Session is s-7, Tags are prod and eval, Name and team metadata are unchanged
4. Filtering the Langfuse traces by user, session or tag now finds the request; a request sent without those fields shows them empty, as before

Relevant issues

Follows #40793 (trace name). Supersedes the stale #35210 for the user, session and tags part; the mutation controls (trace_id, existing_trace_id, update_trace_keys, trace_metadata) and the caller-metadata allowlist from that PR are deliberately not carried

Affected release

Linear ticket

Resolves LIT-7718

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.). All 83 checks pass at the current tip, including Vertex AI / Run tests which failed on the merge base of the earlier run
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live audit run of 2026-09-14/15, one leg per commit, same rig for both. Proxy on :20718 with 4 uvicorn workers (LITELLM_DISABLE_NO_REDIS_WARNING=true uv run --no-sync litellm --config config-dual.yaml --port 20718 --num_workers 4 --use_v2_migration_resolver), real Postgres (db lit7718), real Anthropic (anthropic/claude-haiku-4-5, model alias haiku), a real local Langfuse v3 at :3100 as the destination of record, plus a wire-faithful OTLP/HTTP sink at :20719 that stores exported span attributes verbatim. The sink is an emulator, used only to read the root and generation attributes deterministically; every Langfuse claim is looked up by the request's x-litellm-call-id, never by count. Each leg ran from its own worktree and printed the loaded litellm import root before the first request. Requests use a key that belongs to team lit7718-real-team. $T is the run tag (a3b before, a3 after)

model_list:
  - model_name: haiku
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
litellm_settings:
  callbacks: ["langfuse_otel", "otel"]
general_settings:
  master_key: sk-7718-master
  database_url: os.environ/LIT7718_DATABASE_URL

Shared shell: P=http://localhost:20718; K="Authorization: Bearer $TEAM_KEY"; CT="Content-Type: application/json"; MSG='"messages":[{"role":"user","content":"reply with one word: ok"}],"max_tokens":5'. Every curl is curl -s -D - -o /dev/null <url> -H "$K" -H "$CT" ... and the printed line is the status plus the x-litellm-call-id response header. Streaming calls are read to [DONE] before readback. Readback: GET /api/public/observations?type=GENERATION paginated, then GET /api/public/traces/{traceId} for the trace whose generation id equals the call id, printed as userId / sessionId / tags / name / metadata.team_id; the sink readback prints the Langfuse attributes on the root span and on the generation span. SDK cases use openai 3.13.0 and anthropic 1.5.0 pointed at $P with the same key

Before (7fd541e)

/v1/chat/completions, body metadata: user + session + 3 tags

1. curl ... $P/v1/chat/completions -d "{\"model\":\"haiku\",$MSG,\"metadata\":{\"trace_user_id\":\"$T-u-body\",\"session_id\":\"$T-s-body\",\"tags\":[\"$T-tag-a\",\"$T-tag-b\",\"$T-tag-c\"]}}"
2. status=200 call_id=8262ba80-6050-4d3d-8052-20f8952df18f
3. Langfuse: userId=None sessionId=None tags=[] name='POST /v1/chat/completions' metadata.team_id='lit7718-real-team'
4. Sink: root {}, generation {"langfuse.trace.metadata.team_id": "lit7718-real-team"}

/v1/chat/completions, headers langfuse_trace_user_id + langfuse_session_id

1. curl ... $P/v1/chat/completions -H "langfuse_trace_user_id: $T-u-hdr" -H "langfuse_session_id: $T-s-hdr" -d "{\"model\":\"haiku\",$MSG}"
2. status=200 call_id=902dd95d-a167-4bb4-9b98-b3abe05e174f
3. Langfuse: userId=None sessionId=None tags=[]

/v1/chat/completions, header beats body, trace_name kept

1. curl ... $P/v1/chat/completions -H "langfuse_trace_user_id: $T-u-hdr-wins" -H "langfuse_session_id: $T-s-hdr-wins" -d "{\"model\":\"haiku\",$MSG,\"metadata\":{\"trace_user_id\":\"$T-u-body-loses\",\"session_id\":\"$T-s-body-loses\",\"trace_name\":\"$T-name-body\"}}"
2. status=200 call_id=9304994e-8179-4175-8833-6e45064aeccf
3. Langfuse: userId=None sessionId=None tags=[] name='a3b-name-body' (the fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name #40793 name works, user and session are missing)

/v1/chat/completions, no controls sent

1. curl ... $P/v1/chat/completions -d "{\"model\":\"haiku\",$MSG}"
2. status=200 call_id=384b211c-acec-4549-b80e-5ce982c25ced
3. Langfuse: userId=None sessionId=None tags=[]; sink root {}

/v1/chat/completions, stream: true with body metadata

1. curl ... $P/v1/chat/completions -d "{\"model\":\"haiku\",$MSG,\"stream\":true,\"metadata\":{\"trace_user_id\":\"$T-u-stream\",\"session_id\":\"$T-s-stream\",\"tags\":[\"$T-stream\"]}}", stream consumed to [DONE]
2. status=200 call_id=9447a595-c75e-4bdc-8c25-ccb0478ec7e1
3. Langfuse: userId=None sessionId=None tags=[]

/v1/messages with litellm_metadata

1. curl ... $P/v1/messages -d "{\"model\":\"haiku\",$MSG,\"litellm_metadata\":{\"trace_user_id\":\"$T-u-msgs\",\"session_id\":\"$T-s-msgs\",\"tags\":[\"$T-msgs\"]}}"
2. status=200 call_id=25f45357-a40d-434d-b005-affa2587fdbc
3. Langfuse: userId=None sessionId=None tags=[] name='POST /v1/messages'

/v1/responses with body metadata

1. curl ... $P/v1/responses -d "{\"model\":\"haiku\",\"input\":\"reply with one word: ok\",\"max_output_tokens\":16,\"metadata\":{\"trace_user_id\":\"$T-u-resp\",\"session_id\":\"$T-s-resp\",\"tags\":[\"$T-resp\"]}}"
2. status=200 call_id=4930fe50-2bc8-4aa0-916f-ab6b147ae5f1
3. Langfuse: userId=None sessionId=None tags=[] name='POST /v1/responses'

/v1/chat/completions, caller tries to spoof team and force trace ids

1. curl ... $P/v1/chat/completions -d "{\"model\":\"haiku\",$MSG,\"metadata\":{\"trace_user_id\":\"$T-u-spoof\",\"team_id\":\"spoofed-team\",\"team_alias\":\"spoofed-alias\",\"trace_metadata\":{\"team_id\":\"spoofed-team\"},\"trace_id\":\"forced-trace\",\"existing_trace_id\":\"forced-trace\"}}"
2. status=200 call_id=61932ad6-e291-4d27-9b8a-05c8de9c9cfa
3. Langfuse: userId=None sessionId=None tags=[] metadata.team_id='lit7718-real-team', trace id is the proxy's own
4. Sink generation: {"langfuse.trace.metadata.team_id": "lit7718-real-team"}, no langfuse.trace.id / existing_id / update_keys

OpenAI and Anthropic SDK clients

1. openai sync chat.completions.create(model="haiku", extra_body={"metadata": {"trace_user_id": "$T-u-k1", "session_id": "$T-s-k1", "tags": ["$T-k1"]}}): call_id=1b8be446-56ca-4285-a4ec-5439c23144a2, Langfuse userId=None sessionId=None tags=[]
2. openai async stream=True, consumed: call_id=3f03860c-cc89-488e-8e50-41990ceda47e, Langfuse userId=None sessionId=None tags=[]
3. anthropic sync messages.create(..., extra_body={"litellm_metadata": {...k3...}}): call_id=ed6f8411-218d-46f7-b790-6237c83d188b, Langfuse userId=None sessionId=None tags=[]
4. anthropic async stream, consumed, extra_headers={"langfuse_trace_user_id": "$T-u-k4", "langfuse_session_id": "$T-s-k4"}: call_id=f879db9e-aed5-45d7-a59e-c57fc0f59b1f, Langfuse userId=None sessionId=None
5. openai async responses.create(..., metadata={...k5...}): call_id=cd426761-dc4a-460d-a13a-ed1ff8c7f4b7, Langfuse userId=None sessionId=None tags=[]

direct SDK call (no proxy)

1. litellm.callbacks=["langfuse_otel"]; await litellm.acompletion(model="anthropic/claude-haiku-4-5", messages=[...], max_tokens=5, metadata={"trace_user_id": "$T-u-sdk", "session_id": "$T-s-sdk", "tags": ["$T-sdk-1", "$T-sdk-2"], "trace_name": "$T-sdk-name"}) from the base worktree, call_id=1d225ba0-5294-4714-abb4-73f49d77f94b
2. Exported generation span attributes: "langfuse.trace.name": "a3b-sdk-name" only, no user.id, session.id or langfuse.trace.tags
3. A second call without metadata (call_id=87c0ec22-7cf7-4b9a-a087-507d4de9e3df) exports none of the four
4. Destination readback: the SDK process printed Already shutdown, dropping span and neither span reached the sink or Langfuse, so this case is proven at the exported-span layer only

After (9238530)

The two commits after this hash carry no behavior change: 126e257062 merges current main (one conflict in otel/model/metadata.py, both sides kept) and 79aae7f062 rebuilds the same Langfuse attribute map from a tuple of pairs so the type discipline gate stays within budget. The OTel unit suite (735 tests) passes at the tip, and the Langfuse UI proof was rerun at 79aae7f with the same outcome: #41140 (comment)

/v1/chat/completions, body metadata: user + session + 3 tags

1. Same curl as Before
2. status=200 call_id=29052ee9-76fa-4acc-85a1-108bda031328
3. Langfuse: userId='a3-u-body' sessionId='a3-s-body' tags=['a3-tag-a', 'a3-tag-b', 'a3-tag-c'] name='POST /v1/chat/completions' metadata.team_id='lit7718-real-team'
4. Sink: root {"user.id": "a3-u-body", "session.id": "a3-s-body", "langfuse.trace.tags": ["a3-tag-a", "a3-tag-b", "a3-tag-c"]}, generation the same plus "langfuse.trace.metadata.team_id": "lit7718-real-team"

/v1/chat/completions, headers langfuse_trace_user_id + langfuse_session_id

1. Same curl as Before
2. status=200 call_id=368a9fd6-72c3-4d10-aa51-cff960cc771c
3. Langfuse: userId='a3-u-hdr' sessionId='a3-s-hdr' tags=[], both spans

/v1/chat/completions, header beats body, trace_name kept

1. Same curl as Before
2. status=200 call_id=43de61b9-ff4d-4a48-977e-46850c2a9ad3
3. Langfuse: userId='a3-u-hdr-wins' sessionId='a3-s-hdr-wins' tags=[] name='a3-name-body'

/v1/chat/completions, no controls sent

1. Same curl as Before
2. status=200 call_id=77dde379-75dc-4cb1-8f32-0c6d928a4e47
3. Langfuse: userId=None sessionId=None tags=[]; sink root {}, generation {"langfuse.trace.metadata.team_id": "lit7718-real-team"} (identical to Before)

/v1/chat/completions, stream: true with body metadata

1. Same curl as Before, stream consumed to [DONE]
2. status=200 call_id=a2500c9c-947c-48ab-9c29-19398a25f6f7
3. Langfuse: userId='a3-u-stream' sessionId='a3-s-stream' tags=['a3-stream']

/v1/messages with litellm_metadata

1. Same curl as Before
2. status=200 call_id=beb21225-9a93-41f0-9d78-a0ea24c678a6
3. Langfuse: userId='a3-u-msgs' sessionId='a3-s-msgs' tags=['a3-msgs'] name='POST /v1/messages'

/v1/responses with body metadata

1. Same curl as Before
2. status=200 call_id=75eadbb7-8cc3-4b4c-8495-f0952da770fe
3. Langfuse: userId='a3-u-resp' sessionId='a3-s-resp' tags=['a3-resp'] name='POST /v1/responses'

/v1/chat/completions, caller tries to spoof team and force trace ids

1. Same curl as Before
2. status=200 call_id=5150b2b4-8467-42b1-9bb1-7440aa1e7711
3. Langfuse: userId='a3-u-spoof' sessionId=None tags=[] metadata.team_id='lit7718-real-team', trace id is the proxy's own
4. Sink generation: {"langfuse.trace.metadata.team_id": "lit7718-real-team", "user.id": "a3-u-spoof"}; still no langfuse.trace.id / existing_id / update_keys, no spoofed team

OpenAI and Anthropic SDK clients

1. openai sync, same call: call_id=5118c23b-84f6-45e8-a4f2-59cf605c3fd6, Langfuse userId='a3-u-k1' sessionId='a3-s-k1' tags=['a3-k1']
2. openai async stream=True, consumed: call_id=20f9e5fd-1ad8-456e-83ee-62a434ac9efa, Langfuse userId='a3-u-k2' sessionId='a3-s-k2' tags=['a3-k2']
3. anthropic sync: call_id=e416eb84-2e0d-4df2-8bf9-b3593bcd4a27, Langfuse userId='a3-u-k3' sessionId='a3-s-k3' tags=['a3-k3']
4. anthropic async stream, extra_headers: call_id=5d3262de-99dc-4793-ae1d-1daca47ff33a, Langfuse userId='a3-u-k4' sessionId='a3-s-k4' tags=[] (headers carry the two scalars only, as designed)
5. openai async responses.create: call_id=3729f86b-9d78-47e6-8f59-788379fc54a0, Langfuse userId='a3-u-k5' sessionId='a3-s-k5' tags=['a3-k5']

direct SDK call (no proxy)

1. Same call as Before from the PR worktree, call_id=db3e1ffa-1b0c-4d97-9a91-a93db0bdebe8
2. Exported generation span attributes: "langfuse.trace.name": "a3-sdk-name", "user.id": "a3-u-sdk", "session.id": "a3-s-sdk", "langfuse.trace.tags": ["a3-sdk-1", "a3-sdk-2"]
3. A second call without metadata (call_id=305fe348-5483-434e-b29f-9b38bb402e87) exports none of the four
4. Destination readback: same Already shutdown, dropping span as Before, also after a rerun with a 15 s wait (db1074c7-576f-4c71-89cc-a6ffe1955fbc, 470da94f-29f2-4bf2-a9ed-9a09b131cf74); the direct-SDK case is proven at the exported-span layer only on both legs

Also run at the After hash and only there, all against the contract: empty and null controls emit nothing (c1b8ba5e-7f2b-4bd6-b2ca-ca2f36140ca8); a tags list with non-strings and empties keeps only the strings in order (b06d2716-aa26-4ff7-8e3f-f2427d01fe71); three identical requests give three traces with one generation each; 24 concurrent mixed requests across all three endpoints land exactly once each with controls, 0 duplicates, 0 missing; a master-key request with no team carries the controls and no team_id; an invalid provider key (401 to the caller, dccca70a-573a-4d2c-945d-ffbc5baef5b3) still carries the controls on the root and the failure generation. Chaos at the After hash with the same 4-worker rig: Langfuse stopped for 33 s during a 32-request burst gave 32/32 HTTP 200 and 31/32 traces (1 span batch dropped by the OTLP exporter while the destination was down); Langfuse paused for 18 s gave 32/32 and 32/32 with no duplicates; SIGTERM restart right after a 40-request burst gave 40/40 HTTP 200 and 21/40 traces (spans still queued in-process at SIGTERM are lost); kill -9 of the busiest worker mid 40-request burst gave 28 HTTP 200, 12 disconnected, 28/40 traces. The merge base showed the same exporter-queue loss shape in the same scenarios (32/32, 25/32, 0/40 and 28/40 with 12 disconnected), so those losses are pre-existing and are reported, not excused. Postgres outage chaos was not run (only the shared box service exists). Real Langfuse UI screenshots and the annotated recording of the After traces are in #41140 (comment)

Type

🐛 Bug Fix

Caveats (if any)

Medium

• A request the proxy rejects before calling a provider (bad model, wrong proxy key) gets a root trace carrying the caller's user.id and tags, no generation
  • Matches the legacy langfuse callback, which stamps user, session and tags on its error trace too
  • Means an unauthenticated caller can pick the user id and tags on that failure trace; say if these requests should be excluded
• tags sent as a string instead of a list makes the shared logging payload fail ('str' object has no attribute 'copy'), so no generation reaches any logger
  • Pre-existing on the merge base, identical here, request still returns 200; follow-up ticket
• Under a destination outage or a proxy restart, spans still in the exporter queue are lost; same on the merge base

Low

• Tags come only from the body (metadata.tags / litellm_metadata.tags); there is no langfuse_tags header, same as before
• Non-string or empty tag entries are dropped; scalars other than strings are stringified, like trace_name
• The mutation controls from fix(otel): map Langfuse trace user, session, name, and tags in the v2 mapper #35210 stay out on purpose; a follow-up can add them with their own tests
• A key with no team still falls back to the caller's bare metadata.team_id for langfuse.trace.metadata.team_id; pre-existing, identical on both legs, untouched here
• LLMCallEvent.trace_name became LLMCallEvent.trace, and LLMCallSpanData.from_standard_logging_payload(trace_name=...) became trace=...; both shipped only in v1.102.0-rc.1 via fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name #40793 and every caller lives in litellm/integrations/otel
• Direct SDK (no proxy) is proven on the exported span, not at the destination: the process drops its spans at shutdown on both legs
• The opentelemetry_v2.md mapper table in litellm-docs does not list these attributes yet; docs follow-up
• Veria posted no check or review on this PR (unavailable, not counted as a pass); the Devin Review "No Issues Found" is from the first commit, not re-run at the tip

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

ran /live-pr-risk and found no regressions/backward incompatible risks

Link to Devin session: https://app.devin.ai/sessions/1d3bdadbd958454796014f11b7033854
Open in Devin Desktop: https://app.devin.ai/desktop/session/1d3bdadbd958454796014f11b7033854?variant=devin
Requested by: @yucheng-berri

---

Note

Low Risk
Changes are confined to OTel v2 Langfuse attribute mapping and parsing, with tests covering header precedence and team spoofing; no auth or billing logic changes.

Overview
OTel v2 Langfuse now propagates the caller’s user, session, and tags (not just trace name) onto both the proxy root observation and the generation span.

Parsing is centralized in a new TraceControls type and caller_trace_controls, which read metadata / litellm_metadata and prefer langfuse_* proxy headers for scalars; mutation fields like trace_id are ignored. LangfuseMapper.trace_attributes maps these to user.id, session.id, langfuse.trace.tags, and langfuse.trace.name, used at pre-call on the root span and on LLMCallSpanData.trace when the LLM span closes. Unset controls are omitted; proxy team metadata is unchanged.

^{Reviewed by Cursor Bugbot for commit 79aae7f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_otel_v2_langfuse_user_session_tags (79aae7f) with main (0add8c0)}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness, security, or repository-rule findings

Summary

This PR restores caller-provided Langfuse trace controls in the OTel v2 integration

• Parses trace name, user ID, session ID, and tags into a typed TraceControls value
• Applies the controls consistently to root and generation spans while omitting unset values
• Preserves proxy-owned team identity and adds regression coverage for body, header, streaming, and empty-control cases

_{Reviews (3) · Last reviewed commit: "refactor(otel v2): build Langfuse trace ..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

Langfuse UI at the After hash 9238530: user, session, tags on trace, root and generation, header precedence, team preserved, tag filter

[devin-ai-integration]

Langfuse UI proof rerun at tip 79aae7f: body controls, header precedence, empty fields, streaming Messages and tag filter all match the 9238530 run

Screenshots

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

Bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 79aae7f. Configure here.}

[devin-ai-integration]

Gap rerun at 79aae7f: team spoofing via metadata and headers is rejected; error traces lack a call id on tip and merge base alike

Screenshots