fix(otel): promote litellm.request.model via baggage instead of gen_ai.request.model - #35228
Conversation
…i.request.model Baggage promotion stamped canonical gen_ai.request.model onto SERVER, DB, and service spans, causing GenAI backends to misclassify non-LLM operations. Use litellm.request.model for cross-span correlation; LLM-call spans still receive gen_ai.request.model from the GenAI mapper. Fixes BerriAI#35193
|
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| LiteLLM.KEY_HASH: lambda identity, model, team_metadata_keys: identity.key_hash, | ||
| LiteLLM.END_USER: lambda identity, model, team_metadata_keys: identity.end_user, | ||
| GenAI.REQUEST_MODEL: lambda identity, model, team_metadata_keys: model, | ||
| LiteLLM.REQUEST_MODEL: lambda identity, model, team_metadata_keys: model, |
There was a problem hiding this comment.
When an existing deployment explicitly configures baggage_promoted_keys with gen_ai.request.model, promoted_baggage no longer has an extractor for that configured key and silently omits it, causing downstream non-LLM span processors and correlations to lose the request model after upgrading. Please retain compatibility for the old configured value or reject it with an actionable configuration error.
Rule Used: What: avoid backwards-incompatible changes without... (source)
Greptile SummaryUpdates OpenTelemetry baggage model correlation semantics.
Confidence Score: 4/5The explicit legacy baggage-allowlist behavior needs compatibility handling or validation before this PR is safe to merge. Existing deployments can configure Files Needing Attention: litellm/integrations/otel/model/baggage.py
|
| Filename | Overview |
|---|---|
| litellm/integrations/otel/model/baggage.py | Changes the request-model promotion key, but existing explicit allowlists using the former key are now silently ignored. |
| litellm/integrations/otel/model/semconv.py | Defines and documents the new vendor-specific request-model attribute. |
| tests/test_litellm/integrations/otel/test_otel_v2_baggage.py | Verifies the intended span attributes, but does not cover compatibility with explicitly configured legacy allowlists. |
| tests/test_litellm/integrations/otel/test_otel_v2_sources_of_truth.py | Updates the bounded default-allowlist assertion to the new key. |
Reviews (1): Last reviewed commit: "fix(otel): promote litellm.request.model..." | Re-trigger Greptile
|
recheck |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Deployments that still list gen_ai.request.model in baggage_promoted_keys keep model correlation via litellm.request.model instead of silently dropping it.
|
Addressed the Greptile P1: legacy |
LIT001 rejected list[str]/set[str] locals in _normalize_promoted_keys. Use tuple-only dedupe so the type-discipline budget stays flat.
|
Closing this PR to keep the LiteLLM surface focused on the higher-value streaming fix in #35349. This otel baggage change no longer justifies a second conflicting PR on |
2 participants
Summary
gen_ai.request.modelthrough OpenTelemetry Baggage by default.litellm.request.modelinstead so SERVER / DB / service / guardrail spans can correlate the inbound model without being classified as GenAI operations.gen_ai.request.modelfrom the GenAI mapper.Fixes #35193
Notes
litellm_internal_stagingas required for external fork contributions.Test plan
pytest tests/test_litellm/integrations/otel/test_otel_v2_baggage.pypytest tests/test_litellm/integrations/otel/test_otel_v2_sources_of_truth.py::test_promoted_baggage_is_bounded_allowlistpytest tests/test_litellm/integrations/otel/test_otel_v2_emitter.py::test_llm_call_span_goldenpytest tests/test_litellm/integrations/otel/test_otel_v2_config_baggage_parenting_guardrails.py