[Feat] Honor OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT - #27403
Merged
yuneng-berri merged 2 commits intoMay 9, 2026
Merged
Conversation
|
Michael Riad Zaky seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
Greptile SummaryThis PR adds first-class support for
Confidence Score: 5/5Safe to merge — the change is narrowly scoped to the OTEL content-capture path, and the new capture-mode logic is fully unit-tested with isolation guards on existing tests. All content-capture gates were consistently updated to use the new helpers, the previously-noted gap with "false"/"0" is correctly handled, and no behavioral regression is introduced for users who never set the new env var or config field. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/integrations/opentelemetry.py | Adds _compute_capture_mode_from_init_state / _resolve_capture_mode / _capture_in_span / _capture_in_event helpers, caches the resolved mode at init, and updates every content-capture gate to use the new helpers. Logic is sound; "false"/"0" → NO_CONTENT is correctly handled. |
| tests/test_litellm/integrations/test_opentelemetry.py | Adds TestOpenTelemetryCaptureMessageContent covering precedence, legacy boolean mapping, per-handler isolation, and kill-switch behavior. Existing tests are hardened with @patch.dict / setUp/tearDown to isolate from a shell-set env var without weakening any assertions. |
Reviews (2): Last reviewed commit: "Fix OTEL false content capture mode" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Setting env var to "false" silently enables content capture
- Mapped legacy boolean false values "false" and "0" to NO_CONTENT and added coverage to prevent fallback to message logging.
Preview (492118de25)
diff --git a/litellm/integrations/opentelemetry.py b/litellm/integrations/opentelemetry.py
--- a/litellm/integrations/opentelemetry.py
+++ b/litellm/integrations/opentelemetry.py
@@ -57,7 +57,18 @@
RAW_REQUEST_SPAN_NAME = "raw_gen_ai_request"
LITELLM_REQUEST_SPAN_NAME = "litellm_request"
+CAPTURE_MODE_NO_CONTENT = "NO_CONTENT"
+CAPTURE_MODE_SPAN_ONLY = "SPAN_ONLY"
+CAPTURE_MODE_EVENT_ONLY = "EVENT_ONLY"
+CAPTURE_MODE_SPAN_AND_EVENT = "SPAN_AND_EVENT"
+_VALID_CAPTURE_MODES = {
+ CAPTURE_MODE_NO_CONTENT,
+ CAPTURE_MODE_SPAN_ONLY,
+ CAPTURE_MODE_EVENT_ONLY,
+ CAPTURE_MODE_SPAN_AND_EVENT,
+}
+
@dataclass
class OpenTelemetryConfig:
exporter: Union[str, SpanExporter] = "console"
@@ -71,6 +82,9 @@
ignore_context_propagation: Optional[bool] = None
# When True, create a private TracerProvider instead of reusing or setting the global one.
skip_set_global: bool = False
+ # Programmatic override for OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT.
+ # One of NO_CONTENT, SPAN_ONLY, EVENT_ONLY, SPAN_AND_EVENT (or "true" as legacy alias).
+ capture_message_content: Optional[str] = None
def __post_init__(self) -> None:
# If endpoint is specified but exporter is still the default "console",
@@ -182,6 +196,9 @@
super().__init__(**kwargs)
self._init_metrics(meter_provider)
self._init_logs(logger_provider)
+ # Sample env-var / config / message_logging at init so subsequent
+ # _capture_in_span / _capture_in_event calls are deterministic.
+ self._capture_mode_cached = self._compute_capture_mode_from_init_state()
self._init_otel_logger_on_litellm_proxy()
@staticmethod
@@ -306,6 +323,62 @@
hasattr(self, "callback_name") and self.callback_name == "langfuse_otel"
)
+ def _compute_capture_mode_from_init_state(self) -> Optional[str]:
+ """Sample explicit settings at init. Returns the resolved mode or
+ None if nothing explicit is set (in which case the legacy
+ ``self.message_logging`` flag is consulted dynamically per request).
+
+ ``"true"``/``"1"`` map to ``EVENT_ONLY`` per the contrib convention.
+ ``"false"``/``"0"`` map to ``NO_CONTENT``.
+ Unknown values are ignored.
+ """
+ explicit = self.config.capture_message_content or os.getenv(
+ "OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"
+ )
+ if not explicit:
+ return None
+ normalized = explicit.upper()
+ if normalized in ("TRUE", "1"):
+ return CAPTURE_MODE_EVENT_ONLY
+ if normalized in ("FALSE", "0"):
+ return CAPTURE_MODE_NO_CONTENT
+ if normalized in _VALID_CAPTURE_MODES:
+ return normalized
+ return None
+
+ def _resolve_capture_mode(self) -> str:
+ """Return the active capture mode for this request.
+
+ Precedence:
+ 1. ``litellm.turn_off_message_logging=True`` forces ``NO_CONTENT``
+ (kill-switch checked dynamically).
+ 2. Explicit setting sampled at init from
+ ``OpenTelemetryConfig.capture_message_content`` or
+ ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT``.
+ 3. Legacy ``self.message_logging`` (checked dynamically).
+ """
+ if litellm.turn_off_message_logging:
+ return CAPTURE_MODE_NO_CONTENT
+ if self._capture_mode_cached is not None:
+ return self._capture_mode_cached
+ return (
+ CAPTURE_MODE_SPAN_AND_EVENT
+ if self.message_logging
+ else CAPTURE_MODE_NO_CONTENT
+ )
+
+ def _capture_in_span(self) -> bool:
+ return self._resolve_capture_mode() in (
+ CAPTURE_MODE_SPAN_ONLY,
+ CAPTURE_MODE_SPAN_AND_EVENT,
+ )
+
+ def _capture_in_event(self) -> bool:
+ return self._resolve_capture_mode() in (
+ CAPTURE_MODE_EVENT_ONLY,
+ CAPTURE_MODE_SPAN_AND_EVENT,
+ )
+
def _init_tracing(self, tracer_provider):
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
@@ -825,8 +898,7 @@
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
- # only log raw LLM request/response if message_logging is on and not globally turned off
- if litellm.turn_off_message_logging or not self.message_logging:
+ if not self._capture_in_span():
return
litellm_params = kwargs.get("litellm_params", {})
@@ -1117,9 +1189,14 @@
}
if role == "tool" and msg.get("id"):
attrs["id"] = msg["id"]
- if self.message_logging and msg.get("content"):
+ capture_event_content = self._capture_in_event()
+ if capture_event_content and msg.get("content"):
attrs["gen_ai.prompt"] = msg["content"]
+ body = msg.copy()
+ if not capture_event_content:
+ body.pop("content", None)
+
log_record = SdkLogRecord(
timestamp=self._to_ns(datetime.now()),
trace_id=parent_ctx.trace_id,
@@ -1127,7 +1204,7 @@
trace_flags=parent_ctx.trace_flags,
severity_number=SeverityNumber.INFO,
severity_text="INFO",
- body=msg.copy(),
+ body=body,
attributes=attrs,
)
otel_logger.emit(log_record)
@@ -1141,14 +1218,15 @@
"finish_reason": choice.get("finish_reason"),
}
body_msg = choice.get("message", {})
- if self.message_logging and body_msg.get("content"):
+ capture_event_content = self._capture_in_event()
+ if capture_event_content and body_msg.get("content"):
attrs["message.content"] = body_msg["content"]
body = {
"index": idx,
"finish_reason": choice.get("finish_reason"),
"message": {"role": body_msg.get("role", "assistant")},
}
- if self.message_logging and body_msg.get("content"):
+ if capture_event_content and body_msg.get("content"):
body["message"]["content"] = body_msg["content"]
log_record = SdkLogRecord(
@@ -1674,10 +1752,8 @@
########## LLM Request Medssages / tools / content Attributes ###########
#########################################################################
- if litellm.turn_off_message_logging is True:
+ if not self._capture_in_span():
return
- if self.message_logging is not True:
- return
if optional_params.get("tools"):
tools = optional_params["tools"]
diff --git a/tests/test_litellm/integrations/test_opentelemetry.py b/tests/test_litellm/integrations/test_opentelemetry.py
--- a/tests/test_litellm/integrations/test_opentelemetry.py
+++ b/tests/test_litellm/integrations/test_opentelemetry.py
@@ -442,6 +442,109 @@
)
+class TestOpenTelemetryCaptureMessageContent(unittest.TestCase):
+ """OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT and the
+ OpenTelemetryConfig.capture_message_content programmatic override
+ drive what the handler captures in spans vs events."""
+
+ @staticmethod
+ def _make(env=None, config_value=None, message_logging=True):
+ env_dict = (
+ {"OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT": env}
+ if env is not None
+ else {"OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT": ""}
+ )
+ with patch.dict(os.environ, env_dict):
+ handler = OpenTelemetry(
+ config=OpenTelemetryConfig(
+ exporter="console", capture_message_content=config_value
+ )
+ )
+ handler.message_logging = message_logging
+ return handler, handler._resolve_capture_mode()
+
+ def test_no_explicit_setting_falls_back_to_message_logging_true(self):
+ _, mode = self._make()
+ self.assertEqual(mode, "SPAN_AND_EVENT")
+
+ def test_no_explicit_setting_falls_back_to_message_logging_false(self):
+ _, mode = self._make(message_logging=False)
+ self.assertEqual(mode, "NO_CONTENT")
+
+ def test_env_var_no_content(self):
+ _, mode = self._make(env="NO_CONTENT")
+ self.assertEqual(mode, "NO_CONTENT")
+
+ def test_env_var_span_only(self):
+ _, mode = self._make(env="SPAN_ONLY")
+ self.assertEqual(mode, "SPAN_ONLY")
+
+ def test_env_var_event_only(self):
+ _, mode = self._make(env="EVENT_ONLY")
+ self.assertEqual(mode, "EVENT_ONLY")
+
+ def test_env_var_span_and_event(self):
+ _, mode = self._make(env="SPAN_AND_EVENT")
+ self.assertEqual(mode, "SPAN_AND_EVENT")
+
+ def test_env_var_legacy_true_maps_to_event_only(self):
+ _, mode = self._make(env="true")
+ self.assertEqual(mode, "EVENT_ONLY")
+
+ def test_env_var_legacy_false_maps_to_no_content(self):
+ for env in ("false", "0"):
+ with self.subTest(env=env):
+ _, mode = self._make(env=env)
+ self.assertEqual(mode, "NO_CONTENT")
+
+ def test_env_var_unknown_value_falls_through_to_legacy(self):
+ _, mode = self._make(env="garbage", message_logging=True)
+ self.assertEqual(mode, "SPAN_AND_EVENT")
+
+ def test_config_field_overrides_env(self):
+ _, mode = self._make(env="EVENT_ONLY", config_value="SPAN_ONLY")
+ self.assertEqual(mode, "SPAN_ONLY")
+
+ def test_turn_off_message_logging_forces_no_content(self):
+ with patch("litellm.turn_off_message_logging", True):
+ _, mode = self._make(env="SPAN_AND_EVENT", message_logging=True)
+ self.assertEqual(mode, "NO_CONTENT")
+
+ def test_capture_in_span_and_event_predicates(self):
+ cases = {
+ "NO_CONTENT": (False, False),
+ "SPAN_ONLY": (True, False),
+ "EVENT_ONLY": (False, True),
+ "SPAN_AND_EVENT": (True, True),
+ }
+ for mode, (in_span, in_event) in cases.items():
+ handler, _ = self._make(env=mode)
+ self.assertEqual(handler._capture_in_span(), in_span, msg=mode)
+ self.assertEqual(handler._capture_in_event(), in_event, msg=mode)
+
+ def test_two_handlers_can_have_different_modes(self):
+ # FIL's stated requirement: one handler strips content, the other keeps it.
+ with patch.dict(
+ os.environ, {"OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT": ""}
+ ):
+ stripped = OpenTelemetry(
+ config=OpenTelemetryConfig(
+ exporter="console", capture_message_content="NO_CONTENT"
+ )
+ )
+ kept = OpenTelemetry(
+ config=OpenTelemetryConfig(
+ exporter="console", capture_message_content="SPAN_AND_EVENT"
+ )
+ )
+ self.assertEqual(stripped._resolve_capture_mode(), "NO_CONTENT")
+ self.assertEqual(kept._resolve_capture_mode(), "SPAN_AND_EVENT")
+ self.assertFalse(stripped._capture_in_span())
+ self.assertFalse(stripped._capture_in_event())
+ self.assertTrue(kept._capture_in_span())
+ self.assertTrue(kept._capture_in_event())
+
+
class TestOpenTelemetry(unittest.TestCase):
POLL_INTERVAL = 0.05
POLL_TIMEOUT = 2.0
@@ -1067,6 +1170,7 @@
result = otel._get_span_name(kwargs)
self.assertEqual(result, LITELLM_REQUEST_SPAN_NAME)
+ @patch.dict(os.environ, {"OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT": ""})
@patch("litellm.turn_off_message_logging", False)
def test_maybe_log_raw_request_creates_span(self):
"""Test _maybe_log_raw_request creates span when logging enabled"""
@@ -2194,6 +2298,19 @@
See: https://github.com/BerriAI/litellm/issues/17794
"""
+ def setUp(self):
+ # Insulate from a shell-set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT
+ # so these tests exercise the legacy default path (message_logging=True).
+ self._prev = os.environ.pop(
+ "OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT", None
+ )
+
+ def tearDown(self):
+ if self._prev is not None:
+ os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = (
+ self._prev
+ )
+
def test_input_messages_uses_parts_structure(self):
"""
Test that gen_ai.input.messages uses the OTEL 1.38 parts array structure.You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 581ae14. Configure here.
Contributor
Author
|
@greptile-ai |
yassinkortam
approved these changes
May 8, 2026
yuneng-berri
enabled auto-merge
May 9, 2026 01:12
yuneng-berri
approved these changes
May 9, 2026
yuneng-berri
merged commit May 9, 2026
0824c4c
into
litellm_internal_staging
115 of 119 checks passed
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…eMessageContent [Feat] Honor OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Expands the OpenTelemetry integration to use the OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT environment variable and a per-handler OpenTelemetryConfig.capture_message_content field to control whether prompts and completions are captured on spans, events, both, or neither. This follows the convention used by the official OTEL Python GenAI libraries:
opentelemetry-util-genai PyPI
opentelemetry-instrumentation-openai-v2 PyPI
opentelemetry-instrumentation-google-genai PyPI
opentelemetry-python-contrib GitHub repo
Linear ticket
Resolves part of LIT-2707.
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewScreenshots / Proof of Fix
Output from an example script testing config combinations:
Type
🆕 New Feature
✅ Test
Changes
opentelemetry.py
Note
Medium Risk
Changes how prompts/completions are captured into OpenTelemetry spans vs log events, which impacts what sensitive user content may be exported and can alter observability payloads across deployments.
Overview
Adds first-class support for
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENTandOpenTelemetryConfig.capture_message_contentto control whether prompt/completion message content is captured in spans, events, both, or neither (with legacytrue/falsemapping). The resolved mode is cached per handler for deterministic behavior, whilelitellm.turn_off_message_loggingremains a dynamic kill-switch.Updates raw-request span creation, span attribute population (messages/tools), and semantic log bodies/attributes to respect the new capture mode (including stripping
contentfrom event bodies when disabled). Expands tests to cover precedence, legacy mappings, per-handler isolation, and to insulate existing semantic-convention tests from shell-set env vars.Reviewed by Cursor Bugbot for commit 492118d. Bugbot is set up for automated code reviews on this repo. Configure here.