diff --git a/docs/proxy/config_settings.md b/docs/proxy/config_settings.md index b64d68e7c..5787acd68 100644 --- a/docs/proxy/config_settings.md +++ b/docs/proxy/config_settings.md @@ -38,6 +38,7 @@ litellm_settings: # Debugging - see debugging docs for more options # Use `--debug` or `--detailed_debug` CLI flags, or set LITELLM_LOG env var to "INFO", "DEBUG", or "ERROR" json_logs: boolean # if true, logs will be in json format + request_correlation_in_logs: boolean # if true, stamps every log line with the request's trace_id and session_id # Fallbacks, reliability default_fallbacks: ["claude-opus"] # set default_fallbacks, in case a specific model group is misconfigured / bad. @@ -206,6 +207,7 @@ router_settings: | langfuse_default_tags | array of strings | Default tags for Langfuse Logging. Use this if you want to control which LiteLLM-specific fields are logged as tags by the LiteLLM proxy. By default LiteLLM Proxy logs no LiteLLM-specific fields as tags. [Further docs](./logging#litellm-specific-tags-on-langfuse---cache_hit-cache_key) | | set_verbose | boolean | [DEPRECATED - see debugging docs](./debugging) Use `--debug` or `--detailed_debug` CLI flags, or set `LITELLM_LOG` env var to "INFO", "DEBUG", or "ERROR" instead. | | json_logs | boolean | If true, logs will be in json format. If you need to store the logs as JSON, just set the `litellm.json_logs = True`. We currently just log the raw POST request from litellm as a JSON [Further docs](./debugging) | +| request_correlation_in_logs | boolean | If true, stamps every log line (plaintext or JSON) with the request's `trace_id` and `session_id`, and adds a `session_id` field to `StandardLoggingPayload`. [Further docs](./debugging#request-correlation-ids) | | default_fallbacks | array of strings | List of fallback models to use if a specific model group is misconfigured / bad. [Further docs](./reliability#default-fallbacks) | | request_timeout | integer | The timeout for requests in seconds. If not set, the default value is `6000 seconds`. [For reference OpenAI Python SDK defaults to `600 seconds`.](https://github.com/openai/openai-python/blob/main/src/openai/_constants.py) | | force_ipv4 | boolean | If true, litellm will force ipv4 for all LLM requests. Some users have seen httpx ConnectionError when using ipv6 + Anthropic API | diff --git a/docs/proxy/debugging.md b/docs/proxy/debugging.md index fbcac24a4..eb714cd5e 100644 --- a/docs/proxy/debugging.md +++ b/docs/proxy/debugging.md @@ -113,6 +113,39 @@ $ litellm The proxy will now all logs in json format. +## Request Correlation IDs + +Set `request_correlation_in_logs: true` to stamp every log line with the request's `trace_id` and `session_id`. This lets you filter your log aggregator down to every line tied to a single request, or every request in a single end-user session, without adding logging calls at each call site. Works with both plaintext and JSON logs + +```yaml showLineNumbers +litellm_settings: + request_correlation_in_logs: true +``` + +`trace_id` comes from the `x-litellm-trace-id` request header (or is generated per request if the header isn't set). `session_id` comes from `litellm_session_id` in the request body, or from the `x-litellm-session-id` header; see [Request Headers](./request_headers#litellm-headers) for the full resolution order. It's only added to a log line once a session id has actually been supplied + +If neither of those explicit headers is present, `trace_id`/`session_id` fall back to the standard [W3C Trace Context](https://www.w3.org/TR/trace-context/) `traceparent` header and [W3C Baggage](https://www.w3.org/TR/baggage/) `baggage` header respectively: the trace-id from `traceparent` becomes `trace_id`, and a `session.id` entry in `baggage` becomes `session_id`. This lets a request already carrying real OpenTelemetry trace context correlate litellm's logs with the same trace in your existing observability backend, without needing a separate litellm-specific header. The explicit litellm headers always take precedence when present + +Example log line with `json_logs: true`: + +```json showLineNumbers +{"message": "...", "level": "INFO", "timestamp": "...", "trace_id": "2a5cbcfa-ccdf-493c-858b-eb8e9b07f32c", "session_id": "user-123-session-1"} +``` + +Example log line without `json_logs` (plaintext): + +```bash showLineNumbers +15:30:43 - LiteLLM Proxy:ERROR: common_request_processing.py:848 - some log message [trace_id=2a5cbcfa-ccdf-493c-858b-eb8e9b07f32c session_id=user-123-session-1] +``` + +`request_correlation_in_logs` also adds an independent `session_id` field to `StandardLoggingPayload` (sent to logging integrations like S3 and Langfuse), populated the same way. See the [StandardLoggingPayload spec](./logging_spec#standardloggingpayload) + +Both `trace_id` and `session_id` are sanitized before they're stored: control characters (including `\r`/`\n`) are stripped and the value is capped at 256 characters, so a caller-supplied `litellm_session_id` can't be used to forge fake log lines or bloat log storage + +The flag defaults to `false`, so existing log output is unaffected until you opt in. + +This currently applies to requests handled by the proxy and to `litellm.acompletion()` calls made directly through the SDK. Synchronous SDK calls (`litellm.completion()`) don't stamp `trace_id`/`session_id` yet; support for that path is planned as a follow-up. + ## Control Log Output Turn off fastapi's default 'INFO' logs diff --git a/docs/proxy/logging_spec.md b/docs/proxy/logging_spec.md index 2786a75b4..894185ae2 100644 --- a/docs/proxy/logging_spec.md +++ b/docs/proxy/logging_spec.md @@ -9,6 +9,7 @@ Found under `kwargs["standard_logging_object"]`. This is a standard payload, log |-------|------|-------------| | `id` | `str` | Unique identifier | | `trace_id` | `str` | Trace multiple LLM calls belonging to same overall request | +| `session_id` | `str` | End-user/conversation session id, from `litellm_session_id`. Independent of `trace_id`; only populated when `litellm_settings.request_correlation_in_logs` is enabled. [Further docs](./debugging#request-correlation-ids) | | `call_type` | `str` | Type of call | | `response_cost` | `float` | Cost of the response in USD ($) | | `cost_breakdown` | `Optional[CostBreakdown]` | Detailed cost breakdown object |