Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/proxy/config_settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ litellm_settings:
# Debugging - see debugging docs for more options
# Use `--debug` or `--detailed_debug` CLI flags, or set LITELLM_LOG env var to "INFO", "DEBUG", or "ERROR"
json_logs: boolean # if true, logs will be in json format
request_correlation_in_logs: boolean # if true, stamps every log line with the request's trace_id and session_id

# Fallbacks, reliability
default_fallbacks: ["claude-opus"] # set default_fallbacks, in case a specific model group is misconfigured / bad.
Expand Down Expand Up @@ -206,6 +207,7 @@ router_settings:
| langfuse_default_tags | array of strings | Default tags for Langfuse Logging. Use this if you want to control which LiteLLM-specific fields are logged as tags by the LiteLLM proxy. By default LiteLLM Proxy logs no LiteLLM-specific fields as tags. [Further docs](./logging#litellm-specific-tags-on-langfuse---cache_hit-cache_key) |
| set_verbose | boolean | [DEPRECATED - see debugging docs](./debugging) Use `--debug` or `--detailed_debug` CLI flags, or set `LITELLM_LOG` env var to "INFO", "DEBUG", or "ERROR" instead. |
| json_logs | boolean | If true, logs will be in json format. If you need to store the logs as JSON, just set the `litellm.json_logs = True`. We currently just log the raw POST request from litellm as a JSON [Further docs](./debugging) |
| request_correlation_in_logs | boolean | If true, stamps every log line (plaintext or JSON) with the request's `trace_id` and `session_id`, and adds a `session_id` field to `StandardLoggingPayload`. [Further docs](./debugging#request-correlation-ids) |
| default_fallbacks | array of strings | List of fallback models to use if a specific model group is misconfigured / bad. [Further docs](./reliability#default-fallbacks) |
| request_timeout | integer | The timeout for requests in seconds. If not set, the default value is `6000 seconds`. [For reference OpenAI Python SDK defaults to `600 seconds`.](https://github.com/openai/openai-python/blob/main/src/openai/_constants.py) |
| force_ipv4 | boolean | If true, litellm will force ipv4 for all LLM requests. Some users have seen httpx ConnectionError when using ipv6 + Anthropic API |
Expand Down
33 changes: 33 additions & 0 deletions docs/proxy/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,39 @@ $ litellm

The proxy will now all logs in json format.

## Request Correlation IDs

Set `request_correlation_in_logs: true` to stamp every log line with the request's `trace_id` and `session_id`. This lets you filter your log aggregator down to every line tied to a single request, or every request in a single end-user session, without adding logging calls at each call site. Works with both plaintext and JSON logs

```yaml showLineNumbers
litellm_settings:
request_correlation_in_logs: true
```

`trace_id` comes from the `x-litellm-trace-id` request header (or is generated per request if the header isn't set). `session_id` comes from `litellm_session_id` in the request body, or from the `x-litellm-session-id` header; see [Request Headers](./request_headers#litellm-headers) for the full resolution order. It's only added to a log line once a session id has actually been supplied

If neither of those explicit headers is present, `trace_id`/`session_id` fall back to the standard [W3C Trace Context](https://www.w3.org/TR/trace-context/) `traceparent` header and [W3C Baggage](https://www.w3.org/TR/baggage/) `baggage` header respectively: the trace-id from `traceparent` becomes `trace_id`, and a `session.id` entry in `baggage` becomes `session_id`. This lets a request already carrying real OpenTelemetry trace context correlate litellm's logs with the same trace in your existing observability backend, without needing a separate litellm-specific header. The explicit litellm headers always take precedence when present

Example log line with `json_logs: true`:

```json showLineNumbers
{"message": "...", "level": "INFO", "timestamp": "...", "trace_id": "2a5cbcfa-ccdf-493c-858b-eb8e9b07f32c", "session_id": "user-123-session-1"}
```

Example log line without `json_logs` (plaintext):

```bash showLineNumbers
15:30:43 - LiteLLM Proxy:ERROR: common_request_processing.py:848 - some log message [trace_id=2a5cbcfa-ccdf-493c-858b-eb8e9b07f32c session_id=user-123-session-1]
```

`request_correlation_in_logs` also adds an independent `session_id` field to `StandardLoggingPayload` (sent to logging integrations like S3 and Langfuse), populated the same way. See the [StandardLoggingPayload spec](./logging_spec#standardloggingpayload)

Both `trace_id` and `session_id` are sanitized before they're stored: control characters (including `\r`/`\n`) are stripped and the value is capped at 256 characters, so a caller-supplied `litellm_session_id` can't be used to forge fake log lines or bloat log storage

The flag defaults to `false`, so existing log output is unaffected until you opt in.

This currently applies to requests handled by the proxy and to `litellm.acompletion()` calls made directly through the SDK. Synchronous SDK calls (`litellm.completion()`) don't stamp `trace_id`/`session_id` yet; support for that path is planned as a follow-up.

## Control Log Output

Turn off fastapi's default 'INFO' logs
Expand Down
1 change: 1 addition & 0 deletions docs/proxy/logging_spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Found under `kwargs["standard_logging_object"]`. This is a standard payload, log
|-------|------|-------------|
| `id` | `str` | Unique identifier |
| `trace_id` | `str` | Trace multiple LLM calls belonging to same overall request |
| `session_id` | `str` | End-user/conversation session id, from `litellm_session_id`. Independent of `trace_id`; only populated when `litellm_settings.request_correlation_in_logs` is enabled. [Further docs](./debugging#request-correlation-ids) |
| `call_type` | `str` | Type of call |
| `response_cost` | `float` | Cost of the response in USD ($) |
| `cost_breakdown` | `Optional[CostBreakdown]` | Detailed cost breakdown object |
Expand Down