fix(tracing): unify ClickHouse storage configuration

[yujonglee-berri]

TLDR

Problem this solves:

• One ClickHouse URL leaves trace reads unavailable
• Trace and spend retention use separate environment settings

How it solves it:

• Add tracing.store.type, url, database, and retention_days
• Use one URL for writes and bounded reads
• Apply one 14-day default TTL to trace and spend tables
• Keep literal defaults in litellm/constants.py, read CLICKHOUSE_DATABASE and AGENT_TRACING_RETENTION_DAYS at startup
• Fail tracing startup when a YAML os.environ/ reference is unset

Intentional product change: a separate ClickHouse reader URL is no longer used, AGENT_TRACING_SPEND_LOG_RETENTION_DAYS is gone, and the string form store: clickhouse is rejected. None of these were in a release

User Flow

Before: an admin configures tracing with one ClickHouse URL, but trace reads are unavailable

1. They set general_settings.tracing.store: clickhouse and CLICKHOUSE_URL, then start the proxy
2. They call GET http://localhost:4004/v1/traces with an admin key and receive HTTP 501

After: the same ClickHouse connection serves trace writes and reads

1. They set general_settings.tracing.store.type: clickhouse, store.url, and store.retention_days: 14, then start the proxy
2. They call GET http://localhost:4003/v1/traces with an admin key and receive HTTP 200 with a trace page
3. They send POST http://localhost:4003/v1/traces with an OTLP JSON span and retrieve it by trace ID with HTTP 200

Pre-Submission checklist

• I have added meaningful tests
• The focused Python and Rust test files pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible
• I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Both runs used the local ClickHouse and PostgreSQL Compose services, one CLICKHOUSE_URL, and no CLICKHOUSE_READER_URL

Before (0f6a06a)

1. Start the proxy with general_settings.tracing.store: clickhouse. Startup logs Agent tracing unavailable: 'CLICKHOUSE_READER_URL'
2. Run curl -H "Authorization: Bearer $LITELLM_MASTER_KEY" http://127.0.0.1:4004/v1/traces. The response is HTTP 501 with Agent tracing is not enabled

After (235d270)

1. Start the proxy with tracing.store.type: clickhouse, tracing.store.url: os.environ/CLICKHOUSE_URL, and tracing.store.retention_days: 14. Startup completes without a tracing warning
2. Run the same GET /v1/traces request on port 4003. The response is HTTP 200 with data and next_cursor
3. Send an OTLP JSON span to POST /v1/traces, then request GET /v1/traces/91913eaedfa5643a9a1acd15f512a6f8. Both responses are HTTP 200, and the detail contains single-url-check
4. Query ClickHouse system.tables. agent_traces_by_key, otel_traces, and spend_logs each show toIntervalDay(14)

Type

Bug Fix

Caveats

Severe

• Deployments using a separate reader host must use one ClickHouse URL
• Shorter retention can trigger background ClickHouse mutations on existing rows

Low

• UI setup screenshot is pending before review
• Provider proof is pending: Mistral returned 429 and OpenRouter returned 401

Final Attestation

• The tests and live proof cover every relevant customer path

Link to Devin session: https://app.devin.ai/sessions/88f2936b688f4bdea7f713b41728a437
Open in Devin Desktop: https://app.devin.ai/desktop/session/88f2936b688f4bdea7f713b41728a437?variant=devin
Requested by: @yujonglee-berri

---

[codecov]

Codecov Report

❌ Patch coverage is 79.54545% with 18 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/tracing/config.py	74.07%	14 Missing ⚠️
litellm/tracing/receiver.py	71.42%	2 Missing ⚠️
litellm/integrations/clickhouse/schema.py	50.00%	1 Missing ⚠️
litellm/rust_bridge/traces.py	92.85%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_optional_trace_reader_url (f9f2ced) with main (fe9b6fd)¹}

Footnotes

1. No successful run was found on main (615ed79) during the generation of this report, so fe9b6fd was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[devin-ai-integration]

Devin Review found 3 potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[greptile-apps]

Confidence Score: 5/5

[High risk] Refactors ClickHouse storage configuration across Rust and Python.

The PR appears safe to merge based on the reviewed changes and prior findings.

Summary

The PR unifies ClickHouse tracing configuration, using one connection for writes and bounded reads and one retention setting for trace and spend tables. It updates the Python and Rust configuration paths, deployment examples, setup UI, and tests.

_{Reviews (3) · Last reviewed commit: "fix(ui): split audit log query guard to ..."}

[yujonglee-berri]

@greptile-apps wdyt

[devin-ai-integration]

@greptileai

[yujonglee-berri]