feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks

[ishaan-berri]

Relevant issues

First piece of the litellm-rust realtime work: the Rust gateway owns the connection (Python isn't in the request path), so it needs to push finished log/usage events back to Python. This PR adds the Python side — a single generic endpoint that replays a prebuilt StandardLoggingPayload through LiteLLM's normal success/failure callback fan-out.

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem

Screenshots / Proof of Fix

Started a real proxy against a local Postgres and POSTed a StandardLoggingPayload — it lands in LiteLLM_SpendLogs and triggers the full cost-tracking fan-out (user + team spend), exactly like a normal completion.

1. POST the payload (admin key):

$ curl -X POST http://localhost:4000/v1/callbacks/logs -H "Authorization: Bearer sk-1234" --data @payload.json
{"processed":1,"failed":0}

2. Row count went 379 → 380, and the row is correct:

-[ RECORD 1 ]-------+------------------------
request_id          | cb-logs-test-1782252248
call_type           | acompletion
model               | gpt-4o-mini
api_key             | rust-gateway-test-key
spend               | 0.0123
total_tokens        | 42
prompt_tokens       | 30
completion_tokens   | 12
user                | user-cb-logs-test
team_id             | team-cb-logs-test
custom_llm_provider | openai

3. It went through the real callback path, not a raw insert — spend was attributed to the user and team:

db_spend_update_writer.py - User Spend transactions: {'user-cb-logs-test': 0.0123}
db_spend_update_writer.py - Team Spend transactions: {'team-cb-logs-test': 0.0123}

4. Auth guard:

no auth   -> 401
wrong key -> 401
admin key -> 200
non-admin -> 403

5. Unit tests:

$ pytest tests/test_litellm/proxy/logging_endpoints/test_callback_logs_endpoints.py -q
7 passed

Type

🆕 New Feature

Changes

Adds POST /v1/callbacks/logs (admin-only). It takes a batch of {status, standard_logging_payload, error?} records, and for each one reconstructs a minimal Logging object, sets the prebuilt payload on model_call_details["standard_logging_object"] (so the handler skips rebuilding it), and calls the existing async_success_handler / async_failure_handler. That means spend logs + every registered callback (Langfuse, Datadog, etc.) fire unchanged — there is no spend-logs-specific or callback-specific logic in the endpoint, just the replay.

The endpoint is generic on purpose: the contract is the self-describing StandardLoggingPayload (it carries call_type, model, usage, cost, metadata), so realtime is the first producer but completions/responses can use it later with no changes.

Files:

• litellm/proxy/logging_endpoints/callback_logs_endpoints.py — the endpoint + replay helpers
• litellm/proxy/proxy_server.py — register the router
• tests/test_litellm/proxy/logging_endpoints/test_callback_logs_endpoints.py — replay (success/failure), admin guard, partial-failure isolation

Real E2E Proof it worked

[codecov]

Codecov Report

❌ Patch coverage is 98.71795% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...proxy/logging_endpoints/callback_logs_endpoints.py	98.30%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR adds POST /v1/callbacks/logs (admin-only), enabling external producers like the litellm-rust realtime gateway to push finished StandardLoggingPayload records back into LiteLLM's Python callback fan-out — spend logs, Langfuse, Datadog, etc. — without being in the Python request path. The Rust side adds a RealTimeStreaming collector, a LiteLLMPythonProxyAPILogger background worker, and the auth infrastructure needed to hash credentials correctly before they reach the wire.

• Python endpoint (callback_logs_endpoints.py): Rebuilds a minimal Logging object per record, pre-seeds standard_logging_object so callbacks skip re-building it, and dispatches to async_success_handler / async_failure_handler. Batch size is bounded by MAX_CALLBACK_LOG_RECORDS, per-record failures are returned by index, and auth is enforced to PROXY_ADMIN only.
• Rust gateway: RealTimeStreaming observes upstream→client events only (client events are explicitly not observed, preventing spend-log manipulation). The master key is SHA-256 hashed via auth::hash_token before it enters any payload, matching Python's hash_token exactly — confirmed by a pinned unit test.
• Both previously flagged security items are resolved on HEAD: key hashing and upstream-only observation are present and tested.

Confidence Score: 5/5

Safe to merge. The new endpoint is admin-only with a bounded batch size, and the two critical security concerns from the prior review round are demonstrably fixed in this commit.

The key-hashing and upstream-only-observe fixes are both present and unit-tested. The Python replay path is clean: stream is forced to False, cost attribution metadata is mapped correctly, and failure isolation prevents one bad record from aborting the batch. The only finding is a redundant decorator-level Depends that FastAPI silently deduplicates anyway.

No files require special attention. The most sensitive logic — auth hashing in auth/mod.rs and the observe arm selection in providers/src/realtime.rs — has explicit comments and dedicated tests.

Important Files Changed

Filename	Overview
litellm/proxy/logging_endpoints/callback_logs_endpoints.py	New admin-only POST /v1/callbacks/logs endpoint that replays pre-built StandardLoggingPayload records through LiteLLM's callback fan-out. Auth guard, stream=False override, and per-record failure isolation are all correct. Minor: the decorator-level Depends(user_api_key_auth) is redundant with the function-parameter dependency.
litellm/types/proxy/callback_logs_endpoints.py	Type definitions for the callback-logs endpoint. Batch size is bounded by MAX_CALLBACK_LOG_RECORDS via Field(max_length=...). Per-record failure reporting via CallbackLogFailure is clean.
tests/test_litellm/proxy/logging_endpoints/test_callback_logs_endpoints.py	Unit tests using monkeypatching — no real network calls. Covers success/failure replay, admin-only guard, partial-failure isolation, and batch-size limit. All 7 tests pass per the PR description.
litellm-rust/crates/ai-gateway/src/auth/mod.rs	Adds hash_token (SHA-256 hex digest) matching Python's hash_token exactly, with a unit test that pins the known output for 'sk-1234'. The previous raw-key-in-spend-logs issue is fixed here.
litellm-rust/crates/ai-gateway/src/routes/realtime/mod.rs	Bridge now hashes the master key via auth::hash_token before seeding RequestMetadata.user_api_key_hash, and passes the observe closure to service::run which only invokes it on upstream→client events. Both previous security items are resolved.
litellm-rust/crates/providers/src/realtime.rs	observe(&event) is now called only in the upstream arm. The client arm has a comment explicitly explaining why observe must not be called there. Previous fabricated-response.done manipulation vector is closed.
litellm-rust/crates/ai-gateway/src/realtime/streaming.rs	New RealTimeStreaming collector: O(1) observe accumulates session id, model, and usage; log_messages stamps end_time and fans the payload to all CustomLoggers. Well-tested with CapturingLogger and FailingLogger scenarios.
litellm-rust/crates/ai-gateway/src/integrations/litellm_python_proxy_api.rs	Background worker POSTs batched LogRecords to /v1/callbacks/logs with bearer_auth(master_key). Channel-full/closed errors are non-fatal. Tunables are env-overridable. Clean design.

_{Reviews (5): Last reviewed commit: "feat(proxy): bound callback-logs batch +..." | Re-trigger Greptile}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ishaan-jaff
❌ ishaan-berri
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[ishaan-berri]

Verified end-to-end locally (Rust gateway → /v1/callbacks/logs → DB)

Ran the full stack locally — the Python proxy from this branch + the Rust litellm-ai-gateway binary from this branch — drove a real OpenAI realtime WebSocket session through the gateway, and the usage landed in LiteLLM_SpendLogs:

request_id   | sess_Du948SyGSUnHeSvBexr2C   ← OpenAI realtime session id
call_type    | realtime
model        | gpt-realtime
total_tokens | 118  (prompt 114, completion 4)
api_key      | 88dc28d0…6b4b  (SHA-256 of the master key)

It also shows up in the LiteLLM UI Request Logs (model gpt-realtime, 118 tokens, type LLM/Success). Screenshot below.

cargo build + clippy -D warnings + cargo test -p litellm-ai-gateway all green; observe() is lock-free on the hot path; request_id == the OpenAI session id by rule.

[ishaan-berri]

@greptileai review

Rebased onto current litellm_internal_staging and consolidated: this PR now carries both the Python /v1/callbacks/logs ingest endpoint and the Rust ai-gateway realtime logging glue (CustomLogger callback layer + RealTimeStreaming collector). The gateway base comes from the already-merged #31135 — the Rust diff here is only the logging glue (integrations/ + realtime/streaming.rs + lock-free observe wiring). Verified end-to-end: a live realtime session writes a LiteLLM_SpendLogs row.

[ishaan-berri]

@greptile review

Addressed the review feedback:

• P1 (master key): the gateway now SHA-256-hashes the master key (auth::hash_token, matching the proxy's hash_token) before setting user_api_key_hash — the raw credential never enters a log payload, and the DB hash is unchanged.
• P1 (spend spoofing): observe now fires on the upstream→client arm only, so a client can't fabricate response.done usage.
• P2 (batch size): /v1/callbacks/logs is bounded by MAX_CALLBACK_LOG_RECORDS (default 1000) → 422 over the cap.
• P2 (failure detail): the response now returns per-record {index, error} failures.

Tests added on both sides (rust hash_token parity, batch-limit, failure-detail).

[ishaan-berri]

@greptile review — note: the two P1 security items are already fixed on the current commit (17c5cb6). routes/realtime/mod.rs hashes the key via auth::hash_token (SHA-256, unit-tested), and providers/realtime.rs observes the upstream arm only (the client arm has no observe call). Please re-review HEAD.