fix(spend): give a batch's cost row a primary key of its own

[marty-sullivan]

TLDR

Problem this solves:

• Batch and file spend rows share one hardcoded primary key
• Message redaction makes that key a constant
• Every row after the first is silently dropped
• Batch cost never reaches /spend/logs or chargeback

How it solves it:

• Derive request_id from the response or call id
• Namespace a batch cost row away from its creation row

User Flow

Before: a team running batch jobs through the gateway with message redaction on never sees a completed batch's cost in their spend logs

1. The proxy admin turns on message redaction (litellm_settings: turn_off_message_logging: true) and restarts the proxy
2. A developer uploads a JSONL file with POST https://litellm-domain/v1/files (purpose: batch, target_model_names: <model>) and gets back a gateway file id
3. They send POST https://litellm-domain/v1/batches with that input_file_id and get back a batch id with status: validating
4. They poll GET https://litellm-domain/v1/batches/{batch_id} until it returns status: completed
5. After the batch completes they GET https://litellm-domain/spend/logs (or open https://litellm-domain/ui/?page=logs) and find no entry for the completed batch at all: only the $0 batch creation row and the $0 file upload row, and the upload's request id is the same 32-character value (00fcbef15a3b0097e14b0ca016ed30a0) for every upload and every batch on the proxy, so the batch's tokens and cost never reach the key's or team's spend

After: the same batch lands in the spend logs as its own entry with its real cost

1. The proxy admin turns on message redaction (litellm_settings: turn_off_message_logging: true) and restarts the proxy
2. A developer uploads a JSONL file with POST https://litellm-domain/v1/files (purpose: batch, target_model_names: <model>) and gets back a gateway file id
3. They send POST https://litellm-domain/v1/batches with that input_file_id and get back a batch id with status: validating
4. They poll GET https://litellm-domain/v1/batches/{batch_id} until it returns status: completed
5. After the batch completes, GET https://litellm-domain/spend/logs (or https://litellm-domain/ui/?page=logs) shows an entry for the batch with call_type: aretrieve_batch, request id {batch_id}_batch_cost, and the batch's real token counts and cost, next to the batch creation row keyed by the batch id and the file upload row keyed by the provider's own file id instead of one fixed id shared with every other upload

Relevant issues

Sibling finding from the same investigation as #36638, which fixed message redaction aborting success logging on a batch output body. Independent of it, since this bug predates that PR and neither fix depends on the other.

#36877 is the companion change that stops a batch's cost being accounted twice, or not at all when a caller's retrieve races the cost poller. That one decides whether the cost gets computed; this one decides whether the resulting row can be stored.

Linear ticket

Resolves LIT-5621

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Same end-user flow run twice against live proxies, each on a fresh Postgres database, with message redaction on and a real OpenAI managed batch (openai/gpt-5.4-nano, two chat requests, real $). Config used by both:

model_list:
  - model_name: gpt-5.4-nano
    litellm_params:
      model: openai/gpt-5.4-nano
      api_key: os.environ/OPENAI_API_KEY
litellm_settings:
  turn_off_message_logging: true
general_settings:
  proxy_batch_polling_interval: 20

$ cat input.jsonl
{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-5.4-nano", "messages": [{"role": "user", "content": "Say hello in one word"}], "max_completion_tokens": 200}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-5.4-nano", "messages": [{"role": "user", "content": "Name one color, one word"}], "max_completion_tokens": 200}}

Before, at 6704a10 (merge base), proxy on port 29010

$ curl -sS http://127.0.0.1:29010/v1/files -H "Authorization: Bearer $KEY" -F purpose=batch -F target_model_names=gpt-5.4-nano -F file=@input.jsonl
{
    "id": "bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw2ODk0MzJiMS1mMzA1LTRjYzktYTBkMy05NmMwOGZmYmUzMzU7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LjQtbmFubztsbG1fb3V0cHV0X2ZpbGVfaWQsZmlsZS1SWVZpTVRncU1pZUZFWVc0RGl0RHl6O2xsbV9vdXRwdXRfZmlsZV9tb2RlbF9pZCxkZjZhOGE3NmE1YTdmM2I2ZWI1YWMxZDM5M2I4MmExMjY2YzUyYjc4ZWUyZGE5NjkyYjhhZmZlNmQxZjZiYjlj",
    "object": "file",
    "purpose": "batch",
    "status": "uploaded",
    ...
}

$ curl -sS http://127.0.0.1:29010/v1/batches -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"input_file_id": "<file id above>", "endpoint": "/v1/chat/completions", "completion_window": "24h"}'
{
    "id": "bGl0ZWxsbV9wcm94eTttb2RlbF9pZDpkZjZhOGE3NmE1YTdmM2I2ZWI1YWMxZDM5M2I4MmExMjY2YzUyYjc4ZWUyZGE5NjkyYjhhZmZlNmQxZjZiYjljO2xsbV9iYXRjaF9pZDpiYXRjaF82YTgwYjU1YzhhNTA4MTkwYTU0ZjhlM2EwYWEyZGQ5OA",
    "object": "batch",
    "status": "validating",
    ...
}

$ while true; do curl -sS http://127.0.0.1:29010/v1/batches/$BATCH_ID -H "Authorization: Bearer $KEY" | jq -c '{status, request_counts}'; sleep 20; done   # until status is completed
11:52:13 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:52:33 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:52:53 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:53:14 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:53:34 {"status": "completed", "request_counts": {"completed": 2, "failed": 0, "total": 2}}

# 75s later, past the 20s polling interval plus jitter and the spend log flush
$ curl -sS "http://127.0.0.1:29010/spend/logs" -H "Authorization: Bearer $KEY" | jq -c '.[] | {request_id, call_type, model, spend, total_tokens, prompt_tokens, completion_tokens, startTime}'
{"request_id": "829dc768-cc0e-4f64-9bee-5b47ab6e4286", "call_type": "acreate_batch", "model": "openai/gpt-5.4-nano", "spend": 0.0, "total_tokens": 0, "prompt_tokens": 0, "completion_tokens": 0, "startTime": "2026-08-15T18:52:12.215000Z"}
{"request_id": "00fcbef15a3b0097e14b0ca016ed30a0", "call_type": "acreate_file", "model": "openai/gpt-5.4-nano", "spend": 0.0, "total_tokens": 0, "prompt_tokens": 0, "completion_tokens": 0, "startTime": "2026-08-15T18:52:10.989000Z"}

$ curl -sS "http://127.0.0.1:29010/spend/logs?request_id=${BATCH_ID}_batch_cost" -H "Authorization: Bearer $KEY" | jq -c '.[] | {request_id, call_type, model, spend, total_tokens}'
(no rows)

$ curl -sS "http://127.0.0.1:29010/spend/logs?request_id=00fcbef15a3b0097e14b0ca016ed30a0" -H "Authorization: Bearer $KEY" | jq -c '.[] | {request_id, call_type, model, spend}'
{"request_id": "00fcbef15a3b0097e14b0ca016ed30a0", "call_type": "acreate_file", "model": "openai/gpt-5.4-nano", "spend": 0.0}

No aretrieve_batch row anywhere. The proxy did compute and try to write the batch's cost, under the same fixed id the file upload already occupied, so the flush silently dropped it:

$ grep "Writing spend log to db" proxy_before.log
11:52:12 ... Writing spend log to db - request_id: 00fcbef15a3b0097e14b0ca016ed30a0, spend: 0.0
11:52:13 ... Writing spend log to db - request_id: 829dc768-cc0e-4f64-9bee-5b47ab6e4286, spend: 0.0
11:53:34 ... Writing spend log to db - request_id: 00fcbef15a3b0097e14b0ca016ed30a0, spend: 7.3e-06

After, at 363e3f3 (this PR's tip), proxy on port 51449

$ curl -sS http://127.0.0.1:51449/v1/files -H "Authorization: Bearer $KEY" -F purpose=batch -F target_model_names=gpt-5.4-nano -F file=@input.jsonl
{
    "id": "bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCwzMmMzMjQyNC03ODg4LTRlNmMtOGQ4NC03NWJiMzVhMzhiZmM7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LjQtbmFubztsbG1fb3V0cHV0X2ZpbGVfaWQsZmlsZS00NjFwNTVaQVd0eEp5QkdRa1dVWTZFO2xsbV9vdXRwdXRfZmlsZV9tb2RlbF9pZCxkZjZhOGE3NmE1YTdmM2I2ZWI1YWMxZDM5M2I4MmExMjY2YzUyYjc4ZWUyZGE5NjkyYjhhZmZlNmQxZjZiYjlj",
    "object": "file",
    "purpose": "batch",
    "status": "uploaded",
    ...
}

$ curl -sS http://127.0.0.1:51449/v1/batches -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"input_file_id": "<file id above>", "endpoint": "/v1/chat/completions", "completion_window": "24h"}'
{
    "id": "bGl0ZWxsbV9wcm94eTttb2RlbF9pZDpkZjZhOGE3NmE1YTdmM2I2ZWI1YWMxZDM5M2I4MmExMjY2YzUyYjc4ZWUyZGE5NjkyYjhhZmZlNmQxZjZiYjljO2xsbV9iYXRjaF9pZDpiYXRjaF82YTgwYjU1Yzg4NTg4MTkwOTA2YzBlNzQ2Yzc1YjdlYQ",
    "object": "batch",
    "status": "validating",
    ...
}

$ while true; do curl -sS http://127.0.0.1:51449/v1/batches/$BATCH_ID -H "Authorization: Bearer $KEY" | jq -c '{status, request_counts}'; sleep 20; done   # until status is completed
11:52:13 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:52:33 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:52:53 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:53:14 {"status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}}
11:53:34 {"status": "in_progress", "request_counts": {"completed": 1, "failed": 0, "total": 2}}
11:53:54 {"status": "in_progress", "request_counts": {"completed": 1, "failed": 0, "total": 2}}
11:54:14 {"status": "in_progress", "request_counts": {"completed": 1, "failed": 0, "total": 2}}
11:54:35 {"status": "completed", "request_counts": {"completed": 2, "failed": 0, "total": 2}}

# 75s later, past the 20s polling interval plus jitter and the spend log flush
$ curl -sS "http://127.0.0.1:51449/spend/logs" -H "Authorization: Bearer $KEY" | jq -c '.[] | {request_id, call_type, model, spend, total_tokens, prompt_tokens, completion_tokens, startTime}'
{"request_id": "bGl0ZWxsbV9wcm94eTttb2RlbF9pZDpkZjZhOGE3NmE1YTdmM2I2ZWI1YWMxZDM5M2I4MmExMjY2YzUyYjc4ZWUyZGE5NjkyYjhhZmZlNmQxZjZiYjljO2xsbV9iYXRjaF9pZDpiYXRjaF82YTgwYjU1Yzg4NTg4MTkwOTA2YzBlNzQ2Yzc1YjdlYQ_batch_cost", "call_type": "aretrieve_batch", "model": "openai/gpt-5.4-nano", "spend": 7.3e-06, "total_tokens": 31, "prompt_tokens": 23, "completion_tokens": 8, "startTime": "2026-08-15T18:54:35.040000Z"}
{"request_id": "bGl0ZWxsbV9wcm94eTttb2RlbF9pZDpkZjZhOGE3NmE1YTdmM2I2ZWI1YWMxZDM5M2I4MmExMjY2YzUyYjc4ZWUyZGE5NjkyYjhhZmZlNmQxZjZiYjljO2xsbV9iYXRjaF9pZDpiYXRjaF82YTgwYjU1Yzg4NTg4MTkwOTA2YzBlNzQ2Yzc1YjdlYQ", "call_type": "acreate_batch", "model": "openai/gpt-5.4-nano", "spend": 0.0, "total_tokens": 0, "prompt_tokens": 0, "completion_tokens": 0, "startTime": "2026-08-15T18:52:12.210000Z"}
{"request_id": "file-461p55ZAWtxJyBGQkWUY6E", "call_type": "acreate_file", "model": "openai/gpt-5.4-nano", "spend": 0.0, "total_tokens": 0, "prompt_tokens": 0, "completion_tokens": 0, "startTime": "2026-08-15T18:52:10.982000Z"}

$ curl -sS "http://127.0.0.1:51449/spend/logs?request_id=${BATCH_ID}_batch_cost" -H "Authorization: Bearer $KEY" | jq -c '.[] | {request_id, call_type, model, spend, total_tokens}'
{"request_id": "bGl0ZWxsbV9wcm94eTttb2RlbF9pZDpkZjZhOGE3NmE1YTdmM2I2ZWI1YWMxZDM5M2I4MmExMjY2YzUyYjc4ZWUyZGE5NjkyYjhhZmZlNmQxZjZiYjljO2xsbV9iYXRjaF9pZDpiYXRjaF82YTgwYjU1Yzg4NTg4MTkwOTA2YzBlNzQ2Yzc1YjdlYQ_batch_cost", "call_type": "aretrieve_batch", "model": "openai/gpt-5.4-nano", "spend": 7.3e-06, "total_tokens": 31}

$ curl -sS "http://127.0.0.1:51449/spend/logs?request_id=00fcbef15a3b0097e14b0ca016ed30a0" -H "Authorization: Bearer $KEY" | jq -c '.[] | {request_id, call_type, model, spend}'
(no rows)

The batch cost row lands under {batch_id}_batch_cost with the batch's real usage (23 prompt, 8 completion tokens, the same $7.3e-06 the before proxy computed and lost), the batch creation row is keyed by the batch id, and the file upload row is keyed by the provider's file id. The seven in-progress polls wrote no spend row on either proxy, only the completed poll did:

$ grep "Writing spend log to db" proxy_after.log
11:52:12 ... Writing spend log to db - request_id: file-461p55ZAWtxJyBGQkWUY6E, spend: 0.0
11:52:13 ... Writing spend log to db - request_id: bGl0...YjdlYQ, spend: 0.0
11:54:35 ... Writing spend log to db - request_id: bGl0...YjdlYQ_batch_cost, spend: 7.3e-06

Type

🐛 Bug Fix

Caveats (if any)

• Bedrock Converse batch output still logs $0 usage, tracked separately
• Relies on in-progress polls logging nothing, which they do today

Changes

get_spend_logs_id no longer hashes the response. Every call type now derives its id the same way, preferring the response's own id, then the standard logging payload's id, then litellm_call_id.

The middle term is what keeps this correct under redaction. That payload is built from the unredacted response, so it still carries the batch id after redaction has flattened the body. Without it a redacted batch falls through to the per-call id, which writes a fresh row per retrieve, each carrying the same batch's full cost, and overstates spend by however many times the caller polled.

A batch cost row is namespaced with a _batch_cost suffix so it cannot collide with the acreate_batch row written when the batch was submitted. This follows the existing precedent for cache hits, which already suffix the id to avoid duplicating a request id.

Cost and usage themselves are unaffected by redaction and were verified to stay correct in both modes: the token columns fall back to the standard logging payload and spend comes from its response_cost, neither of which redaction touches. generate_hash_from_response had no other caller and is removed with it.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

^{Cursor Bugbot is generating a summary for commit 363e3f3. Configure here.}

[marty-sullivan]

@greptileai

[greptile-apps]

Greptile Summary

This PR changes spend-log request ID selection so batch cost rows use stable, namespaced batch identities instead of response hashes

• Resolves IDs from the response, standard logging payload, or call ID
• Namespaces retrieved batch cost rows with _batch_cost
• Adds regression coverage for redaction, collision avoidance, and repeated accounting
• Ratchets down lint and type-check budgets

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains

Important Files Changed

Filename	Overview
litellm/proxy/spend_tracking/spend_tracking_utils.py	Replaces response hashing with ordered ID resolution and namespaces batch cost rows to prevent primary-key collisions
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py	Adds focused regression tests for redacted responses, stable batch identity, distinct batches, and creation-row separation
basedpyright-code-budget.json	Lowers resolved type-check diagnostic ceilings
ruff-strict-budget.json	Lowers strict lint ceilings after removing obsolete hashing code
type-discipline-budget.json	Ratchets down type-discipline limits associated with the cleanup

_{Reviews (4): Last reviewed commit: "test(spend): annotate the batch cost row..." | Re-trigger Greptile}

[marty-sullivan]

Added Final to both constants. Worth noting no other test file under tests/test_litellm uses it, so the convention currently lives in source only.

@greptileai

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[marty-sullivan]

@greptileai

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing cu-aaii:litellm_fix_batch_spend_log_constant_request_id (363e3f3) with litellm_internal_staging (6704a10)}

[mateo-berri]

bugbot run

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 363e3f3. Configure here.}

[mateo-berri]

LGTM. Thanks for the contribution!