test(proxy): pin what a failed request records as usage and spend

[yuneng-berri]

TLDR

Problem this solves:

• Six helpers decide the usage a failed request records
• None of them is named anywhere in the suite
• Two of their decisions can be reversed with the file still green

How it solves it:

• Twelve cases asserting those contracts directly
• Both reversible decisions now fail

User Flow

No end-user behavior changes. A proxy admin reading a failed request on
https://litellm-domain/ui/?page=logs keeps seeing the same tokens, and the two
ways that row could silently go wrong now fail in CI first.

Relevant issues

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Shared setup: ten decisions in litellm/proxy/utils.py are rewritten to their
opposite, one at a time, and the mapped test file is run against each.

for each rewrite:
    apply it to litellm/proxy/utils.py
    uv run pytest tests/test_litellm/proxy/test_proxy_utils.py -q
    killed if the run fails

Before (ff02d5c)

1. uv run pytest tests/test_litellm/proxy/test_proxy_utils.py -q

88 passed, 2 warnings in 6.88s

2. The ten rewrites:

KILLED    system blocks concatenated without text
KILLED    system prompt dropped from the count
SURVIVED  estimate keeps a zero count
KILLED    estimate bills completion too
KILLED    recovered usage ignored
SURVIVED  undispatched failures estimated too
KILLED    call type gate dropped
KILLED    dispatched system prompt ignored
KILLED    estimated failure billed at cost
KILLED    None fields lifted too

kill rate: 8/10

3. Eight already fail through ProxyLogging. The two that survive are the ones
   that put a wrong number on a spend row: a request with nothing countable in
   it lifted as a zero-token usage, and a request that never reached a provider
   billed for input it never sent

After (df3bb2f)

1. uv run pytest tests/test_litellm/proxy/test_proxy_utils.py -q

108 passed, 2 warnings in 5.41s

2. The same ten rewrites:

KILLED    system blocks concatenated without text
KILLED    system prompt dropped from the count
KILLED    estimate keeps a zero count
KILLED    estimate bills completion too
KILLED    recovered usage ignored
KILLED    undispatched failures estimated too
KILLED    call type gate dropped
KILLED    dispatched system prompt ignored
KILLED    estimated failure billed at cost
KILLED    None fields lifted too

kill rate: 10/10

Type

✅ Test

Caveats (if any)

• Eight of the ten were already caught, so this closes two gaps
• The value is also that each contract is now named, not inferred
• One test pins the literal litellm_no_upstream_llm_call, deliberately
• Token counts are asserted as relationships, so a tokenizer bump is fine

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

The PR adds focused unit coverage for failed-request usage and spend recording without changing production behavior.

• Exercises system-prompt normalization and request token counting.
• Pins dispatch, recovered-usage, no-upstream-call, and spend-log field behavior.
• Uses local tokenization and does not introduce provider network calls.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
tests/test_litellm/proxy/test_proxy_utils.py	Adds direct regression tests for failure-usage estimation and spend-log lifting; no blocking issue was found.

_{Reviews (2): Last reviewed commit: "test(proxy): pin what a failed request r..." | Re-trigger Greptile}

[greptile-apps]

Avoid pinning catalog pricing

This assertion duplicates the active gpt-4o catalog price as a literal, so a legitimate pricing-data update will fail the test even when streamed usage reconstruction and cost calculation remain correct.

Suggested change

assert cost == pytest.approx(0.0007625)

Knowledge Base Used: Cost Tracking and Budget Enforcement

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yuneng-berri]

@greptileai