fix(anthropic): report real token usage on blocked responses

[seph-barker]

The ModifyResponseException handler in the /v1/messages endpoint synthesizes a "blocked" response reporting zero input and output tokens, even though the request consumed real input tokens and the synthetic block message carries real content. Callers relying on usage (billing, quotas, metrics) under-count every blocked response.

Compute input_tokens from the original request messages (carried on the exception's request_data) and output_tokens from the block message text via litellm.token_counter. Counting is best-effort and falls back to zero on failure so a blocked response is always returned. The streaming synthesis path reuses the same response object, so both paths are fixed by one change.

Adds tests asserting nonzero, correct counts and graceful fallback.

Example responses, showing correct token counts.

From a request to /v1/chat/completions with a moderated response:

{
    "id": "chatcmpl-d673e5b5-2b05-4dd6-a630-036d9d67d505",
    "created": 1782768070,
    "model": "gpt-5.1-2025-11-13",
    "object": "chat.completion",
    "system_fingerprint": null,
    "choices": [
        {
            "finish_reason": "content_filter",
            "index": 0,
            "message": {
                "content": "The response was blocked by Rubrik Agent Cloud (Reference ID: 8c8bbcda-3b49-4d08-9755-c2d901690969)",
                "role": "assistant",
                "tool_calls": null,
                "function_call": null,
                "provider_specific_fields": null
            }
        }
    ],
    "usage": {
        "completion_tokens": 37,
        "prompt_tokens": 75,
        "total_tokens": 112,
        "completion_tokens_details": null,
        "prompt_tokens_details": null
    }
}

From a request to /v1/messages:

{
    "id": "msg_cb824a75-527b-45c7-aa18-8ffa5a921b1c",
    "type": "message",
    "role": "assistant",
    "content": [
        {
            "type": "text",
            "text": "The response was blocked by Rubrik Agent Cloud (Reference ID: 8c04589b-9b3b-4878-87fc-cab4280cc9d2)"
        }
    ],
    "model": "claude-sonnet-4-5",
    "stop_reason": "end_turn",
    "usage": {
        "input_tokens": 13,
        "output_tokens": 38
    }
}

[greptile-apps]

Greptile Summary

This PR fixes synthetic guardrail-blocked responses in the /v1/messages Anthropic endpoint and OpenAI-compatible (/v1/chat/completions, /v1/completions) endpoints to report real token usage instead of hard-coded zeros. Token counting is best-effort via litellm.token_counter with exception handling so blocked responses are always returned.

• Adds _get_blocked_response_usage in anthropic_endpoints/endpoints.py and _blocked_response_usage in proxy_server.py, both of which count input tokens from the original request messages (including the top-level Anthropic system field) and output tokens from the block-message text.
• Both streaming and non-streaming paths are fixed by a single placement of the usage computation before the stream-branch check.
• All new tests are mock-based (patching litellm.token_counter), conforming to the no-real-network-calls requirement for this test folder.

Confidence Score: 5/5

Safe to merge — the change is isolated to blocked-response synthesis paths and falls back gracefully to zero usage on any error.

The fix is well-scoped: it only touches the ModifyResponseException handler branches, which are off the happy path. Both new helpers are pure functions with a blanket exception handler, so the worst case is the same zero-usage behavior as before. Tests are mock-based, cover the main wiring, system-prompt inclusion, and the error fallback. No changes to auth, routing, or database access.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/anthropic_endpoints/endpoints.py	Adds _get_blocked_response_usage helper and wires it into the ModifyResponseException handler; imports AnthropicUsage for the return type; correctly includes the top-level system field in input token counting.
litellm/proxy/proxy_server.py	Adds module-level _blocked_response_usage helper and updates all three ModifyResponseException handler sites (chat_completion streaming + non-streaming, completion streaming + non-streaming) to use real token counts; removes the two hard-coded zero-usage assignments.
tests/test_litellm/proxy/anthropic_endpoints/test_endpoints.py	Adds TestBlockedResponseUsage class with five properly-mocked tests covering nonzero counts, output wiring, system-prompt inclusion, fallback, and an integration smoke test through the full handler.
tests/test_litellm/proxy/test_blocked_response_usage.py	New file with three mock-based tests covering the _blocked_response_usage helper for messages+tools, text-prompt, and error-fallback paths.

_{Reviews (3): Last reviewed commit: "fix(proxy): report real token usage on O..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing predibase:joseph/fix-blocked-token-counts-main (5666852) with main (3818d64)}

[codecov]

Codecov Report

❌ Patch coverage is 89.65517% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	75.00%	2 Missing ⚠️
litellm/proxy/anthropic_endpoints/endpoints.py	92.30%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[seph-barker]

@greptile-apps review

[Sameerlite]

Thanks for this thoughtful fix for blocked-response token counting — the test coverage looks thorough and Greptile gave it a 5/5! A couple of items to address before this can be merged:\n\n1. CI lint is failing — there is a check showing FAILURE. Could you take a look and fix the lint error so all checks are green?\n2. Proof of fix — the PR body describes the change well but it would be great to see some concrete evidence that the fix works end-to-end (e.g. test output showing non-zero token counts returned for a blocked response, or a before/after log snippet). Even a short output snippet would do!\n\nOnce those are addressed this looks close to ready. Thanks!

[seph-barker]

Thanks for this thoughtful fix for blocked-response token counting — the test coverage looks thorough and Greptile gave it a 5/5! A couple of items to address before this can be merged:\n\n1. CI lint is failing — there is a check showing FAILURE. Could you take a look and fix the lint error so all checks are green?\n2. Proof of fix — the PR body describes the change well but it would be great to see some concrete evidence that the fix works end-to-end (e.g. test output showing non-zero token counts returned for a blocked response, or a before/after log snippet). Even a short output snippet would do!\n\nOnce those are addressed this looks close to ready. Thanks!

Thanks for the review! Fixed the lint issues, and I'll add an end-to-end example soon.

[seph-barker]

Thanks for this thoughtful fix for blocked-response token counting — the test coverage looks thorough and Greptile gave it a 5/5! A couple of items to address before this can be merged:\n\n1. CI lint is failing — there is a check showing FAILURE. Could you take a look and fix the lint error so all checks are green?\n2. Proof of fix — the PR body describes the change well but it would be great to see some concrete evidence that the fix works end-to-end (e.g. test output showing non-zero token counts returned for a blocked response, or a before/after log snippet). Even a short output snippet would do!\n\nOnce those are addressed this looks close to ready. Thanks!

Thanks for the review! Fixed the lint issues, and I'll add an end-to-end example soon.

@Sameerlite Thanks again for the review. I've updated the PR to add correct token counting for /v1/chat/completions as well (previously it just covered /v1/messages).

PR description has been updated with example responses.

[Sameerlite]

@greptileai

[Sameerlite]

The proper fix would be:

1. Add an original_response optional field to ModifyResponseException
2. Pass the LLM response into it when raising post-call
3. In the handler that catches ModifyResponseException, use e.original_response.usage directly instead of re-counting tokens
   The token counter approach is a workaround for the symptom (zero usage) rather than fixing the root cause (usage being discarded)

[seph-barker]

Thanks! Addressed both:

1. CI / lint & conflicts — rebased onto the latest litellm_oss_staging (was conflicting after the base moved) and reformatted to the repo standard; ruff check and ruff format --check are clean locally. Squashed to a single commit.

2. Proof of fix — concrete before/after for a guardrail-blocked response (the exact /v1/chat/completions request from the report, plus the Anthropic path):

OpenAI /v1/chat/completions blocked usage:
  before (hard-coded): prompt=0 completion=0 total=0
  after  (this PR):    prompt=55 completion=19 total=74

Anthropic /v1/messages blocked usage:
  before (hard-coded): input_tokens=0 output_tokens=0
  after  (this PR):    input_tokens=27 output_tokens=19

Backed by mock-based unit tests (8 passing) covering chat (messages+tools), text completion (prompt), the Anthropic top-level system field, output tokens from the block message, and the zero fallback on counter error:

proxy/test_blocked_response_usage.py::test_chat_usage_counts_messages_tools_and_block_message PASSED
proxy/test_blocked_response_usage.py::test_text_completion_usage_counts_prompt_and_block_message PASSED
proxy/test_blocked_response_usage.py::test_usage_falls_back_to_zero_on_error PASSED
proxy/anthropic_endpoints/test_endpoints.py::TestBlockedResponseUsage (5 tests) PASSED

[seph-barker]

@Sameerlite good call — you're right that re-counting was treating the symptom. Reworked it to fix the root cause per your suggestion:

1. Added an optional original_response field to ModifyResponseException.
2. The unified guardrail's post-call success hook attaches the blocked LLM response to the exception (e.original_response) when a guardrail blocks post-call.
3. The /v1/messages and OpenAI-format (/v1/chat/completions, /v1/completions) block handlers now report e.original_response.usage directly instead of re-counting tokens.

token_counter is gone. Pre-call blocks (prompt moderation) never invoke the LLM, so there's no original_response and usage is reported as zero (nothing was consumed).

Tests updated to mock-based coverage: the helper returns the original response's usage (and zero when absent), the success hook attaches original_response on a block, and the endpoint reports it end-to-end. Also rebased onto latest litellm_oss_staging and reformatted; ruff check/format clean locally.

[Sameerlite]

LGTM