fix(bedrock): read batch usage by payload shape, not by provider name

[marty-sullivan]

TLDR

Problem this solves:

• Every Bedrock batch line uses the Anthropic usage parser
• Converse models report camelCase counts it cannot read
• Nova batches report zero tokens and bill $0
• An unreadable usage shape fails silently

How it solves it:

• Pick the usage parser by the payload's shape
• Warn when no parser understands a shape

User Flow

Before: a team running Nova batch jobs sees every completed batch logged with zero tokens and $0.00 spend, so the work appears free and never reaches chargeback

1. They upload a JSONL file with POST https://litellm-domain/v1/files (purpose: batch, target_model_names: amazon.nova-lite) and get back a gateway file id
2. They send POST https://litellm-domain/v1/batches with that input_file_id and get back a batch id with status: validating
3. They poll GET https://litellm-domain/v1/batches/{batch_id} until it returns status: completed
4. They download the batch's output with GET https://litellm-domain/v1/files/{output_file_id}/content and count real token usage in it, line by line
5. They open https://litellm-domain/ui/?page=logs and find that same batch recorded as 0 prompt tokens, 0 completion tokens, and $0.00 spend

After: the same batch reports the tokens its own output file shows, and bills accordingly

1. They upload the same JSONL file with POST https://litellm-domain/v1/files and get back a gateway file id
2. They send the same POST https://litellm-domain/v1/batches and get back a batch id with status: validating
3. They poll GET https://litellm-domain/v1/batches/{batch_id} until it returns status: completed
4. They download the same output with GET https://litellm-domain/v1/files/{output_file_id}/content and count the same real token usage
5. https://litellm-domain/ui/?page=logs now shows that batch with those same token counts and non-zero spend

Relevant issues

Linear ticket

Resolves LIT-5668

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Two live Bedrock managed batch runs against real AWS in us-west-2, one per checkpoint, each on its own proxy booted from a fresh Postgres with the batch cost job polling every 30s. Model bedrock/us.amazon.nova-lite-v1:0 (Converse family), the same 110-record JSONL both times, real $$$

Each leg is the exact end-user flow:

curl -s http://localhost:$PORT/v1/files -H "Authorization: Bearer $KEY" \
  -F purpose=batch -F target_model_names=nova-batch -F file=@batch_input.jsonl
curl -s http://localhost:$PORT/v1/batches -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"input_file_id": "<file id>", "endpoint": "/v1/chat/completions", "completion_window": "24h"}'
curl -s http://localhost:$PORT/v1/batches/<batch id> -H "Authorization: Bearer $KEY"   # poll until completed
curl -s http://localhost:$PORT/v1/files/<output file id>/content -H "Authorization: Bearer $KEY" -H "custom-llm-provider: bedrock"

Before, at merge base 973329e: the batch completes and Bedrock's own output file reports real usage on every one of the 110 lines, for example "usage":{"inputTokens":8,"outputTokens":11,"totalTokens":19}, summing to 990 input, 1597 output, and 2587 total tokens, yet the tracked spend row reads zero across the board

    call_type    |          model           | prompt_tokens | completion_tokens | total_tokens | spend
-----------------+--------------------------+---------------+-------------------+--------------+-------
 aretrieve_batch | us.amazon.nova-lite-v1:0 |             0 |                 0 |            0 |     0

After, at PR tip 5fe7793: same flow, same model, same input file. The output file sums to 990 input, 1577 output, and 2567 total tokens, and the spend row now matches it token for token, priced at the 50% batch rate on nova-lite's on-demand cost

    call_type    |          model           | prompt_tokens | completion_tokens | total_tokens |   spend
-----------------+--------------------------+---------------+-------------------+--------------+------------
 aretrieve_batch | us.amazon.nova-lite-v1:0 |           990 |              1577 |         2567 | 0.00021894

The after leg logged zero "billed at $0" warnings, confirming the Converse shape is now parsed rather than warned about. Anthropic-shaped Bedrock batches were additionally verified unaffected across 23 real batches on a deployed gateway, where token counts reconciled exactly against the providers' own output files

Type

🐛 Bug Fix

Changes

Bedrock batch usage was parsed by provider name rather than by the shape of the payload, and the single parser chosen reads Anthropic's snake_case token counts. Any Converse-family model, Nova being the common one, reports camelCase counts instead, so usage aggregated to 0/0/0 and the batch priced at $0 despite real consumption

Usage is now selected by the shape it actually has. A Converse-shaped block goes through the same transform the live Converse path uses, so a batch and an equivalent non-batch call agree on tokens, including cache reads and cache writes. Anthropic-shaped output keeps its existing parser, and the OpenAI-shaped path for other providers is untouched

Reusing the live transform means also exposing it: this makes the Converse usage transform public, since batch parsing is a second legitimate caller from outside that module. That additionally removes the private-member access invoke_handler was already making on it, which is why the basedpyright budget ratchets down by one here

A payload neither parser understands, such as an InvokeModel-native shape from Titan, Cohere, or Llama, still reads zero. That is left as is rather than guessed at, but it no longer does so quietly: it warns with the usage keys it saw, so the next unhandled shape shows up in logs instead of as a silently free batch

Caveats

• Titan, Cohere, and Llama shapes still read zero, now logged
• Fixing tokens alone is not enough; pricing needs the deployment's model

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing cu-aaii:litellm_fix_bedrock_converse_batch_usage (5fe7793) with litellm_internal_staging (973329e)}

[marty-sullivan]

@greptileai

[greptile-apps]

Greptile Summary

Bedrock batch usage is now selected by payload shape, with Converse-shaped counts normalized through the live Converse usage transform and unknown shapes logged.

• Adds Converse batch-usage shape detection and normalization.
• Preserves Anthropic-shaped Bedrock usage parsing.
• Makes the shared Converse usage transform public and updates its callers.
• Adds coverage for token totals, cache tokens, fallback parsing, and warnings.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/batches/batch_utils.py	Selects Converse normalization for camelCase Bedrock usage payloads while preserving Anthropic parsing and warning on unknown shapes.
litellm/llms/bedrock/chat/converse_transformation.py	Adds reusable Converse batch-usage normalization and exposes the usage transform for batch and streaming callers.
litellm/llms/bedrock/chat/invoke_handler.py	Updates streaming usage processing to call the newly public Converse transform.
tests/test_litellm/batches/test_batch_utils.py	Covers Converse and Anthropic batch payloads, cache-token accounting, inferred totals, and unknown-shape warnings.
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py	Covers shape classification, incomplete batch blocks, cache-token handling, and streaming usage conversion.

_{Reviews (3): Last reviewed commit: "refactor(bedrock): own the Converse batc..." | Re-trigger Greptile}

[marty-sullivan]

Both addressed. Shape detection and block normalization now live on AmazonConverseConfig as is_converse_usage_shape and usage_from_batch_output, so the batch layer asks the provider adapter instead of knowing Bedrock field names, and batch and live parsing cannot drift. Banner comment and redundant docstrings removed.

Also added direct provider-layer coverage for the predicate, the incomplete-block completion, cache-count inflation, and the streaming usage event, which closes the uncovered line Codecov flagged in invoke_handler.

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 5fe7793. Configure here.}

[mateo-berri]

LGTM. Thanks for the contribution!