fix(vertex-ai): fix zero cost/usage on completed Vertex AI batch jobs

[Sameerlite]

Problem

Vertex AI batch jobs completed successfully but always recorded spend = 0, prompt_tokens = 0, completion_tokens = 0 in proxy spend logs and the UI.

OpenAI batch jobs in the same proxy attributed cost and tokens correctly.

Fixes #27891
Fixes LIT-3060

Root Cause

Bug 1 — batch_utils.py

_batch_cost_calculator and _get_batch_job_total_usage_from_file_content always routed vertex_ai through calculate_vertex_ai_batch_cost_and_usage, which reads raw response.usageMetadata.* fields. After #25627, those fields no longer exist — the file content is now OpenAI-shaped (response.body.usage.*) — so every line evaluated to 0 tokens.

Bug 2 — cost_calculator.py

batch_cost_calculator skipped the global litellm.get_model_info() lookup when a model_info dict was passed in. CheckBatchCost always passes deployment_info.model_info.model_dump(), which for a standard deployment only contains id and db_model — no pricing fields. The non-None dict blocked the global lookup → all pricing fields None → returned (0.0, 0.0).

Fix

litellm/batches/batch_utils.py — Gate the Vertex-specific reader behind disable_vertex_batch_output_transformation:

• Flag False (default): content is already OpenAI-shaped; fall through to the generic path that reads response.body.usage.* — same as OpenAI/Azure.
• Flag True: content is raw Vertex shape; use the specialized usageMetadata reader.

Cost tracking working with openai format

Cost tracking working with vertex ai format

---

Note

Medium Risk
Changes batch spend/tokens attribution logic for vertex_ai and modifies pricing resolution when model_info is provided, which can affect billing calculations across deployments.

Overview
Fixes Vertex AI batch jobs recording zero cost/usage by only using the Vertex-specific usageMetadata reader when litellm.disable_vertex_batch_output_transformation=True; otherwise Vertex batch outputs are treated as OpenAI-shaped and flow through the generic cost/usage path.

Updates batch_cost_calculator to fall back to global litellm.get_model_info() when a provided model_info lacks pricing fields (while preserving explicit 0.0 pricing), and adds regression tests covering both Vertex output shapes and the pricing fallback behavior.

^{Reviewed by Cursor Bugbot for commit 642269a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

Fixes Vertex AI batch jobs recording zero cost and usage by gating the Vertex-specific usageMetadata reader behind disable_vertex_batch_output_transformation, and adds a fallback in batch_cost_calculator to the global pricing table when the provided model_info carries no pricing fields.

• batch_utils.py: Both _batch_cost_calculator and _get_batch_job_total_usage_from_file_content now only invoke the native Vertex reader when disable_vertex_batch_output_transformation=True; with the default False, OpenAI-shaped Vertex output falls through to the generic response.body.usage.* path shared with OpenAI and Azure.
• cost_calculator.py: The new elif branch detects a model_info with all four pricing fields None (e.g., a deployment stub containing only id/db_model) and replaces it with the global litellm.get_model_info() result, using is not None so explicit 0.0 pricing is correctly preserved as "present".
• Tests: New unit tests cover both Vertex output shapes and the zero-cost-not-overridden-by-global scenario; all are pure mock tests with no network calls.

Confidence Score: 5/5

Safe to merge; the routing change is backward-compatible and the cost-fallback is gated behind a precise is-not-None check.

Both fixes are tightly scoped: the batch_utils.py change is a one-line condition addition that only affects the Vertex AI code path when the flag is absent (default), and the cost_calculator.py fallback fires only when every pricing key is None, leaving all existing behavior unchanged. The new tests correctly exercise both code paths and would catch regression in either direction.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/batches/batch_utils.py	Gates Vertex-specific usageMetadata reader behind disable_vertex_batch_output_transformation flag; default (False) falls through to the generic OpenAI-shaped path, fixing zero-cost attribution.
litellm/cost_calculator.py	Adds elif fallback to global pricing when model_info carries no pricing fields; uses is not None correctly so explicit 0.0 pricing is not mistaken for absent pricing.
tests/batches_tests/test_batch_custom_pricing.py	Adds test verifying explicit 0.0 pricing prevents fallback to global table; monkeypatched global returns non-zero so the assertion correctly detects accidental fallback.
tests/test_litellm/proxy/pass_through_endpoints/test_vertex_ai_batch_passthrough.py	Adds two pure unit regression tests (no real network calls) covering OpenAI-shaped Vertex output and raw Vertex output with transformation disabled; both use hardcoded mock data.

_{Reviews (4): Last reviewed commit: "fix(cost-calculator): treat explicit zer..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Sameerlite]

@greptile re review

[Sameerlite]

@greptile re review

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is ON, but it could not run because the spend limit has been reached. To enable Bugbot Autofix, have a team admin raise the spend limit in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f77b5c3. Configure here.}

[Sameerlite]

@greptile re review

[mateo-berri]

LGTM; thanks!