fix(cost): price Vertex AI DeepSeek OCR by token usage by mateo-berri · Pull Request #39414 · BerriAI/litellm

mateo-berri · 2026-09-02T21:50:48Z

TLDR

Problem this solves:

Vertex AI DeepSeek OCR responses report tokens, never pages, so they were never priced
Cost-map name vertex_ai/deepseek-ai/deepseek-ocr-maas: no x-litellm-response-cost header at all
Short name vertex_ai/deepseek-ocr-maas: cost 0.0 in the headers and $0 in the spend logs

How it solves it:

OCR cost falls back to token pricing when the model has token rates and reports no pages
The DeepSeek OCR response carries the deepseek-ai/ model name, so the short deployment name finds its price
Drop the unsourced ocr_cost_per_page from the DeepSeek entry: Google bills DeepSeek-OCR on Vertex per token
The Rust OCR port reports the same deepseek-ai/ response model

User Flow

Before: a developer OCRs an image through the gateway's Vertex AI DeepSeek OCR deployment and the request is never priced, so the logs page and key spend stay at $0

They send POST https://litellm-domain/v1/ocr with {"model": "vertex_ai/deepseek-ai/deepseek-ocr-maas", "document": {"type": "image_url", "image_url": "data:image/png;base64,..."}}
200 OK comes back with the OCR markdown and usage_info reporting prompt_tokens: 901, completion_tokens: 1057, pages_processed: null, but no x-litellm-response-cost header at all and x-litellm-key-spend: 0.0
They send the same POST with "model": "vertex_ai/deepseek-ocr-maas", the short name their admin deployed
200 OK again with the same kind of markdown and token counts, this time with every x-litellm-response-cost-* breakdown header at 0.0 and x-litellm-key-spend: 0.0
They open https://litellm-domain/ui/?page=logs (or GET https://litellm-domain/spend/logs?request_id=...) and both requests sit at spend: 0.0, so the usage page and the key's budget never move

After: the same two requests come back priced from Google's published per-token DeepSeek-OCR rates, and the spend shows up in the logs

They send POST https://litellm-domain/v1/ocr with {"model": "vertex_ai/deepseek-ai/deepseek-ocr-maas", "document": {"type": "image_url", "image_url": "data:image/png;base64,..."}}
200 OK comes back with the OCR markdown and the same kind of token counts, now with x-litellm-response-cost: 0.0003879 (901 prompt tokens at $0.30 per 1M plus 98 completion tokens at $1.20 per 1M), x-litellm-response-cost-input: 0.0002703, x-litellm-response-cost-output: 0.0001176, and x-litellm-key-spend moving by the same amount
They send the same POST with "model": "vertex_ai/deepseek-ocr-maas"
200 OK again, now with x-litellm-response-cost: 0.0007143 for 901 prompt and 370 completion tokens
They open https://litellm-domain/ui/?page=logs (or GET https://litellm-domain/spend/logs?request_id=...) and both requests show that same non-zero spend

Relevant issues

Linear ticket

Resolves LIT-6727

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Setup shared by both legs: a proxy booted with --num_workers 2 on a fresh Postgres database, VERTEX_AI_API_KEY set to a gcloud auth print-access-token value, and this config

model_list:
  - model_name: vertex_ai/deepseek-ai/deepseek-ocr-maas
    litellm_params:
      model: vertex_ai/deepseek-ai/deepseek-ocr-maas
      vertex_project: <project>
      vertex_location: us-central1
      api_key: os.environ/VERTEX_AI_API_KEY
  - model_name: vertex_ai/deepseek-ocr-maas
    litellm_params:
      model: vertex_ai/deepseek-ocr-maas
      vertex_project: <project>
      vertex_location: us-central1
      api_key: os.environ/VERTEX_AI_API_KEY
general_settings:
  master_key: sk-1234

The same loop runs on both legs against a 100 KB PNG of a rendered text page, then reads each request back through the spend logs API

b64=$(base64 -i ocr-input.png | tr -d '\n')
for M in vertex_ai/deepseek-ai/deepseek-ocr-maas vertex_ai/deepseek-ocr-maas; do
  printf '{"model": "%s", "document": {"type": "image_url", "image_url": "data:image/png;base64,%s"}}' "$M" "$b64" > ocr.json
  echo "== POST /v1/ocr model=$M"
  curl -s -D headers.txt -o body.json -X POST http://localhost:4000/v1/ocr -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" --data-binary @ocr.json
  grep -i "^HTTP\|x-litellm-response-cost\|x-litellm-key-spend\|x-litellm-call-id" headers.txt
  python3 -c "import json; d=json.load(open('body.json')); u=d['usage_info']; print(d['model'], {k: u.get(k) for k in ('pages_processed', 'prompt_tokens', 'completion_tokens')})"
done
sleep 12
for id in <the two x-litellm-call-id values>; do
  echo "== GET /spend/logs?request_id=$id"
  curl -s "http://localhost:4000/spend/logs?request_id=$id" -H "Authorization: Bearer sk-1234" | python3 -c "import json, sys; print([{k: r.get(k) for k in ('model', 'spend', 'prompt_tokens', 'completion_tokens', 'call_type')} for r in json.load(sys.stdin)])"
done

Before (`ba2e5d2`)

Cost-map name vertex_ai/deepseek-ai/deepseek-ocr-maas

POST /v1/ocr, no x-litellm-response-cost header at all

HTTP/1.1 200 OK
x-litellm-call-id: 89c64bc3-0ca3-4107-a8a1-74ed37e7aee6
x-litellm-key-spend: 0.0
vertex_ai/deepseek-ai/deepseek-ocr-maas {'pages_processed': None, 'prompt_tokens': 901, 'completion_tokens': 1057}

GET /spend/logs?request_id=89c64bc3-0ca3-4107-a8a1-74ed37e7aee6

[{'model': 'vertex_ai/deepseek-ai/deepseek-ocr-maas', 'spend': 0.0, 'prompt_tokens': 901, 'completion_tokens': 1057, 'call_type': 'aocr'}]

Short name vertex_ai/deepseek-ocr-maas

POST /v1/ocr, every cost breakdown header at 0.0

HTTP/1.1 200 OK
x-litellm-call-id: cfabd49d-847d-4adb-8f40-e92879d6fbc7
x-litellm-response-cost-original: 0.0
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0
x-litellm-response-cost-input: 0.0
x-litellm-response-cost-output: 0.0
x-litellm-response-cost-tool-usage: 0.0
x-litellm-key-spend: 0.0
vertex_ai/deepseek-ocr-maas {'pages_processed': None, 'prompt_tokens': 901, 'completion_tokens': 115}

GET /spend/logs?request_id=cfabd49d-847d-4adb-8f40-e92879d6fbc7

[{'model': 'vertex_ai/deepseek-ocr-maas', 'spend': 0.0, 'prompt_tokens': 901, 'completion_tokens': 115, 'call_type': 'aocr'}]

After (`24299c3`)

Head is 4f99996 since this run: the only commit after 24299c3 wraps a docstring line in litellm/cost_calculator.py, which cannot change behavior

Cost-map name vertex_ai/deepseek-ai/deepseek-ocr-maas

POST /v1/ocr, priced at 901 x $0.30/1M + 98 x $1.20/1M

HTTP/1.1 200 OK
x-litellm-call-id: d6adfcce-77db-468d-9d81-ae90541ebc0a
x-litellm-response-cost: 0.0003879
x-litellm-response-cost-original: 0.0003879
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0
x-litellm-response-cost-input: 0.0002703
x-litellm-response-cost-output: 0.0001176
x-litellm-response-cost-tool-usage: 0.0
x-litellm-key-spend: 0.0003879
vertex_ai/deepseek-ai/deepseek-ocr-maas {'pages_processed': None, 'prompt_tokens': 901, 'completion_tokens': 98}

GET /spend/logs?request_id=d6adfcce-77db-468d-9d81-ae90541ebc0a

[{'model': 'vertex_ai/deepseek-ai/deepseek-ocr-maas', 'spend': 0.0003879, 'prompt_tokens': 901, 'completion_tokens': 98, 'call_type': 'aocr'}]

Short name vertex_ai/deepseek-ocr-maas

POST /v1/ocr, priced at 901 x $0.30/1M + 370 x $1.20/1M

HTTP/1.1 200 OK
x-litellm-call-id: 7af206e7-e2a2-456b-bc60-ef8029a9b8c3
x-litellm-response-cost: 0.0007143
x-litellm-response-cost-original: 0.0007143
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0
x-litellm-response-cost-input: 0.0002703
x-litellm-response-cost-output: 0.000444
x-litellm-response-cost-tool-usage: 0.0
x-litellm-key-spend: 0.0007143
vertex_ai/deepseek-ocr-maas {'pages_processed': None, 'prompt_tokens': 901, 'completion_tokens': 370}

GET /spend/logs?request_id=7af206e7-e2a2-456b-bc60-ef8029a9b8c3

[{'model': 'vertex_ai/deepseek-ocr-maas', 'spend': 0.0007143, 'prompt_tokens': 901, 'completion_tokens': 370, 'call_type': 'aocr'}]

QA notes, none caused or changed by this PR:

Master key x-litellm-key-spend shows the request's own cost, not a running total
Before leg's short name emits only the -* breakdown headers, no bare total
The two legs got different completion token counts for the same image

Type

🐛 Bug Fix

Caveats (if any)

Medium

Keys and teams near their budget now get 429s on DeepSeek OCR calls, since those calls are finally charged
- Seen live: a key with max_budget: 0.0002 made one OCR call, then its next one got 429 Budget has been exceeded

Low

The OCR response's model field is now always deepseek-ai/deepseek-ocr-maas
- Before, it was whatever Vertex echoed back, which was already that name in every run here
- Proxy clients still see the name they requested, unless the deployment sets return_raw_model_name
The Vertex OCR docs page still quotes a flat "$0.0005 per page" that never applied to DeepSeek; docs follow-up
x-litellm-response-cost can carry float noise (0.00041430000000000004), the same formatting other providers' cost headers already have
Reducto OCR and the Rust OCR path were not exercised live (no key, litellm_rust not installed); the page-priced branch was verified live with Mistral OCR and is unchanged

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
24299c3 passes /live-pr-risk

Note

Medium Risk
Changes OCR spend calculation and enables non-zero charges for DeepSeek OCR (including budget 429s); page-priced OCR paths are preserved when pages are reported.

Overview
Fixes $0 spend on Vertex AI DeepSeek OCR when responses report token usage but not pages_processed.

OCR billing now uses _ocr_token_cost in ocr_cost: if the model has no applicable per-page (or annotation) pricing but has input_cost_per_token / output_cost_per_token and integer prompt_tokens / completion_tokens, cost is computed from tokens. Page-based pricing still wins when pages_processed and ocr_cost_per_page apply (e.g. Mistral OCR).

DeepSeek OCR responses (Python and Rust) normalize the returned model to the canonical deepseek-ai/deepseek-ocr-maas via _provider_model_name / deepseek_model_name, so short deployment names still match the cost map. The vertex_ai/deepseek-ai/deepseek-ocr-maas entry drops the unsourced ocr_cost_per_page so billing aligns with Google’s per-token rates.

Tests cover token fallback, page-priority, and end-to-end pricing for both model name forms.

^{Reviewed by Cursor Bugbot for commit 4f99996. Bugbot is set up for automated code reviews on this repo. Configure here.}

DeepSeek OCR on Vertex AI reports prompt and completion tokens and never a page count, so the OCR cost calculator either raised (cost-map name, no cost header at all) or returned 0.0 (short name, no cost-map entry). OCR cost now falls back to the model's token rates when no page pricing applies, the DeepSeek transform reports the canonical deepseek-ai/ model on the response so the short deployment name resolves the cost-map entry, and the unsourced ocr_cost_per_page is dropped from that entry since Google bills it per token. The Rust OCR port mirrors the canonical response model.

greptile-apps · 2026-09-02T21:53:01Z

Greptile Summary

This PR prices Vertex AI DeepSeek OCR responses from reported token usage when page usage is unavailable.

Adds token-based OCR cost fallback while preserving page-based pricing precedence.
Canonicalizes DeepSeek OCR response model names in the Python and Rust transformations.
Removes the obsolete per-page rate from both model-price maps.
Adds transformation and cost-calculation regression tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/cost_calculator.py	Adds prompt and completion token-cost calculation for OCR responses that lack applicable page pricing.
litellm/llms/vertex_ai/ocr/deepseek_transformation.py	Canonicalizes DeepSeek OCR response model names so token pricing resolves for short deployment names.
litellm-rust/crates/core/src/providers/vertex_ai/ocr/transformation.rs	Aligns Rust DeepSeek OCR response model normalization with the Python implementation.
model_prices_and_context_window.json	Removes the obsolete DeepSeek OCR per-page rate while retaining token rates.
litellm/model_prices_and_context_window_backup.json	Keeps the backup model-price map synchronized with the primary map.
tests/test_litellm/llms/vertex_ai/ocr/test_deepseek_transformation.py	Covers canonical response naming and nonzero token pricing for both supported model-name forms.
tests/test_litellm/test_cost_calculator.py	Covers token fallback, page-pricing precedence, cost splitting, and missing-token behavior.

_{Reviews (2): Last reviewed commit: "style(cost): wrap an overlong docstring ..." | Re-trigger Greptile}

greptile-apps · 2026-09-02T21:53:04Z

+
+
+@pytest.mark.parametrize("model", ["deepseek-ocr-maas", "deepseek-ai/deepseek-ocr-maas"])
+def test_response_is_priced_from_token_usage_for_either_model_name(local_model_cost_map: None, model: str) -> None:


Test lines exceed limit

Several newly added test declarations and assertions exceed the repository's 120-character Python line limit, including this line, line 45, and tests/test_litellm/test_cost_calculator.py:4480; wrap them to keep formatting and lint checks passing.

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

codspeed-hq · 2026-09-02T21:53:07Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_vertex_deepseek_ocr_pricing (4f99996) with litellm_internal_staging (6c5fb0e)¹}

No successful run was found on litellm_internal_staging (3c6b070) during the generation of this report, so 6c5fb0e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

codecov · 2026-09-02T22:03:34Z

Codecov Report

❌ Patch coverage is 91.30435% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/cost_calculator.py	94.73%	1 Missing ⚠️
...ellm/llms/vertex_ai/ocr/deepseek_transformation.py	75.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

The Returns line of the ocr_cost docstring ran to 126 columns, past the repo's 120-column limit. Wrapped it; no behavior change.

mateo-berri · 2026-09-02T23:24:08Z

@greptileai

mateo-berri · 2026-09-02T23:37:24Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4f99996. Configure here.}

greptile-apps Bot reviewed Sep 2, 2026

View reviewed changes

style(cost): wrap an overlong docstring line in ocr_cost

4f99996

The Returns line of the ocr_cost docstring ran to 126 columns, past the repo's 120-column limit. Wrapped it; no behavior change.

cursor Bot reviewed Sep 2, 2026

View reviewed changes

mateo-berri requested a review from yucheng-berri September 2, 2026 23:43

mateo-berri enabled auto-merge September 2, 2026 23:44

devin-ai-integration Bot mentioned this pull request Sep 3, 2026

fix(model_prices): verified registry audit, Databricks Sep-2026 catalog, realtime image pricing, deprecation dates #39388

Merged

7 tasks

yucheng-berri approved these changes Sep 3, 2026

View reviewed changes

devin-ai-integration Bot mentioned this pull request Sep 5, 2026

fix(pricing): rolling model registry update: Gemini lyria-3.5, Voyage multilingual-2, ChatGPT GPT-5.5/5.6, Vertex Haiku 4.5 output, Bedrock Mantle gpt-oss rates, Scaleway deprecation dates #31884

Open

7 tasks

Navigation Menu

Uh oh!

Uh oh!

fix(cost): price Vertex AI DeepSeek OCR by token usage - #39414

fix(cost): price Vertex AI DeepSeek OCR by token usage#39414
mateo-berri wants to merge 2 commits into
litellm_internal_stagingfrom
litellm_vertex_deepseek_ocr_pricing

mateo-berri commented Sep 2, 2026 •

edited by cursor Bot

Loading

Uh oh!

greptile-apps Bot commented Sep 2, 2026 •

edited

Loading

Important Files Changed

Uh oh!

greptile-apps Bot Sep 2, 2026

Uh oh!

codspeed-hq Bot commented Sep 2, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 2, 2026

Uh oh!

mateo-berri commented Sep 2, 2026

Uh oh!

mateo-berri commented Sep 2, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants



		@pytest.mark.parametrize("model", ["deepseek-ocr-maas", "deepseek-ai/deepseek-ocr-maas"])
		def test_response_is_priced_from_token_usage_for_either_model_name(local_model_cost_map: None, model: str) -> None:

Uh oh!

Uh oh!

Conversation

mateo-berri commented Sep 2, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (ba2e5d2)

Cost-map name vertex_ai/deepseek-ai/deepseek-ocr-maas

Short name vertex_ai/deepseek-ocr-maas

After (24299c3)

Cost-map name vertex_ai/deepseek-ai/deepseek-ocr-maas

Short name vertex_ai/deepseek-ocr-maas

Type

Caveats (if any)

Medium

Low

Final Attestation

Uh oh!

greptile-apps Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

greptile-apps Bot Sep 2, 2026

Choose a reason for hiding this comment

Uh oh!

codspeed-hq Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

codecov Bot commented Sep 2, 2026

Codecov Report

Uh oh!

mateo-berri commented Sep 2, 2026

Uh oh!

mateo-berri commented Sep 2, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mateo-berri commented Sep 2, 2026 •

edited by cursor Bot

Loading

Before (`ba2e5d2`)

After (`24299c3`)

greptile-apps Bot commented Sep 2, 2026 •

edited

Loading

codspeed-hq Bot commented Sep 2, 2026 •

edited

Loading