Skip to content

Fix spend tracking for OCR/aOCR requests (log pages_processed + recognize OCRResponse) - #16070

Merged
ishaan-jaff merged 2 commits into
BerriAI:mainfrom
OrionCodeDev:main
Oct 30, 2025
Merged

Fix spend tracking for OCR/aOCR requests (log pages_processed + recognize OCRResponse)#16070
ishaan-jaff merged 2 commits into
BerriAI:mainfrom
OrionCodeDev:main

Conversation

@OrionCodeDev

Copy link
Copy Markdown
Contributor

Type

🐛 Bug Fix

Changes

This merge request fixes an issue where OCR requests did not accrue spend in the database—the spend columns remained unchanged because OCR responses don’t expose token-based usage like text models. After this change, OCR costs are correctly computed and persisted.

What was wrong

  • OCR/AOCR responses report usage_info (with pages_processed) instead of the usual token-based usage.
  • Since the logger and spend tracker only looked for usage, OCR requests were treated as having no billable usage, leaving spend at 0.

What this MR does

  1. Recognize OCR responses in logging pipeline

    • litellm/litellm_core_utils/litellm_logging.py
      • Add OCRResponse to _is_recognized_call_type_for_logging, ensuring OCR calls are handled by the standard logging path.
  2. Translate OCR usage_info → normalized usage for spend tracking

    • litellm/proxy/spend_tracking/spend_tracking_utils.py
      • In get_logging_payload():
        • Normalize response_obj to a dict early.
        • When call_type in ["ocr", "aocr"], read usage_info (supports dict, Pydantic .model_dump(), or __dict__).
        • Build a synthetic usage object with:
          • prompt_tokens = 0, completion_tokens = 0, total_tokens = 0
          • pages_processed populated from usage_info.pages_processed (default 0).
        • For non-OCR calls, keep existing token-based behavior.

Result / Impact

  • Spend tracking now correctly accounts for OCR workloads on a per-page basis.
  • The database spend fields update as expected for OCR/AOCR calls.
  • No impact on non-OCR code paths.

Test plan (proposed)

  • Add a unit test under tests/litellm/spend_tracking/test_ocr_spend_tracking.py that:
    1. Mocks an OCR response carrying usage_info = {"pages_processed": N}.
    2. Invokes get_logging_payload(call_type="ocr", ...).
    3. Asserts the returned payload includes usage.pages_processed == N and that spend is computed from pages (not tokens).
  • Include a local run screenshot of the new test passing and make test-unit green.

Backwards compatibility

  • Fully backward compatible. Only extends logging/spend tracking for ocr/aocr calls and does not alter token-based accounting.

Notes

  • This implementation intentionally keeps token counts at 0 for OCR and relies on pages_processed to drive cost computation.
  • If future OCR providers introduce token-like metrics, the normalization layer can be extended similarly.

@vercel

vercel Bot commented Oct 29, 2025

Copy link
Copy Markdown

@OrionCodeDev is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Oct 29, 2025

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ishaan-jaff ishaan-jaff left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed, minor changes pls

Comment thread litellm/proxy/spend_tracking/spend_tracking_utils.py
@OrionCodeDev

Copy link
Copy Markdown
Contributor Author

@ishaan-jaff i push updated version with helper and some unit test (unit tests is not my best pros)

@ishaan-jaff ishaan-jaff left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ishaan-jaff
ishaan-jaff merged commit 5f52533 into BerriAI:main Oct 30, 2025
3 of 7 checks passed
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants