feat(batches): support Mistral files/batches and per-page OCR batch cost tracking

[mubashir1osmani]

TLDR

Problem this solves:

• Mistral cannot be used as a Files or Batches provider through LiteLLM
• /v1/ocr is rejected as a batch endpoint, so OCR batches are impossible
• Batch cost tracking is token-based; OCR is billed per page

How it solves it:

• Adds MistralFilesConfig and MistralBatchesConfig on the shared HTTP handler path
• Accepts /v1/ocr as a batch endpoint
• Prices OCR batch output lines per page at a new ocr_cost_per_page_batches rate
• Fixes GET /v1/files/{id} sending model-routed non-OpenAI ids to api.openai.com
• Enforces key/team/org/project model grants before model-routed and unified-id file and batch credentials are resolved, including before a unified batch id is served from the database's cached terminal result
• Fixes a TypeError in pre-call logging when a presigned batch retrieve passes api_base=None
• Masks the auth header a pre-signed batch or file request embeds in its body before pre-call raw-request logging, so the provider key never lands in raw_request logs or callbacks

User Flow

Before: a developer who wants Mistral's discounted OCR batches cannot get past the upload, so the whole job runs outside LiteLLM's auth and spend tracking

1. They send POST http://localhost:4000/v1/files with purpose=batch, model=mistral-ocr and a JSONL of /v1/ocr requests and get 400 LiteLLM doesn't support mistral for 'create_file'. Only ['openai', 'azure', 'vertex_ai', 'manus', 'anthropic'] are supported.
2. No file id comes back, so GET http://localhost:4000/v1/files/{id} and the output download never happen
3. They send POST http://localhost:4000/v1/batches with endpoint: /v1/ocr, model: mistral-ocr anyway and get 400 LiteLLM doesn't support custom_llm_provider=mistral for 'create_batch'
4. They send GET http://localhost:4000/v1/files?purpose=user_data&provider=mistral and get 400 LiteLLM doesn't support mistral for 'file_list'
5. https://litellm-domain/ui/?page=logs shows only $0 Failure rows, and they call api.mistral.ai directly, so none of the OCR spend shows up there

After: the same developer runs the whole OCR batch lifecycle through the gateway and sees per-page spend

1. They send POST http://localhost:4000/v1/files with purpose=batch, model=mistral-ocr and the same JSONL
2. The upload succeeds and hands back a file-... id
3. They send GET http://localhost:4000/v1/files/{that id} and get 200 with the file's metadata
4. They send POST http://localhost:4000/v1/batches with endpoint: /v1/ocr, model: mistral-ocr and get back a batch object with status: validating
5. They poll GET http://localhost:4000/v1/batches/{id} until status: completed, then GET http://localhost:4000/v1/files/{output_file_id}/content to download the OCR results
6. https://litellm-domain/ui/?page=logs shows the batch at the per-page batch rate (3 one-page documents on mistral-ocr-latest = $0.006, half the $0.012 sync price)
7. A teammate whose key is restricted to claude-haiku sends GET http://localhost:4000/v1/files/{that id} and gets 403 The requested model 'mistral-ocr' is not available for this API key, or the model name is invalid, so the Mistral deployment's key is never used on their behalf

Relevant issues

Fixes #29914

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

/qa verdict: PASS (before = fail, after = succeeds). Two legs, each in its own worktree, venv and Postgres database, booted with python litellm/proxy/proxy_cli.py --config config.yaml --port <port> --num_workers 2 and LITELLM_LOCAL_MODEL_COST_MAP=True, against the real Mistral API with a real key, so the After batch cost real money. Before is the merge base faed57f on :21905, After is 5783a38 on :33468. Both legs use the same config and the same 3-line input, and the flow sends each request exactly once

model_list:
  - model_name: mistral-ocr
    litellm_params:
      model: mistral/mistral-ocr-latest
      api_key: os.environ/MISTRAL_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

ocr_batch_input.jsonl is 3 lines, each a one-page PDF as a base64 data URI:

{"custom_id": "doc-0", "method": "POST", "url": "/v1/ocr", "body": {"document": {"type": "document_url", "document_url": "data:application/pdf;base64,JVBERi0x..."}}}

Before (faed57f)

1. Upload (User Flow step 1):

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:21905/v1/files -H 'Authorization: Bearer sk-1234' -F purpose=batch -F file=@ocr_batch_input.jsonl -F model=mistral-ocr
{"error":{"message":"litellm.BadRequestError: LiteLLM doesn't support mistral for 'create_file'. Only ['openai', 'azure', 'vertex_ai', 'manus', 'anthropic'] are supported.","type":"invalid_request_error","param":null,"code":"400"}}
http=400

2. No file id came back, so the retrieve (step 3) has nothing to fetch. Batch create (step 4) with the empty id:

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:21905/v1/batches -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"input_file_id": "", "endpoint": "/v1/ocr", "completion_window": "24h", "model": "mistral-ocr"}'
{"error":{"message":"litellm.BadRequestError: LiteLLM doesn't support custom_llm_provider=mistral for 'create_batch'","type":"internal_server_error","param":null,"code":"400"}}
http=400

3. Polling and the output download (step 5) cannot run without a batch. Spend (step 6): GET /spend/logs/v2 over the run's window returned [] at query time and the logs page shows the three calls as $0 Failure rows, nothing billable:

4. Listing files under the OCR purpose:

$ curl -s -w '\nhttp=%{http_code}\n' 'http://localhost:21905/v1/files?purpose=user_data&provider=mistral' -H 'Authorization: Bearer sk-1234'
{"error":{"message":"litellm.BadRequestError: LiteLLM doesn't support mistral for 'file_list'. Only 'openai', 'azure', 'manus', and 'anthropic' are supported.","type":"invalid_request_error","param":null,"code":"400"}}
http=400

5. The restricted key (step 7) has no file to be refused on; with the empty id the route falls through to the admin-only check:

$ curl -s http://localhost:21905/key/generate -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"models":["claude-haiku"]}'
{"key_alias":null,"duration":null,"models":["claude-haiku"], ... ,"key":"sk-j7Y...", ... }

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:21905/v1/files/ -H 'Authorization: Bearer sk-j7Y...'
{"error":{"message":"Authentication Error, Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/v1/files/. Your role=unknown. Your user_id=*******","type":"auth_error","param":"None","code":"401"}}
http=401

After (5783a38)

1. Upload (steps 1 and 2):

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:33468/v1/files -H 'Authorization: Bearer sk-1234' -F purpose=batch -F file=@ocr_batch_input.jsonl -F model=mistral-ocr
{"id":"file-bGl0ZWxsbTphOGJkMzEwNS0xNGY3LTQ5ZTUtYjVmZC01NmQ2YzVhMzg3ZTg7bW9kZWwsbWlzdHJhbC1vY3I","bytes":3198,"created_at":1789795890,"filename":"ocr_batch_input.jsonl","object":"file","purpose":"batch","status":"uploaded","expires_at":null,"status_details":null}
http=200

2. Retrieve (step 3), which at the merge base could not even be reached:

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:33468/v1/files/$FILE_ID -H 'Authorization: Bearer sk-1234'
{"id":"file-bGl0ZWxsbTphOGJkMzEwNS0xNGY3LTQ5ZTUtYjVmZC01NmQ2YzVhMzg3ZTg7bW9kZWwsbWlzdHJhbC1vY3I","bytes":3198,"created_at":1789795890,"filename":"ocr_batch_input.jsonl","object":"file","purpose":"batch","status":"uploaded","expires_at":null,"status_details":null}
http=200

3. Create the OCR batch (step 4):

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:33468/v1/batches -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d "{\"input_file_id\": \"$FILE_ID\", \"endpoint\": \"/v1/ocr\", \"completion_window\": \"24h\", \"model\": \"mistral-ocr\"}"
{"id":"batch_bGl0ZWxsbTo0OTQzMWRiMS1lMmYzLTQ5NTUtYmYwOC0zZmJlMTc0MTEwNWU7bW9kZWwsbWlzdHJhbC1vY3I","completion_window":"24h","created_at":1789795891,"endpoint":"/v1/ocr","input_file_id":"file-bGl0ZWxsbTphOGJkMzEwNS0xNGY3LTQ5ZTUtYjVmZC01NmQ2YzVhMzg3ZTg7bW9kZWwsbWlzdHJhbC1vY3I","object":"batch","status":"validating","cancelled_at":null,"cancelling_at":null,"completed_at":null,"error_file_id":null,"errors":null,"expired_at":null,"expires_at":null,"failed_at":null,"finalizing_at":null,"in_progress_at":null,"metadata":null,"model":null,"output_file_id":null,"request_counts":{"completed":0,"failed":0,"total":0},"usage":null}
http=200

4. Poll (step 5), one GET /v1/batches/$BATCH_ID every 15 s: in_progress at 05:31:31Z, in_progress at 05:31:47Z, completed at 05:32:02Z:

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:33468/v1/batches/$BATCH_ID -H 'Authorization: Bearer sk-1234'
{"id":"batch_bGl0ZWxsbTo0OTQzMWRiMS1lMmYzLTQ5NTUtYmYwOC0zZmJlMTc0MTEwNWU7bW9kZWwsbWlzdHJhbC1vY3I","completion_window":"24h","created_at":1789795891,"endpoint":"/v1/ocr","input_file_id":"file-bGl0ZWxsbTphOGJkMzEwNS0xNGY3LTQ5ZTUtYjVmZC01NmQ2YzVhMzg3ZTg7bW9kZWwsbWlzdHJhbC1vY3I","object":"batch","status":"completed","cancelled_at":null,"cancelling_at":null,"completed_at":1789795912,"error_file_id":null,"errors":null,"expired_at":null,"expires_at":null,"failed_at":null,"finalizing_at":null,"in_progress_at":1789795891,"metadata":null,"model":null,"output_file_id":"file-bGl0ZWxsbTo2ODkyODE1My01MTcyLTRmZmItYjIwOC05MDkxZTNlYzQ0MTQ7bW9kZWwsbWlzdHJhbC1vY3I","request_counts":{"completed":3,"failed":0,"total":3},"usage":null}
http=200

5. Download the OCR results (step 5):

$ curl -s http://localhost:33468/v1/files/$OUTPUT_FILE_ID/content -H 'Authorization: Bearer sk-1234' | head -c 700
{"id":"batch-49431db1-a8bd3105-0","custom_id":"doc-0","response":{"status_code":200,"body":{"pages":[{"index":0,"markdown":"Invoice 1001 total 42 USD","images":[],"dimensions":{"dpi":93,"height":1023,"width":791},"tables":[],"hyperlinks":[],"header":null,"footer":null,"confidence_scores":null,"blocks":[{"top_left_x":89,"top_left_y":92,"bottom_right_x":457,"bottom_right_y":130,"content":"Invoice 1001 total 42 USD","confidence_scores":null,"type":"text"}]}],"model":"mistral-ocr-latest","usage_info":{"pages_processed":1,"doc_size_bytes":600},"document_annotation":null}},"error":null}
{"id":"batch-49431db1-a8bd3105-1066","custom_id":"doc-1","response":{"status_code":200,"body":{"pages":[{"index"

6. Spend (step 6): GET /spend/logs/v2 over the run's window, one line per row, shows the batch billed at $0.006, which is 3 pages at the $0.002 per-page batch rate and half the $0.012 sync price:

$ curl -s "http://localhost:33468/spend/logs/v2?start_date=2026-09-19%2004:32:09&end_date=2026-09-19%2006:32:09&page=1&page_size=50" -H 'Authorization: Bearer sk-1234' | python3 -c 'import json,sys; d=json.load(sys.stdin); rows=d.get("data", d); print(json.dumps([{k: r.get(k) for k in ("request_id","call_type","model","spend","status")} for r in rows], indent=1))'
[
 {"request_id": "dcd68c2b-4fc8-4cf4-b985-2b7c80aaad14", "call_type": "afile_content", "model": "", "spend": 0.0, "status": "success"},
 {"request_id": "955d881d-7368-443e-aed6-b42f4c065b2d", "call_type": "afile_content", "model": "", "spend": 0.0, "status": "success"},
 {"request_id": "batch_bGl0ZWxsbTo0OTQzMWRiMS1lMmYzLTQ5NTUtYmYwOC0zZmJlMTc0MTEwNWU7bW9kZWwsbWlzdHJhbC1vY3I_batch_cost", "call_type": "aretrieve_batch", "model": "mistral-ocr", "spend": 0.006, "status": "success"},
 {"request_id": "batch_bGl0ZWxsbTo0OTQzMWRiMS1lMmYzLTQ5NTUtYmYwOC0zZmJlMTc0MTEwNWU7bW9kZWwsbWlzdHJhbC1vY3I", "call_type": "acreate_batch", "model": "mistral/mistral-ocr-latest", "spend": 0.0, "status": "success"},
 {"request_id": "file-bGl0ZWxsbTphOGJkMzEwNS0xNGY3LTQ5ZTUtYjVmZC01NmQ2YzVhMzg3ZTg7bW9kZWwsbWlzdHJhbC1vY3I", "call_type": "acreate_file", "model": "mistral/mistral-ocr-latest", "spend": 0.0, "status": "success"}
]

The logs page shows the same batch row at $0.006000, and its detail view shows 3 successful requests on mistral-ocr with cost $0.00600000:

7. Listing files under the OCR purpose (400 at the merge base, and the bugbot finding fixed on an earlier tip):

$ curl -s -w '\nhttp=%{http_code}\n' 'http://localhost:33468/v1/files?purpose=user_data&provider=mistral' -H 'Authorization: Bearer sk-1234'
{"object":"list","data":[],"first_id":null,"last_id":null,"has_more":false}
http=200

8. The restricted key (step 7) is refused before any Mistral call:

$ curl -s http://localhost:33468/key/generate -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"models":["claude-haiku"]}'
{"key_alias":null,"duration":null,"models":["claude-haiku"], ... ,"key":"sk-...REDACTED", ... }

$ curl -s -w '\nhttp=%{http_code}\n' http://localhost:33468/v1/files/$FILE_ID -H 'Authorization: Bearer sk-...REDACTED'
{"error":{"message":"The requested model 'mistral-ocr' is not available for this API key, or the model name is invalid. Check the models available to you and try again.","type":"key_model_access_denied","param":"model","code":"403"}}
http=403

Observations from both legs, none of them a defect in what this PR changes:

• Batch model, usage, expires_at always null: PR causes
• $0.006 lands on the _batch_cost row: pre-existing, leaves alone
• By-batch-id spend lookup shows only the $0 row: leaves alone
• Cost row model mistral-ocr, create row mistral/mistral-ocr-latest: PR causes
• Two afile_content rows, empty model, $0: leaves alone
• 403 body is the generic access-denied text: leaves alone
• File list logs call_type alist_fine_tuning_jobs: pre-existing, leaves alone
• Restricted key exercised on GET /v1/files/{id} only

Type

🆕 New Feature
🐛 Bug Fix

Caveats (if any)

Medium

• The 50% OCR batch discount comes from Mistral's blanket batch statement, not an OCR-specific price

Low

• Model-grant check also 403s cross-deployment file access on OpenAI, Azure, Vertex and Bedrock
• list_batches and cancel_batch are not wired for Mistral, same as Bedrock
• Mistral batch input lines need method and url, which Mistral ignores
• Fresh Mistral batches show request_counts.total: 0 until validation finishes
• Mistral batch objects come back with model, usage and expires_at null, and the batch cost row names the model group (mistral-ocr) where the create row names the deployment (mistral/mistral-ocr-latest)
• OCR files read back as purpose: user_data; user_data maps back onto ocr on upload and list
• assistants, vision and evals uploads get 400 instead of silently becoming batch

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches proxy authorization for file/batch credential routing and batch billing logic for OCR; both are security- and spend-sensitive but are covered by new tests.

Overview
Adds Mistral as a first-class Files and Batches provider (create/retrieve jobs, multipart uploads, purpose mapping for batch/ocr/fine-tune) and extends batch APIs to accept /v1/ocr with mistral in provider literals.

Batch spend accounting now detects OCR output lines via usage_info and prices them with new ocr_batch_cost using ocr_cost_per_page_batches / annotation_cost_per_page_batches (with sync-rate fallback); Mistral OCR models in the cost map gain those fields and /v1/batch in supported_endpoints.

On the proxy, model-encoded file/batch IDs and x-litellm-model routing go through get_authorized_credentials_for_model so restricted keys cannot borrow another deployment’s credentials; afile_retrieve forwards custom_llm_provider for model-routed retrieves (fixes Mistral ids hitting OpenAI). Logging tolerates api_base=None on presigned batch retrieves and masks auth headers embedded in presigned request bodies before raw-request logs.

^{Reviewed by Cursor Bugbot for commit f3b198c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing mubashir1osmani:litellm_mistral_ocr_batches (5783a38) with main (12ddb35)}

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no outstanding correctness, security, or repository-rule violations remain.

Findings

1.   Model Grants Are Bypassed ▶

Summary

Adds Mistral file and batch support, including OCR batch creation, retrieval, output handling, and per-page cost accounting.

• Extends shared file and batch HTTP handling with Mistral transformations.
• Adds OCR batch pricing fields and aggregates completed OCR output by processed page count.
• Hardens model-routed file and batch credential resolution and masks embedded authorization headers in request logging.
• Adds coverage for Mistral transformations, proxy routing, authorization, logging, and batch costing.

_{Reviews (8) · Last reviewed commit: "fix(proxy): enforce the unified batch mo..."}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 3 · PR risk: 0/10

[codecov]

Codecov Report

❌ Patch coverage is 95.53265% with 13 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/cost_calculator.py	83.33%	4 Missing ⚠️
litellm/llms/mistral/files/transformation.py	96.00%	4 Missing ⚠️
litellm/llms/custom_httpx/llm_http_handler.py	75.00%	2 Missing ⚠️
litellm/utils.py	66.66%	2 Missing ⚠️
...lm/proxy/openai_files_endpoints/files_endpoints.py	80.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mateo-berri]

Requesting changes only for the two required reds and the test banners, all this PR's own; diff and live QA (body) look right

[mubashir1osmani]

@greptileai

[mubashir1osmani]

@greptileai

[mubashir1osmani]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mubashir1osmani]

@mateo-berri please review

[mubashir1osmani]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f3b198c. Configure here.}

[mateo-berri]

@greptileai

[greptile-apps]

Model Grants Are Bypassed

For a model derived from the vector-store registry, this branch calls get_credentials_for_model without authorizing that model for the caller. Access to a vector store does not guarantee access to its backing deployment: teamless stores are available to every caller, and object permissions can grant store access independently of model grants. A key restricted to other models can therefore use credentials for a deployment it is not allowed to access. The model grant must be checked before resolving its credentials. This violates the repository directive to reject authentication-layer changes that may cause a security incident.

How this was verified: Registry-derived model hints reach the credential resolver without a model-grant check, while vector-store access can be granted independently through teamless stores, object permissions, or team membership.

Rule Used: What: Fail any PR which may contains a security incident on litellm's authentication layer Why: Do not cause security incidents Bad: ```python # Check cache first cache_key = ( f"oidc_userinfo_{token[:20]}" # Use fi... (source)

Knowledge Base Used: Proxy authentication and authorization

[mateo-berri]

Unchanged from main: #29739 authorizes only caller-supplied hints. Registry-bound stores are admin config gated by vector store permissions. Authorizing them would 403 existing store-permitted teams

[greptile-apps]

That distinction is valid. should_authorize_model_hint is only true when the model came from ?model/x-litellm-model; a model injected from the admin-controlled vector-store registry is intentionally not treated as a caller-supplied hint. Access is gated earlier by assert_user_can_access_vector_store_id, and adding the model-grant check here would change the established behavior for teams authorized to use that store. Since this matches the existing #29739 boundary and the diff does not introduce a new authorization bypass for registry-bound stores, I’m withdrawing this finding.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

Superseded by #41934, an internal-branch copy of this exact tip so CircleCI can run. Your commits and authorship carry over unchanged