feat(ocr): add Cohere Parse support for cohere and azure_ai - #39862
feat(ocr): add Cohere Parse support for cohere and azure_ai#39862mateo-berri wants to merge 3 commits into
Conversation
Greptile SummaryAdds Cohere Parse OCR support for direct Cohere and Azure AI Foundry deployments, including request and response transformations, image inlining for Foundry, native responses, health-check inputs, endpoint metadata, pricing, and focused tests
Confidence Score: 5/5The PR appears safe to merge with no new actionable failures identified The changes since the previous review correctly add provider-specific health-check documents and capability metadata without introducing a confirmed regression. The earlier Azure routing thread was manually resolved without explanation
|
| Filename | Overview |
|---|---|
| litellm/llms/cohere/ocr/transformation.py | Implements Cohere Parse request validation, transformation, response normalization, native payload handling, and image-only health checks |
| litellm/llms/azure_ai/ocr/cohere_parse_transformation.py | Adapts Cohere Parse for Azure AI Foundry authentication, endpoint construction, and remote-image inlining |
| litellm/litellm_core_utils/health_check_helpers.py | Selects health-check documents through each provider’s OCR configuration |
| litellm/llms/azure_ai/ocr/common_utils.py | Adds Cohere Parse selection while retaining existing Document Intelligence and Mistral routing |
| litellm/ocr/main.py | Lets provider OCR configurations opt out of the Rust bridge |
| model_prices_and_context_window.json | Adds OCR mode and per-page pricing for direct and Azure-hosted Cohere Parse models |
Reviews (3): Last reviewed commit: "fix(ocr): send each provider a health-ch..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
bugbot run |
Health checks probed every OCR deployment with a PDF, which Cohere Parse rejects, so /health, background health checks, and the UI Test Connection button marked Cohere Parse deployments unhealthy. BaseOCRConfig gains a get_health_check_document hook (PDF by default) that CohereParseConfig overrides with a 1x1 PNG data URI. cohere also gains ocr in the provider endpoint matrix
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 004a820. Configure here.
1 participant
TLDR
Problem this solves:
/v1/ocrcannot serve Cohere Parse:cohere/returns 500, Azure Foundry deployments 404How it solves it:
cohereOCR config posting to Cohere'sPOST /v2/parseazure_airoutes deployments named with bothcohereandparse(the catalog defaultCohere-parse-v5) to Foundry's Cohere Parse API, inlining remote images/health, background health checks, the UI Test Connection button) probe each OCR deployment with a document its provider accepts: a newBaseOCRConfig.get_health_check_document()hook keeps the shared PDF for Mistral and Document Intelligence and sends a 1x1 PNG data URI for Cohere Parse, so Cohere Parse deployments no longer show as unhealthycoheregainsocrin the provider endpoint matrix served by/public/providersUser Flow
Before: a proxy admin adds Cohere Parse to
model_listand every/v1/ocrrequest fails before any text comes backcohere-parse(cohere/parse-v5.0with their Cohere API key) andazure-cohere-parse(azure_ai/Cohere-parse-v5with their Foundry endpoint and key) tomodel_listand start the proxyPOST https://litellm-domain/v1/ocrwith{"model": "cohere-parse", "document": {"type": "image_url", "image_url": "https://.../receipt.png"}}litellm.APIConnectionError: OCR is not supported for provider: coherePOST https://litellm-domain/v1/ocrwith"model": "azure-cohere-parse"NotFoundError: Azure_aiException - {"error":{"code":"api_not_supported","message":"Requested API is currently not supported"}}GET https://litellm-domain/healthlists both deployments underunhealthy_endpointswith those same errorsAfter: the same two requests return the parsed page as markdown with a page count and a per-page cost
cohere-parse(cohere/parse-v5.0with their Cohere API key) andazure-cohere-parse(azure_ai/Cohere-parse-v5with their Foundry endpoint and key) tomodel_listand start the proxyPOST https://litellm-domain/v1/ocrwith{"model": "cohere-parse", "document": {"type": "image_url", "image_url": "https://.../receipt.png"}}{"pages": [{"index": 0, "markdown": "# PLACE FACE UP ON DASH ..."}], "model": "parse-v5.0", "usage_info": {"pages_processed": 1}, "object": "ocr"}and the headerx-litellm-response-cost: 0.0015POST https://litellm-domain/v1/ocrwith"model": "azure-cohere-parse"https://image URLs"output_format": "blocks"returns Cohere's block layout underpages[].blocks, and"req_format": "native"returns Cohere's own response body unchangeddocument_urlor a PDF gets HTTP 400Cohere Parse only accepts image_url documents (an image URL or a base64 image data URI); document_url and PDF inputs are not supported.instead of a provider error, and every successful call carriesx-litellm-response-cost: 0.0015, the per-page price spend tracking usesfilefield instead of a URL (-F model=cohere-parse -F file=@receipt.png, or-F model=azure-cohere-parse) returns the same page at the same cost from either deploymentGET https://litellm-domain/healthlists both deployments underhealthy_endpoints(the UI Test Connection button runs that same probe)Relevant issues
Supersedes #39427
Linear ticket
Resolves LIT-6992
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Setup (shared by both legs): one proxy per leg, each started with
python litellm/proxy/proxy_cli.py --config config.yaml --port <port> --num_workers 2(two uvicorn workers, single instance, no database,LITELLM_LOCAL_MODEL_COST_MAP=True, keys read from.env). The Before leg is a detached worktree at the merge base 2151dcb on port 56915, the After leg is the PR tip 004a820 on port 37572. Both legs run the same curl script; the only difference is which commit the proxy was booted fromThe Azure side is a Foundry resource with the catalog model
Cohere-parse-v5deployed under that same name (GlobalStandard, 3 requests a minute, so both legs sleep between Azure calls). The document is the public receipt imagehttps://raw.githubusercontent.com/mistralai/cookbook/refs/heads/main/mistral/ocr/receipt.pngBefore (2151dcb)
cohere_markdown
cohere_blocks
cohere_native
cohere_pdf_rejected
cohere_bad_output_format
azure_markdown
azure_blocks
azure_native
azure_pdf_rejected
cohere_multipart_file
curl -s -D - http://localhost:56915/v1/ocr -H 'Authorization: Bearer $LITELLM_MASTER_KEY' -F model=cohere-parse -F file=@receipt.pnghealth
curl -s http://localhost:56915/health -H 'Authorization: Bearer $LITELLM_MASTER_KEY'After (004a820)
cohere_markdown
cohere_blocks
cohere_native
cohere_pdf_rejected
cohere_bad_output_format
azure_markdown
azure_blocks
azure_native
azure_pdf_rejected
cohere_multipart_file
curl -s -D - http://localhost:37572/v1/ocr -H 'Authorization: Bearer $LITELLM_MASTER_KEY' -F model=cohere-parse -F file=@receipt.pngazure_cohere_multipart_file
curl -s -D - http://localhost:37572/v1/ocr -H 'Authorization: Bearer $LITELLM_MASTER_KEY' -F model=azure-cohere-parse -F file=@receipt.pnghealth
curl -s http://localhost:37572/health -H 'Authorization: Bearer $LITELLM_MASTER_KEY'Health check at the previous tip d3a179f
Before the health-check hook, every Cohere Parse deployment failed its
ocrhealth probe because the shared PDF document was sent to Parse. The same two SDK calls (real keys, no proxy) at the previous tip:Type
🆕 New Feature
Caveats (if any)
Medium
cohereandparsegoes to Cohere Parse, anything else (parse-v5,north-parse) stays on the Mistral OCR path and gets the Mistral 404 it gets today. Foundry takes the body'smodelas the deployment name, so the LiteLLM model name has to be the deployment name and there is no per-deployment override; the workaround is a deployment name with both words, which the catalog defaultCohere-parse-v5has. Cost tracking keys on the name too: onlyazure_ai/Cohere-parse-v5is in the cost map, so a deployment under another name records $0 spend with a warning log unless its entry setsmodel_info.base_model: azure_ai/Cohere-parse-v5. The docs (litellm-docs#1215) state both rules/azure_ai/...with the deployment's own URL) is not touched here: that failure is owned by fix(azure_ai): add passthrough config so router-model relays reach the deployment's own endpoint #39863 (LIT-7022). Until it lands,/v1/ocris the working path for Foundry Cohere ParseLow
document_urlinputs return 400 by design because the live Cohere Parse API rejects anything but PNG, JPEG, WebP, and GIF today; if Cohere adds PDF support the guard has to be relaxedcohere/prefix routes to Parse;cohere_chat/parse-v5.0is not acceptedapi_basethat already ends in/v1gets/v2/parseappended verbatim, so Cohere-compatible gateways must be configured with the host onlyx-litellm-response-costheader onlyazure_ai/Cohere-parse-v5cost map entry carries Cohere's list price ($1.50 per 1,000 pages): Azure's Foundry Cohere pricing page has no Parse row and the Foundry catalog's View pricing link points at Cohere's blog, so the entry needs updating if Azure publishes its own row, and until then a Foundry invoice can differ from what LiteLLM recordsahealth_checkwith nomode, was called live for both models and came back healthy after resolvingocrfrom the cost mapazure_ai/my-parse-deploy) health-checks aschatunlessmodel_info.mode: ocris set, the same convention every other non-chat model followsdeprecation_datefor/model/deprecations) and intermittently answersapi_not_supportedorRateLimitReachedon valid requests; LiteLLM surfaces those asNotFoundErrorandRateLimitError, and the same call succeeds on retrytests/e2e/case or coverage-registry row is added; coverage is unit tests plus the live curl legs belowcohere/...model sent to/v1/ocrgets the Parse config, so a non-Parse Cohere model fails at Cohere's API rather than with a LiteLLM errorCOHERE_API_BASEis not read for OCR (only an explicitapi_baseis), matching the Mistral OCR config, which ignoresMISTRAL_API_BASEthe same waysupports_rust_bridge()returning False for both configs) is covered by unit tests only; the live legs ran the Python pathfileupload has an After leg on both providers but a Before leg only on Cohere; at the merge base the Foundry deployment takes the same Mistral OCR path as the JSON Before leg and gets the sameapi_not_supported404proxy_store_model_in_db_testsis red on this branch only: staging's test(store_model_in_db): assert the 400 contract in the unknown-model spend log test #39842 relaxed that spend-log assertion after this branch's merge base (LIT-6949 tracks the flake), so the merge onto staging fixes itFinal Attestation