Skip to content

feat(cloudflare): add embeddings and rerank support - #35054

Open
prdai wants to merge 8 commits into
BerriAI:mainfrom
prdai-archive:feat/cloudflare-workers-ai-21115
Open

prdai wants to merge 8 commits into
BerriAI:mainfrom
prdai-archive:feat/cloudflare-workers-ai-21115

Conversation

@prdai

@prdai prdai commented Jul 29, 2026

Copy link
Copy Markdown

What changed

  • add Cloudflare Workers AI embeddings through the OpenAI-compatible embeddings endpoint
  • add Cloudflare reranking through the native Workers AI run endpoint
  • translate LiteLLM rerank documents and top_n into Cloudflare contexts and top_k
  • normalize Cloudflare rerank responses into LiteLLM's standard response shape
  • update provider capability metadata and add focused transformation and routing tests

Why

The Cloudflare provider only exposed chat behavior even though Workers AI now supports additional API surfaces. This adds embeddings and reranking while keeping image and audio work separately scoped.

Validation

  • 13 Cloudflare embedding and reranking tests passed
  • Ruff passed
  • strict-rule budget gate passed
  • type-discipline budget gate passed

Related to #21115

Documentation PR: BerriAI/litellm-docs#704

@CLAassistant

CLAassistant commented Jul 29, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.32432% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/main.py 88.88% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@prdai
prdai force-pushed the feat/cloudflare-workers-ai-21115 branch 3 times, most recently from 50d09ae to ca1b841 Compare July 29, 2026 06:27
@codspeed

codspeed Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing prdai-archive:feat/cloudflare-workers-ai-21115 (86acd9d) with litellm_internal_staging (c2c2a62)

Open in CodSpeed

@prdai
prdai marked this pull request as ready for review July 29, 2026 09:48
Comment thread litellm/llms/cloudflare/rerank/transformation.py
@greptile-apps

greptile-apps Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds Cloudflare Workers AI embedding and rerank support

  • Routes embeddings through Cloudflare's OpenAI-compatible endpoint
  • Translates unified rerank requests and native Cloudflare responses
  • Registers provider configurations, capability metadata, lazy imports, and focused tests

Confidence Score: 4/5

The missing Cloudflare return_documents behavior should be fixed before merging because the default rerank response no longer satisfies the requested normalized result shape

Cloudflare calls are reachable through the new public rerank route, but the adapter drops a defaulted unified parameter and omits document data even though the submitted contexts remain available for reconstruction

Files Needing Attention: litellm/llms/cloudflare/rerank/transformation.py

Important Files Changed

Filename Overview
litellm/llms/cloudflare/embedding/transformation.py Adds an embedding configuration that resolves Cloudflare's OpenAI-compatible endpoint and reuses the shared HTTP handler
litellm/llms/cloudflare/rerank/transformation.py Adds native rerank request and response translation, but drops the public return_documents behavior
litellm/main.py Adds Cloudflare to the shared embedding dispatch path with API-key and base-URL resolution
litellm/utils.py Registers the new Cloudflare embedding and rerank provider configurations
litellm/rerank_api/main.py Extends the public rerank provider type declarations to include Cloudflare

Reviews (1): Last reviewed commit: "test(cloudflare): cover embedding dispat..." | Re-trigger Greptile

@prdai
prdai force-pushed the feat/cloudflare-workers-ai-21115 branch 2 times, most recently from a47dd01 to 56de9db Compare July 29, 2026 09:58
Comment thread litellm/llms/cloudflare/rerank/transformation.py Outdated
@veria-ai

veria-ai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

@prdai
prdai force-pushed the feat/cloudflare-workers-ai-21115 branch from 56de9db to 21034da Compare July 29, 2026 10:48
@yuneng-berri
yuneng-berri deleted the branch BerriAI:main September 13, 2026 04:36
@prdai

prdai commented Sep 13, 2026 •

Copy link
Copy Markdown
Author

hi @yuneng-berri is there a plan to add cloudflare updated api support to litellm? if so let me know how i can help out, tnx

@yuneng-berri

Copy link
Copy Markdown
Contributor

@prdai Hey sorry for the friction. reopening the PR now

@yuneng-berri yuneng-berri reopened this Sep 13, 2026
@prdai
prdai force-pushed the feat/cloudflare-workers-ai-21115 branch from 21034da to 880e4a4 Compare September 13, 2026 09:36
@prdai

prdai commented Sep 13, 2026

Copy link
Copy Markdown
Author

hi @yuneng-berri i rebased this branch, the ci/cd failing seems to be resolved from #40996 let me know how we can work on getting this merged thanks! :)

@devin-ai-integration
devin-ai-integration Bot changed the base branch from litellm_internal_staging to main September 23, 2026 15:25
@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 23, 2026 15:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants