Skip to content

feat: add LiteLLM Rust workspace with Mistral OCR bridge - #31032

Closed
ishaan-berri wants to merge 1 commit into
mainfrom
codex/litellm-rust-mistral-ocr
Closed

feat: add LiteLLM Rust workspace with Mistral OCR bridge#31032
ishaan-berri wants to merge 1 commit into
mainfrom
codex/litellm-rust-mistral-ocr

Conversation

@ishaan-berri

Copy link
Copy Markdown
Contributor

What

  • Add minimal top-level litellm-rust/ workspace with only the crates needed for the first bridge: core, providers, and python-bridge.
  • Shape Rust around LiteLLM route/provider responsibilities:
    • core/src/ocr/ owns OCR shared types and the OcrProviderConfig template trait.
    • providers/src/mistral/ocr/transformation.rs mirrors litellm/llms/mistral/ocr/transformation.py.
    • providers/src/ocr.rs owns Rust-side OCR provider/operation dispatch.
  • Keep the PyO3 bridge intentionally thin: it exports one OCR function, ocr(payload), and delegates OCR routing to Rust.
  • Add litellm/rust_bridge/ as the Python-side bridge package with an explicit OCR provider rollout enum and Python fallback behavior.
  • Keep litellm/ocr/main.py small: after normal provider resolution, it asks litellm.rust_bridge.ocr for a Rust-backed config; disabled, unavailable, or unsupported Rust paths return the existing Python config unchanged.
  • Add Rust CLAUDE.md/README rules for pure transforms, provider tree mapping, bridge shape, parity tests, and future host I/O boundaries.
  • Add a path-scoped Rust GitHub Actions job that runs cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, and cargo test --workspace on Rust changes.
  • Add Python bridge tests for disabled, enabled, unavailable module, and unknown-provider fallback behavior.

Verification

  • uv run black litellm/rust_bridge litellm/ocr/main.py tests/test_litellm/llms/mistral/ocr/test_mistral_ocr_rust_bridge.py
  • uv run ruff check litellm/rust_bridge litellm/ocr/main.py tests/test_litellm/llms/mistral/ocr/test_mistral_ocr_rust_bridge.py
  • uv run pytest tests/test_litellm/llms/mistral/ocr/test_mistral_ocr_transformation.py tests/test_litellm/llms/mistral/ocr/test_mistral_ocr_rust_bridge.py -q
  • cd litellm-rust && cargo fmt --check
  • cd litellm-rust && cargo clippy --workspace --all-targets -- -D warnings
  • cd litellm-rust && cargo test --workspace

Scope

This PR keeps the rollout narrow: Mistral OCR only, route-level Python hook, Python fallback for every disabled/unavailable Rust path. Future OCR providers should add providers/src/<provider>/ocr/transformation.rs and register the provider in the Rust OCR dispatcher plus the Python rollout enum.

@ishaan-berri ishaan-berri changed the title Add LiteLLM Rust workspace with Mistral OCR bridge feat: add LiteLLM Rust workspace with Mistral OCR bridge Jun 22, 2026
@codspeed-hq

codspeed-hq Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing codex/litellm-rust-mistral-ocr (ce27d80) with main (dcf1b44)

Open in CodSpeed

@ishaan-berri

Copy link
Copy Markdown
Contributor Author

Refiled from the CI-accepted branch name as #31033: #31033

@ishaan-berri

Copy link
Copy Markdown
Contributor Author

Closing this codex-branch PR in favor of #31033 from litellm_hotfix_rust_mistral_ocr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants