refactor(litellm-rust): 3-crate layer structure (core / ai-gateway / python-bridge) + AGENTS.md + enforcement test

[ishaan-berri]

Relevant issues

Restructures litellm-rust so the crate boundaries match what they actually are — layers, not routes — before more providers/routes land. This is the shape every mature Rust gateway uses (TensorZero, ruff, uv): a pure core, an I/O host, and the binding.

Pre-Submission checklist

• I have added meaningful tests (enforcement test for the crate set + existing transform/route tests still pass)
• cargo test --workspace → 27 passed, 1 ignored (the ignored one is the pre-existing live-OpenAI realtime test)
• cargo clippy --workspace --all-targets -- -D warnings, cargo fmt --all --check, cargo build --workspace all green
• No Python changes

Type

🧹 Refactoring

Changes

Collapses {core, providers, python-bridge} into three layer crates:

Crate	Role	Pure / I/O
litellm-core	Translation layer — types, route contracts (traits), and provider transforms (modules under providers/)	Pure
litellm-ai-gateway	Routes/host — the only crate that touches the network; HTTP/WS I/O + the end-to-end route fns (run_ocr, realtime)	I/O
litellm-python-bridge	PyO3 cdylib exposing Rust to the litellm Python SDK	Binding

Dependency line (acyclic): litellm-core ← litellm-ai-gateway ← litellm-python-bridge

What moved:

• Provider transforms (mistral, openai) providers/ → core/src/providers/ (they're pure)
• The network code (ocr.rs run_ocr, realtime.rs) providers/ → new ai-gateway/ crate
• python-bridge now imports litellm_ai_gateway::ocr::run_ocr
• providers crate deleted

Why: the pure/I-O split is now compiler-enforced — core doesn't list reqwest/tokio, so it can't do network I/O. Faster incremental builds (change one layer, rebuild one layer) and the layering can't rot into a cycle.

Guardrails added:

• litellm-rust/AGENTS.md + per-crate AGENTS.md documenting the rule: a crate is a layer; routes/providers are modules; add a crate only on a real trigger (separate artifact / proc-macro / shared foundation / publishable).
• crates/core/tests/workspace_crate_allowlist.rs — fails CI if the crate set ever drifts from the 3 allowlisted crates, so humans and agents get told the rule.

Note: added litellm-rust/.cargo/config.toml with the standard pyo3 -undefined dynamic_lookup flag (apple-darwin only) so cargo build --workspace links the cdylib locally without maturin. macOS-only, no effect on Linux/CI or crate logic.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR restructures litellm-rust from a {core, providers, python-bridge} layout into a clean three-layer architecture: litellm-core (pure transforms), litellm-ai-gateway (network I/O), and litellm-python-bridge (PyO3 binding). No logic changes are made — files are moved, import paths are updated, and the dependency arrow is now compiler-enforced (core ← ai-gateway ← python-bridge).

• The providers crate is deleted; its pure transform modules (mistral/, openai/) move into litellm-core/src/providers/ and its I/O modules (ocr.rs, realtime.rs) move into the new litellm-ai-gateway crate.
• A new enforcement test (workspace_crate_allowlist.rs) uses a hand-rolled TOML parser to fail CI if the three-crate allowlist ever drifts, and AGENTS.md files are added at every layer to document the rule.

Confidence Score: 4/5

Safe to merge — no logic changes, all moves are mechanical renames with updated import paths confirmed by the passing test suite.

The refactor is purely mechanical with no runtime behavior changes. Two minor follow-ups remain: stale litellm_providers references in unchanged CLAUDE.md files, and the section-unaware TOML parser in the enforcement test.

crates/python-bridge/CLAUDE.md (stale litellm_providers reference) and crates/core/tests/workspace_crate_allowlist.rs (section-unaware TOML parser).

Important Files Changed

Filename	Overview
litellm-rust/Cargo.toml	Workspace manifest updated: crates/providers to crates/ai-gateway, all references consistent with the rename.
litellm-rust/crates/core/tests/workspace_crate_allowlist.rs	New enforcement test using a hand-rolled TOML parser; works correctly for the current fixed file layout but would silently mismatch if a members key appears in a non-[workspace] section before the real one.
litellm-rust/crates/ai-gateway/src/ocr.rs	Renamed from providers; import paths updated to litellm_core::providers::mistral — logic unchanged.
litellm-rust/crates/ai-gateway/src/realtime.rs	Renamed from providers; import path for OPENAI_REALTIME_CONFIG updated; doc comment and ignored-test command updated to match new crate name.
litellm-rust/crates/python-bridge/CLAUDE.md	Unchanged file with a stale reference to the deleted litellm_providers crate name that was not updated alongside this rename.
litellm-rust/.cargo/config.toml	New macOS-only config.toml adding the standard pyo3 -undefined dynamic_lookup linker flag for both x86_64 and arm64; no effect on Linux/CI.
litellm-rust/crates/providers/CLAUDE.md	Deleted along with the providers crate; the detailed test requirements and implementation rules it contained are not all carried over to the new AGENTS.md files.

_{Reviews (1): Last reviewed commit: "docs(litellm-rust): add litellm-python-b..." | Re-trigger Greptile}

[greptile-apps]

Hand-rolled parser lacks section-scope tracking

parse_workspace_members matches the first line starting with members in the entire file, regardless of which TOML section it is in. If a new section (e.g., [workspace.metadata], [lints.workspace], or a tool section) ever gains a members = [...] key that appears before [workspace], the test would silently validate the wrong list and fail to catch a real workspace-membership change. Adding a check that resets in_members when a new [section] header is encountered would make the parser section-aware.

[ishaan-berri]

Superseded — staging has since added the ai-gateway crate, so this 3-crate restructure no longer applies. Redoing from current staging (dissolve providers into core + ai-gateway).