refactor(rust): isolate legacy callback contract by devin-ai-integration[bot] · Pull Request #41752 · BerriAI/litellm

devin-ai-integration · 2026-09-18T04:17:13Z

TLDR

Problem this solves:

OCR requests need the established legacy callback behavior
Callback behavior should remain consistent across OCR request paths

How it solves it:

Reuses one callback contract for OCR requests
Preserves legacy callback results and cleanup behavior

User Flow

Before: an OCR request using legacy callbacks could complete without the expected callback events

A client sends an OCR request through the public OCR API
The client configures the established legacy callback integration
The OCR request completes without a consistent callback lifecycle

After: the same OCR request preserves the established callback lifecycle

A client sends the same OCR request through the public OCR API
The client configures the same legacy callback integration
The OCR request completes with the expected callback lifecycle and cleanup

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

I have added meaningful tests
The handful of test files covering my change pass locally
My PR passes all required CI/CD checks
My PR's scope is as isolated as possible
I have received a Greptile Confidence Score of at least 4/5

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review)

Screenshots / Proof of Fix

Local and CI validation are pending for the current PR tip

Type

Refactoring

Caveats (if any)

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/afea2121eeac4ba180934abde5b39896
Open in Devin Desktop: https://app.devin.ai/desktop/session/afea2121eeac4ba180934abde5b39896?variant=devin
Requested by: @yujonglee-berri

@client

… a legacy Logging adapter Extracted from #41733 without the router loop, the cache machine layer, streaming, or the error, timeout and route-pruning work that moved to #41745 litellm-callbacks holds the contract a native call and its host share: Machine, HostOp, CallEvent, the in-process run loop, and Passthrough, which is built only by comparing the caller's inputs with the body the route sends, so a route can never mark a key it rewrote. litellm-host-python (formerly python-interop) owns the CPython driver and the Execution handle, and litellm-callbacks-legacy is the @client wrapper as the native call sees it: function_setup, the deployment hooks, pre_call and post_call, the success and failure fan-out and the deferred proxy release. OCR is the one route on it, and the old core and bridge lifecycles are gone The passthrough rule is the structural fix for the bug #41719 patched in core and #41716 reworks: an inlined remote document no longer counts as the caller's value, so the legacy adapter never hands the caller's URL back into the body. core/tests/ocr/passthrough.rs pins it for every route and document source, including that unchanged values stay passthrough, and callbacks-legacy/tests/payload.rs pins the adapter side with a real pre_call callback Python OCR integration tests that only exercised core behavior now live as Rust tests, so tests/test_litellm_rust keeps the cases that need the full Python stack

devin-ai-integration · 2026-09-18T04:17:16Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment, CI, and merge conflict monitoring

codspeed · 2026-09-18T04:21:15Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_ocr_callbacks_legacy_contract (1d88ca1) with main (db37977)}

greptile-apps · 2026-09-18T04:22:55Z

Confidence Score: 4/5

The implementation appears functionally coherent, but the explicit repository requirement against weakening modified tests must be satisfied before merging.

Findings

Public OCR Coverage Removed ▶

Summary

This PR restructures Rust OCR execution around a route-neutral machine, a shared Python host driver, and a dedicated adapter implementing the legacy Logging callback contract.

Adds callback, legacy-callback, and Python-host crates.
Moves OCR projection, lifecycle events, request interception, token acquisition, and response construction onto the new machine/host boundary.
Preserves legacy Python callback dispatch through litellm.rust_bridge.legacy_callbacks.
Refactors provider and bridge tests, but removes substantial public Python regression coverage that should be retained.

sequenceDiagram
    participant Python as Python OCR entrypoint
    participant Adapter as Legacy callback adapter
    participant Driver as Python host driver
    participant Machine as OCR route machine
    participant Provider as OCR provider
    Python->>Adapter: capture args, kwargs, request
    Adapter->>Driver: prepared keyword view
    Driver->>Machine: start call
    Machine->>Driver: project request / acquire token / read file
    Driver->>Python: execute caller-owned operation
    Machine->>Adapter: before-send event
    Adapter->>Python: legacy pre_call
    Machine->>Provider: rewritten request
    Provider-->>Machine: raw response
    Machine->>Adapter: response and terminal events
    Adapter->>Python: post_call and success/failure callbacks
    Driver-->>Python: finalized response or mapped exception

_{Reviews (1) · Last reviewed commit: "refactor(rust): run OCR through a route-..."}

codecov · 2026-09-18T04:30:07Z

Codecov Report

❌ Patch coverage is 92.10526% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/rust_bridge/legacy_callbacks.py	91.89%	6 Missing ⚠️

📢 Thoughts on this report? Let us know!

…replace The Python/Rust parity cases behind the ocr_backend fixture are back as they were on main: the malformed-document matrix, Azure invalid options, native format for every provider and the unknown Reducto model. They are the only check that the Python opt-out path and the native path agree test_native_failures_raise_the_public_exception_class drives every native failure kind through litellm.ocr and litellm.aocr and pins the exception class callers catch. That class is chosen in Python by route_host.map_failure, so no Rust test can cover it; bypassing the mapping fails all 26 cases. The nested document edit and metadata failure tests run sync again, since the sync path skips deployment hooks and dispatches success on the executor legacy_callbacks.callbacks_needed now takes a Literal phase and ends its match with assert_never, and setup imports from litellm.utils instead of mixing import styles

+]
+
+
+def callbacks_needed(logger: Logging, phase: Phase) -> bool:


devin-ai-integration Bot requested a review from a team September 18, 2026 04:17

devin-ai-integration Bot assigned yujonglee-berri Sep 18, 2026

yujonglee-berri approved these changes Sep 18, 2026

View reviewed changes

yujonglee-berri enabled auto-merge September 18, 2026 04:18

yujonglee-berri changed the title ~~refactor(rust): run OCR through a legacy callback contract~~ refactor(rust): isolate legacy callback contract Sep 18, 2026

greptile-apps Bot reviewed Sep 18, 2026

View reviewed changes

Comment thread tests/test_litellm_rust/ocr/test_requests.py Outdated

github-advanced-security Bot found potential problems Sep 18, 2026

View reviewed changes

Comment thread litellm/rust_bridge/legacy_callbacks.py Fixed

Comment thread litellm/rust_bridge/legacy_callbacks.py Fixed

Comment thread litellm/rust_bridge/legacy_callbacks.py Fixed

Comment thread litellm/rust_bridge/legacy_callbacks.py Fixed

yujonglee-berri disabled auto-merge September 18, 2026 04:27

yujonglee-berri enabled auto-merge September 18, 2026 04:39

github-advanced-security Bot found potential problems Sep 18, 2026

View reviewed changes

Comment thread litellm/rust_bridge/legacy_callbacks.py

]

def callbacks_needed(logger: Logging, phase: Phase) -> bool:

yujonglee-berri merged commit b1f9da7 into main Sep 18, 2026
91 checks passed

yujonglee-berri deleted the litellm_ocr_callbacks_legacy_contract branch September 18, 2026 04:46

yujonglee-berri mentioned this pull request Sep 18, 2026

fix(ocr): keep inlined remote documents when the host restores retained fields #41716

Closed

5 of 6 tasks

mateo-berri mentioned this pull request Sep 18, 2026

feat(a2a): reach Microsoft Foundry agents with Entra auth and versioned card discovery #41511

Merged

10 tasks done

Navigation Menu

Repository navigation

refactor(rust): isolate legacy callback contract - #41752

refactor(rust): isolate legacy callback contract#41752
yujonglee-berri merged 2 commits into
mainfrom
litellm_ocr_callbacks_legacy_contract

devin-ai-integration Bot commented Sep 18, 2026 •

edited

Loading

Uh oh!

devin-ai-integration Bot commented Sep 18, 2026

Uh oh!

codspeed Bot commented Sep 18, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Sep 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		]


		def callbacks_needed(logger: Logging, phase: Phase) -> bool:

Conversation

devin-ai-integration Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Type

Caveats (if any)

Final Attestation

Uh oh!

devin-ai-integration Bot commented Sep 18, 2026

🤖 Devin AI Engineer

Uh oh!

codspeed Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

greptile-apps Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 4/5

Findings

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

devin-ai-integration Bot commented Sep 18, 2026 •

edited

Loading

codspeed Bot commented Sep 18, 2026 •

edited

Loading

greptile-apps Bot commented Sep 18, 2026 •

edited

Loading

codecov Bot commented Sep 18, 2026 •

edited

Loading