test(ocr): record provider fixtures in the migration harness

[yujonglee-berri]

TLDR

Problem this solves:

• Existing credentials skip Azure and Vertex fixture recording
• Transient provider failures can become cached fixtures

How it solves it:

• Discover gcloud OAuth tokens and Azure deployment aliases
• Default to two workers and reject transient recordings
• Add 136 sanitized recordings across four providers
• Keep fixtures in docs: define Rust Python harness structure #39456's API-owned strategy layout

User Flow

Before: the fixture corpus covers only Mistral and Reducto targets

1. Inspect the available recorded OCR cases
2. Observe 31 recordings across three targets
3. Run the OCR recording command with Azure credentials and a gcloud login
4. Observe that those credentials do not populate both additional providers

After: the fixture corpus covers all four OCR providers

1. Inspect the available recorded OCR cases
2. Observe 167 recordings across seven targets
3. Run the OCR recording command with Azure credentials and a gcloud login
4. Observe Azure and Vertex targets using discovered credentials

Relevant issues

Second PR in the two-PR stack, directly based on #38765. Both PRs follow the specification in #39456

OCR inputs, provider configuration, recording commands, and all 167 cassettes live under tests/rust-python-harness/strategies/e2e_parity/sdk/ocr/. Shared recording and replay live under shared/parity/. This PR inherits the independent E2E, trace, unit, and existing live SDK runners and combined reports from #38765

Run OCR checks with uv run python -m tests.rust-python-harness --strategy e2e_parity --function ocr --plain. Generate recordings with uv run python -m tests.rust-python-harness.strategies.e2e_parity.sdk.ocr.fixtures.record --examples 1000

The stack is local-only with no CI execution. Existing Python tests and native Rust unit tests remain in their original locations. All 167 cassettes retain identical Git blob hashes from this PR's original tip. This alignment changes paths and imports without refreshing recordings or changing provider assertions

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• All affected tests pass, including expanded corpus parity
• My PR passes all required CI/CD checks
• My PR's scope is isolated to fixture generation and recordings
• I have received a current Greptile Confidence Score of at least 4/5

Validation

Merged the updated #38765 at 0e4232b468, including staging's new SDK test matrix. The final child checkout passes 110 focused tests covering harness CLI entry points, ledger consistency, Python/Cargo runner behavior, recursive directory selectors, reporting, trace comparison, and CrowdStrike guardrails

The previous tip 08b8db37a4 passed all GitHub Actions checks, including 8,954 passing proxy-endpoints tests. CI is running again for this latest merge

All 167 cassette blobs are unchanged. The corpus still contains seven targets across four providers. The full expanded OCR parity corpus was not rerun for this merge; the earlier Azure request mismatch and existing expected failures remain outside this CI fix

Screenshots / Proof of Fix

Run this corpus-inspection command from the repository root:

python -c "from pathlib import Path; from collections import Counter; print(dict(sorted(Counter(p.parent.name for p in Path('tests/rust-python-harness/strategies/e2e_parity/sdk/ocr/fixtures/data').rglob('*.yaml')).items())))"

Before (0e4232b)

1. Run the corpus-inspection command
2. Observe {'mistral-ocr': 18, 'reducto-legacy': 4, 'reducto-v3': 9}

After (6619f08)

1. Run the corpus-inspection command
2. Observe {'azure-document-intelligence': 17, 'azure-mistral': 15, 'mistral-ocr': 49, 'reducto-legacy': 6, 'reducto-v3': 63, 'vertex-deepseek': 2, 'vertex-mistral': 15}

Type

Test

Bug Fix

Caveats (if any)

Medium

• Existing provider-error and Azure request parity mismatches remain
• Available native extension was reused for local replay
• Full expanded corpus parity was not rerun
• One Vertex annotation case still returns HTTP 500
• Four Vertex parameter cases retain upstream 404/422 rejections
• Two recording workers can still hit provider quotas

Low

• Finite strategies do not cover every schema value
• Reducto retains the parent's expected-failure marks
• Trace, gateway, and complete unit mappings remain planned

Final Attestation

• Every expanded provider case is proven equivalent end to end

[greptile-apps]

Too many files changed for review (148 files, 100 file limit).

Bypass the limit by tagging @greptile-apps to review.

[devin-ai-integration]

PR #39425 labeled risk:high (148 files, +54,321/-38, touches litellm/litellm_core_utils/prompt_templates/image_handling.py). No enterprise label, so Linear routing was skipped.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_ocr_fixture_generation (a31f3f7) with litellm_ocr_sdk_parity_tests (5f9936e)¹}

Footnotes

1. No successful run was found on litellm_ocr_sdk_parity_tests (5792e0c) during the generation of this report, so 669542c was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩