fix: restore Python compatibility and test 3.10 through 3.14 - #39399
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile SummaryThe PR restores runtime compatibility across Python 3.10 through 3.14 and expands existing unit-test shards to cover all supported versions.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| .github/workflows/_test-unit-base.yml | Expands every reusable unit-test shard across Python 3.10–3.14, scopes dependency caches by interpreter, and retains coverage publication from Python 3.12. |
| litellm/types/llms/openai.py | Resolves the reasoning-summary TypedDict directly and marks the summary field read-only to avoid incomplete Pydantic models. |
| litellm/llms/base_llm/ocr/transformation.py | Qualifies object annotations through builtins so the OCR model's object field cannot shadow their runtime resolution. |
| litellm/vector_stores/main.py | Qualifies list annotations through builtins to preserve runtime introspection when the module also exports a list function. |
| litellm/utils.py | Simplifies UTC timestamp construction to consistently return timezone-aware datetime values. |
| pyproject.toml | Raises the proxy uvloop floor for Python 3.14 and declares the Python 3.10 tomli development dependency. |
| schema.prisma | Configures recursive Prisma type generation to avoid excessive bounded expansion on newer Python versions. |
| tests/sdk_function_trace/profiler.py | Reconstructs qualified function names on Python versions without code-object co_qualname support. |
Reviews (5): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile
…itellm_python_version_ci
…itellm_python_version_ci # Conflicts: # basedpyright-code-budget.json
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit a581399. Configure here.
…itellm_python_version_ci Resolves two conflicts: - tests/test_litellm/vector_stores/test_main.py: staging moved search() to a RouterVectorStoreEmbeddingExecutor while this branch parametrized the same test over query; keep both the executor assertions and the parametrize. - tests/logging_callback_tests/test_bedrock_knowledgebase_hook.py: staging carries a duplicate embedding_executor kwarg that makes the file a SyntaxError; drop the trailing duplicate.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 71f5b89. Configure here.
…itellm_python_version_ci # Conflicts: # basedpyright-code-budget.json # tests/sdk_function_trace/profiler.py # tests/sdk_function_trace/test_profiler.py
2e73400
into
litellm_internal_staging
TLDR
Problem this solves:
How it solves it:
The import changes incorporate all six runtime fixes proposed in #38512 and #39374, plus the same fix for general upload validation added to staging while this PR was running. The additional response fix removes an unnecessary quoted reference to a type already defined above it. On Python 3.10, Pydantic otherwise resolves that name in the consuming model's namespace and leaves both Message and Delta incomplete
The existing reusable unit workflow runs each of its 26 SDK and proxy shards on all five supported versions. Both setup-python and uv use the selected interpreter. Only uv downloads/build artifacts are cached, using the actual UV_CACHE_DIR and version-specific keys. Each job builds a fresh virtual environment. Older interpreters use coverage's C tracer, and existing Python 3.12 check names and Codecov reports remain intact. There is no separate compatibility workflow
User Flow
Before: supported Python versions fail at different points in an SDK or proxy request
litellm --config config.yamlexits withImportError: cannot import name 'BaseDefaultEventLoopPolicy' from 'asyncio.events', OCR returns an incomplete-model error, and vector search can raiseTypeError: 'function' object is not subscriptableAfter: those imports and response paths work on the same Python versions
/v1/chat/completionson 3.14, and OCR response validation and vector-search cache handling succeed there tooRelevant issues
Fixes #38202
Fixes #36384
Includes the import-source fixes proposed in #38512 and #39374
Compatibility findings and fixes
These are observed failures and constraints, not a claim that the matrix catches every possible compatibility problem
assert_never,Never, andRequiredmust come fromtyping_extensions. Fix every affected module because an early import failure masks the next one.compact.pywas already fixed upstreamMessageandDeltaincomplete on 3.10. Resolve the already-defined type directly and test normal and streaming serializationtyping_extensionswhile importing those types.recursive_type_depth = -1, the documented setting for Pyright, reduces this to 164,539 lines. Import and construction took about eight seconds locally. Both first generation and regeneration were verified.venvrestores the old generated Prisma clientUV_CACHE_DIR, not necessarily~/.cache/uvschema.prismais canonical; the SDK proxy and proxy-extras copies must match byte-for-byte. Update all three even for a generator-only setting. The first follow-up CI run caught this driftprisma migrate diffbetween the old and new schemas reports an empty migration. The change affects generated Python typesdatetime.UTCis absent on 3.10; the fallback returned naiveutcnow(). Usedatetime.now(timezone.utc)on every version and verify an aware current timestampresponsesfunction orcachingboolean instead of the same-named package. Import the actual module withimport_moduleand patch its attribute. The same target can fail only after another test changes the exported valuegeneral_upload_validation.pywith a new stdlibassert_neverimport while this PR was open. Local checks at the old base passed, but CI merged with1bb9b175e2and failed on 3.10. Bring in that base, fix the import, and rerun against the actual merged code--maxfaillimit stopped shards before they reached more mock targets, OCR behavior, and dependency-specific tests. After fixing a blocker, rerun the complete affected shard; one green targeted test is insufficient. Swallowed import errors also produced misleading mock-call and metrics assertions, so inspect the earliest tracebackOCRResponse.object = "ocr"shadowed builtinobjectin annotations for tables and key-value pairs, leaving Pydantic with an unresolvedocrreference. Usebuiltins.objectand verify nested strings, numbers, and booleans survive the proxy responselistalso shadow annotation builtins on 3.14list(), so deferredlist[...]annotations resolve to that function. Search then fails when cache handling callsinspect.signature, also breaking RAG billing tests. The models and teams management clients reproduce the same signature failure. Usebuiltins.list; verify string and multi-query searches, downstream RAG billing, and the public call signaturesExceptionGroupon 3.10, caught the resultingNameError, and then failed an unrelated assertion. Import theexceptiongroupbackport on 3.10 so it exercises the intended treeAsyncMockexposes mock code/name attributes that Python 3.10'sinspectand APScheduler cannot use. Build the scheduled async functions withcreate_autospecso signatures and names are realasyncio.get_running_loopglobally breaks Redis testsasync_timeout, which needs that function. Construct the clients in a worker thread to avoid startup tasks, then exercise real Redis failures on the actual event looptyping_extensions.get_type_hintsto unwrap backportedRequiredconsistentlytomllib. Usetomlion 3.10 and declare the existing locked version as a direct development dependencysemantic-routerandaurelio-sdkon 3.14. The encoder module and 35 dependency-specific routing/MCP tests follow that exclusion and remain tested on 3.10–3.13. The same routing/MCP files still run 672 dependency-independent tests on 3.14. This is a documented feature and coverage gapasyncio.events.BaseDefaultEventLoopPolicy, which uvloop 0.21.0 imports at module scope.uvicornonly imports its loop factory when the server starts, so every import and unit-test check passed whilelitellm --configdied at boot. Requireuvloop>=0.22.1and assert the selected loop factory imports on the running interpreterco_qualnamedoes not exist before 3.11ProviderConfigManager.get_provider_chat_configagainst bare code names on 3.10, so five parity tests failed there. Rebuild the qualified name from module declarations and enclosing frames, following classes, accessors, wrappers, and nested functions. Resolve explicit function references from portable__qualname__tomliCoverage boundaries
The matrix tests normal CPython 3.10–3.14 on Linux with locked dependencies, fresh virtual environments, Prisma generation, and the existing SDK/proxy behavior tests. The minor-version selectors track the patch version supplied by setup-python, and each job prints and verifies its actual interpreter. Local checks used 3.10.20 and 3.14.5; the initial CI failures used 3.10.21 and 3.14.7
This does not test every patch release, free-threaded
3.13t/3.14tbuilds, macOS/Windows, fresh dependency resolution, lowest dependencies, or an installed release wheel outside the checkout. Metadata already selects different Lunary releases on 3.10 and newer Python versionsThe Python 3.14 porting notes also call out annotation evaluation and Linux's default
forkserverbehavior. No direct multiprocessing/ProcessPoolExecutor use was found inlitellm/; xdist passing would not independently prove multi-process server startup. The custom-code guardrail already supplies an explicit namespace toexec, avoiding implicit-locals assumptionsOther dependency combinations can still encounter removed standard-library modules, native-extension ABI changes, or older ASGI/async libraries. Those are coverage considerations, not failures established here. Pydantic's 2.12 announcement adds 3.14 support to V2 and explicitly says V1 is unsupported on 3.14
Findings carried over from #38512
Compared against #38512 at
0748f4f01772ed1d76ccc1c742d7f9c3d95e2415, including its discussion and inline reviewsAll six remaining runtime import fixes are included: websearch interception, MCP tool search, agent endpoints, budget resets, batch-file validation, and Gemini transcription types. The earlier
compact.pyfix is already in our baseThe contributor's rebase findings show why a one-time import fix is insufficient: typing cleanups introduced new incompatible imports while that PR was open, and the first import failure concealed later ones. Our supported-version matrix runs the existing SDK and proxy test suites, covering imports and behavior such as the separate Pydantic response failure
Their static scanner adds something the runtime matrix cannot guarantee: checking imports in files and functions tests never execute. It is not included here because its version-guard handling accepts imports in the wrong branch, and its API catalog omits
asyncio.timeoutandtimeout_at. Local probes confirmed those misses, plus qualifiedtyping.assert_neveraccess and silently ignored syntax errors. Its fixed-schema violation records also need precise types, as the third review comment notesThe compatibility gate in this PR is the actual Python 3.10 through 3.14 test run. No hard-coded API scanner or replacement scanner is planned. Running on each interpreter exercises its real standard library, installed dependencies, import paths, and runtime behavior without maintaining a separate list of incompatible names
Coverage still matters: an import inside a function only fails when that function executes. Missing coverage should be addressed with tests that exercise the affected behavior. The matrix does not prove compatibility for untested paths or dependency combinations
basedpyright still targets 3.12. Targeting the minimum supported version could provide additional static feedback, but it is not a prerequisite for replacing #38512's import fixes and is outside this PR
Linear ticket
Resolves LIT-6777
Pre-Submission checklist
Greptile scored this 5/5 at
a581399027. Bugbot re-reviewed the earlier staging-merged tip71f5b89499directly and found no new issues. The current staging merge relocates the profiler fix into the replacement Rust/Python harness and preserves Python 3.10 support thereAll 130 matrix jobs pass at
71f5b89499, the tip that mergeslitellm_internal_stagingin, and they passed ata581399027before it. On the merge's first run seven shards failed onInstall dependencies has timed out after 8 minuteswith zero test failures. That 8-minute step timeout is pre-existing on the base branch anduv.lockis byte-identical across the merge, so the download cache key never changed; all seven went green on re-run with no code change, which is what puts the tip at 200 passing checks and no failures. The matrix is 26 SDK/proxy shards on each of Python 3.10, 3.11, 3.12, 3.13, and 3.14. The three container checks (ui-image,runtime-image,image-scan) failed ata581399027on the base branch's dashboard build rather than on anything here:npm run buildinside the ui image could not resolve../../../../litellm/proxy/public_endpoints/autorouter_presets.jsonfrom its Docker context, and the other two images depend on it. Merginglitellm_internal_stagingin picked up the fix for that, so all three pass at71f5b89499CircleCI was re-run at
71f5b89499and 39 of its 43 jobs pass.build_docker_database_imagegoes green here, which is what lets the eleven jobs downstream of it run at all; they were all skipped on the earlier run. The three red jobs are red onlitellm_internal_stagingitself, one test each and the same test name on both sides:litellm_router_testingontest_router::test_router_timeout,local_testing_part2ontest_timeout::test_timeout_streaming, ande2e_openai_endpointsontest_e2e_openai_responses_api::test_cancel_streaming_response. None of the three is a required checkThe unit matrix does not reach
tests/llm_translationor the bulk oftests/local_testing; those run only on CircleCI, which builds one Python version. So both were collected locally on 3.10 and on 3.14 at71f5b89499, and each collects the same set on both: 2,799 tests intests/llm_translationand 1,576 intests/local_testing, with no collection errors.tests/local_testing/test_tpm_rpm_routing_v2.pyis the only CircleCI-only file that calls the one function whose behavior changed,get_utc_datetime, and it reports 10 passed on both versions. Its two failures are the Redis-backed cases, which needREDIS_HOSTand are the same on either versionThe previously failing Python 3.14 miscellaneous shard now reports 5,388 passed and 37 skipped. The nine vector-store/RAG failures are resolved. The version-specific semantic-router exclusions are described above
The final local checks include 19 vector-store/RAG tests on both 3.10 and 3.14, 46 management-client tests on 3.14, and the five direct runtime reproductions below. After the current staging merge, the affected harness suites report 50 passed on both Python 3.10 and 3.12.
make checkpasses, including lint/type/quality gates, import safety, and API-schema synchronization. The Prisma generator change has an empty database migration diff. All budget changes lower existing ceilingsDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Commands below use Python 3.10.20 for the first three cases and Python 3.14.7 for OCR, runtime annotations, and proxy boot, with locked dependencies and
LITELLM_LOCAL_MODEL_COST_MAP=True. Each run imports the source from the indicated revision. The proxy runs use this config and a real OpenAI key:Before (1bb9b17)
1bb9b175e2was the merge base when these were captured. Every symptom below still reproduces at the current merge base3cac5e5cd4on both 3.10.20 and 3.14.7Proxy import
python -c 'import litellm.proxy.proxy_server; print("Proxy import OK")'ImportError: cannot import name 'assert_never' from 'typing'Completion responses
python -c 'from litellm.types.utils import ModelResponse, ModelResponseStream; ModelResponse(); ModelResponseStream(); print("Normal and streaming responses OK")'PydanticUserError: Message is not fully definedUTC timestamps
python -c 'from litellm.utils import get_utc_datetime; print(get_utc_datetime().utcoffset())'None, a timezone-naive timestampOCR response
python -c 'from litellm.llms.base_llm.ocr.transformation import OCRResponse; print(OCRResponse(pages=[], model="test", tables=[{"amount": 42}], keyValuePairs=[{"approved": True}]).model_dump_json(exclude_none=True))'PydanticUserError: OCRResponse is not fully defined; you should define ocrRuntime annotations
Run this with Python 3.14:
Output:
Proxy boot on Python 3.14
Run
python litellm/proxy/proxy_cli.py --config config.yaml --port <port>on Python 3.14The proxy exits before serving anything:
After (71f5b89)
Proxy import
python -c 'import litellm.proxy.proxy_server; print("Proxy import OK")'Proxy import OKCompletion responses
python -c 'from litellm.types.utils import ModelResponse, ModelResponseStream; ModelResponse(); ModelResponseStream(); print("Normal and streaming responses OK")'Normal and streaming responses OKUTC timestamps
python -c 'from litellm.utils import get_utc_datetime; print(get_utc_datetime().utcoffset())'0:00:00, an aware UTC timestampOCR response
python -c 'from litellm.llms.base_llm.ocr.transformation import OCRResponse; print(OCRResponse(pages=[], model="test", tables=[{"amount": 42}], keyValuePairs=[{"approved": True}]).model_dump_json(exclude_none=True))'{"pages":[],"model":"test","tables":[{"amount":42}],"keyValuePairs":[{"approved":true}],"object":"ocr"}Runtime annotations
Run this with Python 3.14:
Output:
Proxy boot and live completions on Python 3.10 and 3.14
Start the proxy on each interpreter:
LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config config.yaml --port <port>curl -s http://127.0.0.1:<port>/health/livelinessreturns"I'm alive!"on bothSend a real completion:
Output on Python 3.10.20 (port 27431) and Python 3.14.7 (port 31882):
Repeat with
"stream": true; both interpreters stream chunks and finish withdata: [DONE]Type
Bug Fix
Infrastructure
Test
Caveats (if any)
Medium
a581399027, the two unit workflows cost 503 runner-minutes across 157 jobs against 87 minutes before, so the four added interpreters are about 416 extra runner-minutes per push. Dropping 3.11 and 3.13 to post-merge runs is the knob if that spend ever outweighs catching a break pre-mergeInstall dependenciesstep timeout easier to hit. The staging-merge run showed this: seven shards timed out on that step with no test failures and went green on re-run. Raising that step timeout is the knob if it recursFinal Attestation
The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
a581399 passes /live-pr-risk
71f5b89 passes /live-pr-risk: the staging merge leaves every
litellm/contribution byte-identical toa581399027, so that graph walk still holds. The delta is two conflict-resolved test files, both re-run on 3.10, 3.12 and 3.14 and re-driven against a live proxy on 3.10 and 3.146675e1f merges the current staging base, passes
make check, and passes the affected harness suites on Python 3.10 and 3.12Note
Medium Risk
Touches proxy boot dependencies, Prisma client generation, and broad CI matrix expansion; changes are mostly compatibility and test infrastructure but affect every unit shard and generated DB client types.
Overview
Expands the reusable unit-test workflow so each SDK/proxy shard runs on Python 3.10 through 3.14 (matrix, per-version
UV_PYTHON, version-scoped uv download cache, interpreter assertion after sync). Coverage upload stays on 3.12 only, with ctrace on 3.10/3.11 and sysmon on newer interpreters.Runtime and dependency fixes for those versions: Prisma generator
recursive_type_depth = -1on all schema copies;get_utc_datetime()always returns timezone-aware UTC; OCR/vector-store/proxy client annotations usebuiltins.object/builtins.listso methods namedlistandobjectfields do not break 3.14; reasoning summary typing usesReadOnly[...]with new serialization tests; proxy extra requiresuvloop>=0.22.1for 3.14 boot; dev addstomlion Python < 3.11 for scripts/tests that readpyproject.toml.Tests and tooling are adjusted for portability:
import_module+patch.objectfor stable mocks on 3.10, Redis circuit-breaker tests construct clients off-loop, semantic-router suites skip on 3.14, SDK function-trace profiler resolves qualified names withoutco_qualname, and lint/type budget JSON ceilings are ratcheted down slightly.Reviewed by Cursor Bugbot for commit a581399. Bugbot is set up for automated code reviews on this repo. Configure here.