Repository navigation
fix: backport eight backport-stable fixes to rc/1.102.0 (#40596, #41046, #41086, #41171, #41178, #41283, #41495, #41689) - #41854
Conversation
…le on team list Backport of #41086 to rc/1.102.0. Cherry-picked from merge commit 4123b4b (main), originally by app/devin-ai-integration. rc/1.102.0 has no litellm/proxy/auth/resolvers/grants.py, so its one-line import change is dropped; auth_checks.py imports UserNotFoundError from the new types module, so every importer on this line still resolves.
Backport of #41495 to rc/1.102.0. Cherry-picked from merge commit 0add8c0 (main), originally by app/devin-ai-integration. The only conflict was the test import header: the rc line never gained the logging_executor import that main carries next to this change, so only the MockResponseIterator import comes along.
…equests Backport of #41689 to rc/1.102.0. Cherry-picked from merge commit 07b5051 (main), originally by app/devin-ai-integration. Both conflicts were in test files: the rc line lacks the Codex additional_tools tests that sit next to the new test in test_handler.py on main, and its test_utils.py imports all_litellm_params on a separate line, so only this PR's own additions (the imports, the recording handler, and the two new tests) are taken.
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| async def _get_user_team_ids_from_db( | ||
| user_id: str, | ||
| prisma_client: PrismaClient, | ||
| user_api_key_cache: UserApiKeyCache, | ||
| proxy_logging_obj: ProxyLogging, | ||
| ) -> tuple[str, ...]: | ||
| try: | ||
| user: Final = await get_user_object( | ||
| user_id=user_id, | ||
| prisma_client=prisma_client, | ||
| user_api_key_cache=user_api_key_cache, | ||
| user_id_upsert=False, | ||
| proxy_logging_obj=proxy_logging_obj, | ||
| check_db_only=True, | ||
| ) | ||
| except UserNotFoundError: | ||
| return () | ||
| return tuple(user.teams or ()) if user is not None else () | ||
|
|
||
|
|
There was a problem hiding this comment.
The new org-admin self-query path calls get_user_object(..., check_db_only=True), adding a cache-bypassing database lookup before the paginated team query. This violates the repository directive to avoid creating new database requests in request paths. Reuse membership data already loaded for authorization or combine this lookup with the existing query. This repository requirement must be satisfied before merging.
Rule Used: What: Avoid creating new database requests or Router objects in the critical request path. Why: Creating these objects on every request causes performance degradation and unnecessary resource consumption. (source)
There was a problem hiding this comment.
Cherry-pick of #41086 as merged on main. The lookup runs only when an org admin lists their own teams, a management route, not LLM traffic.
There was a problem hiding this comment.
That’s a fair distinction. The added lookup is confined to the management-only list_team_v2 path, runs only for an org admin’s own-team query, and uses the existing get_user_object helper. It is not on the LLM request path covered by the critical-path database-query concern, so I’m withdrawing this finding; no change is required here.
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 3a7a204. Configure here.
1 participant
TLDR
Problem this solves:
backport-stablefixes only landed on mainrc/1.102.0(4fd5bc6, v1.102.0-rc.2) still ships every one of those regressionslitellm[cli]cannot even import on Python 3.10How it solves it:
rc/1.102.0, one commit per PR, provenance in the bodyUser Flow
The flow below is the one fix reproduced by hand on this line (Python 3.10 CLI). The other seven keep the User Flow written in their original PR, at code identical to what this branch carries: #41086, #41178, #41283, #41171, #40596, #41495, #41689
Before: a developer on Python 3.10 installs the CLI from the rc and cannot run a single
litecommandpip install 'litellm[cli]==1.102.0rc2'on Python 3.10 and the install succeedslite --versionand get a traceback ending inImportError: cannot import name 'StrEnum' from 'enum', exit code 1lite login,lite keys list, andlite models listall die with the same traceback before any argument is read, so the CLI never reachesGET http://localhost:4000/health/readinessor any other proxy routeAfter: the same developer runs every
litecommand on Python 3.10pip install 'litellm[cli]==1.102.0'on Python 3.10 and the install succeedslite --versionand seeLiteLLM Proxy CLI Version: 1.102.0, the configured server URL, and the server version fromGET http://localhost:4000/health/readiness(or(unavailable)when no proxy is running), exit code 0lite login,lite keys list, andlite models listreach the proxy routes they always didBackported PRs
In pick order (each row is one commit on this branch, cherry-picked from the PR's merge commit on main):
routing_strategyoverride routes blind/v1/responsesfollow-up turns return 400Conflict notes, each resolved to the merged upstream behavior with nothing new:
litellm/proxy/auth/resolvers/grants.pyis dropped because that file does not exist on this line;UserNotFoundErrorstill moves tolitellm/types/proxy/auth/auth_checks.pyandauth_checks.pyimports it from there, exactly as on mainlogging_executorimport that sits next to the change on main, so only the PR's ownMockResponseIteratorimport is takenadditional_toolstests that sit next to the new test intest_handler.pyon main, and itstest_utils.pyimportsall_litellm_paramson a separate line, so only the PR's own imports, recording handler, and two new tests are takenStill open on main with the label and not in this batch: #41235 and #41347.
stable/1.101.xis untouchedRelevant issues
Affected release
Every fix here is a regression fix already labeled
backport-stableon main. The oldest regressed in v1.83.10 (#41086), the newest in v1.102.0-rc.1 (#41046, #41689); all eight are present in v1.102.0-rc.2, the current tip ofrc/1.102.0Linear ticket
Resolves LIT-8123
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or moreproxy_multi_instance_testswent green on a rerun after its first run timed out cloning from github.com before any test ran, and the three red jobs are the pre-existing Together AI rot described under Caveats@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup for the CLI case: a Python 3.10.20 venv holding litellm's base and
clidependencies straight fromuv.lock(uv export --frozen --no-emit-project --no-dev --no-hashes --extra cli, thenuv pip install -r), with thelitellmpackage loaded from a source tree throughPYTHONPATHrather than pip-installed, which is why the CLI's own version line readsunknown. Theliteconsole script islitellm.proxy.client.cli:cli, sopython3.10 -c "from litellm.proxy.client.cli import cli; cli()" --versionis the same entrypointlite --versionruns. Before is thelitellm/tree atrc/1.102.04fd5bc6 (git archive), After is this branchShared setup for the regression-test case: this branch's 13 touched test files (which carry the 32 tests the eight PRs add) run in the worktree's venv (Python 3.14.3). After runs them from the worktree; Before runs the same 13 files copied into a detached checkout of the rc tip 4fd5bc6, with
litellmverified to import from that checkout, sincepython -m pytestputs the working directory first onsys.pathand a Before run started from the branch worktree would test the branchBefore (4fd5bc6)
lite --version on Python 3.10
PYTHONPATH=<rc/1.102.0 tip 4fd5bc6b22 litellm tree> python3.10 -c "from litellm.proxy.client.cli import cli; cli()" --versionThe eight PRs' regression tests on the rc tip
<worktree venv>/python -m pytest -q --continue-on-collection-errors <the 13 touched test files>from the detached rc-tip checkouttest_utils.pyand the 1 intest_handler.pyimportADDRESSED_RESPONSE_ID_FIELD, which fix(responses): keep the addressed response id off bridged provider requests #41689 introduces), and 5 already pass there: fix(cli): drop enum.StrEnum so the CLI imports on Python 3.10 #41046'stest_listing_failure_is_str_enum(Python 3.14 hasStrEnum, which is why the CLI case above runs on Python 3.10), fix(router): bind per-request routing_strategy override selectors to the request's callbacks #41178'stest_override_matching_the_router_strategy_is_not_bound_twice, fix(proxy): retry rate-limit fallbacks from a pristine request snapshot #40596'stest_key_metadata_disable_fallbacks_returns_429_instead_of_retrying, and fix(logging): track spend for streams a deployment hook converted to non-streaming #41171's two cache-hit tests intests/test_litellm/caching/test_caching_handler.py. Two existing tests the picks change also fail on the rc tip:test_list_team_v2_org_admin_own_user_id_sees_all_org_teams(fix(proxy): keep org admins' own team memberships in other orgs visible on team list #41086) andtest_should_defer_streaming_cache_hit_callbacks_for_any_streaming_request(fix(logging): track spend for streams a deployment hook converted to non-streaming #41171)After (3a7a204)
lite --version on Python 3.10
PYTHONPATH=<this branch> python3.10 -c "from litellm.proxy.client.cli import cli; cli()" --versionThe eight PRs' regression tests on this branch
.venv/bin/python -m pytest -q <the 13 touched test files>from the worktree rootBASE_REF=origin/rc/1.102.0 make check: Python lint, the ruff-strict, type-discipline, test-quality, and basedpyright budget gates (all measured againstorigin/rc/1.102.0), the circular-import check, andruff format --checkall pass. The lazy OpenAPI snapshot regenerated with zero diff under Python 3.12, the version CI usesType
🐛 Bug Fix
Caveats (if any)
Low
/v1/responsesturns) at two commits each does not fit before the v1.102.0 tag expected Sat, Sep 19, and missing that tag ships all eight regressions in stable, the exact harm this batch exists to prevent(unavailable); the fix is import-time and the readiness round trip that follows is the same code at both commits.github/workflows/test-code-quality.ymlhunk (alite versionsmoke step on Python 3.10) so the commit stays the merge commit verbatim; that workflow does not run on PRs into this linemake checkran under Python 3.14 flagged the lazy OpenAPI snapshot as stale; the only diff was Python 3.13+ docstring dedent, and a Python 3.12 regeneration matches the committed file byte for byte, so the regenerated file was discardedbackport-stableand are open on main; they are not in this batch and need their own pick once mergedlocal_testing_part1,local_testing_part2, andllm_translation_testingare red at the tip only on Together AI tests hard-coded toopenai/gpt-oss-20b, which Together no longer serves serverless; fix(license): backport the wildcard license auto_router grant to rc/1.102.0 (#41684) #41701, the last PR merged into this line, failed the same tests, and main is green because it rewrote those tests after the rc cut (1aa2e19 and follow-ups, test-only). Not a pick of this batch, which carries no test change outside the eight PRsFinal Attestation
The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
3a7a204 passes /live-pr-risk