chore(typing): clear 1.6k basedpyright Any errors across 56 files - #36543
Merged
mateo-berri merged 4 commits intoAug 11, 2026
Conversation
reportAny 16720 -> 15482 and reportExplicitAny 5689 -> 5316 with real types only: no casts, no ignores, no new Any. Whole-tree basedpyright drops 2173 diagnostics with zero per-rule or per-file regressions. Budgets ratcheted: basedpyright -2173, ruff-strict -188, type-discipline -55
Contributor
Greptile SummaryThe PR replaces broad or implicit typing across provider handlers, proxy request processing, streaming, and logging paths while ratcheting the static-analysis budgets downward.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains within the eligible follow-up review scope. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| litellm/main.py | Refines types around completion dispatch and defensively accepts the Azure AD token provider only when callable. |
| litellm/completion_extras/litellm_responses_transformation/handler.py | Introduces typed bridge inputs and consistently carries the optional encoding value through response transformation. |
| litellm/completion_extras/litellm_responses_transformation/transformation.py | Replaces broad response-transformation types with typed dictionaries, sequences, mappings, and concrete choice types. |
| litellm/integrations/generic_api/generic_api_callback.py | Types the Generic API callback’s request path and explicitly forwards its configured timeout. |
| litellm/proxy/hooks/parallel_request_limiter.py | Narrows rate-limiter state and handles responses whose usage or total-token data is absent. |
| litellm/responses/streaming_iterator.py | Corrects the asynchronous iterator protocol declaration while preserving delegated stream consumption. |
| litellm/litellm_core_utils/streaming_handler.py | Removes a concrete logging-object validator so existing duck-typed logging seams retain their prior compatibility. |
| basedpyright-code-budget.json | Ratchets static-analysis ceilings downward to retain the typing improvements. |
Reviews (2): Last reviewed commit: "fix: remove over-strict stream logging v..." | Re-trigger Greptile
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Contributor
Author
Contributor
Author
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: TPM undercount for image responses
- Extended
_response_total_tokensto acceptImageResponsewithImageUsage, so image generation successes now contribute theirtotal_tokensto per-key/user/team/customer TPM counters, and added a parametrized regression test covering that path.
- Extended
Or push these changes by commenting:
@cursor push f43045f357
Preview (f43045f357)
diff --git a/litellm/proxy/hooks/parallel_request_limiter.py b/litellm/proxy/hooks/parallel_request_limiter.py
--- a/litellm/proxy/hooks/parallel_request_limiter.py
+++ b/litellm/proxy/hooks/parallel_request_limiter.py
@@ -20,7 +20,7 @@
from litellm.proxy.auth.budget_throttle import throttled_limit
from litellm.proxy.common_utils.proxy_rate_limit_error import ProxyRateLimitError
from litellm.proxy.hooks.rate_limiter_utils import resolve_llm_provider_for_rate_limit
-from litellm.types.utils import Usage
+from litellm.types.utils import ImageResponse, ImageUsage, Usage
if TYPE_CHECKING:
from opentelemetry.trace import Span as _Span
@@ -35,10 +35,10 @@
def _response_total_tokens(response_obj: object) -> int:
- if not isinstance(response_obj, (ModelResponse, EmbeddingResponse, TextCompletionResponse)):
+ if not isinstance(response_obj, (ModelResponse, EmbeddingResponse, TextCompletionResponse, ImageResponse)):
return 0
response_usage: Final = getattr(response_obj, "usage", None)
- return response_usage.total_tokens if isinstance(response_usage, Usage) else 0
+ return response_usage.total_tokens if isinstance(response_usage, (Usage, ImageUsage)) else 0
class CacheObject(TypedDict):
diff --git a/tests/test_litellm/proxy/hooks/test_parallel_request_limiter.py b/tests/test_litellm/proxy/hooks/test_parallel_request_limiter.py
--- a/tests/test_litellm/proxy/hooks/test_parallel_request_limiter.py
+++ b/tests/test_litellm/proxy/hooks/test_parallel_request_limiter.py
@@ -11,7 +11,14 @@
_PROXY_MaxParallelRequestsHandler,
)
from litellm.proxy.utils import InternalUsageCache, hash_token
-from litellm.types.utils import EmbeddingResponse, TextCompletionResponse, Usage
+from litellm.types.utils import (
+ EmbeddingResponse,
+ ImageResponse,
+ ImageUsage,
+ ImageUsageInputTokensDetails,
+ TextCompletionResponse,
+ Usage,
+)
@pytest.mark.parametrize(
@@ -25,14 +32,24 @@
model="gpt-3.5-turbo-instruct",
usage=Usage(prompt_tokens=20, completion_tokens=30, total_tokens=50),
),
+ ImageResponse(
+ usage=ImageUsage(
+ input_tokens=20,
+ input_tokens_details=ImageUsageInputTokensDetails(
+ image_tokens=0, text_tokens=20
+ ),
+ output_tokens=30,
+ total_tokens=50,
+ ),
+ ),
],
)
@pytest.mark.asyncio
async def test_async_log_success_event_counts_non_chat_response_tokens(response_obj):
"""
- Embedding and text completion responses must increment the per key, user,
- team, and end user TPM counters, not just chat completion ModelResponse
- objects.
+ Embedding, text completion, and image responses must increment the per key,
+ user, team, and end user TPM counters, not just chat completion
+ ModelResponse objects.
"""
_api_key = hash_token("sk-12345")
user_id = "ishaan"You can send follow-ups to the cloud agent here.
Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 262d1b4. Configure here.
| if not isinstance(response_obj, (ModelResponse, EmbeddingResponse, TextCompletionResponse)): | ||
| return 0 | ||
| response_usage: Final = getattr(response_obj, "usage", None) | ||
| return response_usage.total_tokens if isinstance(response_usage, Usage) else 0 |
Contributor
There was a problem hiding this comment.
TPM undercount for image responses
Medium Severity
_response_total_tokens only accepts ModelResponse, EmbeddingResponse, and TextCompletionResponse with a litellm Usage instance. ImageResponse carries ImageUsage.total_tokens, which the previous unguarded response_obj.usage.total_tokens read counted. Image (and similar) successes now add 0 TPM, so keys can exceed configured TPM limits on those endpoints.
Reviewed by Cursor Bugbot for commit 262d1b4. Configure here.
Contributor
Author
There was a problem hiding this comment.
This is a pre-existing gap. We don't count TPM for image tokens. Belongs in a separate PR
Contributor
Author
There was a problem hiding this comment.
Also this is the legacy limiter which you have to opt into via env flag
5 tasks
…itellm_decrease_anys_fable5 # Conflicts: # litellm/proxy/common_request_processing.py # litellm/proxy/pass_through_endpoints/streaming_handler.py
5 tasks
yucheng-berri
approved these changes
Aug 11, 2026
…itellm_decrease_anys_fable5 # Conflicts: # ruff-strict-budget.json # type-discipline-budget.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
How it solves it:
User Flow
Before: the flow succeeds, and every response is byte-identical to what this branch produces
POST https://litellm-domain/v1/chat/completionswith{"model": "gpt-4o", "messages": [{"role": "user", "content": "hi"}]}and a proxy key in theAuthorizationheader200with achatcmpl-...completion, token usage filled in, and the spend for the call visible shortly after atGET https://litellm-domain/ui/?page=logs"stream": trueand watchdata:chunks arrive, ending indata: [DONE]429with the retry guidance textAfter: the flow succeeds identically at every step because this PR only changes type annotations
POST https://litellm-domain/v1/chat/completionswith{"model": "gpt-4o", "messages": [{"role": "user", "content": "hi"}]}and a proxy key in theAuthorizationheader200with achatcmpl-...completion, token usage filled in, and the spend for the call visible shortly after atGET https://litellm-domain/ui/?page=logs"stream": trueand watchdata:chunks arrive, ending indata: [DONE]429with the retry guidance textRelevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Whole-tree basedpyright before and after this branch, measured with the gate's own command (
basedpyright --outputjsonoverlitellm/), counting in-tree diagnostics:No basedpyright rule increased, repo-wide or in any individual file; a per-file-per-rule diff of the two whole-tree runs shows zero regression rows.
make lint-budget-updateoutput confirming the ceilings now hold the fixes:make checkis green on the full staged diff. The mapped test suites for every touched module pass: 8,082 passed with 7 failures that reproduce identically on the unmodified base (a missing optionalvertexaimodule in the local venv and one env-sensitive test), plus 137 passing follow-up runs covering the files edited after the main sweep and a rerun of the rate limiter's mapped tests after its final refactor. After the first CI round flagged a streaming logging validator this branch had introduced as too strict for duck-typed logging objects, the validator was removed and those seams restored to their base expressions; the three CI-failing test files plus the endpoint modules touched in the fix now pass locally (277 passed, 1 xfailed)Type
🧹 Refactoring
Changes
Typing-only changes concentrated where reportAny/reportExplicitAny density was highest: the LLM handler seams (azure, anthropic, openai, openai_like, codestral, predibase, replicate, sagemaker, oci, github_copilot handlers now carry real
LiteLLMLoggingObjand typed parameter annotations instead of implicit Any), the passthrough logging pipeline, the responses-MCP streaming wrapper, the parallel request limiter, and the completion dispatch path inmain.py. No runtime behavior changes: annotations, narrowed locals, and isinstance guards that are runtime-equivalent to the attribute accesses they replace, converting only would-be crash paths into graceful defaultsForbidden constructs were not used anywhere in the diff: no
cast(), no# type: ignore, no# pyright: ignore, no# noqa, no newAnyannotations, no new**kwargs. Diagnostics that could not be fixed without one of those were left in place rather than hiddenBudget files are ratcheted by
make lint-budget-updateso the cleared headroom cannot silently grow backCaveats (if any)
__aiter__was wrongly async, breaking directasync forazure_ad_token_provideris now ignored instead of crashingFinal Attestation
Note
Low Risk
Changes are overwhelmingly annotations and budget ratchets on non-critical paths; any runtime delta is limited to defensive guards on edge-case inputs (tokens, usage-less responses) rather than core request routing.
Overview
This PR tightens static typing across the LLM call path, HTTP layer, and integrations, and lowers
basedpyright-code-budget.jsonceilings (notablyreportAnyandreportExplicitAny) so the cleared headroom cannot regress silently.Handler and bridge seams now use concrete types instead of implicit
Any:logging_objis annotated asLiteLLMLoggingObjon Azure, Anthropic, OpenAI-family, Codestral, Predibase, Replicate, SageMaker, and related handlers; the responses→chat bridge addsencodingto its validatedTypedDictand passes it throughtransform_responseinstead of reading rawkwargs.Core utilities swap many
dict[str, Any]/ bareAnyparameters forobject,Mapping, or smallTypedDicts (reasoning items, OTEL span kwargs, web-search tool config, streaming chunk shapes, HTTPcontentbodies, aiohttp connector kwargs).litellm_loggingtypes in-memory logger lists and usesTYPE_CHECKINGaliases for enterprise callback factories soisinstancechecks stay accurate without import-timeAny.Small behavior-adjacent fixes ride along where typing exposed gaps: Azure image
azure_ad_tokenis only used when it is a non-empty string;GenericAPILoggerposts with explicit keyword args; OpenAIrun_thread_streampasses named arguments instead of a loosely typed**datadict.Reviewed by Cursor Bugbot for commit 262d1b4. Bugbot is set up for automated code reviews on this repo. Configure here.