[litellm-agent] Staging → litellm_internal_staging (5/13/2026) - #27809
[litellm-agent] Staging → litellm_internal_staging (5/13/2026)#27809oss-pr-review-agent-shin[bot] wants to merge 5 commits into
Conversation
… OpenAI-compatible endpoint (#27508) Squash-merged by litellm-agent from yimao's PR.
|
@greptile please review |
Greptile SummaryThis staging merge bundles four PRs: DashScope embeddings and reranks via the OpenAI-compatible endpoint, Vertex AI/Gemini
Confidence Score: 5/5Safe to merge — all four merged PRs implement clean, well-tested incremental changes with no breaking modifications to existing call paths. The GCS metadata lookup, async offload via asyncify, streaming error detection, and null-guard fixes are all implemented correctly with comprehensive unit tests. No existing behavior is altered without an explicit opt-in (litellm_params defaults to None). The DashScope integration follows the same pattern as existing providers. litellm/llms/vertex_ai/gemini/transformation.py contains the most new logic (GCS metadata fetch, asyncify heuristic, MIME normalization) and is worth a second look before deploying to high-traffic environments.
|
| Filename | Overview |
|---|---|
| litellm/llms/vertex_ai/gemini/transformation.py | Major rework: adds extension-less gs:// MIME resolution via authenticated GCS metadata API, asyncify offload for async path, mid-stream error chunk detection, and BadRequestError for missing image_url/file fields. Logic is sound but the extension-based gs:// path redundantly runs _normalize_and_validate_gemini_mime_type on an already-validated canonical MIME type. |
| litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py | Adds _check_streaming_error static method to ModelResponseIterator that raises VertexAIError on mid-stream error chunks; called at the top of chunk_parser before JSON parsing. Only json.JSONDecodeError is caught in chunk_parser so VertexAIError propagates correctly. |
| litellm/llms/dashscope/rerank/transformation.py | New DashScopeRerankConfig for qwen3-rerank. URL construction, auth, and response parsing all look correct. RerankBilledUnits/RerankTokens accept Optional[int] so None total_tokens is safe. |
| litellm/llms/dashscope/embed/transformation.py | New DashScopeEmbeddingConfig using OpenAI-compatible endpoint. Clean implementation that delegates to base_llm_http_handler; error handling, URL construction, and response transformation are correct. |
| litellm/litellm_core_utils/prompt_templates/factory.py | Adds null guard for missing 'file' sub-field in anthropic_process_openai_file_message and both BedrockConverseMessagesProcessor._process_file_message/_async_process_file_message, raising BadRequestError instead of KeyError. |
| litellm/main.py | Adds DashScope branch to the embedding() dispatcher, delegates to base_llm_http_handler.embedding with an explicit dashscope_key extracted from api_key/litellm.api_key/env. |
Reviews (5): Last reviewed commit: "Fix Gemini MIME detection for extensionl..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| @@ -0,0 +1,141 @@ | |||
| """ | |||
There was a problem hiding this comment.
__init__.py in test directory
The new tests/test_litellm/llms/dashscope/ directory is missing __init__.py files. Every other provider test directory in this tree (e.g. llms/anthropic/, llms/volcengine/, llms/voyage/rerank/, etc.) ships an __init__.py. Without it, pytest may fail to discover these tests in certain configurations, and relative imports inside the package will break entirely. Both tests/test_litellm/llms/dashscope/ and any parent directory that also lacks the file need __init__.py added.
| def _resp(self, body, status_code=200): | ||
| return httpx.Response( | ||
| status_code=status_code, content=json.dumps(body).encode() | ||
| ) |
There was a problem hiding this comment.
_resp helper constructs httpx.Response without a request= argument, while the embedding test suite consistently passes request=httpx.Request(...). While the current code paths only call .json(), .status_code, and .text (none of which strictly require a bound request), inconsistency across the sibling test file means a future caller that accesses .request will get a RuntimeError. Providing the request parameter is the defensive choice and aligns with the embedding tests.
| def _resp(self, body, status_code=200): | |
| return httpx.Response( | |
| status_code=status_code, content=json.dumps(body).encode() | |
| ) | |
| def _resp(self, body, status_code=200): | |
| return httpx.Response( | |
| status_code=status_code, | |
| content=json.dumps(body).encode(), | |
| request=httpx.Request("POST", DEFAULT_RERANK_URL), | |
| ) |
| def test_non_json_response_raises(self): | ||
| bad = httpx.Response(status_code=500, content=b"<html>bad gateway</html>") |
There was a problem hiding this comment.
httpx.Response is also missing a request= argument. For consistency with both the _resp helper fix above and the embedding test file, a dummy request object should be supplied.
| def test_non_json_response_raises(self): | |
| bad = httpx.Response(status_code=500, content=b"<html>bad gateway</html>") | |
| def test_non_json_response_raises(self): | |
| bad = httpx.Response( | |
| status_code=500, | |
| content=b"<html>bad gateway</html>", | |
| request=httpx.Request("POST", DEFAULT_RERANK_URL), | |
| ) |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
#24550) Squash-merged by litellm-agent from krisxia0506's PR.
|
@greptile please review |
… silently swallowing (#23711) Squash-merged by litellm-agent from krisxia0506's PR.
|
@greptile please review |
#24503) Squash-merged by litellm-agent from krisxia0506's PR.
|
@greptile please review |
Squash-merged by litellm-agent from krisxia0506's PR.
|
@greptile please review |
3 participants
Merged PRs (4)
Auto-updated by litellm-agent on each merge.