fix(vertex_ai/gemini): MIME detection for extensionless GCS URIs + async event-loop fix - #27715
Closed
krrish-berri-2 wants to merge 5 commits into
Closed
fix(vertex_ai/gemini): MIME detection for extensionless GCS URIs + async event-loop fix#27715krrish-berri-2 wants to merge 5 commits into
krrish-berri-2 wants to merge 5 commits into
Conversation
Propagate litellm_params across Gemini transformers, resolve MIME from GCS metadata for extensionless gs:// objects, normalize file MIME aliases (format/mime_type/content_type), improve error mapping to BadRequestError, and add regression coverage for explicit MIME, metadata success, and file_data error-message behavior. Made-with: Cursor
…nt loop Address PR #27278 review feedback: - async_transform_request_body now runs the sync _transform_request_body (which may issue an httpx.get with up to 5s timeout for GCS metadata) in a worker thread via asyncify, so extensionless gs:// URIs no longer block the event loop on the async path. - Drop redundant `pass` in _get_gcs_object_content_type's except block. - Mock _get_gcs_object_content_type in test_process_gemini_media_gcs_without_extension_raises_clear_error so the test no longer makes real network calls. - Add a regression test that verifies async_transform_request_body does not block the event loop while the sync GCS metadata fetch is slow.
Address CodeQL alerts introduced in #27278: - Defer VertexBase import into a lazy helper to break the module-level cyclic import between transformation and vertex_llm_base. This also clears the downstream cyclic-import alerts on vertex_and_google_ai_studio_gemini.py. _GCS_METADATA_VERTEX_BASE is kept as a module attribute so existing patch.object tests still work. - Move get_file_extension_from_mime_type to a function-local import to drop the cyclic-import alert on litellm.types.files. - Build the GCS metadata URL with httpx.URL and url-encode the bucket so the fully-qualified host is obvious to CodeQL, clearing the partial SSRF alert.
- mypy: cast image_url/file dict to Dict[str, Any] before reading mime_type/content_type, since those keys are not declared on the ChatCompletionImageUrlObject / ChatCompletionFileObjectFile TypedDicts and mypy was inferring `object` for the fallback-or chain, failing the lint job. - security (veria-ai review): in _get_gcs_object_content_type, only call VertexBase.get_access_token when the caller explicitly passes vertex_project/vertex_credentials. Without explicit credentials we now skip the authenticated call entirely and only attempt an anonymous metadata lookup, so a Gemini API-key / Google AI Studio caller can no longer trick the proxy into querying GCS with the server's default Google credentials. - Add regression test covering the no-credentials path (no ADC token requested, no Authorization header on the outbound metadata call).
Translate the Chinese comments and docstrings introduced in this PR to English to match the repository convention for upstream BerriAI/litellm. No behavior change.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Fixes Gemini MIME detection for extensionless
gs://media URIs, and propagateslitellm_paramsso private GCS objects can be resolved via authenticated metadata lookup.Addresses Greptile feedback on #27278.
Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🐛 Bug Fix
✅ Test
Changes
Problem
gs://URI has no file extension, Gemini's current MIME inference fails, which causes the media payload to be rejected or mistyped.litellm_params(includingvertex_project/vertex_credentials) was not propagated into the message transformation layer, so authenticated GCS metadata lookups could not be performed for private objects.Fix
Core logic:
contentTypefrom GCS object metadata when thegs://URI is extensionless using a new_get_gcs_object_content_typehelperformat,mime_type,content_typelitellm_paramsfrom the request body down into the message transformation layer, carryingvertex_project/vertex_credentialsthroughGreptile feedback addressed:
httpx.getGCS metadata call is now offloaded to a worker thread viaasyncify(_transform_request_body)inasync_transform_request_body, so it cannot block the event loop. A dedicated test (test_async_transform_request_body_does_not_block_event_loop) verifies this with a 0.5s simulated blocking call and confirms a concurrentasyncio.sleep(0.05)completes without delay._get_gcs_object_content_typeorhttpx.get— no real outbound HTTP calls are made.Security
httpx.URLwith URL-encoded bucket/object names to prevent SSRF.Affected files
litellm/llms/vertex_ai/gemini/transformation.pylitellm/llms/gemini/chat/transformation.pylitellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.pytests/test_litellm/llms/vertex_ai/test_vertex.pyTests
All 40 unit tests in
tests/test_litellm/llms/vertex_ai/test_vertex.pypass, including:image/jpg→image/jpegalias normalizationBadRequestError