gemini imageConfig forwarding in /images/generations and /images/edits - #28192
Conversation
|
@greptileai please review |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR fixes Gemini
Confidence Score: 5/5The change is safe to merge; it is well-scoped to Gemini image generation/editing paths and does not touch the authentication or routing layers. All previously identified issues have been addressed: the 21:9 mapping typo is corrected to 792x336, backward-compat entries for 1280x896/896x1280 are preserved, the malformed-imageConfig JSON path now returns a 400 instead of a 500, and explicit imageConfig takes precedence over size in both transformation classes. The Imagen path now correctly forwards aspectRatio and imageSize as top-level parameters via the updated GeminiImageGenerationParameters model. Test coverage is thorough and strengthened, not weakened. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/llms/gemini/common_utils.py | Adds GEMINI_IMAGE_SIZE_TO_ASPECT_RATIO lookup table and helper functions for mapping OpenAI size strings to Gemini imageConfig dicts; includes backward-compat entries for 1280x896 and 896x1280, and the corrected 792x336 for 21:9. |
| litellm/llms/gemini/image_generation/transformation.py | Replaces the old 5-entry aspect-ratio map with the new imageConfig helpers; correctly branches on 'gemini' vs Imagen for how the config is applied, with explicit imageConfig-over-size precedence logic. |
| litellm/llms/gemini/image_edit/transformation.py | Replaces _map_size_to_aspect_ratio with map_openai_size_to_gemini_image_config, adds multipart JSON string parsing for imageConfig with proper UnsupportedParamsError on malformed input, and handles imageSize exclusion for 2.5-flash models. |
| litellm/types/llms/gemini.py | Adds imageSize field to GeminiImageGenerationParameters so the Pydantic model accepts the new field for Imagen (non-Gemini) model requests. |
| litellm/utils.py | Registers imageConfig as an optional parameter with None default in get_optional_params_image_gen so it reaches the provider layer without triggering UnsupportedParamsError for non-Gemini providers. |
| tests/llm_translation/test_gemini.py | Adds comprehensive parametrized tests for the full size-to-imageConfig mapping table, snapping logic, auto/invalid size pass-through, and end-to-end integration with mock HTTP handler. |
Reviews (4): Last reviewed commit: "Format Gemini image generation transform" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| def supports_gemini_image_size(model: str) -> bool: | ||
| return "2.5-flash" not in model |
There was a problem hiding this comment.
supports_gemini_image_size hard-codes "2.5-flash" as a string guard. Per the project's custom rule, model-specific capability flags must live in model_prices_and_context_window.json and be queried through get_model_info, so that support for future models can be enabled without a code change. As written, any new model that also lacks imageSize support (or any future Gemini 2.5-flash variant with a slightly different name) will silently receive imageConfig objects that include imageSize, potentially causing API errors.
Rule Used: What: Do not hardcode model-specific flags in the ... (source)
There was a problem hiding this comment.
I’m going to keep the 2.5-flash guard local here rather than adding a new model_prices_and_context_window.json capability flag for this PR. This matches existing Gemini transformation patterns in this area: VertexGeminiConfig._is_gemini_3_or_newer() uses model-name detection for Gemini 3 behavior, _supports_penalty_parameters() has a model-name exception list, and the Gemini thinking mapping branches on gemini-2.5-flash-lite / gemini-2.5-pro / gemini-2.5-flash substrings.
For image generation, this is also an older-model exception: Gemini 2.5 Flash image supports aspectRatio but not imageSize, while newer Gemini image models are expected to support imageSize. Adding a new global model metadata flag for that narrow exception would add more surface area than this PR needs. I’ll add an inline code comment to make this intentional.
| elif k == "imageConfig": | ||
| mapped_params["imageConfig"] = v |
There was a problem hiding this comment.
isinstance guard to match the pattern used in GeminiImageEditConfig.map_openai_params, which validates imageConfig is a dict before forwarding it. Without this check a non-dict value (e.g. a string passed by mistake) would be silently forwarded and then dropped in transform_image_generation_request without any error, while the image-edit path would just skip it cleanly.
| elif k == "imageConfig": | |
| mapped_params["imageConfig"] = v | |
| elif k == "imageConfig": | |
| if isinstance(v, dict): | |
| mapped_params["imageConfig"] = v |
There was a problem hiding this comment.
Fixed in force-pushed commit 2dfdd6f. GoogleImageGenConfig.map_openai_params now uses explicit size then imageConfig precedence, and only forwards explicit imageConfig when it is a dict.
Greptile SummaryThis PR adds proper
Confidence Score: 3/5Not safe to merge as-is: the image-generation transformation no longer passes any aspect ratio to non-Gemini Imagen models when a size is specified. The image-generation transformation now writes imageConfig into optional_params for every model, but GeminiImageGenerationParameters only has an aspectRatio field. Pydantic silently ignores imageConfig, so any user passing size to an Imagen model gets a request with no aspect ratio — a silent regression. The image-edit path is clean and the new lookup table is well-tested. litellm/llms/gemini/image_generation/transformation.py — map_openai_params and transform_image_generation_request need to route Imagen models to aspectRatio while routing Gemini models to imageConfig.
|
| Filename | Overview |
|---|---|
| litellm/llms/gemini/image_generation/transformation.py | Switches from writing aspectRatio to writing imageConfig in optional_params; breaks Imagen (non-Gemini) models because GeminiImageGenerationParameters has no imageConfig field |
| litellm/llms/gemini/common_utils.py | Adds size-to-imageConfig mapping with a large lookup table and log-distance fallback; hardcodes 2.5-flash model name for imageSize support, and silently changes aspect-ratio for sizes 1280x896/896x1280 |
| litellm/llms/gemini/image_edit/transformation.py | Correctly refactors image-edit mapping to use imageConfig; imageConfig from request overrides size-derived config deterministically |
| litellm/proxy/common_utils/http_parsing_utils.py | Adds Gemini-specific imageConfig to the list of JSON-decoded form fields in a shared proxy utility |
| litellm/utils.py | Adds imageConfig as a recognized optional param in get_optional_params_image_gen; minimal and correct |
| litellm/types/images/main.py | Adds imageConfig to ImageEditOptionalRequestParams TypedDict; straightforward type addition |
| tests/test_litellm/llms/gemini/image_edit/test_gemini_image_edit_transformation.py | New unit tests cover imageConfig forwarding for image edit; pure mock-based |
| tests/llm_translation/test_gemini.py | Comprehensive parametrized tests for size-to-imageConfig mapping; mocked HTTP calls |
Comments Outside Diff (1)
-
litellm/llms/gemini/image_generation/transformation.py, line 55-66 (link)Aspect-ratio mapping silently dropped for Imagen (non-Gemini) models
The old code set
mapped_params["aspectRatio"], which is a recognized field onGeminiImageGenerationParameters. The new code setsmapped_params["imageConfig"]for every model — butGeminiImageGenerationParametershas noimageConfigfield, so whentransform_image_generation_requestreaches the non-Gemini branch and callsGeminiImageGenerationParameters(**optional_params), Pydantic silently drops the key. Any user currently passingsizeto an Imagen model (e.g.imagen-3.0-generate-002) will get a request with no aspect ratio at all, whereas before they received the correctaspectRatiovalue.
Reviews (2): Last reviewed commit: "Fix Gemini image config mapping" | Re-trigger Greptile
| def supports_gemini_image_size(model: str) -> bool: | ||
| return "2.5-flash" not in model |
There was a problem hiding this comment.
imageSize support
supports_gemini_image_size hard-codes the string "2.5-flash" to gate feature availability. Per the team's rule, model-specific capabilities should live in model_prices_and_context_window.json and be queried via get_model_info, so that a new model that also lacks imageSize support doesn't require a code change and a LiteLLM release to be handled correctly.
| def supports_gemini_image_size(model: str) -> bool: | |
| return "2.5-flash" not in model | |
| def supports_gemini_image_size(model: str) -> bool: | |
| # TODO: move this capability flag to model_prices_and_context_window.json | |
| # and query via get_model_info so new models work without a code change. | |
| return "2.5-flash" not in model |
Rule Used: What: Do not hardcode model-specific flags in the ... (source)
There was a problem hiding this comment.
I’m going to keep the 2.5-flash guard local here rather than adding a new model_prices_and_context_window.json capability flag for this PR. This matches existing Gemini transformation patterns in this area: VertexGeminiConfig._is_gemini_3_or_newer() uses model-name detection for Gemini 3 behavior, _supports_penalty_parameters() has a model-name exception list, and the Gemini thinking mapping branches on gemini-2.5-flash-lite / gemini-2.5-pro / gemini-2.5-flash substrings.
For image generation, this is also an older-model exception: Gemini 2.5 Flash image supports aspectRatio but not imageSize, while newer Gemini image models are expected to support imageSize. Adding a new global model metadata flag for that narrow exception would add more surface area than this PR needs. I’ll add an inline code comment to make this intentional.
| for json_field in ("metadata", "imageConfig"): | ||
| if json_field in parsed_body and isinstance( | ||
| parsed_body[json_field], str | ||
| ): | ||
| parsed_body[json_field] = json.loads(parsed_body[json_field]) |
There was a problem hiding this comment.
imageConfig is a Gemini API concept; adding it to this shared proxy parsing file couples the provider-agnostic request layer to a single provider's naming convention. Provider-specific deserialization logic should live in the provider's own transformation layer, not here.
Rule Used: What: Avoid writing provider-specific code outside... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Fixed in force-pushed commit 2dfdd6f. I removed imageConfig JSON parsing from the shared proxy parsing utility and moved multipart string parsing into the Gemini image edit transformation path.
| def _map_dimensions_to_gemini_aspect_ratio(width: int, height: int) -> str: | ||
| if (width, height) in GEMINI_IMAGE_SIZE_TO_ASPECT_RATIO: | ||
| return GEMINI_IMAGE_SIZE_TO_ASPECT_RATIO[(width, height)] | ||
|
|
||
| requested_ratio = width / height | ||
| return min( | ||
| GEMINI_IMAGE_ASPECT_RATIOS, | ||
| key=lambda aspect_ratio: abs( | ||
| math.log(GEMINI_IMAGE_ASPECT_RATIOS[aspect_ratio] / requested_ratio) | ||
| ), | ||
| ) |
There was a problem hiding this comment.
1280x896 and 896x1280
The old _map_size_to_aspect_ratio mapped "1280x896" → "4:3" and "896x1280" → "3:4". These dimensions are absent from GEMINI_IMAGE_SIZE_TO_ASPECT_RATIO, so they fall through to the log-distance snapper, which now picks "3:2" and "2:3" respectively. Users who relied on these specific sizes will get a different aspect ratio after the upgrade without any warning.
Rule Used: What: avoid backwards-incompatible changes without... (source)
There was a problem hiding this comment.
Fixed in force-pushed commit 2dfdd6f. The mapping table now preserves the prior behavior for 1280x896 -> 4:3 and 896x1280 -> 3:4, with regression coverage added.
9a4ec8e to
2dfdd6f
Compare
|
@greptileai please re-review |
| (1376, 768): "16:9", | ||
| (2752, 1536): "16:9", | ||
| (5504, 3072): "16:9", | ||
| (792, 168): "21:9", |
There was a problem hiding this comment.
21:9 entry (792, 168) is mathematically incorrect: 792 / 168 = 4.714, which is nowhere near 21 / 9 = 2.333. Every other 21:9 group divides the larger resolution by 2 to get the next row — (3168, 1344) → (1584, 672) → half again should be (792, 336), not (792, 168). As a result, any request with size="792x168" will hit the exact-match path and return aspectRatio: "21:9" for a 4.7:1 image. The companion test has the same value so it would need to be corrected too.
| (792, 168): "21:9", | |
| (792, 336): "21:9", |
There was a problem hiding this comment.
Fixed in 82a9bba880 by correcting the 21:9 512-row lookup to 792x336.
| image_config_param = filtered_params.get("imageConfig") | ||
| if isinstance(image_config_param, str): | ||
| image_config_param = json.loads(image_config_param) | ||
| if isinstance(image_config_param, dict): | ||
| mapped_params["imageConfig"] = image_config_param |
There was a problem hiding this comment.
json.loads here is uncaught. When a multipart form request supplies a malformed string for imageConfig (e.g. "{bad") this raises json.JSONDecodeError, which propagates all the way up as an unhandled exception and surfaces to the caller as a 500 error instead of a 400 validation error.
| image_config_param = filtered_params.get("imageConfig") | |
| if isinstance(image_config_param, str): | |
| image_config_param = json.loads(image_config_param) | |
| if isinstance(image_config_param, dict): | |
| mapped_params["imageConfig"] = image_config_param | |
| image_config_param = filtered_params.get("imageConfig") | |
| if isinstance(image_config_param, str): | |
| try: | |
| image_config_param = json.loads(image_config_param) | |
| except json.JSONDecodeError: | |
| image_config_param = None | |
| if isinstance(image_config_param, dict): | |
| mapped_params["imageConfig"] = image_config_param |
There was a problem hiding this comment.
Fixed in 82a9bba880 by converting malformed string imageConfig into an UnsupportedParamsError/400 instead of allowing a raw JSONDecodeError/500, with regression coverage.
| ("1376x768", "16:9", "1K"), | ||
| ("2752x1536", "16:9", "2K"), | ||
| ("5504x3072", "16:9", "4K"), | ||
| ("792x168", "21:9", "512"), |
There was a problem hiding this comment.
"792x336" (ratio ≈ 2.36, matching 21:9) so the test actually validates the correct mapping rather than confirming the bug.
| ("792x168", "21:9", "512"), | |
| ("792x336", "21:9", "512"), |
Rule Used: What: Flag any modifications to existing tests and... (source)
There was a problem hiding this comment.
Fixed in 82a9bba880 by updating the test expectation to 792x336.
|
🤖 litellm-agent: This PR is currently BLOCKED from merge. Score: 2/5 ❌ Why blocked:
Details: Score docked for: 1 PR-related CI failure (Greptile gate: score 3/5 below required 4/5 — request a Greptile review ( Fix the issues above and push an update — the bot will re-review automatically.
|
|
@greptileai fixes pushed (82a9bba + lint fix: c066090), please re-review. BTW the mapping table for supported and asserted gemini resolutions aspect sizes was put together based on https://ai.google.dev/gemini-api/docs/image-generation#aspect_ratios_and_image_size and now re-validated. The issue here is that google's published documentation contains what appears to be a typo 
Our inaccuracy is forgivable since it's based on Google's.Agreed that our code still needs to be valid, 792x168 is likely not a thing. If size "792x168" is legitimately passed by the client, let it snap to whatever closest supported aspect ratio is (4:1) via the fallback math. |
|
@oss-pr-review-agent-shin please re-review. Gemini's feedback taken into account, now 5/5 |
|
@greptileai you left two separate comments one with 5/5 and another with 3/5; The finalizing PR review agent (Shin) is now confused. If you believe the 3/5 comment was already addressed, can you please edit it as well. |
|
Clarification re: conflicting scores The 3/5 comment was posted against an earlier commit ( That issue has been fully resolved in the subsequent commits ( The 5/5 comment reflects the current state of the PR after those fixes. The 3/5 score is stale and should be disregarded — the current score is 5/5 and this PR is safe to merge. Tip: You can customize Greptile's behavior for this repo with |
|
🤖 litellm-agent: This PR is currently BLOCKED from merge. Score: 2/5 ❌ Why blocked:
Details: Score docked for: 1 PR-related CI failure (Greptile gate: score 3/5 below required 4/5 — request a Greptile review ( Fix the issues above and push an update — the bot will re-review automatically.
|
1 participant
Relevant issues
Fixes #24621
Prior PRs were incomplete:
@Chesars FYI
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand will receiv a Confidence Score of at least 4/5 before requesting a maintainer reviewScreenshots / Proof of Fix
Live local repro against a locally built LiteLLM container/proxy on localhost.
Tested option coverage:
/v1/images/generationswith OpenAIsizemapped to GeminiimageConfig./v1/images/generationswith native GeminiimageConfigforwarded as-is./v1/images/generationswithimagen-4.0-generate-001, verifying Imagen still routes through the:predictresponse shape./v1/images/editswith multipart image upload plus multipartimageConfig, verifying the uploaded image is included and edit usage records input image tokens.Gemini
/v1/images/generationswith OpenAIsizeGemini
/v1/images/generationswith nativeimageConfigImagen
/v1/images/generationsGemini
/v1/images/editswith multipart image upload andimageConfigType
🐛 Bug Fix
Changes
For for
/v1/images/generationsand/v1/images/editsendpoints:sizefor Gemini image generation by mapping it to GeminigenerationConfig.imageConfig. Use the documented Gemini image aspect ratio and resolution table for exact matches, then snap unsupported dimensions to the closest known aspect ratio/resolution.imageConfigas-is, if provided in the bodyimageConfigentirely forsize=autoor unparseable sizes so Gemini defaults remain in control.aspectRatioand omitimageSize.