Skip to content

🐛 fix(image_gen): route nano-banana text-to-image through /v1/images/generations (4K) - #63

Merged
cwest merged 1 commit into
cwest/integrationfrom
topic/nano-banana-images-path-4k
Jul 12, 2026
Merged

🐛 fix(image_gen): route nano-banana text-to-image through /v1/images/generations (4K)#63
cwest merged 1 commit into
cwest/integrationfrom
topic/nano-banana-images-path-4k

Conversation

@cwest

@cwest cwest commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Why

gemini-3-pro-image silently returned ~1K (1408x768) regardless of the
configured resolution. Root cause (verified against LiteLLM 1.91.2 and
1.92.0 source): the /chat/completions path has no imageConfig
generationConfig mapping, so image_size on chat is inert for Pro. LiteLLM
maps resolution only on /v1/images/generations.

What

Route text-to-image generation through /v1/images/generations with a
nested imageConfig, so Pro (and all image models) honor 4K.

Exact text-to-image request shape implemented

POST {base_url}/v1/images/generations
Authorization: Bearer <proxy token>
Content-Type: application/json

{
  "model": "<model_id>",
  "prompt": "<prompt>",
  "imageConfig": {
    "imageSize": "4K",        // configured resolution (1K/2K/4K)
    "aspectRatio": "16:9"     // proxy string (1:1 / 16:9 / 9:16)
  }
}
  • Nested imageConfig — maps to Vertex generationConfig.imageConfig for
    pro/flash/lite (model-agnostic, no flag). Flat imageSize/image_size at the
    top level are dropped by the proxy, so they are not sent.
  • Response is the images-API shape: data[0].b64_json (preferred, decoded via
    save_b64_image) and/or data[0].url (fetched via save_url_image).
  • base_url already ends in /v1, so the code joins {base_url}/images/generations
    {host}/v1/images/generations.

Edit / reference path — kept on /chat/completions (unchanged)

Design decision per spec change #2: the images endpoint has no clean
input-image contract
at LiteLLM 1.92.0 (image input lives on the multipart
/v1/images/edits route, unverified against this proxy). The chat edit path
works today, and resolution matters less for edits because the model preserves
the source image's dimensions regardless. So text-to-image — where 4K actually
matters — moves to the images path, and edit/reference stays on chat. The
split is documented in the module docstring and inline comments.

Resilience

If an older proxy rejects the nested imageConfig (pre-1.92.0), the request is
retried once without it so generation still lands (falls back to default
geometry rather than hard-failing). Reuses _resolve_model /
_resolve_resolution / _clamp_resolution and the aspect map unchanged; the
Lite 1K cap still clamps on the images path.

Tests (mocks only — no running proxy)

New/updated behavior-contract tests assert:

  • text-to-image hits /v1/images/generations with a nested imageConfig
    carrying imageSize (= configured resolution) and aspectRatio (this
    contract would have caught the original ~1K bug);
  • response parse: b64_json → saved path; url → saved path; b64_json
    preferred when both present;
  • imageConfig-rejection fallback → retry without it;
  • edit/reference path still routes to /chat/completions with the chat payload
    shape (and does NOT carry the nested imageConfig);
  • existing nano-banana tests remain green.
$ uv run pytest tests/plugins/image_gen/test_nano_banana_provider.py -q
.....................................................                    [100%]
53 passed in 1.09s

Wider regression scope (no regressions):

$ uv run pytest tests/plugins/ -q
1696 passed in 167.98s

$ uv run pytest tests/tools/test_image_generation.py tests/tools/test_image_generation_image_to_image.py -q
81 passed in 1.82s

Live verification — DEFERRED to proxy restart (not done here)

Live 4K pixel proof requires the proxy on LiteLLM >=1.92.0 (proxy PR #10).
Until the proxy restarts onto 1.92.0, live 4K cannot be verified — that is
Casey's restart step. This PR does not restart the proxy or gateway.

Post-restart verification command (expect ~4K, not ~1376x768):

# Generate at 4K via the tool, then decode the PNG IHDR for dimensions.
python - <<'PY'
import struct
from plugins.image_gen import  # (load nano-banana provider via importlib)
# ... run image_generate at 4K, then:
def png_dims(path):
    with open(path, "rb") as f:
        f.read(16)                      # skip PNG sig + IHDR len/type
        w, h = struct.unpack(">II", f.read(8))
    return w, h
# expect ~4K (e.g. 4K+16:9 -> 5504x3072, 4K+1:1 -> 4096x4096), NOT 1376x768
PY

Concretely: set image_gen.provider: nano-banana + image_gen.nano-banana.resolution: 4K,
run an image_generate at aspect landscape, and confirm the decoded PNG is
~4K (5504×3072) rather than the old ~1K (1376×768).

…generations

gemini-3-pro-image silently returned ~1K (1408x768) regardless of the
configured resolution because the chat/completions path has NO
imageConfig->generationConfig mapping (verified against LiteLLM 1.91.2
and 1.92.0 source). LiteLLM maps resolution ONLY on /v1/images/generations.

Route text-to-image to POST {base_url}/v1/images/generations with a NESTED
imageConfig ({"imageSize": resolution, "aspectRatio": ratio}), which maps to
Vertex generationConfig.imageConfig for pro/flash/lite (model-agnostic, no
flag), so Pro (and all image models) honor 4K. Parse the images-API response
(b64_json preferred, url fallback) and save it.

Keep the EDIT / reference path on chat/completions: the images endpoint has
no clean input-image contract at 1.92.0 (image input lives on the multipart
/v1/images/edits route, unverified against this proxy), the chat edit path
works today, and resolution matters less for edits since the model preserves
source dimensions. The split is documented in the module docstring + comments.

Preserve resolution-fallback resilience: if an older proxy rejects the nested
imageConfig, retry once without it so generation still lands. Reuse
_resolve_model/_resolve_resolution/_clamp_resolution and the aspect map
unchanged; the Lite 1K cap still clamps on the images path.

Tests mock the HTTP layer (no running proxy): assert text-to-image hits
/v1/images/generations with the nested imageConfig carrying imageSize +
aspectRatio, response parse (b64_json/url), imageConfig-rejection fallback,
and that the edit/reference path still routes to chat/completions unchanged.

@cwest cwest left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed from the remote at 1d40f3a. The split is the right call and the diff does exactly what the description says.

Text-to-image now POSTs to {base_url}/images/generations (base_url ends in /v1, so it lands on /v1/images/generations) with a nested imageConfig {imageSize, aspectRatio} and no flat imageSize/image_size at the top level. The response parser reads data[0].b64_json first and falls back to data[0].url, saving via save_b64_image / save_url_image. The edit/reference path is untouched on /chat/completions and still carries its flat image_config, not the nested one. The imageConfig-rejection fallback retries once without the block, and _is_resolution_field_error was widened to catch imageConfig/imageSize spellings while still letting an unrelated 400 surface as a real error. _resolve_model, _resolve_resolution, _clamp_resolution and the aspect map are reused unchanged, and the Lite 1K cap still clamps on the images path. save_url_image moved to a top-level import and is used on both paths. It stays inside the plugin: no core tool-schema change, no per-call param, no new env var.

The tests earn their keep. test_posts_to_images_generations_endpoint pins the URL and test_payload_shape_text_to_image_nested_image_config asserts the nested imageConfig with imageSize == configured resolution and aspectRatio, plus that no flat resolution keys and no chat-only keys leak — that pair is what would have caught the original ~1K bug. Response parse is covered for b64_json, url, and b64_json-preferred-when-both. The fallback test asserts the two-call sequence (first with imageConfig, retry without). The edit path has both a routing test (still hits chat/completions) and a payload-shape test (keeps image_config, does not carry imageConfig). These are behavior contracts on request shape and outcome, not snapshot/change-detector tests.

I ran the suites at this SHA in a throwaway checkout rather than trusting the counts: tests/plugins/image_gen/test_nano_banana_provider.py -> 53 passed; the twelve load-bearing contract tests pass by name; tests/tools/test_image_generation.py + test_image_generation_image_to_image.py -> 81 passed; tests/plugins/ -> 1696 passed, no failures, no new regressions against the cwest/integration baseline. The commit is signed (good signature), Conventional Commits + emoji, no attribution trailers.

Live 4K pixel proof is not exercised here — it needs the proxy on LiteLLM >=1.92.0 (proxy PR #10). Correctness is established from source and mocked tests; the pixel confirmation is the post-restart step.

No changes needed.

@cwest
cwest marked this pull request as ready for review July 12, 2026 16:36
@cwest
cwest merged commit 24a741c into cwest/integration Jul 12, 2026
31 checks passed
@cwest
cwest deleted the topic/nano-banana-images-path-4k branch July 12, 2026 16:36
cwest added a commit that referenced this pull request Jul 26, 2026
…generations (#63)

gemini-3-pro-image silently returned ~1K (1408x768) regardless of the
configured resolution because the chat/completions path has NO
imageConfig->generationConfig mapping (verified against LiteLLM 1.91.2
and 1.92.0 source). LiteLLM maps resolution ONLY on /v1/images/generations.

Route text-to-image to POST {base_url}/v1/images/generations with a NESTED
imageConfig ({"imageSize": resolution, "aspectRatio": ratio}), which maps to
Vertex generationConfig.imageConfig for pro/flash/lite (model-agnostic, no
flag), so Pro (and all image models) honor 4K. Parse the images-API response
(b64_json preferred, url fallback) and save it.

Keep the EDIT / reference path on chat/completions: the images endpoint has
no clean input-image contract at 1.92.0 (image input lives on the multipart
/v1/images/edits route, unverified against this proxy), the chat edit path
works today, and resolution matters less for edits since the model preserves
source dimensions. The split is documented in the module docstring + comments.

Preserve resolution-fallback resilience: if an older proxy rejects the nested
imageConfig, retry once without it so generation still lands. Reuse
_resolve_model/_resolve_resolution/_clamp_resolution and the aspect map
unchanged; the Lite 1K cap still clamps on the images path.

Tests mock the HTTP layer (no running proxy): assert text-to-image hits
/v1/images/generations with the nested imageConfig carrying imageSize +
aspectRatio, response parse (b64_json/url), imageConfig-rejection fallback,
and that the edit/reference path still routes to chat/completions unchanged.

(cherry picked from commit 24a741c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant