Skip to content

✨ feat(image_gen): nano-banana Gemini image backend (generate + edit + reference) - #60

Merged
cwest merged 1 commit into
cwest/integrationfrom
topic/nano-banana-image-backend
Jul 11, 2026
Merged

✨ feat(image_gen): nano-banana Gemini image backend (generate + edit + reference)#60
cwest merged 1 commit into
cwest/integrationfrom
topic/nano-banana-image-backend

Conversation

@cwest

@cwest cwest commented Jul 11, 2026

Copy link
Copy Markdown
Owner

What

Adds a first-class nano-banana image_gen backend serving Google's Gemini image models (Nano Banana Pro / Nano Banana 2) through the local OpenAI-compatible proxy. It joins the existing ImageGenProvider framework and mirrors the shipped openrouter backend's chat-completions image protocol — one new backend on an existing framework, not a from-scratch plugin.

Capability (v1)

  • Text-to-image, image edit (image_url source), and reference-image grounding (reference_image_urls, multiple, clamped to 3) for character/subject consistency. Local files are inlined as data-URI image_url parts.
  • Reads the image from choices[0].message.images[0].image_url.url (message.content is null on this protocol); POSTs {base_url}/chat/completions with modalities:["image","text"].
  • Pro (gemini-3-pro-image) is the quality default; Flash (gemini-3.1-flash-image) is a one-parameter fast path. Model routing is config-driven: model kwarg → NANO_BANANA_IMAGE_MODEL env → image_gen.nano-banana.modelimage_gen.model (known ids only) → default. Nano Banana 2 Lite (gemini-3.1-flash-lite-image) is a documented catalog slot that drops in with zero code change when the proxy serves it.
  • image_config.aspect_ratio is honored on the generate path (1:1→1024×1024, 16:9→1376×768, 9:16→768×1376); on the edit path output dims track the source image (documented).

Auth (rubric-compliant)

Proxy/OpenAI-compatible by construction: credentials resolve via the shared runtime resolver against the vertex-llm-proxy custom_providers entry (overridable via image_gen.nano-banana.runtime). No Google credential is handled client-side — the proxy authenticates upstream. is_available() degrades gracefully (False, no crash) when the proxy/token is absent; generate() returns error_response for every failure mode (missing creds, HTTP error, timeout, connection error, empty response, IO error).

Framework cache GC (shared janitor)

The cache dir $HERMES_HOME/cache/images/ is shared by every backend, so GC is framework-level, not per-backend. A small janitor runs opportunistically inside save_b64_image / save_url_image (so all backends get GC for free), pruning oldest-first when total size exceeds max_total_mb (default 2048) or a file's age exceeds max_age_days (default 30), never deleting the just-written file, emitting one INFO line per prune, best-effort (never fails a save). Caps are config-overridable under image_gen.cache in config.yaml.

Verification

  • TDD throughout: tests/agent/test_image_cache_gc.py (12) + tests/plugins/image_gen/test_nano_banana_provider.py (31). Behavior/invariant assertions, no change-detector snapshots.
  • Full affected suites green: image_gen plugins + GC + registry/picker/dispatch/save + config + plugin-scanner = 409 tests, 0 failures.
  • E2E against the live proxy (real calls, no mocks): text-to-image (Pro landscape + Flash portrait), edit (image_url → modality=image), multi-reference grounding, real PNGs saved to the framework cache, and the GC janitor pruning-on-save with the INFO log line — all PASS. Generated image visually confirmed as a coherent render.
  • Backend discovered + registered by the real plugin loader alongside the six existing backends.

Config

New image_gen section in DEFAULT_CONFIG documents cache.max_age_days / cache.max_total_mb and the per-backend model/runtime slots. New key → deep-merge handles it; no _config_version bump.

Spec: docs/specs/nano-banana-image-backend.md · Plan: docs/plans/nano-banana-image-backend.md.

Out of scope (separate follow-ups)

nano-banana-prompting skill; illustrator persona; multi-turn stateful editing; upstreaming to core (stays a plugin).

…+ reference)

Add a first-class `nano-banana` image_gen backend serving Google's Gemini
image models (Nano Banana Pro / Nano Banana 2) through the local
OpenAI-compatible proxy. It joins the existing ImageGenProvider framework and
mirrors the shipped openrouter backend's chat-completions image protocol.

Capability (v1, verified E2E against the live proxy):
- Text-to-image, image editing (image_url source), and reference-image
  grounding (reference_image_urls, multiple, clamped to 3) for character/subject
  consistency. Local files are inlined as data-URI image_url parts.
- Reads the generated image from choices[0].message.images[0].image_url.url
  (message.content is null on this protocol); POSTs to
  {base_url}/chat/completions with modalities:["image","text"].
- Pro (gemini-3-pro-image) is the quality default; Flash
  (gemini-3.1-flash-image) is a one-parameter fast path. Model routing is
  config-driven (kwarg > NANO_BANANA_IMAGE_MODEL env > image_gen.nano-banana.model
  > image_gen.model > default); Nano Banana 2 Lite is a documented catalog slot
  that drops in with zero code change when the proxy serves it.
- image_config.aspect_ratio is honored on the generate path
  (1:1→1024x1024, 16:9→1376x768, 9:16→768x1376); edit-path dims track the
  source image.

Auth is proxy/OpenAI-compatible by construction: credentials resolve via the
shared runtime resolver against the vertex-llm-proxy custom_providers entry
(overridable via image_gen.nano-banana.runtime). No Google credential is handled
client-side. is_available() degrades gracefully (False, no crash) when the proxy
or token is absent; generate() surfaces error_response for every failure mode.

Also add a framework-level GC janitor over the shared cache dir
($HERMES_HOME/cache/images/, used by every backend). It runs opportunistically
inside save_b64_image / save_url_image — so all backends get GC for free —
pruning oldest-first when total size exceeds max_total_mb (default 2048) or a
file's age exceeds max_age_days (default 30), never deleting the just-written
file, emitting one INFO line per prune, best-effort (never fails a save). Caps
are config-overridable under image_gen.cache in config.yaml.

@cwest cwest left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No changes needed.

This adds one backend to the existing image_gen framework rather than a from-scratch plugin, which is the right shape for it. The chat-completions protocol is built to the documented spec: modalities set to ["image","text"], prompt and any source/reference images inlined as content parts, and the result read from choices[0].message.images[0].image_url.url with message.content null. The generate/edit/reference paths collapse into one call keyed on whether any source image is present, references clamp to 3, and local files are inlined as data URIs so the proxy never needs to reach a path on our disk.

Model routing follows the documented precedence (kwarg, env, image_gen.nano-banana.model, image_gen.model when it names a known id, then the Pro default). The top-level image_gen.model is only honored when it matches a known model, so it can't hijack this backend from another one. The Lite id sits in the catalog as a config-only slot, so it becomes selectable with no code change once the proxy serves it. Auth is proxy-only by construction: credentials resolve through the shared runtime resolver against the vertex-llm-proxy entry, no Google credential is handled client-side, is_available() returns False cleanly when the token is absent, and every failure mode in generate() returns error_response instead of raising.

The cache GC is placed at the framework level inside save_b64_image and save_url_image, so all backends inherit it with no per-backend code. The prune is oldest-first, protects the just-written file in both the age and size passes, emits one INFO line only when it actually deletes, and is wrapped so a GC error can never fail a save.

Verified locally: the two new test files pass (43/43) run through scripts/run_tests.sh, and the backend is discovered and registered through the real plugin loader alongside the other seven backends, with the Pro default, the [text,image] capabilities, the three-model catalog, and is_available() returning False with no proxy token configured. The live-proxy behavior (real image output, aspect-ratio dimensions, edit/reference grounding) is covered by the e2e CI check and matches the code paths; I did not re-run it against the proxy here. Commit is signed, attribution is clean, and the merge state is CLEAN with zero open threads.

@cwest
cwest marked this pull request as ready for review July 11, 2026 19:40
@cwest
cwest merged commit 08e4e5d into cwest/integration Jul 11, 2026
31 checks passed
@cwest
cwest deleted the topic/nano-banana-image-backend branch July 11, 2026 19:45
cwest added a commit that referenced this pull request Jul 26, 2026
…+ reference) (#60)

Add a first-class `nano-banana` image_gen backend serving Google's Gemini
image models (Nano Banana Pro / Nano Banana 2) through the local
OpenAI-compatible proxy. It joins the existing ImageGenProvider framework and
mirrors the shipped openrouter backend's chat-completions image protocol.

Capability (v1, verified E2E against the live proxy):
- Text-to-image, image editing (image_url source), and reference-image
  grounding (reference_image_urls, multiple, clamped to 3) for character/subject
  consistency. Local files are inlined as data-URI image_url parts.
- Reads the generated image from choices[0].message.images[0].image_url.url
  (message.content is null on this protocol); POSTs to
  {base_url}/chat/completions with modalities:["image","text"].
- Pro (gemini-3-pro-image) is the quality default; Flash
  (gemini-3.1-flash-image) is a one-parameter fast path. Model routing is
  config-driven (kwarg > NANO_BANANA_IMAGE_MODEL env > image_gen.nano-banana.model
  > image_gen.model > default); Nano Banana 2 Lite is a documented catalog slot
  that drops in with zero code change when the proxy serves it.
- image_config.aspect_ratio is honored on the generate path
  (1:1→1024x1024, 16:9→1376x768, 9:16→768x1376); edit-path dims track the
  source image.

Auth is proxy/OpenAI-compatible by construction: credentials resolve via the
shared runtime resolver against the vertex-llm-proxy custom_providers entry
(overridable via image_gen.nano-banana.runtime). No Google credential is handled
client-side. is_available() degrades gracefully (False, no crash) when the proxy
or token is absent; generate() surfaces error_response for every failure mode.

Also add a framework-level GC janitor over the shared cache dir
($HERMES_HOME/cache/images/, used by every backend). It runs opportunistically
inside save_b64_image / save_url_image — so all backends get GC for free —
pruning oldest-first when total size exceeds max_total_mb (default 2048) or a
file's age exceeds max_age_days (default 30), never deleting the just-written
file, emitting one INFO line per prune, best-effort (never fails a save). Caps
are config-overridable under image_gen.cache in config.yaml.

(cherry picked from commit 08e4e5d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant