feat(plugins): add Google Vertex AI image and video generation providers

[conernar]

Adds native support for Vertex AI's Veo and Gemini image/video models (Omni included) via GCP credentials.
Shared vertex_adapter is used for auth resolution.

What does this PR do?

Hermes has pluggable image_gen / video_gen backends, but none for Google Vertex AI — users who already run their chat model through the vertex provider (service-account auth) had no way to use Google's media models (Gemini image, Imagen, Veo, Gemini Omni) from the image_generate / video_generate tools.

This PR adds two bundled kind: backend plugins. Per the footprint ladder, no core files are touched — the plugins register through the existing provider registries and reuse agent/vertex_adapter.py for credential resolution, so a user with a working vertex chat setup needs zero new secrets: just image_gen.provider: vertex / video_gen.provider: vertex in config.yaml.

Routing covers all four Vertex media API surfaces, dispatched by model-id prefix:

Surface	Models	Endpoint
generateContent (image out)	gemini-* image models	global
:predict	imagen-*	regional (us-central1 default)
:predictLongRunning + poll	veo-*	regional (us-central1 default)
Interactions API (v1beta1)	gemini-* video (Omni)	global

Design choice (deliberate): model IDs pass through unvalidated, with google/ publisher-prefix normalization (the vertex chat endpoint convention). Google's media catalog moves fast — Veo previews were removed 2026-04, Imagen 2–4 and Veo ≤3.0 retired 2026-06-30, and gemini-omni-flash-preview shipped 2026-06-30. Validating against a hardcoded catalog would break users on every Google release cycle; with passthrough, any current or future model on these four surfaces works via config alone. The built-in catalog (list_models) is advisory.

Feature coverage: text-to-image, image editing (Gemini multimodal contents), text-to-video and image-to-video (Veo and Omni), audio generation, resolution/aspect-ratio/duration handling with provider-limit clamping, negative prompt / seed / person-generation passthrough, GCS output download, and RAI safety-filter reasons surfaced as structured errors instead of silent empty results.

Related Issue

N/A — no existing issue. Gap: image_gen/video_gen had no Vertex backend despite chat-side Vertex support.

Type of Change

• ✨ New feature (non-breaking change that adds functionality)

Changes Made

• plugins/image_gen/vertex/__init__.py + plugin.yaml — VertexImageGenProvider: prefix routing (gemini-* → generateContent, imagen-* → predict) with image_gen.vertex.api escape hatch for future prefixes; edit requests on predict-only models auto-reroute to a generateContent model
• plugins/video_gen/vertex/__init__.py + plugin.yaml — VertexVideoGenProvider: veo-* → predictLongRunning/fetchPredictOperation polling; gemini-* → synchronous Interactions API (/v1beta1/.../locations/global/interactions); handles both response.videos[] and generatedSamples[] operation shapes; downloads gcsUri outputs with the same bearer token
• tests/plugins/image_gen/test_vertex_provider.py — 17 tests
• tests/plugins/video_gen/test_vertex_plugin.py — 22 tests
• All tests are offline: credentials and HTTP are monkeypatched; they assert full request shape (URL, headers, payload) and response/error parsing, including safety blocks and RAI-filtered results
• cli-config.yaml.example — new commented "Image / Video Generation" section documenting image_gen.provider/model, video_gen.provider/model, and the optional image_gen.vertex.* / video_gen.vertex.* keys
• Omni requests declare generation_config.video_config.task (text/image/reference_to_video) explicitly; uri-delivered video outputs are parsed defensively (download with bearer token, URI fallback) — Vertex currently inlines all outputs and rejects response_format.delivery, so no request-side delivery field is sent

How to Test

1. scripts/run_tests.sh tests/plugins/image_gen/test_vertex_provider.py tests/plugins/video_gen/test_vertex_plugin.py — 39 tests, no network or credentials needed
2. Live: set VERTEX_CREDENTIALS_PATH (service-account JSON) in .env, vertex.project_id in config.yaml, then:

   image_gen: { provider: vertex, model: google/gemini-3.1-flash-image }
   video_gen: { provider: vertex, model: veo-3.1-generate-001 }   # or google/gemini-omni-flash-preview

3. hermes -q "generate an image of a lighthouse" and hermes -q "generate a short video of ocean waves" — verify media lands in the media cache and the tool result includes model/location metadata

Verified end-to-end on a production GCP project (Debian 12 vps, Telegram gateway): Gemini image gen + edit, Veo 3.1 t2v/i2v, and Omni t2v/i2v via the Interactions API.Request fields were validated against the live Vertex endpoint, not just docs — video_config.task is confirmed accepted, response_format.delivery is confirmed rejected (the Gemini API and Vertex schemas have diverged; the plugin only sends fields verified on Vertex).

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (feat(plugins): ...)
• I searched existing PRs — no duplicate
• My PR contains only changes related to this feature
• I've run the test suite and all tests pass
• I've added tests for my changes
• I've tested on my platform: Debian 12 (VPS, gateway mode)

Documentation & Housekeeping

• Docstrings document routing, endpoints, and config keys in both plugins
• I've updated cli-config.yaml.example for the new optional image_gen.vertex.* / video_gen.vertex.* keys
• Architecture unchanged (plugin-only) — AGENTS.md/CONTRIBUTING.md N/A
• Cross-platform: pure requests + stdlib, explicit encoding= on file IO, no POSIX-only calls
• Tool behavior unchanged (backend plugins only) — schemas N/A

Screenshots / Logs

hermes logs

[alt-glitch]

This was generated by AI during triage.

Related: the native Vertex/Gemini provider cluster (#55742 Claude-on-Vertex, #29611 Gemini Express Mode, #59938 Gemini built-in tools) — those cover the chat surface; this PR adds the distinct media-generation (image/video) surface via the same agent/vertex_adapter.py auth. Not a duplicate.

[teknium1]

Thanks for the substantial implementation and for keeping the media integration on the existing backend-provider interfaces. The media-generation scope is distinct from the existing Vertex chat-provider work noted in the triage discussion, but this cannot land as a bundled plugin.

• PR commit c52f5068e6c74aae2c5bff20e65467f0b5f85ce2 adds Google Vertex AI backend directories under plugins/image_gen/vertex/ and plugins/video_gen/vertex/.
• The standing policy in AGENTS.md:126-135 excludes new third-party vendor integrations from this repository's plugins/ tree, irrespective of implementation quality.
• The existing provider interfaces support the intended distribution model: user-installed backend plugins under ~/.hermes/plugins/image_gen/<name>/ and ~/.hermes/plugins/video_gen/<name>/ (agent/image_gen_provider.py:10-12, agent/video_gen_provider.py:10-12).

Please publish this as a standalone plugin repository or pip entry point, where users can install it into ~/.hermes/plugins/; it can then be promoted in #plugins-skills-and-skins. This is an automated hermes-sweeper review.

---

Closed as not-planned per standing maintainer policy (in-tree-provider-integration). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.