Skip to content

feat(providers): add Google Vertex AI as a first-class inference provider - #16010

Closed
Zephyr596 wants to merge 4 commits into
NousResearch:mainfrom
Zephyr596:dev-0425
Closed

feat(providers): add Google Vertex AI as a first-class inference provider#16010
Zephyr596 wants to merge 4 commits into
NousResearch:mainfrom
Zephyr596:dev-0425

Conversation

@Zephyr596

Copy link
Copy Markdown

What does this PR do?

Adds Google Vertex AI as a first-class inference provider for Gemini models, using GCP's OpenAI-compatible endpoint with dynamic OAuth2 token resolution. This is similar in intent to #8427 but addresses several correctness and UX issues found during implementation and testing.

Issues corrected vs #8427:

  • Adds hermes_cli/vertex_adapter.py (not just agent/vertex_adapter.py) so the CLI credential path is fully decoupled from the agent subsystem
  • Adds a dedicated _model_flow_vertex() in main.py — the generic _model_flow_api_key_provider() was misidentifying the service account JSON path as an "API key" and unconditionally prompting for a "Base URL" that is dynamically computed and should not be manually entered
  • Fixes validate_requested_model() to strip the mandatory google/ publisher prefix before catalog lookup, preventing a false "model not found in curated catalog" warning on every session start
  • Model catalog restricted to official stable chat models only (gemini-2.5-pro/flash/flash-lite, gemini-2.0-flash-001/flash-lite-001) — no preview/audio/image/embedding models

Related Issue

See also: #8427, #4375, #6491, #3569

Type of Change

  • ✨ New feature (non-breaking change that adds functionality)
  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

New files:

  • hermes_cli/vertex_adapter.py — credential resolution (SA JSON or ADC), token caching with 5-min refresh window, base URL construction. Falls back from ADC to SA file if ADC refresh fails.

Provider registration:

  • hermes_cli/auth.py — register vertex in PROVIDER_REGISTRY with VERTEX_CREDENTIALS_PATH / GOOGLE_APPLICATION_CREDENTIALS env var priority
  • hermes_cli/providers.py — register vertex in HERMES_OVERLAYS (openai_chat transport); add aliases vertex-ai, google-vertex
  • hermes_cli/models.py — add vertex to _PROVIDER_MODELS and CANONICAL_PROVIDERS; fix validate_requested_model to strip google/ prefix for catalog lookup
  • hermes_cli/model_normalize.py — inject mandatory google/ publisher prefix for Vertex requests transparently
  • hermes_cli/setup.py — add vertex to setup model picker catalog

Runtime:

  • hermes_cli/runtime_provider.py — replace 30-line inline credential block with a 5-line get_vertex_config() call; supports both SA file and ADC; token cached and auto-refreshed
  • hermes_cli/main.py — dedicated _model_flow_vertex() flow: shows credential status without prompting, prompts for region only, suppresses spurious normalization warning for the mandatory google/ prefix
  • run_agent.py — exclude vertex from the generic "API key not found" fail-fast check (vertex uses file-based credentials, not a key env var)

Dependencies:

  • pyproject.toml — lightweight [vertex] optional dependency group (google-auth)

How to Test

  1. Install the optional dependency: pip install hermes-agent[vertex]
  2. Set GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json (or run gcloud auth application-default login)
  3. Run hermes model, select Vertex AI — should show credential path with ✓, prompt for region only, then show model picker
  4. Select a model (e.g. gemini-2.5-pro) and start a session — no "model not found in catalog" warning should appear

Checklist

Code

Documentation & Housekeeping

  • N/A — no new config keys beyond what's described above
  • Cross-platform impact considered (credential path handling uses os.path.exists, works on Windows/macOS/Linux)

Screenshots / Logs

$ hermes model
  Current model:    gemini-2.5-pro
  Active provider:  Vertex AI

  Credentials: ...service-account-key.json ✓

  Region [us-central1]:
Select default model:
-> gemini-2.5-pro  ← currently in use
   gemini-2.5-flash
   gemini-2.5-flash-lite
   gemini-2.0-flash-001
   gemini-2.0-flash-lite-001
   Enter custom model name
   Skip (keep current)

  Default model set to: gemini-2.5-pro (via Google Vertex AI, us-central1)

Zephyr596 and others added 4 commits April 26, 2026 17:44
This commit introduces Google Vertex AI as a first-class inference provider, leveraging its OpenAI-compatible endpoint.

Key implementations:
- Auth: Dynamically generates short-lived Bearer tokens using `google-auth` via `GOOGLE_APPLICATION_CREDENTIALS` (Service Account JSON).
- Model Normalization: Transparently injects the mandatory `google/` publisher prefix for Vertex requests in `model_normalize.py` while maintaining clean, unprefixed model slugs in the TUI and config.
- UX: Suppresses the implicit normalization warning in `main.py` specifically for the Vertex prefix to prevent noisy CLI output.
- Dependencies: Introduces a lightweight `[vertex]` optional dependency group in `pyproject.toml` to avoid bloating the base installation.
- Add hermes_cli/vertex_adapter.py with token caching, ADC support,
  and auto-refresh (mirrors agent/vertex_adapter.py pattern)
- Replace 30-line inline auth block in runtime_provider.py with a
  5-line call to get_vertex_config()
- Add _model_flow_vertex() in main.py — shows creds status, prompts
  for region only (no misleading API key / Base URL prompts)
- Fix validate_requested_model: strip google/ prefix before catalog
  lookup so google/gemini-2.5-pro matches gemini-2.5-pro in catalog
- Update vertex model list to gemini-3.x preview + 2.5 series
- Align env var names: VERTEX_CREDENTIALS_PATH > GOOGLE_APPLICATION_CREDENTIALS,
  VERTEX_REGION for region override

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix _save_model_choice call signature (takes model_id only)
- Save provider="vertex" to config after model selection
- Replace gemini-3.x preview models with official stable catalog:
  gemini-2.5-pro/flash/flash-lite, gemini-2.0-flash-001/flash-lite-001
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/gemini Google Gemini (AI Studio, Cloud Code) labels Apr 26, 2026
slawt added a commit to slawt/hermes-agent that referenced this pull request Apr 28, 2026
Adopts the legitimate UX improvements from the competing PR while
keeping our broader scope (auxiliary client integration, global region
default, _provider_matches_config refactor, Gemini 3.x preview support).

- hermes_cli/main.py: dedicated _model_flow_vertex() — the generic
  api-key flow misidentifies a service-account JSON path as an 'API
  key' and prompts for a Base URL that's dynamically computed from the
  project_id. The new flow shows credential status, prompts only for
  region, then runs the model picker.
- hermes_cli/model_normalize.py: auto-inject the mandatory google/
  publisher prefix so users can configure 'gemini-2.5-pro' and have it
  routed correctly to Vertex's OpenAPI endpoint.
- cli.py: suppress the spurious 'normalized X to google/X' warning
  for the silent prefix injection above.
- hermes_cli/models.py: vertex-ai / google-vertex aliases; strip the
  google/ prefix in validate_requested_model() so the catalog lookup
  doesn't false-warn at session start.
- hermes_cli/models.py setup catalog: add stable 2.0/2.5 chat
  completions models alongside our existing 3.x previews.
- tests/test_model_normalize.py: lock in the prefix-injection
  behaviour and alias resolution.
- pyproject.toml: ship google-auth as an optional [vertex] extra so
  users who don't use Vertex don't pull it in.
@teknium1

teknium1 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Superseded by #56363, which adds Google Vertex AI as a first-class Gemini provider on current main (OAuth2 via service-account JSON / ADC on the OpenAI-compatible endpoint, per-turn token refresh, setup/model/desktop pickers, docs + tests).

That PR salvaged the earliest submission in this cluster (#8427 by @slawt) and preserved its authorship. Closing this as a duplicate — thanks for working on the same gap; the merged implementation covers the Gemini-via-Vertex use case. If you spot something it misses, please open a focused follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have provider/gemini Google Gemini (AI Studio, Cloud Code) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants