Skip to content

feat(vertex): add Google Vertex AI provider for Gemini models - #36253

Closed
Harshavardhan-Yaddalapuri wants to merge 3 commits into
NousResearch:mainfrom
Harshavardhan-Yaddalapuri:feat/vertex-gemini-provider
Closed

feat(vertex): add Google Vertex AI provider for Gemini models#36253
Harshavardhan-Yaddalapuri wants to merge 3 commits into
NousResearch:mainfrom
Harshavardhan-Yaddalapuri:feat/vertex-gemini-provider

Conversation

@Harshavardhan-Yaddalapuri

@Harshavardhan-Yaddalapuri Harshavardhan-Yaddalapuri commented Jun 1, 2026

Copy link
Copy Markdown

What does this PR do?

Adds Google Vertex AI as a first-class inference provider for Gemini models. Users with GCP free credits ($300) or enterprise billing can now use Hermes with Vertex AI instead of being stuck with the Google AI Studio free tier.

Mirrors the existing Bedrock provider pattern — aws_sdkgcp_sdk auth type. Uses the standard OpenAI-compatible chat/completions endpoint (no custom SDK or message translation needed).

Related Issue

Fixes #13484

Related: #8427 (stale PR for same feature, this is a clean rewrite against current main), #4375 (Vertex Express via custom provider workaround)

Type of Change

  • ✨ New feature (non-breaking change that adds functionality)

Changes Made

Production code

  • NEW agent/vertex_adapter.py (189 lines) — credential detection via SA JSON or gcloud ADC, module-scoped lazy google-auth import for testability, OAuth2 token refresh with 55-min caching, base URL construction from project_id+region
  • hermes_cli/auth.py (+25 lines) — ProviderConfig registration with auth_type="gcp_sdk", aliases (vertex-ai, google-vertex, gcp-vertex), provider auto-detection, auth_status handler
  • hermes_cli/runtime_provider.py (+57 lines) — vertex runtime resolution block with proper AuthError messages and config.yaml vertex.region support
  • agent/auxiliary_client.py (+46 lines) — vertex handler in resolve_provider_client using standard OpenAI() client
  • run_agent.py (+4/-1 lines) — vertex added to safe_providers + aiplatform.googleapis.com URL defense-in-depth
  • agent/model_metadata.py (+1 line) — aiplatform.googleapis.com hostname mapping

Tests (45 new tests, all passing)

  • NEW tests/agent/test_vertex_adapter.py (32 tests) — credential path detection, project ID resolution from SA JSON and env vars, region resolution, auth source labeling, token caching/refresh behavior, base URL construction, full chain integration test
  • NEW tests/hermes_cli/test_vertex_provider.py (13 tests) — provider registration, alias resolution (parametrized: vertex-ai, google-vertex, gcp-vertex, google-vertex-ai), auth_status reporting (credentials present, absent, import error fallback)

How to Test

  1. Set up a GCP service account with Vertex AI User + Service Usage Consumer roles
  2. Set GOOGLE_APPLICATION_CREDENTIALS and VERTEX_PROJECT_ID in ~/.hermes/.env
  3. Run hermes model → select Vertex AI → pick a Gemini model
  4. Send a test message — verify it routes through Vertex
  5. Run uv run python -m pytest tests/agent/test_vertex_adapter.py tests/hermes_cli/test_vertex_provider.py -v (45/45 pass)

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run the test suite: 27,281 tests across full scripts/run_tests.sh run — 99 pre-existing failures (kanban workers, ACP adapters, GMI provider OpenAI UnboundLocalError) confirmed present on main before our changes. Zero new failures.
  • I've added tests for my changes: 45 new tests covering vertex_adapter + provider registration
  • I've tested on my platform: macOS 26.5 (Apple Silicon) — live end-to-end with SA JSON auth confirmed working

Documentation & Housekeeping

  • I've updated relevant documentation — provider appears in picker automatically via PROVIDER_REGISTRY, model_metadata maps aiplatform hostname
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A (no new required config keys; vertex.region in config.yaml is optional)
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact — google-auth is pure Python, cross-platform. Lazy imports keep startup fast for non-Vertex users.

Notes

  • Requires pip install google-auth (not added to [all] extras — follows Bedrock pattern of lazy dependency with _import_google_auth())
  • OAuth2 tokens expire after 60 min; cached with 55-min refresh window. Module-scoped _ga_* globals allow tests to mock without importing google-auth
  • Region defaults to global (required for Gemini 3.x previews; us-central1 silently breaks them)
  • This PR does NOT add native Gemini SDK transport — uses OpenAI-compatible endpoint exclusively

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have provider/gemini Google Gemini (AI Studio, Cloud Code) comp/cli CLI entry point, hermes_cli/, setup wizard comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Jun 1, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to existing open Vertex AI provider PRs: #16010 (first-class Vertex), #29611 (Vertex Express Mode), and #8427, all addressing #13484. Maintainers should consolidate on one approach before merging.

@Harshavardhan-Yaddalapuri

Copy link
Copy Markdown
Author

Update: Added UX improvements from #16010:

  • _model_flow_vertex() — dedicated provider flow in the hermes model picker (credential display, region override, curated model selection). No API key prompt — detects SA JSON or ADC automatically.
  • Model catalog — 9 models in _PROVIDER_MODELS["vertex"]: both GA (2.0 flash, 2.5 pro/flash/flash-lite) and previews (3.1 flash-lite, 3.1 pro, 3 pro, 3.5 flash). Users pick what they want — not just what the provider decides.
  • google/ prefix normalization — users can type bare gemini-3.5-flash and it auto-resolves to google/gemini-3.5-flash. Already-prefixed names pass through.
  • ProviderEntry in model picker UI — "Google Vertex AI (Gemini models, GCP Service Account JSON or gcloud ADC)"
  • HermesOverlay registration — proper env var surface for VERTEX_CREDENTIALS_PATH and GOOGLE_APPLICATION_CREDENTIALS

All 193 provider-adjacent tests pass. The two GMI failures are pre-existing on main (UnboundLocalError in resolve_provider_client).

This should address the UX gap vs #16010 while keeping the correct architecture (agent/vertex_adapter.py not hermes_cli/vertex_adapter.py).

@slawt

slawt commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Thanks for putting work into this — more hands on Vertex is genuinely welcome. A couple of clarifications for anyone evaluating, though, because the "clean rewrite" framing isn't quite accurate.

This tracks #8427's design pretty directly: same agent/vertex_adapter.py location, the VERTEX_CREDENTIALS_PATH-takes-precedence scheme, and — most tellingly — the global-region default with the "us-central1 silently breaks Gemini 3.x previews" rationale. That global-region decision is specific to #8427; #16010 defaults to us-central1. And per your own update note, the picker flow and catalog come from #16010. So this reads as #8427's core plus #16010's UX rather than an independent rewrite.

On "against current main" specifically:

Credit where it's due: the _ga_* module-scoped lazy-import seam for test mocking is a nice touch, and I'd be glad to fold that into #8427.

Given @alt-glitch's point about consolidating on one approach — #8427 has been open since April, is in live production use, conforms to the current lazy-install policy, and is merge-ready against current main today. The pragmatic path seems to be converging there rather than restarting the clock.

@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: part of the open native-Vertex provider cluster for #13484#8427 (earliest, in production, lazy-deps-compliant, merge-ready), #16010 (first-class Vertex), #29611 (Vertex Express Mode). Per @slawt this PR tracks #8427's design plus #16010's UX rather than an independent rewrite, and currently asks users to pip install google-auth by hand (vs main's lazy-install policy), drops #8427's ADC→SA fallback, and hardcodes aiplatform.googleapis.com for all regions (regional locations need {region}-aiplatform.googleapis.com). Not a duplicate — these are competing approaches; maintainers should consolidate on one before merging.

Harshvardhan Yaddalapuri added 3 commits June 30, 2026 20:33
Adds Vertex AI as a first-class inference provider with dynamic OAuth2
token resolution from service account JSON or gcloud ADC. Mirrors the
existing Bedrock provider pattern (aws_sdk → gcp_sdk auth_type).

- agent/vertex_adapter.py: credential detection, OAuth2 token refresh
  with 55-min caching, base URL construction from project_id+region
- hermes_cli/auth.py: ProviderConfig registration, aliases
  (vertex-ai, google-vertex, gcp-vertex), auto-detection
- hermes_cli/runtime_provider.py: vertex handler block with proper
  auth error messages and config.yaml vertex.region support
- agent/auxiliary_client.py: vertex handler in resolve_provider_client
  using standard OpenAI client (chat/completions — no custom SDK)
- run_agent.py: vertex in safe_providers + aiplatform URI defense
- agent/model_metadata.py: aiplatform.googleapis.com hostname mapping

Requires: pip install google-auth
Credentials: GOOGLE_APPLICATION_CREDENTIALS or VERTEX_CREDENTIALS_PATH
  plus VERTEX_PROJECT_ID (or project_id from SA JSON).

Model: google/gemini-3.1-flash-lite-preview (and all Gemini variants).
Region defaults to 'global' (required for Gemini 3.x previews).
…tration

- tests/agent/test_vertex_adapter.py (32 tests): credential path
  detection, project ID resolution from SA JSON and env vars,
  region resolution, auth source labeling, token caching/refresh,
  base URL construction, integration test
- tests/hermes_cli/test_vertex_provider.py (13 tests): provider
  registration in PROVIDER_REGISTRY, alias resolution, auth_status
  reporting for credential states
- Refactored agent/vertex_adapter.py: module-scoped lazy google-auth
  import via _ga_auth/_ga_transport/_ga_service_account globals,
  making token refresh logic testable via standard mock.patch

All 45 new tests pass. No regressions in existing suite — confirmed
that GMI provider failures (test_resolve_provider_client_* 2 failures)
are pre-existing on main.
…malization

- hermes_cli/main.py: _model_flow_vertex() — dedicated provider flow with
  credential display, region override, and curated model picker (mirrors
  _model_flow_bedrock pattern, no API key prompt)
- hermes_cli/model_normalize.py: auto-prepend google/ prefix for Vertex
  provider — users can type bare 'gemini-3.5-flash' and it resolves
  correctly
- hermes_cli/models.py: curated Vertex model catalog in _PROVIDER_MODELS
  (9 models from GA 2.0 through preview 3.5), ProviderEntry for model
  picker UI listing
- hermes_cli/providers.py: HermesOverlay registration + aliases
  (vertex-ai, google-vertex)
- hermes_cli/setup.py: model list for setup wizard

Includes both GA models (for production workloads) and preview models
(for users who want latest capabilities via GCP credits).
@Harshavardhan-Yaddalapuri
Harshavardhan-Yaddalapuri force-pushed the feat/vertex-gemini-provider branch from 0e699bf to b5fd475 Compare July 1, 2026 00:51
@alt-glitch alt-glitch added the area/auth Authentication, OAuth, credential pools label Jul 1, 2026
@teknium1

teknium1 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Superseded by #56363, which adds Google Vertex AI as a first-class Gemini provider on current main (OAuth2 via service-account JSON / ADC on the OpenAI-compatible endpoint, per-turn token refresh, setup/model/desktop pickers, docs + tests).

That PR salvaged the earliest submission in this cluster (#8427 by @slawt) and preserved its authorship. Closing this as a duplicate — thanks for working on the same gap; the merged implementation covers the Gemini-via-Vertex use case. If you spot something it misses, please open a focused follow-up.

@teknium1 teknium1 closed this Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have provider/gemini Google Gemini (AI Studio, Cloud Code) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: native Google Cloud Vertex AI provider support

4 participants