Skip to content

feat(vertex): add Google Vertex AI provider for Gemini (OAuth2) - #56363

Merged
teknium1 merged 2 commits into
mainfrom
hermes/hermes-4604390b
Jul 1, 2026
Merged

feat(vertex): add Google Vertex AI provider for Gemini (OAuth2)#56363
teknium1 merged 2 commits into
mainfrom
hermes/hermes-4604390b

Conversation

@teknium1

@teknium1 teknium1 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds Google Vertex AI as a first-class provider for Gemini models via Vertex's OpenAI-compatible endpoint. The missing piece behind the recurring requests (#13484, #12639, #56259) was authentication: Vertex has no static API key — every request needs a short-lived OAuth2 access token (~1h TTL) minted from a service-account JSON or Application Default Credentials (ADC), which is why pasting a token into a custom provider dies mid-session. Hermes now mints and auto-refreshes those tokens.

Salvages and modernizes #8427 by @slawt onto current main: rewired from the legacy PROVIDER_REGISTRY path to the current provider-profile architecture, moved non-secret config out of .env into config.yaml, and added per-turn 401 token refresh the original lacked.

Changes

  • agent/vertex_adapter.py — OAuth2 token minting + refresh-on-expiry (5-min margin), ADC→service-account fallback, global vs regional endpoint URLs. Config precedence: env var > config.yaml > default.
  • plugins/model-providers/vertex/ — provider profile (auth_type="vertex"), reuses Gemini's extra_body.google.thinking_config translation.
  • hermes_cli/runtime_provider.py — vertex short-circuit before the credential pool so a credentials-file path is never mistaken for a static API key; mints a fresh token + computes base_url per resolve.
  • run_agent.py + agent/conversation_loop.py_try_refresh_vertex_client_credentials() re-mints the token and rebuilds the client on a mid-session 401, so a long-lived gateway agent survives token expiry.
  • agent/auxiliary_client.py — vertex auth_type branch for side-LLM tasks.
  • hermes_cli/config.pyvertex.project_id / vertex.region (non-secret, bridged to env); credential path stays in .env (VERTEX_CREDENTIALS_PATH).
  • setup wizard + model picker — dedicated _model_flow_vertex, curated google/gemini-* model list, --provider choices, aliases (google-vertex, vertex-ai, gcp-vertex).
  • pricing / metadata — Vertex prices off the gemini docs snapshot; endpoint host auto-maps to the vertex provider (no context-length probe spam).
  • tools/lazy_deps.py + pyproject.toml [vertex]google-auth, opt-in (not in [all]).
  • docsguides/google-vertex.md + providers page; tests for adapter + runtime resolution.

Validation

Result
tests/agent/test_vertex_adapter.py + tests/hermes_cli/test_vertex_provider.py + test_turn_retry_state.py 27/27 passing
tests/hermes_cli/test_runtime_provider_resolution.py + test_model_normalize.py 222/222 passing (no regressions)
tests/hermes_cli/test_models.py + test_usage_pricing.py + test_model_metadata.py 206/206 passing
E2E (real imports, mocked google-auth) token mint, config→env bridge, alias resolution, regional endpoint, thinking_config, friendly AuthError all pass
ruff clean

Verified against Google's official Vertex OpenAI-compatibility docs (endpoint shape, api_key=credentials.token, 1h token TTL, extra_body.google.thinking_config).

Closes the Gemini-via-Vertex request cluster. Claude-via-Vertex (AnthropicVertex SDK, #55742/#6491/#3569) is a separate family, out of scope here.

Infographic

Google Vertex AI provider infographic

@teknium1
teknium1 requested a review from a team July 1, 2026 11:44
slawt and others added 2 commits July 1, 2026 04:56
Adds Vertex AI as a first-class provider for Gemini models via Vertex's
OpenAI-compatible endpoint. Vertex authenticates with short-lived OAuth2
access tokens (service-account JSON or ADC), not a static API key — the
missing piece behind the recurring requests (#13484, #12639, #56259).

- agent/vertex_adapter.py: OAuth2 token minting + refresh-on-expiry
  (5-min margin), ADC->service-account fallback, global vs regional
  endpoint URLs. Config precedence: env var > config.yaml > default.
- plugins/model-providers/vertex/: provider profile (auth_type=vertex),
  reuses Gemini's extra_body.google.thinking_config translation.
- runtime_provider: vertex short-circuit BEFORE the credential pool so a
  credentials-file path is never mistaken for a static API key; mints a
  fresh token + computes base_url per resolve.
- run_agent + conversation_loop: _try_refresh_vertex_client_credentials()
  re-mints the token and rebuilds the client on a mid-session 401, so a
  long-lived gateway agent survives token expiry (~1h).
- auxiliary_client: vertex auth_type branch for side-LLM tasks.
- config.yaml: vertex.project_id / vertex.region (non-secret, bridged to
  env); credential path stays in .env (VERTEX_CREDENTIALS_PATH).
- setup wizard + model picker: dedicated _model_flow_vertex; curated
  google/gemini-* model list; --provider choices.
- pricing/metadata: Vertex prices off the gemini docs snapshot; endpoint
  host auto-maps to the vertex provider (no probe spam).
- lazy_deps + pyproject [vertex] extra: google-auth, opt-in only.
- docs: guides/google-vertex.md + providers page; tests for adapter +
  runtime resolution.

Salvages and modernizes #8427 by @slawt onto current main: rewired from
the legacy PROVIDER_REGISTRY path to the provider-profile architecture,
moved non-secret config out of .env into config.yaml, and added the
per-turn 401 token-refresh the original lacked.
The provider-parity contract (tests/hermes_cli/test_provider_parity.py)
requires every hermes model provider to be configurable in the desktop
Providers tabs. Vertex authenticates via OAuth2 (service-account JSON /
ADC) and has no api_key_env_vars, so — like bedrock's aws_sdk — it needs
its credential env var tagged to the provider card explicitly. Tag
VERTEX_CREDENTIALS_PATH to the vertex card in _catalog_provider_env_metadata().
@teknium1
teknium1 force-pushed the hermes/hermes-4604390b branch from bc53fa1 to d77353c Compare July 1, 2026 11:57
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard provider/gemini Google Gemini (AI Studio, Cloud Code) area/auth Authentication, OAuth, credential pools sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data P3 Low — cosmetic, nice to have labels Jul 1, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: salvage of #8427 (@slawt, earliest/canonical) onto current main. Competing OPEN native-Vertex-Gemini cluster for #13484: #8427, #16010, #29611 (Express Mode api-key variant), #36253 (prior rewrite). Not a duplicate — a maintainer should pick the canonical implementation from the cluster.

@EgorLu

EgorLu commented Jul 1, 2026

Copy link
Copy Markdown

Great to see this land so quickly! My PR #55742 covers the complementary piece: Claude on Vertex via the AnthropicVertex SDK. Both share the vertex provider name and plugin path, so they'll conflict on a few files. Happy to rebase onto this once it merges.

@teknium1
teknium1 merged commit 3f6c6bd into main Jul 1, 2026
31 checks passed
@teknium1
teknium1 deleted the hermes/hermes-4604390b branch July 1, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have provider/gemini Google Gemini (AI Studio, Cloud Code) sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants