Skip to content

feat(providers): add Google Vertex AI provider for Claude - #3569

Open
eilonc-pillar wants to merge 1 commit into
NousResearch:mainfrom
eilonc-pillar:feat/vertex-ai-provider
Open

feat(providers): add Google Vertex AI provider for Claude#3569
eilonc-pillar wants to merge 1 commit into
NousResearch:mainfrom
eilonc-pillar:feat/vertex-ai-provider

Conversation

@eilonc-pillar

@eilonc-pillar eilonc-pillar commented Mar 28, 2026

Copy link
Copy Markdown

Summary

Adds vertex-ai as a first-class provider, enabling Claude models via Google Cloud Vertex AI with Application Default Credentials (ADC).

  • Uses AnthropicVertex from anthropic[vertex] SDK
  • Reuses the existing Anthropic Messages API adapter — zero new adapters needed
  • Supports main agent, auxiliary tasks, compression, delegation, and credential pool routing

Configuration

provider: vertex-ai
model: claude-sonnet-4-6
# Environment
export VERTEX_PROJECT=my-gcp-project   # required
export VERTEX_REGION=us-east5           # optional, default: us-east5

# Auth (any of these):
# 1. Attached service account on GCP VMs (ADC) — zero config
# 2. GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
# 3. gcloud auth application-default login (local dev)

Changes

File Lines What
agent/anthropic_adapter.py +52 build_vertex_client() + resolve_vertex_credentials()
agent/auxiliary_client.py +34 _try_vertex(), auto-resolver guard, resolve_provider_client routing, aux model entry
hermes_cli/auth.py +7 Provider registry entry
hermes_cli/runtime_provider.py +19 Vertex-ai resolution + credential pool api_mode fix
run_agent.py +18 Provider detection + client init
pyproject.toml +2 vertex optional extra

What changed since v1

  • Added hermes_cli/runtime_provider.py — vertex-ai resolution was missing, causing the credential pool to fall back to chat_completions and create an OpenAI client instead of AnthropicVertex
  • Added credential pool api_mode fix in _resolve_runtime_from_pool_entry()
  • Rebased onto current main (resolved auxiliary_client.py conflict from upstream's _resolve_forced_provider removal)

Tested

Locally and on GCP with a Vertex AI service account using ADC.

Install

pip install 'hermes-agent[vertex]'

@eilonc-pillar

Copy link
Copy Markdown
Author

Hi all - @leon @intertwine @timknip @rewbs

Very good job on the Hermes agent! It delivers a lot of value and innovation.
I've opened this PR to enable powering hermes agent with Google's vertex AI juice (both ACP and SA json). Would appreciate if you consider to add this.

Best,
Eilon, Pillar Security

@DeadMeme5441

Copy link
Copy Markdown

Would like this feature to be merged, would work great.

@eilonc-pillar
eilonc-pillar force-pushed the feat/vertex-ai-provider branch from 7b735b2 to 20ec877 Compare April 11, 2026 17:37
Add vertex-ai as a first-class provider using AnthropicVertex SDK
with GCP Application Default Credentials. Supports main agent,
auxiliary, compression, delegation, and credential pool routing.

- anthropic_adapter: build_vertex_client() and resolve_vertex_credentials()
- auxiliary_client: _try_vertex(), forced provider, and resolve_provider_client
- runtime_provider: vertex-ai resolution + credential pool api_mode fix
- auth: vertex-ai ProviderConfig entry
- run_agent: vertex-ai client initialization and api_mode detection
- pyproject.toml: vertex optional extra (anthropic[vertex])

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@eilonc-pillar
eilonc-pillar force-pushed the feat/vertex-ai-provider branch from 20ec877 to 4c49036 Compare April 11, 2026 17:40
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 2, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the Claude-on-Vertex contribution. The requested behavior is not present on current main, but this April implementation predates the current Vertex provider and needs a targeted salvage rather than a direct application.

Problems

  • Current main already owns vertex and maps vertex-ai to it; its runtime path is Gemini/OpenAI-compatible chat_completions at hermes_cli/runtime_provider.py:1573-1604. The PR's separate vertex-ai Anthropic path must coexist with that established Gemini path.
  • The PR's VERTEX_PROJECT / VERTEX_REGION configuration bypasses current vertex: config and profile-aware secret handling (agent/vertex_adapter.py:50-103, hermes_cli/config.py:3287-3303).
  • The diff adds no tests while changing runtime selection, initialization, auxiliary routing, and credential handling. Existing Vertex coverage is in tests/agent/test_vertex_adapter.py and tests/hermes_cli/test_vertex_provider.py.

Suggested changes

  • Implement Claude model detection as a second transport path of the existing vertex provider, preserving Gemini support.
  • Reuse the current config and profile-safe credential-resolution mechanisms.
  • Add coverage for Claude routing/client construction, auxiliary use, and the existing Gemini regression path.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
EgorLu added a commit to EgorLu/hermes-agent that referenced this pull request Jul 29, 2026
Add native support for running Claude models on Google Vertex AI using
Anthropic's AnthropicVertex SDK adapter. The provider is named 'vertex'
(following the Bedrock pattern) with model-based routing: Claude models
use the AnthropicVertex SDK for full feature parity (prompt caching,
thinking budgets), while non-Claude models raise a clear error noting
that Gemini/other model support is planned.

Configuration uses Google Application Default Credentials (ADC):

    export ANTHROPIC_VERTEX_PROJECT_ID=your-gcp-project
    export CLOUD_ML_REGION=global  # optional, defaults to 'global'
    gcloud auth application-default login

Provider registration:
- plugins/model-providers/vertex/ — ProviderProfile with gcp_adc auth
- hermes_cli/auth.py — ProviderConfig, auth status, credential resolver
- hermes_cli/runtime_provider.py — model-based routing with
  _is_anthropic_vertex_model() (mirrors is_anthropic_bedrock_model)
- hermes_cli/models.py — ProviderEntry, aliases, static model catalog
- hermes_cli/main.py + model_setup_flows.py — 'hermes model' wizard

Runtime wiring:
- agent/anthropic_adapter.py — build_anthropic_vertex_client()
- agent/agent_init.py — Vertex client construction branch
- run_agent.py — _rebuild_anthropic_client Vertex dispatch
- agent/agent_runtime_helpers.py — runtime recovery + switch_model
- agent/chat_completion_helpers.py — fallback activation
- agent/auxiliary_client.py — vision backend + auxiliary resolution

Dependencies:
- pyproject.toml — anthropic-vertex optional extra
- tools/lazy_deps.py — lazy install entry

Tests: 7 new tests covering adapter, runtime resolution, client init.
Docs: vertex-ai.md guide + providers table entry.

Aliases: google-vertex, gcp-vertex, vertex-ai, vertex-anthropic.

Closes NousResearch#13484, closes NousResearch#12639.
Related: NousResearch#3569, NousResearch#6491 (prior attempts, smaller scope).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants