Skip to content

feat: add Google Vertex AI as a first-class Claude provider - #6491

Open
elbertwang wants to merge 1 commit into
NousResearch:mainfrom
elbertwang:feat/vertex-ai-provider
Open

feat: add Google Vertex AI as a first-class Claude provider#6491
elbertwang wants to merge 1 commit into
NousResearch:mainfrom
elbertwang:feat/vertex-ai-provider

Conversation

@elbertwang

Copy link
Copy Markdown

Add support for running Claude models on Google Vertex AI, using the Anthropic Python SDK's AnthropicVertex client which shares the same .messages.create() / .messages.stream() interface as the regular Anthropic client — making it a near drop-in replacement.

Configuration is fully compatible with Claude Code's Vertex AI setup:

export CLAUDE_CODE_USE_VERTEX=1
export ANTHROPIC_VERTEX_PROJECT_ID=<project-id>
export CLOUD_ML_REGION=us-east5   # or "global"

Key changes:

  • agent/anthropic_adapter.py: new build_vertex_client() function
  • hermes_cli/auth.py: register vertex provider with gcloud_adc auth type, aliases, and auto-detection via CLAUDE_CODE_USE_VERTEX
  • hermes_cli/providers.py: HermesOverlay, aliases, display label
  • hermes_cli/runtime_provider.py: vertex resolution (skips credential pooling since Vertex uses Google ADC)
  • hermes_cli/main.py: add Vertex AI to the top-level provider menu in hermes model / hermes setup with interactive project/region configuration
  • run_agent.py: vertex-aware client construction in AIAgent init, switch_model, error recovery, and fallback paths via new _build_anthropic_client_for_provider() helper
  • cli.py: skip api_key/base_url validation for vertex provider; pass project_id/region through to AIAgent
  • gateway/run.py: forward project_id/region in runtime kwargs
  • agent/auxiliary_client.py: _try_vertex() for sub-tasks
  • agent/model_metadata.py: add vertex to provider prefixes
  • agent/models_dev.py: map vertex → anthropic in models.dev registry
  • pyproject.toml: anthropicanthropic[vertex] (adds google-auth)

Tests: 16 new tests (6 client + 10 provider), 192 total passing.

What does this PR do?

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

How to Test

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform:

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

  • This skill is broadly useful to most users (if bundled) — see Contributing Guide
  • SKILL.md follows the standard format (frontmatter, trigger conditions, steps, pitfalls)
  • No external dependencies that aren't already available (prefer stdlib, curl, existing Hermes tools)
  • I've tested the skill end-to-end: hermes --toolsets skills -q "Use the X skill to do Y"

Screenshots / Logs

Add support for running Claude models on Google Vertex AI, using the
Anthropic Python SDK's `AnthropicVertex` client which shares the same
`.messages.create()` / `.messages.stream()` interface as the regular
`Anthropic` client — making it a near drop-in replacement.

Configuration is fully compatible with Claude Code's Vertex AI setup:

    export CLAUDE_CODE_USE_VERTEX=1
    export ANTHROPIC_VERTEX_PROJECT_ID=<project-id>
    export CLOUD_ML_REGION=us-east5   # or "global"

Key changes:
- `agent/anthropic_adapter.py`: new `build_vertex_client()` function
- `hermes_cli/auth.py`: register `vertex` provider with `gcloud_adc`
  auth type, aliases, and auto-detection via CLAUDE_CODE_USE_VERTEX
- `hermes_cli/providers.py`: HermesOverlay, aliases, display label
- `hermes_cli/runtime_provider.py`: vertex resolution (skips credential
  pooling since Vertex uses Google ADC)
- `hermes_cli/main.py`: add Vertex AI to the top-level provider menu
  in `hermes model` / `hermes setup` with interactive project/region
  configuration
- `run_agent.py`: vertex-aware client construction in AIAgent init,
  switch_model, error recovery, and fallback paths via new
  `_build_anthropic_client_for_provider()` helper
- `cli.py`: skip api_key/base_url validation for vertex provider;
  pass project_id/region through to AIAgent
- `gateway/run.py`: forward project_id/region in runtime kwargs
- `agent/auxiliary_client.py`: `_try_vertex()` for sub-tasks
- `agent/model_metadata.py`: add vertex to provider prefixes
- `agent/models_dev.py`: map vertex → anthropic in models.dev registry
- `pyproject.toml`: `anthropic` → `anthropic[vertex]` (adds google-auth)

Tests: 16 new tests (6 client + 10 provider), 192 total passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have provider/anthropic Anthropic native Messages API comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard labels Apr 30, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #3569 — both add Google Vertex AI as a Claude provider. Also related to #16010 (Vertex AI for Gemini) and #13484 (feature request).

@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #3569

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the Claude-on-Vertex implementation. The requested Claude behavior remains absent on current main, but this April branch predates the established Vertex/Gemini provider and needs a targeted salvage.

Problems

  • Current main already maps vertex and vertex-ai to Gemini's OpenAI-compatible chat_completions route at hermes_cli/runtime_provider.py:1573-1604. This PR instead unconditionally routes vertex to anthropic_messages at hermes_cli/runtime_provider.py:754-772, so it would replace Gemini rather than add a Claude path.
  • The Vertex fallback block at run_agent.py:4957 cannot run: the preceding mode selection only marks Anthropic or /anthropic endpoints as anthropic_messages (run_agent.py:4938-4955).
  • hermes_cli/main.py:2655-2660 writes project, region, and an enable flag to .env; current main places non-secret Vertex routing settings in vertex: config (hermes_cli/config.py:3287-3303).

Suggested changes

  • Add model-based dual routing within the existing vertex provider, preserving the Gemini route and choosing AnthropicVertex for Claude IDs.
  • Reuse the current config/profile-safe credential path and add Gemini-regression plus Vertex-fallback coverage.

This is an automated hermes-sweeper review.

}

# Google Vertex AI (Claude via Anthropic Messages API)
if provider == "vertex":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current main already uses vertex/vertex-ai for Gemini through the OAuth2 OpenAI-compatible path. This unconditional route must become model-based so Claude selects AnthropicVertex without replacing the existing Gemini behavior.

Comment thread run_agent.py
self._anthropic_base_url = fb_base_url
self._anthropic_client = build_anthropic_client(effective_key, self._anthropic_base_url)
self._is_anthropic_oauth = _is_oauth_token(effective_key)
if fb_provider == "vertex":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch is unreachable for a Vertex fallback: above, fb_api_mode becomes anthropic_messages only for anthropic or a base URL ending in /anthropic; Vertex falls through as chat_completions and never enters this block.

Comment thread hermes_cli/main.py
region = new_region or default_region

# Save env values
save_env_value("ANTHROPIC_VERTEX_PROJECT_ID", project_id)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Project ID and region are non-secret routing settings. Persist them under the existing vertex: config section rather than .env, matching the repository configuration policy and current Vertex provider.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
EgorLu added a commit to EgorLu/hermes-agent that referenced this pull request Jul 29, 2026
Add native support for running Claude models on Google Vertex AI using
Anthropic's AnthropicVertex SDK adapter. The provider is named 'vertex'
(following the Bedrock pattern) with model-based routing: Claude models
use the AnthropicVertex SDK for full feature parity (prompt caching,
thinking budgets), while non-Claude models raise a clear error noting
that Gemini/other model support is planned.

Configuration uses Google Application Default Credentials (ADC):

    export ANTHROPIC_VERTEX_PROJECT_ID=your-gcp-project
    export CLOUD_ML_REGION=global  # optional, defaults to 'global'
    gcloud auth application-default login

Provider registration:
- plugins/model-providers/vertex/ — ProviderProfile with gcp_adc auth
- hermes_cli/auth.py — ProviderConfig, auth status, credential resolver
- hermes_cli/runtime_provider.py — model-based routing with
  _is_anthropic_vertex_model() (mirrors is_anthropic_bedrock_model)
- hermes_cli/models.py — ProviderEntry, aliases, static model catalog
- hermes_cli/main.py + model_setup_flows.py — 'hermes model' wizard

Runtime wiring:
- agent/anthropic_adapter.py — build_anthropic_vertex_client()
- agent/agent_init.py — Vertex client construction branch
- run_agent.py — _rebuild_anthropic_client Vertex dispatch
- agent/agent_runtime_helpers.py — runtime recovery + switch_model
- agent/chat_completion_helpers.py — fallback activation
- agent/auxiliary_client.py — vision backend + auxiliary resolution

Dependencies:
- pyproject.toml — anthropic-vertex optional extra
- tools/lazy_deps.py — lazy install entry

Tests: 7 new tests covering adapter, runtime resolution, client init.
Docs: vertex-ai.md guide + providers table entry.

Aliases: google-vertex, gcp-vertex, vertex-ai, vertex-anthropic.

Closes NousResearch#13484, closes NousResearch#12639.
Related: NousResearch#3569, NousResearch#6491 (prior attempts, smaller scope).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have provider/anthropic Anthropic native Messages API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants