Skip to content

feat(vertex): add Google Cloud Vertex provider - #27356

Open
r-bit-rry wants to merge 4 commits into
NousResearch:mainfrom
r-bit-rry:feat/vertex-ai-provider
Open

feat(vertex): add Google Cloud Vertex provider#27356
r-bit-rry wants to merge 4 commits into
NousResearch:mainfrom
r-bit-rry:feat/vertex-ai-provider

Conversation

@r-bit-rry

@r-bit-rry r-bit-rry commented May 17, 2026

Copy link
Copy Markdown

Summary

  • Add Vertex as a native provider using the SDK's Vertex adapter with ADC (Application Default Credentials) auth
  • Mirrors the existing Bedrock provider pattern — same feature parity: prompt caching, thinking budgets, adaptive thinking, 1M context
  • Auto-fallback: when provider: anthropic is configured but no API key is found, detects VERTEX_PROJECT_ID env var and routes through Vertex
  • Explicit provider selection via provider: vertex / vertex-ai / google-vertex

Configuration

Environment variables (priority order):

  • VERTEX_PROJECT_ID or ANTHROPIC_VERTEX_PROJECT_ID or GOOGLE_CLOUD_PROJECT — GCP project
  • VERTEX_REGION or CLOUD_ML_REGION — region (default: global)

Auth uses the standard GCP ADC chain (gcloud login, service account, workload identity).

Files Changed

  • agent/anthropic_adapter.pybuild_anthropic_vertex_client() factory (mirrors build_anthropic_bedrock_client)
  • hermes_cli/runtime_provider.py — Vertex provider resolution + auto-fallback
  • agent/agent_init.py — Vertex client initialization alongside Bedrock
  • run_agent.py — Vertex client rebuild in _rebuild_anthropic_client

Test Plan

  • python -m py_compile agent/anthropic_adapter.py agent/agent_init.py hermes_cli/runtime_provider.py run_agent.py
  • Verify provider resolution with VERTEX_PROJECT_ID set
  • End-to-end: configure provider: vertex with valid GCP credentials, verify streaming works

r-bit-rry added 2 commits May 17, 2026 11:30
Mirrors the existing Bedrock provider pattern — uses AnthropicVertex SDK
with ADC auth, full feature parity (prompt caching, thinking, 1M context).
@cardtest15-coder

This comment was marked as spam.

@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard provider/anthropic Anthropic native Messages API P3 Low — cosmetic, nice to have labels May 17, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #3569 (and also #6491) — all three PRs add Google Vertex AI as a native provider for Anthropic/Claude models. #3569 was opened first and addresses the same feature request #13484.

@r-bit-rry r-bit-rry changed the title feat(vertex): add Google Cloud Vertex AI provider for Anthropic models feat(vertex): add Google Cloud Vertex provider May 17, 2026
@teknium1

teknium1 commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Thanks for adding Vertex support. I found a couple of integration blockers against current main.

Problems

  • Explicit provider: vertex / vertex-ai / google-vertex will not resolve as written. The PR adds handling in hermes_cli/runtime_provider.py after resolve_provider(), but current hermes_cli/auth.py:1552-1562 rejects unknown non-auto providers, and the alias table at hermes_cli/auth.py:1501-1534 does not include Vertex.
  • The PR’s project_id / region runtime fields are not propagated into AIAgent. Current CLI runtime application copies selected fields at hermes_cli/cli_agent_setup_mixin.py:78-144, and AIAgent.__init__ has no project_id or region parameters at run_agent.py:343-414.

Suggested changes

  • Wire Vertex into the canonical provider registry/alias path, or short-circuit it before resolve_provider() rejects it.
  • Propagate project/region through every agent construction path, or make agent_init resolve the same project/region chain directly.
  • Add tests for explicit Vertex provider aliases and runtime-to-agent propagation.

This is an automated hermes-sweeper review.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the Claude-on-Vertex contribution. The requested native Anthropic transport is not on current main, but this implementation predates—and conflicts with—the established Gemini Vertex provider.

Problems

  • Current vertex is already Gemini/OpenAI-compatible: plugins/model-providers/vertex/__init__.py:3-17,65-73 and hermes_cli/runtime_provider.py:1579-1610 map vertex-ai/google-vertex to chat_completions. This patch replaces that behavior with anthropic_messages (hermes_cli/runtime_provider.py:1394-1419) rather than preserving both transports.
  • The resolver returns project_id/region (hermes_cli/runtime_provider.py:839-855), but agent/agent_init.py:542-543 reads agent attributes that current runtime application does not propagate (hermes_cli/cli_agent_setup_mixin.py:78-128). GOOGLE_CLOUD_PROJECT can therefore resolve successfully and still reach AnthropicVertex as an empty project.
  • Main’s profile-aware Vertex configuration lives in agent/vertex_adapter.py:50-87; the patch instead uses raw environment reads. It also adds no tests or AnthropicVertex dependency integration.
  • Auxiliary Vertex routing is still OpenAI/Gemini-only at agent/auxiliary_client.py:5024-5058.

Suggested changes

  • Add a model-aware Claude path alongside the existing Gemini Vertex transport, reusing current Vertex config/credential resolution.
  • Cover primary, rebuild, auxiliary, alias, configuration-precedence, and Gemini-regression paths.

This is an automated hermes-sweeper review.

Comment thread agent/agent_init.py Outdated
print(f"🤖 AI Agent initialized with model: {agent.model} (AWS Bedrock + AnthropicBedrock SDK, {_br_region})")
elif _is_vertex_anthropic:
from agent.anthropic_adapter import build_anthropic_vertex_client
_vertex_project = getattr(agent, "project_id", None) or os.environ.get("VERTEX_PROJECT_ID") or os.environ.get("ANTHROPIC_VERTEX_PROJECT_ID") or ""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runtime_provider returns project_id, but no current agent-construction path assigns it to agent.project_id; this also omits the resolver's GOOGLE_CLOUD_PROJECT fallback. Resolve the project through the shared Vertex configuration/credential helper, or explicitly propagate it through every construction path before invoking AnthropicVertex.

Comment thread hermes_cli/runtime_provider.py Outdated
return runtime

# Google Cloud Vertex AI (AnthropicVertex SDK via ADC)
if provider in {"vertex", "vertex-ai", "google-vertex"}:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current main already owns these aliases for the Gemini OpenAI-compatible Vertex transport. Routing all vertex selections through anthropic_messages would break that existing provider; select the AnthropicVertex path only for supported Claude models while retaining the current Gemini path.

@alt-glitch alt-glitch added the duplicate This issue or pull request already exists label Jul 13, 2026
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Address review feedback from teknium1:
- Model-aware routing: Claude models use AnthropicVertex SDK
  (anthropic_messages), Gemini/other models use existing
  OpenAI-compatible path (chat_completions)
- Use vertex_adapter.py for credential/config resolution
  (profile-safe) instead of raw os.environ reads
- Add lazy_deps group provider.vertex.anthropic for on-demand install
- Cover auxiliary client (vision, reflection, etc.) for Claude-on-Vertex
- Preserve existing Gemini-on-Vertex behavior unchanged
@alt-glitch alt-glitch added area/auth Authentication, OAuth, credential pools needs-decision Awaiting maintainer decision before any implementation and removed duplicate This issue or pull request already exists labels Aug 2, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Correction: current head uses the existing Vertex provider's dual-route/runtime and auxiliary-client integration, whereas #3569 registers a separate vertex-ai provider. These are competing mechanisms, so this is related rather than a duplicate; maintainer selection is needed.

@r-bit-rry

Copy link
Copy Markdown
Author

@teknium1 Addressed your review feedback. Merged upstream main and pushed a fix commit on top:

Changes in the fix commit:

  1. Model-aware dual routing in runtime_provider.py -- removed the standalone vertex block that forced anthropic_messages for all vertex users. Instead, the existing Gemini vertex block now checks the model name: Claude models route through AnthropicVertex SDK (anthropic_messages), everything else keeps the existing OpenAI-compatible path (chat_completions). No regression for Gemini-on-Vertex users.

  2. vertex_adapter.py for credentials -- replaced raw os.environ.get() calls with _resolve_project_override() and _resolve_region() from vertex_adapter.py. This respects config.yaml, secret scoping, and the established precedence chain.

  3. Model-aware check in agent_init.py -- changed agent.provider == "vertex" to also require "claude" in model so Gemini users are not routed through AnthropicVertex.

  4. Auxiliary client support -- added Claude model detection in the credential refresh handler and provider client resolver in auxiliary_client.py.

  5. Lazy deps -- added provider.vertex.anthropic group in lazy_deps.py with anthropic[vertex], and a lazy ensure call in build_anthropic_vertex_client().

  6. run_agent.py rebuild path -- same model-aware fix with vertex_adapter credential resolution.

Ready for re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard needs-decision Awaiting maintainer decision before any implementation P3 Low — cosmetic, nice to have provider/anthropic Anthropic native Messages API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants