Feat/anthropic on vertex

[haizaar]

What does this PR do?

Adds first-class Anthropic Claude on Google Vertex AI as a supported runtime path under the existing vertex provider name. Users who already run Gemini on Vertex can flip to Claude with a one-line config change:

model:
  default: anthropic/claude-opus-4-8   # was google/gemini-3.1-pro-preview
  provider: vertex

vertex:
  project_id: my-gcp-project
  region: global

Anthropic and Google document the offering on both vendor sides:

• Anthropic: Claude on Vertex AI docs
• Google: Vertex AI — Use Claude
• Anthropic's 1M context GA announcement explicitly lists Vertex AI alongside native Anthropic and Microsoft Foundry.

Related Issue

• Auxiliary-tooling bug (surfaced while integrating this feature) [Bug]: Every auxiliary task silently fails on provider: vertex #61852. The middle commit here fixes it; the last commit adds Anthropic-specific dispatch on top.

Integrating this feature on a Vertex-only deployment surfaced a pre-existing bug in agent/auxiliary_client.py that silently disabled every auxiliary task (vision, compression, curator, session_search, title_generation, web_extract) on any provider: vertex deployment, regardless of model family. The main-chat surface has its own dispatch path (hermes_cli/runtime_provider.py) that special-cases vertex up front, so Gemini-on-Vertex worked at the main-chat level while auxiliary tasks were quietly failing.

That fix is filed as sibling #61853. The middle commit of this PR (fix(auxiliary_client): route vertex through the auth_type dispatch) is bit-identical to the above's single commit — either PR can land first:

Both PRs stand alone. Merging just this PR is fully self-contained; merging just #61853 fixes Gemini-on-Vertex aux tooling without the Claude feature.

Type of Change

• ✨ New feature (non-breaking change that adds functionality) — commits e978baec3 and 0addf0c8e
• 🐛 Bug fix (non-breaking change that fixes an issue) — commit d211336a5 (also filed as fix(auxiliary_client): route vertex through the auth_type dispatch #61853)

Non-breaking. Routes through the existing vertex provider; users who don't set model.default to an anthropic/-prefixed name see zero behavior change.

Changes Made

Commit 1 — feat(providers): Anthropic on Google Vertex AI (e978baec3)

New adapter module — agent/anthropic_vertex_adapter.py:

• build_anthropic_vertex_client(project_id, region, timeout) constructs an anthropic.AnthropicVertex with a fresh google-auth Credentials object. Beta-header policy mirrors build_anthropic_bedrock_client: the common Anthropic betas are attached (interleaved-thinking-2025-05-14, fine-grained-tool-streaming-2025-05-14); context-1m-2025-08-07 is not attached — see Design decisions Architecture planning #3. SDK-level retries are disabled; Hermes's outer loop handles Retry-After.
• get_anthropic_vertex_config(), has_anthropic_vertex_credentials(), build_anthropic_vertex_base_url() are helpers for resolve_runtime_provider and the doctor / setup-status UI. They reuse agent/vertex_adapter.py's credential-resolution helpers (_resolve_credentials_path, _resolve_project_override, _resolve_region) so both wire paths under vertex share one config surface.
• is_anthropic_vertex_model(model_id) is the runtime dispatch classifier. Matches anthropic/<model> only (case-insensitive, whitespace-tolerant); non-string inputs safely return False. See Design decisions Support passing morph snapshot id #2.

Runtime dispatch — hermes_cli/runtime_provider.py:

Extended the vertex branch of resolve_runtime_provider with a model-name dispatch. When is_anthropic_vertex_model(target_model) is true, we return an anthropic_messages runtime dict (placeholder api_key="vertex-adc" since the SDK mints its own tokens per request; vertex_project_id / vertex_region fields for the client-construction sites). Everything else takes the existing chat_completions path. Raises AuthError with an actionable message pointing at the Vertex Model Garden enablement flow when credentials or the project can't be resolved for a Claude request.

The model config is sourced via _get_model_config() into a locally-scoped variable, deliberately named _vertex_model_cfg rather than the more obvious model_cfg — Python's static scoping makes any bare model_cfg name in this function local for the whole function body (because a later fallback branch assigns to it), which would trip UnboundLocalError on any call path that passes target_model=None (cron scheduler + gateway per-turn agent resolve both do). Regression test in test_vertex_dispatch_when_target_model_is_none.

Client construction and rebuild sites — parallel branches mirroring the existing bedrock branches:

• agent/agent_init.py — new branch inside the anthropic_messages transport-selection block constructs AnthropicVertex when agent.provider == "vertex". Stashes _vertex_project_id / _vertex_region on the agent for rebuild-site consumption.
• run_agent.py::_rebuild_anthropic_client — parallel branch reconstructs AnthropicVertex on interrupt / stale-call recovery.
• agent/agent_runtime_helpers.py (2 rebuild sites plus the switch_model snapshot) — parallel branches so restore / fallback / mid-session model swap all recreate the client from the _primary_runtime snapshot without hitting disk.

Model-name normalization — hermes_cli/model_normalize.py:

New vertex branch strips the anthropic/ vendor prefix on the way to the wire. AnthropicVertex substitutes the request body's model field verbatim into .../publishers/anthropic/models/{model}:rawPredict, so a leading anthropic/ corrupts the URL. Gemini's google/gemini-… naming passes through unchanged — the OpenAI-compat aggregator wants the prefix intact.

Small housekeeping — hermes_cli/models.py, tools/vision_tools.py:

ProviderEntry description on vertex updated to reflect that the provider now hosts Gemini + Claude on the same platform. No new aliases, no new auth-type entries, no picker changes — the existing vertex auth flow covers both wire paths.

Docs — website/docs/:

• guides/anthropic-vertex.md — new guide with quick start, configuration, dispatch table, per-region model-availability caveat, feature parity notes, and common failure modes. Explicitly documents that 1M context is GA on Vertex-hosted Opus 4.6+ / Sonnet 4.6+ with no beta header required.
• integrations/providers.md — Anthropic-on-Vertex entry pointing at the same vertex provider row as Gemini.

Commit 2 — fix(auxiliary_client): route vertex through the auth_type dispatch (d211336a5)

Bit-identical to #61853 single commit. Summary: resolve_provider_client("vertex", …) silently returned (None, None) on any Vertex deployment because PROVIDER_REGISTRY.get("vertex") is None (the auto-extension in hermes_cli/auth.py filters out non-api_key providers), so the existing elif pconfig.auth_type == "vertex": handler below was unreachable dead code.

The fix is a plugin-catalog fallback for pconfig=None: when the registry lookup misses, consult providers.get_provider_profile(name) and synthesize a minimal pconfig so the downstream auth_type dispatch runs. Genuinely unknown providers still bail cleanly.

This commit alone fixes Gemini-on-Vertex aux tooling for every existing user. It's a prerequisite for the Anthropic-on-Vertex aux dispatch in commit 3.

Commit 3 — feat(auxiliary_client): route Anthropic-on-Vertex through the native SDK (0addf0c8e)

Extends the vertex handler (which commit 2 made reachable) with an Anthropic-vs-Gemini split, mirroring bedrock_adapter.is_anthropic_bedrock_model's aws_sdk-branch pattern:

try:
    from agent.anthropic_vertex_adapter import is_anthropic_vertex_model
except ImportError:
    def is_anthropic_vertex_model(_m: str) -> bool:
        return False

_model_lc = (model or "").strip().lower()
_is_anthropic = (
    is_anthropic_vertex_model(model)
    or _model_lc.startswith("claude-")
)

if _is_anthropic:
    # Claude on Vertex → AnthropicVertex SDK (Anthropic Messages wire),
    # wrapped in AnthropicAuxiliaryClient with api_key="vertex-adc" as the
    # sentinel (matching the runtime_provider convention).
    ...
else:
    # Existing Gemini OpenAI-compat aggregator path.
    ...

The lazy try/except ImportError keeps commit 2 fully standalone — on main (or with #61853 merged alone), the classifier defaults to lambda: False and every vertex aux call routes to Gemini. Once this commit is applied, the classifier activates.

Why the aux dispatch widens to bare claude-* too: is_anthropic_vertex_model intentionally requires the anthropic/<model> vendor prefix on the main-agent side — that's the loud-fail contract for config typos (see Design decisions #2). But the auxiliary path sees the model AFTER agent_init.py::normalize_model_for_provider(model, "vertex") has stripped the prefix and set_runtime_main has recorded the bare form. _read_main_model() returns bare claude-opus-4-8. If the aux dispatch also required the prefix, every Vertex-Anthropic aux call would silently misroute to the Gemini path and 400 with "Malformed publisher model". The widening mirrors is_anthropic_bedrock_model's dual-form acceptance in the one place where it's actually necessary — the strict main-agent classifier is untouched.

Test file extension — tests/agent/test_auxiliary_client_vertex_dispatch.py (adds 11 cases on top of the 10 from commit 2):

• TestVertexAnthropicDispatch (9) — anthropic/-prefixed model routes to AnthropicAuxiliaryClient wrapping AnthropicVertex; model normalized to bare form on the wire; placeholder api_key="vertex-adc"; base_url shape matches the Vertex publisher endpoint (for billing attribution); missing-creds / missing-project / missing-SDK bail paths; uppercase prefix; bare claude-* widening (case-insensitive); async wrapper.

Total for the vertex dispatch test file: 21 cases, all hermetic (mock the credential seams and SDK factory only).

How to Test

Focused test files:

scripts/run_tests.sh tests/agent/test_anthropic_vertex_adapter.py \
                     tests/hermes_cli/test_anthropic_vertex_provider.py \
                     tests/hermes_cli/test_model_normalize.py \
                     tests/agent/test_auxiliary_client_vertex_dispatch.py \
                     -q
# ─── All tests pass.

Broader regression sweep across sibling test files (provider dispatch, runtime resolution, anthropic_messages transport, primary-runtime restore, model switch, vision routing, provider profiles, bedrock parity):

scripts/run_tests.sh tests/hermes_cli/test_anthropic_vertex_provider.py \
                     tests/agent/test_anthropic_vertex_adapter.py \
                     tests/agent/test_auxiliary_client_vertex_dispatch.py \
                     tests/agent/test_auxiliary_client.py \
                     tests/agent/test_auxiliary_main_first.py \
                     tests/agent/test_vertex_adapter.py \
                     tests/agent/test_auxiliary_client_azure_foundry.py \
                     tests/agent/test_auxiliary_client_anthropic_custom.py \
                     tests/agent/test_auxiliary_client_resolve_dedup.py \
                     tests/agent/test_bedrock_adapter.py \
                     tests/agent/test_bedrock_integration.py \
                     tests/hermes_cli/test_model_normalize.py \
                     tests/hermes_cli/test_models.py \
                     tests/hermes_cli/test_gemini_provider.py \
                     tests/hermes_cli/test_setup.py \
                     tests/hermes_cli/test_cli_provider_resolution.py \
                     tests/hermes_cli/test_status_model_provider.py \
                     tests/providers/test_provider_profiles.py \
                     tests/providers/test_profile_wiring.py \
                     tests/tools/test_vision_tools.py \
                     tests/tools/test_vision_native_fast_path.py \
                     tests/agent/test_vision_routing_31179.py \
                     tests/agent/test_context_compressor.py \
                     tests/tools/test_session_search.py \
                     tests/agent/test_auxiliary_config_bridge.py \
                     tests/run_agent/test_provider_fallback.py \
                     tests/run_agent/test_primary_runtime_restore.py \
                     tests/run_agent/test_streaming.py \
                     tests/agent/transports/ \
                     tests/gateway/test_runtime_footer.py \
                     -q
# ─── 37 files, 1776/1776 tests pass in ~25s.

Real-world validation. Deployed on a Vertex-only GCE fleet (NixOS, ADC via VM service-account) with model.default: anthropic/claude-opus-4-8. hermes -z "what LLM are you?" confirms claude-opus-4-8 served via the Vertex AI provider. The main-chat loop handles live tool-use turns; prompt caching and interleaved thinking on Opus verified via session traces. vision_analyze, title_generation, compression, session_search, curator, and web_extract all round-trip through the Anthropic path in <10s each.

Large-context sanity check. A 40-turn multi-turn accumulator test drove a peak single-call payload of ~675K tokens through hermes → AnthropicVertex → Vertex → claude-opus-4-8. Latency stayed flat at ~27s per turn across all 40 turns (prompt caching dominant); the model correctly recalled sentinel tokens from both the first and last chunks in its final answer. That's 3.4× over the 200K pre-GA cap and matches Anthropic's public GA statement that the beta header is a no-op on Vertex.

Manual reproduction on a fresh checkout:

# One-time GCP-side setup:
#   1. gcloud auth application-default login
#      (or export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa.json)
#   2. Enable at least one Claude SKU (Opus 4.8 / Sonnet 4.5 / Haiku 4.5)
#      in Vertex Model Garden for your GCP project — one-time TOS-accept
#      click, starts the Marketplace subscription.

cat >> ~/.hermes/config.yaml <<'YAML'
model:
  default: anthropic/claude-opus-4-8
  provider: vertex

vertex:
  project_id: your-gcp-project-id
  region: global
YAML

hermes -z "One sentence: what model are you and what provider does your runtime report?"
# → I'm the claude-opus-4-8 model, served via the Vertex AI provider.

Checklist

Code

• I've read the Contributing Guide.
• My commit messages follow Conventional Commits (feat(providers): …, fix(auxiliary_client): …).
• I searched existing PRs to make sure this isn't a duplicate.
• My PR contains only changes related to this feature + the aux-dispatch fix it depends on. Three focused commits, no unrelated changes.
• I've run scripts/run_tests.sh on the affected files (all pass — 1776/1776 across the broader sweep).
• I've added tests — new adapter, runtime-dispatch, normalization, and aux-dispatch coverage. Including regression guards for each of the three design decisions above and a TestHistoricalRegression that pins the aux-dispatch invariant against future refactors of hermes_cli/auth.py.
• I've tested on my platform: NixOS 25.05 (GCE VMs, real Vertex backend), plus local dev on macOS 15 against the same Vertex project.

Documentation & Housekeeping

• Documentation updated: new guide at website/docs/guides/anthropic-vertex.md; new entry in website/docs/integrations/providers.md.
• cli-config.yaml.example — N/A. No new config keys; model.default / model.provider / vertex.* all already exist.
• CONTRIBUTING.md / AGENTS.md — N/A. No architecture or workflow changes.
• Cross-platform impact considered — no OS-specific primitives; google-auth + anthropic SDKs are cross-platform.
• Tool descriptions / schemas — N/A. This is a model-provider path, not a tool.

Follow-ups (out of scope for this PR)

Filing here for reviewer visibility, not asking to bundle in:

1. Setup-wizard's curated Vertex model list. hermes_cli/setup.py::_VERTEX_MODELS currently lists only google/gemini-* SKUs. Extending it to also surface anthropic/claude-opus-4-8 / -sonnet-4-5 / -haiku-4-5 in the picker would make the wizard aware of both families. Would benefit from a small UI hint (which SKUs are Anthropic vs. Google) so users understand the Model-Garden-enablement gate applies to Anthropic entries.
2. Compression cost. With the aux dispatch fix (commit 2 + 3), compression on Vertex-Anthropic deployments now routes to the main Claude model by default (via _resolve_auto Step 1). That's the intended behaviour, but Opus is expensive for what's essentially summarisation. Deployments where compression fires frequently may want to pin auxiliary.compression.provider: vertex, auxiliary.compression.model: google/gemini-3.5-flash explicitly. Not this PR's concern — it's a config-time choice for operators.

Screenshots / Logs

Startup banner on the running agent:

🤖 AI Agent initialized with model: anthropic/claude-opus-4-8 (Anthropic on Vertex AI, <project>/<region>)

Turn round-trip against Opus 4.8 via Vertex:

$ hermes -z "One sentence: what model are you?"
I'm the claude-opus-4-8 model, served via the Vertex AI provider.

[teknium1]

Thanks for the focused Vertex integration and for identifying the auxiliary resolver gap.

Problems

• Blocking: agent/agent_runtime_helpers.py:1891-1927 still constructs every anthropic_messages switch with build_anthropic_client(...). The added Vertex snapshot block in this PR is later in the same function (agent/agent_runtime_helpers.py, new diff line 2077) and does not change that construction. A running session switched to Vertex Claude would therefore not create AnthropicVertex. Please add the parallel Vertex branch here and test a Gemini-on-Vertex → Claude-on-Vertex switch plus restore.
• website/docs/guides/anthropic-vertex.md:13 says bare claude-* routes through AnthropicVertex, contradicting the intended strict-prefix behavior documented at lines 76 and 90. The implementation classifier also only accepts anthropic/.

Suggested changes

• Build AnthropicVertex in the switch_model() Anthropic Messages branch using the resolved Vertex project/region, then snapshot the same fields for restore/recovery.
• Correct the guide’s introductory dispatch statement.

Automated hermes-sweeper review.

[haizaar]

@teknium1 Thanks for the review! - all should be fixed now. Can you please have another look?

[ch-muhammad-asim]

While this PR is still pending, I’ve implemented and tested a working alternative that runs Hermes with Claude on Vertex AI through a dedicated compatibility bridge.

It supports tool calls, prompt caching, retries, Workload Identity, and production-style GKE deployment without requiring changes to Hermes core:

https://github.com/ch-muhammad-asim/hermes-claude-code-bridge/tree/main/vertex-ai

[haizaar]

@teknium1 Did you have a chance to review my updates?

[haizaar]

@teknium1 pinging your again too see how we can advance this PR.

[haizaar]

Rebased onto current main (was conflicting). Two things had to change beyond conflict resolution, and one of them was a real hole in this PR — details below so you don't have to take the rebase on faith.

The rebase conflict

One conflict, in agent/agent_runtime_helpers.py::restore_primary_runtime: main added the moa virtual-provider branch (#53802) ahead of the anthropic_messages branch this PR rewrites. Kept main's moa branch intact and made ours the elif; the build_anthropic_client import moved into the non-Vertex sub-branch so the Vertex path no longer pulls in the direct-Anthropic adapter.

Added: the request-local client was still undispatched

main has since gained AIAgent._create_request_anthropic_client (#67142) — the per-request client that now carries every in-flight anthropic_messages call. It dispatches Bedrock but not Vertex, and this PR did not cover it, so as it stood, merging this would still have 404'd every conversation turn against …/publishers/anthropic/v1/messages. Added the dispatch plus a regression test.

That gap is invisible to a clean rebase, so I verified it mechanically rather than by reading. Classifying every build_anthropic_* construction site by whether its enclosing function dispatches on provider:

	total sites	flagged
this branch, before	27	8
this branch, now	28	7

The removed flag is exactly run_agent.py::_create_request_anthropic_client. The remaining flags are the pre-existing undispatched sites on main — chat_completion_helpers.try_activate_fallback (bedrock-only) and run_agent._swap_credential (no dispatch) — which this PR deliberately leaves alone: both are only reachable with a credential pool or a fallback provider chain configured, and fixing them is a separate change with its own reachability argument. Happy to fold them in here if you'd rather.

Added: the empty-base_url guard blocks Claude on Vertex

main also gained a guard in switch_model (#47828) that raises when a provider change resolves an empty base_url. The premise — "empty means resolution failed upstream" — doesn't hold for the Anthropic cloud partner SDKs: AnthropicVertex and AnthropicBedrock build their endpoint from project/region internally, so there is legitimately no URL to pass. On current main + this PR, /model anthropic/claude-* on provider: vertex raised:

ValueError: switch_model: no base_url resolved for provider 'vertex'
(switching from 'openrouter'); refusing to keep the previous provider's endpoint

Exempted, scoped to api_mode == "anthropic_messages" and provider in {vertex, bedrock}. Gemini-on-Vertex uses the OpenAI-compat aggregator and does need a real base_url, so it stays under the guard. Nothing is inherited in the exempted path: agent.base_url is left untouched and the client is rebuilt from project/region.

Tests pin it from both sides — the Vertex switch survives an empty base_url, and the guard still fires for an ordinary provider change and for Vertex outside anthropic_messages. Your original guard test (tests/run_agent/test_switch_model_stale_base_url.py) passes unchanged.

Your earlier review

Both points from the 2026-07-10 review are in and still hold after the rebase:

• the switch_model() Anthropic Messages branch builds AnthropicVertex from the resolved project/region and snapshots the same fields for restore/recovery (ce6474075), and the request-local site above is now covered too;
• website/docs/guides/anthropic-vertex.md no longer contradicts itself — the intro now states the anthropic/ prefix is required and that bare claude-… falls through to the aggregator and 404s on purpose, matching the dispatch table.

Verification

scripts/run_tests.sh over the PR's suites plus your guard suite and the transports suite: 164 passed, 0 failed across

tests/agent/test_anthropic_vertex_adapter.py
tests/agent/test_auxiliary_client_vertex_dispatch.py
tests/agent/test_switch_model_anthropic_provider_dispatch.py
tests/agent/test_request_anthropic_client_vertex_dispatch.py
tests/hermes_cli/test_anthropic_vertex_provider.py
tests/run_agent/test_switch_model_vertex_anthropic.py
tests/run_agent/test_switch_model_stale_base_url.py
tests/agent/transports/test_chat_completions.py

This is running in production on a two-VM fleet against anthropic/claude-opus-4-8 on Vertex with ADC from the GCE metadata server, no API key anywhere in the tree.