Skip to content

fix(auxiliary_client): route vertex through the auth_type dispatch - #61853

Open
haizaar wants to merge 2 commits into
NousResearch:mainfrom
zarmory:fix/vertex-auxiliary-dispatch
Open

fix(auxiliary_client): route vertex through the auth_type dispatch#61853
haizaar wants to merge 2 commits into
NousResearch:mainfrom
zarmory:fix/vertex-auxiliary-dispatch

Conversation

@haizaar

@haizaar haizaar commented Jul 10, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes a pre-existing bug in agent/auxiliary_client.py::resolve_provider_client that silently disabled every auxiliary task (vision_analyze, context compression, session_search, curator's LLM review pass, title_generation, web_extract) on any provider: vertex deployment.

The main-chat surface is unaffected because hermes_cli/runtime_provider.py::resolve_runtime_provider special-cases the vertex name up front. Auxiliary tasks all go through resolve_provider_client and inherit the bug.

Failure signatures:

  • vision_analyze is tool-gated: check_vision_requirements() uses the same resolver, gets None, returns False, so the tool is stripped from the model-facing schema. The agent narrates its own confusion at not finding the tool.
  • Other aux tasks fall through their fallback chain (openrouter → nous → local/custom → api-key) and terminate with RuntimeError("No LLM provider configured for task=<task> provider=auto. Run: hermes setup").
  • Fleets that keep an aggregator fallback (OPENROUTER_API_KEY set, or a Nous auth token cached) don't notice — Step 3 of _resolve_auto catches the miss. Vertex-only deployments hit it terminally.

Bug is Vertex-wide, not Anthropic-specific. Reproduces on plain provider: vertex, default: "google/gemini-3.1-pro-preview" against current main.

Related Issue

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Non-breaking. The change makes an existing (dead-code) handler reachable; nothing that previously worked can regress. Verified with a TestHistoricalRegression case that pins the invariant against future refactors of hermes_cli/auth.py's PROVIDER_REGISTRY filter.

Root Cause

Two independent issues stack.

Issue 1 — PROVIDER_REGISTRY auto-extension filter excludes non-api_key providers.

hermes_cli/auth.py iterates the plugin catalog and adds providers to PROVIDER_REGISTRY, but only if auth_type == "api_key" and env_vars is non-empty:

# hermes_cli/auth.py, ~line 450
for _pp in _list_providers_for_registry():
    if _pp.name in PROVIDER_REGISTRY:
        continue
    if _pp.auth_type != "api_key" or not _pp.env_vars:   # ← filter
        continue
    ...

The vertex ProviderProfile (plugins/model-providers/vertex/__init__.py) declares auth_type="vertex" (OAuth2 via ADC — no static key) and env_vars=(). Both clauses reject it, so PROVIDER_REGISTRY.get("vertex") is None.

Issue 2 — the elif pconfig.auth_type == "vertex": handler is unreachable dead code.

agent/auxiliary_client.py::resolve_provider_client:

pconfig = PROVIDER_REGISTRY.get(provider)
if pconfig is None:
    if provider not in _LOGGED_UNKNOWN_PROVIDER_KEYS:
        _LOGGED_UNKNOWN_PROVIDER_KEYS.add(provider)
        logger.debug("resolve_provider_client: unknown provider %r", provider)
    return None, None                              # ← always fires for vertex

if pconfig.auth_type == "api_key":                 # ← never reached for vertex
    ...
elif pconfig.auth_type == "vertex":                # ← DEAD CODE
    # This branch already knows how to build a Vertex client via
    # has_vertex_credentials() + get_vertex_config() and serves Gemini
    # over the OpenAI-compat endpoint.
    ...

_resolve_auto's Step 1 correctly calls resolve_provider_client(resolved_provider, main_model, …) per its documented intent ("use my main chat model for side tasks as well"). For vertex, that call returns (None, None), so the chain falls through to Step 3's aggregators — which usually have no credentials on a Vertex-only deployment, and the whole chain terminates with the RuntimeError.

Same shape applies to any future non-api_key provider profile.

Changes Made

One hunk in agent/auxiliary_client.py. No hermes_cli/auth.py touch — keeping PROVIDER_REGISTRY api-key-only preserves invariants elsewhere in the codebase.

Plugin-catalog fallback for pconfig=None

Right where the pconfig is None bail sits today, add a fallback that consults the plugin catalog directly. If the missing provider resolves via providers.get_provider_profile(name) and its auth_type is one of the well-known non-api_key families (vertex, aws_sdk, oauth_device_code, oauth_external), synthesize a SimpleNamespace(auth_type=<profile.auth_type>) and let the existing dispatch branches downstream fire. Genuinely unknown providers still bail cleanly with the unchanged "unknown provider" debug log.

This makes the existing elif pconfig.auth_type == "vertex": handler reachable for Vertex users. The handler was already written to build an openai.OpenAI client pointed at Vertex's OpenAI-compat endpoint with a fresh OAuth2 bearer token — exactly what Gemini traffic needs. The same fallback also opens the door for any other non-api_key provider profile (aws_sdk / bedrock, oauth_*) to be reached the same way.

How to Test

Focused test file:

scripts/run_tests.sh tests/agent/test_auxiliary_client_vertex_dispatch.py -q
# ─── 10/10 pass in ~1s.

The 10 cases cover:

  • TestPluginCatalogFallback (3) — vertex reaches dispatch through the fallback; aliases (google-vertex, vertex-ai, gcp-vertex) resolve too; genuinely-unknown providers still bail.
  • TestVertexGeminiDispatch (6)google/-prefixed model builds an openai.OpenAI client with the right base_url + token; no-model default picks up the aux Gemini slug; bare gemini-* still routes to the Gemini handler (Vertex's 404 for the missing publisher stays the loud-fail diagnostic); missing-credentials / missing-oauth-token paths bail; async wrapper.
  • TestHistoricalRegression (1) — pins the invariant with vertex forcibly removed from PROVIDER_REGISTRY, so a future refactor of the auth.py auto-extension filter cannot silently re-break the aux path.

Broader regression sweep (auxiliary + vision + compression + session_search + config bridge + related adapters):

scripts/run_tests.sh tests/agent/test_auxiliary_client_vertex_dispatch.py \
                     tests/agent/test_auxiliary_client.py \
                     tests/agent/test_auxiliary_main_first.py \
                     tests/agent/test_auxiliary_client_azure_foundry.py \
                     tests/agent/test_auxiliary_client_anthropic_custom.py \
                     tests/agent/test_auxiliary_client_resolve_dedup.py \
                     tests/agent/test_vertex_adapter.py \
                     tests/agent/test_bedrock_adapter.py \
                     tests/agent/test_bedrock_integration.py \
                     tests/tools/test_vision_tools.py \
                     tests/agent/test_context_compressor.py \
                     tests/tools/test_session_search.py \
                     tests/agent/test_auxiliary_config_bridge.py \
                     -q
# ─── 13 files, 865/865 tests pass in ~20s.

Real-world validation. Deployed on a Vertex-only GCE fleet (NixOS, ADC via VM service-account, both Gemini-on-Vertex and Anthropic-on-Vertex configurations). Pre-fix, every auxiliary task terminated with the RuntimeError above; post-fix, every task round-trips through the Vertex handler and returns real content in <10s. vision_analyze on real image uploads via the messaging gateway round-trips cleanly (pre-fix: the tool was silently absent from the model's tool schema).

Manual reproduction on a fresh checkout (no Anthropic setup needed — this reproduces the bug on plain Gemini-on-Vertex against main):

# One-time GCP-side setup:
#   1. gcloud auth application-default login
#      (or export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa.json)

cat >> ~/.hermes/config.yaml <<'YAML'
model:
  default: google/gemini-3.1-pro-preview
  provider: vertex

vertex:
  project_id: your-gcp-project-id
  region: global
YAML

# Do NOT set OPENROUTER_API_KEY, do NOT run `hermes auth` for Nous.
hermes -z "Summarize: foo bar baz" \
  --logs-level DEBUG 2>&1 | grep -iE "vision|no provider|auxiliary"
# Pre-fix (upstream/main): check_vision_requirements returned False +
#                          Auxiliary auto-detect: no provider available
# Post-fix (this PR):      silence (aux tasks route through the main
#                          provider cleanly)

Checklist

Code

  • I've read the Contributing Guide.
  • My commit message follows Conventional Commits (fix(auxiliary_client): …).
  • I searched existing PRs to make sure this isn't a duplicate.
  • My PR contains only changes related to this fix. Single focused commit.
  • I've run scripts/run_tests.sh on the affected files (all pass).
  • I've added tests — 10 new cases in tests/agent/test_auxiliary_client_vertex_dispatch.py, including a TestHistoricalRegression that pins the aux-dispatch invariant against future refactors of hermes_cli/auth.py.
  • I've tested on my platform: NixOS 25.05 (GCE VMs, real Vertex backend), plus local dev on macOS 15.

Documentation & Housekeeping

  • Documentation updated — no user-facing docs to change; this is a silent-failure bug fix with no config surface.
  • cli-config.yaml.example — N/A.
  • CONTRIBUTING.md / AGENTS.md — N/A.
  • Cross-platform impact considered — no OS-specific primitives.
  • Tool descriptions / schemas — N/A.

Screenshots / Logs

Pre-fix INFO-level agent.log signature on a Vertex-only deployment (every turn):

WARNING tools.registry: check_fn check_vision_requirements returned False;
    dependent tools will be unavailable this turn
WARNING agent.auxiliary_client: Auxiliary auto-detect: no provider available
    (tried: openrouter, nous, local/custom, api-key). Compression,
    summarization, and memory flush will not work.
WARNING agent.title_generator: Title generation failed: No LLM provider
    configured for task=title_generation provider=auto. Run: hermes setup

Post-fix INFO log line that never fired before the fix:

INFO agent.auxiliary_client: Auxiliary auto-detect: using main provider vertex (<model>)

Silent regression on any `provider: vertex` deployment:
`resolve_provider_client("vertex", ...)` returned `(None, None)` because
`PROVIDER_REGISTRY.get("vertex")` was None, and the existing
`elif pconfig.auth_type == "vertex":` handler below was unreachable
dead code.

Every auxiliary task on a Vertex-only deployment — `vision_analyze`,
context compression, curator's LLM review pass, `session_search`,
`title_generation`, `web_extract` — fell through to the aggregator
fallback chain (openrouter → nous → local/custom → api-key) and
terminated with:

    RuntimeError: No LLM provider configured for
        task=<task> provider=auto. Run: hermes setup

Vision was hit hardest: `check_vision_requirements()` uses the same
resolver, so it returned False and `vision_analyze` was stripped from
the model-facing tool schema. Fleets with `OPENROUTER_API_KEY` set
never noticed because Step 3 of `_resolve_auto` caught the miss;
Vertex-only deployments hit it terminally.

Root cause is in `hermes_cli/auth.py`'s auto-extension of
`PROVIDER_REGISTRY` from the provider-plugin catalog:

    if _pp.auth_type != "api_key" or not _pp.env_vars:
        continue

The vertex `ProviderProfile` declares `auth_type="vertex"` (OAuth2 via
ADC, not a static key) and `env_vars=()`, so both clauses reject it.
Same shape applies to any future non-api_key provider profile
(aws_sdk, oauth_device_code, oauth_external).

Fix: plugin-catalog fallback for `pconfig=None`. When the registry
lookup misses, consult `providers.get_provider_profile(provider)`. If
the profile's `auth_type` is one of the well-known non-api_key
families (`vertex`, `aws_sdk`, `oauth_device_code`, `oauth_external`),
synthesize a `SimpleNamespace(auth_type=<profile.auth_type>)` and let
the existing dispatch branches downstream fire. Genuinely unknown
providers still bail cleanly with the unchanged "unknown provider"
debug log.

No touch to `hermes_cli/auth.py` — keeping `PROVIDER_REGISTRY`
api-key-only preserves invariants elsewhere in the codebase.

Once the fallback is in place, the pre-existing
`elif pconfig.auth_type == "vertex":` handler serves Gemini traffic
via the OpenAI-compat endpoint as it was originally written to do —
no other changes needed here.

Tests: new `tests/agent/test_auxiliary_client_vertex_dispatch.py` (10
cases, all hermetic — mock the credential seams only):

  - `TestPluginCatalogFallback` (3): vertex reaches dispatch through
    the fallback; aliases (`google-vertex`, `vertex-ai`, `gcp-vertex`)
    resolve; genuinely-unknown providers still bail.
  - `TestVertexGeminiDispatch` (6): `google/`-prefixed model builds an
    OpenAI client with the right base_url + token; no-model default;
    bare `gemini-*` still routes to the Gemini handler (Vertex's
    404 for the missing publisher stays the loud-fail diagnostic);
    credential / token failure paths; async wrapper.
  - `TestHistoricalRegression` (1): pins the invariant with `vertex`
    forcibly removed from `PROVIDER_REGISTRY`, so a future refactor
    of the `auth.py` auto-extension filter cannot silently
    reintroduce the bug.

No user-facing behaviour change on non-Vertex deployments.
@haizaar haizaar mentioned this pull request Jul 10, 2026
14 tasks
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/gemini Google Gemini (AI Studio, Cloud Code) area/auth Authentication, OAuth, credential pools duplicate This issue or pull request already exists labels Jul 10, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #56861 (earlier, still open) — both patch the resolve_provider_client registry-None branch in agent/auxiliary_client.py to fall back to providers.get_provider_profile for non-api_key providers (vertex/aws_sdk/oauth_*), making the existing auth_type == "vertex" dispatch reachable. Same code site and mechanism; #56861 is the earlier canonical fix. Note: #56688 addresses a distinct layer (registering vertex in PROVIDER_REGISTRY + HERMES_OVERLAYS) — related, not a duplicate. Cross-linking the bug issue #61852.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the current registry-miss defect: on main, agent/auxiliary_client.py:4858-4865 returns before the existing Vertex branch at :5012, while hermes_cli/auth.py:455 deliberately excludes the Vertex profile from the API-key registry.

Problems

  • The fallback at agent/auxiliary_client.py:4885-4892 accepts all aws_sdk and OAuth profile types, but the downstream implementations are provider-specific: the AWS path is Bedrock-specific (:5048) and OAuth only handles named providers (:5094-5107). Provider profiles are user-overridable (providers/__init__.py:53-62), so constrain this fallback to canonical Vertex handling unless generic dispatch exists.
  • Cached Vertex clients cannot recover from an expired OAuth token. The auth-retry paths call _refresh_provider_credentials() (agent/auxiliary_client.py:6743-6773, :7291-7320), but that helper has no Vertex branch (:3488-3558), and auto-route endpoint inference omits Vertex (:3561-3583).

Suggested changes

  • Scope the profile fallback to Vertex (including aliases), and add sync/async stale-token refresh coverage plus an _resolve_auto() regression test.

Automated hermes-sweeper review.

Comment thread agent/auxiliary_client.py Outdated
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 11, 2026
 Addresses @teknium1 review feedback on this PR:

 1. The plugin-catalog fallback allow-list was too broad. Narrow it
  from {vertex, aws_sdk, oauth_device_code, oauth_external} to
  {vertex} only. The downstream branches for the removed auth
  families are provider-specific rather than generic:

  - auth_type == aws_sdk builds Bedrock-specific clients
  (AnthropicBedrockClient / BedrockAuxiliaryClient); a future
  non-Bedrock aws_sdk profile would misroute here.
  - auth_type in {oauth_device_code, oauth_external} matches by
  provider name (nous / openai-codex / xai-oauth) inside the
  elif chain; any third-party OAuth profile with a novel name
  falls through to the not-directly-supported branch.

  Provider profiles are user-overridable (last-writer-wins in
  providers/__init__.py), so a user plugin re-declaring one of
  those auth types would land in a branch that cannot build its
  client. Vertex is the one auth family with genuinely generic
  downstream dispatch (OAuth2 token + OpenAI-compat endpoint), and
  its aliases (google-vertex, vertex-ai, gcp-vertex) resolve
  through the same handler unchanged.

 2. Cached Vertex clients could not recover from a stale token.
  _refresh_provider_credentials() had no vertex branch — auth-retry
  paths would call it, get False back, and the aux task would fail
  permanently until process restart. Add a vertex branch that:
  - clears vertex_adapter._creds_cache so google-auth re-mints,
  - re-resolves via get_vertex_config() to verify a fresh token
  can be produced,
  - evicts cached aux clients so they pick up the new token.

 3. Auto-routed aux calls could not infer provider=vertex from the
  selected clients base URL. Extend _auth_refresh_provider_for_route
  to recognise both host shapes:
  - aiplatform.googleapis.com (global location)
  - {region}-aiplatform.googleapis.com (regional)
  base_url_host_matches uses strict subdomain-of matching, which
  would miss the regional case (no dot between region and
  aiplatform), so match on hostname suffix directly.

 Tests: extend tests/agent/test_auxiliary_client_vertex_dispatch.py
 with three new classes covering the requested regression areas:

  - TestResolveAutoVertex: end-to-end via _resolve_auto Step 1 with
  _read_main_provider=vertex, _read_main_model=google/gemini-*.
  Confirms the fallback is reachable through the full aux chain,
  not just direct resolve_provider_client entry.
  - TestRefreshProviderCredentialsVertex (3 cases): cache-clear +
  aux-client eviction on success, False on mint failure, graceful
  bail when vertex_adapter is unimportable.
  - TestAuthRefreshProviderRouteVertex (4 cases): global + regional
  hosts return vertex; look-alike host does not; concrete resolved
  provider wins over URL inference.

 Suite green: 18/18 in the vertex-dispatch file, 484/484 across
 aux client + vertex/bedrock adapter surface via scripts/run_tests.sh.
@haizaar

haizaar commented Jul 13, 2026

Copy link
Copy Markdown
Author

@teknium1 Thanks for the review - both fixed. LMK if you prefer me to remove the long comment in the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/gemini Google Gemini (AI Studio, Cloud Code) sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants