feat(vertex): add Google Vertex AI provider for Gemini models - #36253
feat(vertex): add Google Vertex AI provider for Gemini models#36253Harshavardhan-Yaddalapuri wants to merge 3 commits into
Conversation
|
Update: Added UX improvements from #16010:
All 193 provider-adjacent tests pass. The two GMI failures are pre-existing on main (UnboundLocalError in This should address the UX gap vs #16010 while keeping the correct architecture ( |
f737e34 to
0e699bf
Compare
|
Thanks for putting work into this — more hands on Vertex is genuinely welcome. A couple of clarifications for anyone evaluating, though, because the "clean rewrite" framing isn't quite accurate. This tracks #8427's design pretty directly: same On "against current main" specifically:
Credit where it's due: the Given @alt-glitch's point about consolidating on one approach — #8427 has been open since April, is in live production use, conforms to the current lazy-install policy, and is merge-ready against current main today. The pragmatic path seems to be converging there rather than restarting the clock. |
Related: part of the open native-Vertex provider cluster for #13484 — #8427 (earliest, in production, lazy-deps-compliant, merge-ready), #16010 (first-class Vertex), #29611 (Vertex Express Mode). Per @slawt this PR tracks #8427's design plus #16010's UX rather than an independent rewrite, and currently asks users to |
Adds Vertex AI as a first-class inference provider with dynamic OAuth2 token resolution from service account JSON or gcloud ADC. Mirrors the existing Bedrock provider pattern (aws_sdk → gcp_sdk auth_type). - agent/vertex_adapter.py: credential detection, OAuth2 token refresh with 55-min caching, base URL construction from project_id+region - hermes_cli/auth.py: ProviderConfig registration, aliases (vertex-ai, google-vertex, gcp-vertex), auto-detection - hermes_cli/runtime_provider.py: vertex handler block with proper auth error messages and config.yaml vertex.region support - agent/auxiliary_client.py: vertex handler in resolve_provider_client using standard OpenAI client (chat/completions — no custom SDK) - run_agent.py: vertex in safe_providers + aiplatform URI defense - agent/model_metadata.py: aiplatform.googleapis.com hostname mapping Requires: pip install google-auth Credentials: GOOGLE_APPLICATION_CREDENTIALS or VERTEX_CREDENTIALS_PATH plus VERTEX_PROJECT_ID (or project_id from SA JSON). Model: google/gemini-3.1-flash-lite-preview (and all Gemini variants). Region defaults to 'global' (required for Gemini 3.x previews).
…tration - tests/agent/test_vertex_adapter.py (32 tests): credential path detection, project ID resolution from SA JSON and env vars, region resolution, auth source labeling, token caching/refresh, base URL construction, integration test - tests/hermes_cli/test_vertex_provider.py (13 tests): provider registration in PROVIDER_REGISTRY, alias resolution, auth_status reporting for credential states - Refactored agent/vertex_adapter.py: module-scoped lazy google-auth import via _ga_auth/_ga_transport/_ga_service_account globals, making token refresh logic testable via standard mock.patch All 45 new tests pass. No regressions in existing suite — confirmed that GMI provider failures (test_resolve_provider_client_* 2 failures) are pre-existing on main.
…malization - hermes_cli/main.py: _model_flow_vertex() — dedicated provider flow with credential display, region override, and curated model picker (mirrors _model_flow_bedrock pattern, no API key prompt) - hermes_cli/model_normalize.py: auto-prepend google/ prefix for Vertex provider — users can type bare 'gemini-3.5-flash' and it resolves correctly - hermes_cli/models.py: curated Vertex model catalog in _PROVIDER_MODELS (9 models from GA 2.0 through preview 3.5), ProviderEntry for model picker UI listing - hermes_cli/providers.py: HermesOverlay registration + aliases (vertex-ai, google-vertex) - hermes_cli/setup.py: model list for setup wizard Includes both GA models (for production workloads) and preview models (for users who want latest capabilities via GCP credits).
0e699bf to
b5fd475
Compare
|
Superseded by #56363, which adds Google Vertex AI as a first-class Gemini provider on current That PR salvaged the earliest submission in this cluster (#8427 by @slawt) and preserved its authorship. Closing this as a duplicate — thanks for working on the same gap; the merged implementation covers the Gemini-via-Vertex use case. If you spot something it misses, please open a focused follow-up. |
What does this PR do?
Adds Google Vertex AI as a first-class inference provider for Gemini models. Users with GCP free credits ($300) or enterprise billing can now use Hermes with Vertex AI instead of being stuck with the Google AI Studio free tier.
Mirrors the existing Bedrock provider pattern —
aws_sdk→gcp_sdkauth type. Uses the standard OpenAI-compatible chat/completions endpoint (no custom SDK or message translation needed).Related Issue
Fixes #13484
Related: #8427 (stale PR for same feature, this is a clean rewrite against current main), #4375 (Vertex Express via custom provider workaround)
Type of Change
Changes Made
Production code
agent/vertex_adapter.py(189 lines) — credential detection via SA JSON or gcloud ADC, module-scoped lazy google-auth import for testability, OAuth2 token refresh with 55-min caching, base URL construction from project_id+regionhermes_cli/auth.py(+25 lines) — ProviderConfig registration withauth_type="gcp_sdk", aliases (vertex-ai,google-vertex,gcp-vertex), provider auto-detection, auth_status handlerhermes_cli/runtime_provider.py(+57 lines) — vertex runtime resolution block with proper AuthError messages and config.yamlvertex.regionsupportagent/auxiliary_client.py(+46 lines) — vertex handler inresolve_provider_clientusing standardOpenAI()clientrun_agent.py(+4/-1 lines) — vertex added to safe_providers +aiplatform.googleapis.comURL defense-in-depthagent/model_metadata.py(+1 line) —aiplatform.googleapis.comhostname mappingTests (45 new tests, all passing)
tests/agent/test_vertex_adapter.py(32 tests) — credential path detection, project ID resolution from SA JSON and env vars, region resolution, auth source labeling, token caching/refresh behavior, base URL construction, full chain integration testtests/hermes_cli/test_vertex_provider.py(13 tests) — provider registration, alias resolution (parametrized: vertex-ai, google-vertex, gcp-vertex, google-vertex-ai), auth_status reporting (credentials present, absent, import error fallback)How to Test
GOOGLE_APPLICATION_CREDENTIALSandVERTEX_PROJECT_IDin~/.hermes/.envhermes model→ select Vertex AI → pick a Gemini modeluv run python -m pytest tests/agent/test_vertex_adapter.py tests/hermes_cli/test_vertex_provider.py -v(45/45 pass)Checklist
Code
scripts/run_tests.shrun — 99 pre-existing failures (kanban workers, ACP adapters, GMI providerOpenAIUnboundLocalError) confirmed present on main before our changes. Zero new failures.Documentation & Housekeeping
cli-config.yaml.exampleif I added/changed config keys — N/A (no new required config keys;vertex.regionin config.yaml is optional)CONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/Agoogle-authis pure Python, cross-platform. Lazy imports keep startup fast for non-Vertex users.Notes
pip install google-auth(not added to [all] extras — follows Bedrock pattern of lazy dependency with_import_google_auth())_ga_*globals allow tests to mock without importing google-authglobal(required for Gemini 3.x previews; us-central1 silently breaks them)