feat(providers): add Google Vertex AI as a first-class inference provider - #16010
Closed
Zephyr596 wants to merge 4 commits into
Closed
feat(providers): add Google Vertex AI as a first-class inference provider#16010Zephyr596 wants to merge 4 commits into
Zephyr596 wants to merge 4 commits into
Conversation
This commit introduces Google Vertex AI as a first-class inference provider, leveraging its OpenAI-compatible endpoint. Key implementations: - Auth: Dynamically generates short-lived Bearer tokens using `google-auth` via `GOOGLE_APPLICATION_CREDENTIALS` (Service Account JSON). - Model Normalization: Transparently injects the mandatory `google/` publisher prefix for Vertex requests in `model_normalize.py` while maintaining clean, unprefixed model slugs in the TUI and config. - UX: Suppresses the implicit normalization warning in `main.py` specifically for the Vertex prefix to prevent noisy CLI output. - Dependencies: Introduces a lightweight `[vertex]` optional dependency group in `pyproject.toml` to avoid bloating the base installation.
- Add hermes_cli/vertex_adapter.py with token caching, ADC support, and auto-refresh (mirrors agent/vertex_adapter.py pattern) - Replace 30-line inline auth block in runtime_provider.py with a 5-line call to get_vertex_config() - Add _model_flow_vertex() in main.py — shows creds status, prompts for region only (no misleading API key / Base URL prompts) - Fix validate_requested_model: strip google/ prefix before catalog lookup so google/gemini-2.5-pro matches gemini-2.5-pro in catalog - Update vertex model list to gemini-3.x preview + 2.5 series - Align env var names: VERTEX_CREDENTIALS_PATH > GOOGLE_APPLICATION_CREDENTIALS, VERTEX_REGION for region override Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix _save_model_choice call signature (takes model_id only) - Save provider="vertex" to config after model selection - Replace gemini-3.x preview models with official stable catalog: gemini-2.5-pro/flash/flash-lite, gemini-2.0-flash-001/flash-lite-001
10 tasks
slawt
added a commit
to slawt/hermes-agent
that referenced
this pull request
Apr 28, 2026
Adopts the legitimate UX improvements from the competing PR while keeping our broader scope (auxiliary client integration, global region default, _provider_matches_config refactor, Gemini 3.x preview support). - hermes_cli/main.py: dedicated _model_flow_vertex() — the generic api-key flow misidentifies a service-account JSON path as an 'API key' and prompts for a Base URL that's dynamically computed from the project_id. The new flow shows credential status, prompts only for region, then runs the model picker. - hermes_cli/model_normalize.py: auto-inject the mandatory google/ publisher prefix so users can configure 'gemini-2.5-pro' and have it routed correctly to Vertex's OpenAPI endpoint. - cli.py: suppress the spurious 'normalized X to google/X' warning for the silent prefix injection above. - hermes_cli/models.py: vertex-ai / google-vertex aliases; strip the google/ prefix in validate_requested_model() so the catalog lookup doesn't false-warn at session start. - hermes_cli/models.py setup catalog: add stable 2.0/2.5 chat completions models alongside our existing 3.x previews. - tests/test_model_normalize.py: lock in the prefix-injection behaviour and alias resolution. - pyproject.toml: ship google-auth as an optional [vertex] extra so users who don't use Vertex don't pull it in.
23 tasks
8 tasks
12 tasks
Contributor
|
Superseded by #56363, which adds Google Vertex AI as a first-class Gemini provider on current That PR salvaged the earliest submission in this cluster (#8427 by @slawt) and preserved its authorship. Closing this as a duplicate — thanks for working on the same gap; the merged implementation covers the Gemini-via-Vertex use case. If you spot something it misses, please open a focused follow-up. |
19 tasks
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds Google Vertex AI as a first-class inference provider for Gemini models, using GCP's OpenAI-compatible endpoint with dynamic OAuth2 token resolution. This is similar in intent to #8427 but addresses several correctness and UX issues found during implementation and testing.
Issues corrected vs #8427:
hermes_cli/vertex_adapter.py(not justagent/vertex_adapter.py) so the CLI credential path is fully decoupled from the agent subsystem_model_flow_vertex()inmain.py— the generic_model_flow_api_key_provider()was misidentifying the service account JSON path as an "API key" and unconditionally prompting for a "Base URL" that is dynamically computed and should not be manually enteredvalidate_requested_model()to strip the mandatorygoogle/publisher prefix before catalog lookup, preventing a false "model not found in curated catalog" warning on every session startRelated Issue
See also: #8427, #4375, #6491, #3569
Type of Change
Changes Made
New files:
hermes_cli/vertex_adapter.py— credential resolution (SA JSON or ADC), token caching with 5-min refresh window, base URL construction. Falls back from ADC to SA file if ADC refresh fails.Provider registration:
hermes_cli/auth.py— registervertexinPROVIDER_REGISTRYwithVERTEX_CREDENTIALS_PATH/GOOGLE_APPLICATION_CREDENTIALSenv var priorityhermes_cli/providers.py— registervertexinHERMES_OVERLAYS(openai_chattransport); add aliasesvertex-ai,google-vertexhermes_cli/models.py— add vertex to_PROVIDER_MODELSandCANONICAL_PROVIDERS; fixvalidate_requested_modelto stripgoogle/prefix for catalog lookuphermes_cli/model_normalize.py— inject mandatorygoogle/publisher prefix for Vertex requests transparentlyhermes_cli/setup.py— add vertex to setup model picker catalogRuntime:
hermes_cli/runtime_provider.py— replace 30-line inline credential block with a 5-lineget_vertex_config()call; supports both SA file and ADC; token cached and auto-refreshedhermes_cli/main.py— dedicated_model_flow_vertex()flow: shows credential status without prompting, prompts for region only, suppresses spurious normalization warning for the mandatorygoogle/prefixrun_agent.py— excludevertexfrom the generic "API key not found" fail-fast check (vertex uses file-based credentials, not a key env var)Dependencies:
pyproject.toml— lightweight[vertex]optional dependency group (google-auth)How to Test
pip install hermes-agent[vertex]GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json(or rungcloud auth application-default login)hermes model, select Vertex AI — should show credential path with ✓, prompt for region only, then show model pickergemini-2.5-pro) and start a session — no "model not found in catalog" warning should appearChecklist
Code
Documentation & Housekeeping
os.path.exists, works on Windows/macOS/Linux)Screenshots / Logs