Skip to content

feat: support custom base_url for Gemini native adapter - #21747

Open
li0near wants to merge 1 commit into
NousResearch:mainfrom
li0near:fix/gemini-custom-base-url
Open

feat: support custom base_url for Gemini native adapter#21747
li0near wants to merge 1 commit into
NousResearch:mainfrom
li0near:fix/gemini-custom-base-url

Conversation

@li0near

@li0near li0near commented May 8, 2026

Copy link
Copy Markdown
Contributor

Summary

When provider: "gemini" is explicitly configured, use the native Gemini REST adapter (models/{model}:generateContent) for any base_url that doesn't end in /openai — matching how other providers work unconditionally in _create_openai_client().

Gemini was the only provider with a domain-based gate:

Provider Client creation Gated?
copilot-acp CopilotACPClient No
google-gemini-cli GeminiCloudCodeClient No
gemini GeminiNativeClient Yes (relaxed by this PR)
openai/custom OpenAI() No
anthropic build_anthropic_client() No

Problem

Previously, is_native_gemini_base_url() checked for generativelanguage.googleapis.com in the URL before using the native adapter. With a custom base_url (e.g. a local proxy that speaks Gemini's native REST protocol), the check would fail and silently fall through to the OpenAI-compatible client — sending /v1/chat/completions format instead of /models/{model}:generateContent.

Fix

Replaced the domain-based is_native_gemini_base_url() check with a simpler /openai suffix check:

# Before:
if is_native_gemini_base_url(base_url):  # requires googleapis.com domain

# After:
if not base_url.rstrip('/').lower().endswith('/openai'):  # only skip for OpenAI-compat endpoints
  • Any base_url (including custom proxies) → native Gemini adapter
  • URLs ending in /openai (e.g. generativelanguage.googleapis.com/v1beta/openai) → OpenAI client (these speak OpenAI format)
  • Empty base_url → native adapter (defaults to Google's endpoint internally)

Example config (cli-config.yaml)

model:
  default: "gemini-2.5-flash"
  provider: "gemini"
  base_url: "http://localhost:8900/gemini"
  api_key: "your-proxy-api-key"

Test plan

  • Updated test_gemini_custom_base_url to assert native client is used
  • test_gemini_openai_compat_base_url_keeps_openai_client unchanged (verifies /openai suffix fallback)
  • Verify provider: "gemini" with no base_url still defaults to Google's endpoint
  • Verify provider: "gemini" with custom base_url uses native adapter
  • Verify other providers are unaffected

@li0near
li0near force-pushed the fix/gemini-custom-base-url branch 2 times, most recently from 4078f6c to 7f4c3ef Compare May 8, 2026 08:01
@alt-glitch alt-glitch added type/feature New feature or request provider/gemini Google Gemini (AI Studio, Cloud Code) comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels May 8, 2026
@li0near
li0near force-pushed the fix/gemini-custom-base-url branch 2 times, most recently from 1b8da05 to 49a5e27 Compare May 8, 2026 08:53
When provider is explicitly set to "gemini", use the native Gemini REST
adapter for any base_url that doesn't end in /openai — matching how
other providers (Anthropic, OpenAI, Copilot) work unconditionally.

Previously, is_native_gemini_base_url() gated the native adapter on
the presence of "generativelanguage.googleapis.com" in the URL. This
made Gemini the only provider with a domain-based gate in
_create_openai_client(). With a custom base_url (e.g. a local proxy),
the domain check would fail and silently fall through to the OpenAI-
compatible client, sending the wrong request format to endpoints
expecting Gemini's native models/{model}:generateContent REST API.

URLs ending in /openai (e.g. generativelanguage.googleapis.com/v1beta/
openai) continue to fall through to the OpenAI client, as those
endpoints speak OpenAI format by design.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for documenting the proxy use case. Current main has an explicit contrary routing contract, so this needs a maintainer design decision before salvage.

Problems

  • Commit d393104bad deliberately limited the native adapter to the canonical Gemini endpoint and kept custom base URLs on the OpenAI-compatible path. That contract remains in agent/gemini_native_adapter.py:55-62 and is used by the live client factory at agent/agent_runtime_helpers.py:1671-1693.
  • The PR changes the former implementation in run_agent.py; current run_agent.py:3987-3990 is only a forwarder after c42fa94af extracted the implementation.
  • The predicate also controls auxiliary routing (agent/auxiliary_client.py:1855-1859, 1895-1899, 4904-4911), streaming (agent/chat_completion_helpers.py:2105-2112), and native request filtering (agent/transports/chat_completions.py:621-630). A one-site port would not establish a coherent endpoint contract.

Suggested changes

  • First resolve whether the intentional custom-endpoint policy from d393104bad should change. If approved, update the shared classification contract and all consumers, with an end-to-end proxy-path test.

Automated hermes-sweeper review.

Comment thread run_agent.py

base_url = str(client_kwargs.get("base_url", "") or "")
if is_native_gemini_base_url(base_url):
# URLs ending in /openai are OpenAI-compatible endpoints (e.g.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current main has extracted this implementation: run_agent.py:3987-3990 now forwards to agent/agent_runtime_helpers.py:1643. Port the change to the live helper only after resolving the explicit custom-endpoint routing policy in d393104bad; changing this former implementation cannot be salvaged as-is.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/gemini Google Gemini (AI Studio, Cloud Code) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants