Skip to content

feat(image_gen): support configurable endpoint, credential routing, and proxy bypass for OpenAI image generation - #65323

Open
asdlem wants to merge 2 commits into
NousResearch:mainfrom
asdlem:fix/image-gen-config-endpoint
Open

feat(image_gen): support configurable endpoint, credential routing, and proxy bypass for OpenAI image generation#65323
asdlem wants to merge 2 commits into
NousResearch:mainfrom
asdlem:fix/image-gen-config-endpoint

Conversation

@asdlem

@asdlem asdlem commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Fixes #65309

Summary

Single canonical fix that gives the OpenAI image gen provider three
capabilities it currently lacks:

  1. Configurable endpointimage_gen.openai.base_url sets the
    API base URL (falls back to OPENAI_BASE_URL → SDK default).
    This enables routing through local gateways, task-scoped proxies,
    and third-party OpenAI-compatible backends. Related: fix(plugins): support OPENAI_IMAGE_API_MODEL env var + base_url config for third-party backends #18796.

  2. Credential routing via key_envimage_gen.openai.key_env
    points to an env var holding the API key. The token itself is
    never written to config.yaml. This enables task-scoped tokens
    from gateway/injection systems (e.g., Workbench).

  3. Proxy-bypass client — the image client now uses the shared
    build_keepalive_http_client() transport so macOS system proxy
    settings no longer intercept localhost image endpoints.

Config example

image_gen:
  openai:
    model: gpt-image-2-medium
    base_url: http://localhost:18081/v1
    key_env: IMAGE_GATEWAY_TOKEN

Precedence (per-call)

Base URL: image_gen.openai.base_url → OPENAI_BASE_URL → SDK default
API key:  image_gen.openai.key_env → OPENAI_API_KEY

Related

Allow the OpenAI image generation provider to read endpoint and
credential source from config.yaml:

  image_gen.openai.base_url  → explicit endpoint (fallback: OPENAI_BASE_URL)
  image_gen.openai.key_env   → env var holding the API key (fallback: OPENAI_API_KEY)

Also fixes the proxy-bypass gap: the image client now uses
build_keepalive_http_client() (same helper the main LLM and auxiliary
clients use), so macOS system proxy settings no longer intercept
localhost image endpoint calls.

Fixes NousResearch#65309
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins tool/vision Vision analysis and image generation provider/openai OpenAI / Codex Responses API area/config Config system, migrations, profiles sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Jul 16, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: this combines the scoped OpenAI image configuration overlap in #18796 with the image-client proxy-bypass work in #64893. Its broader key_env/config scope makes it a competing implementation, not a duplicate.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing a verified configuration gap in the OpenAI image provider.

Problems

  • The new resolution and transport path has no regression coverage. Current tests cover only OPENAI_API_KEY and tier resolution (tests/plugins/image_gen/test_openai_provider.py:74-121), while this PR changes only plugins/image_gen/openai/__init__.py.
  • The new image_gen.openai.base_url and key_env contract is not documented. The current image-generation guide still describes OpenAI credentials only as OPENAI_API_KEY (website/docs/user-guide/features/image-generation.md:193-196).

Suggested changes

  • Add temporary-HERMES_HOME tests for endpoint/key precedence, is_available(), configured client arguments, and the macOS system-proxy-bypass case. The analogous proxy test pattern is tests/agent/test_auxiliary_client_proxy_env.py:61-79.
  • Document the two new image-provider settings and their fallback order.

Automated hermes-sweeper review.

@@ -118,6 +123,52 @@ def _resolve_model() -> Tuple[str, Dict[str, Any]]:
return DEFAULT_MODEL, _MODELS[DEFAULT_MODEL]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add focused temporary-HERMES_HOME coverage for this resolver: config-vs-environment precedence, key_env availability/generation behavior, and propagation into openai.OpenAI(). The current provider tests only cover OPENAI_API_KEY and tier resolution.

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 16, 2026
Address review feedback from NousResearch#65323 — add TestCredentialResolution
class covering base_url/key_env precedence, fallback chains, and
proxy-bypass transport logic. Document the two new config options
(base_url, key_env) with their fallback order in the image-generation
user guide.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data tool/vision Vision analysis and image generation type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Configure the OpenAI image endpoint and credential source in config.yaml

3 participants