Skip to content

fix(plugins): support OPENAI_IMAGE_API_MODEL env var + base_url config for third-party backends - #18796

Open
y0shua1ee wants to merge 1 commit into
NousResearch:mainfrom
y0shua1ee:fix/openai-image-api-model-override
Open

fix(plugins): support OPENAI_IMAGE_API_MODEL env var + base_url config for third-party backends#18796
y0shua1ee wants to merge 1 commit into
NousResearch:mainfrom
y0shua1ee:fix/openai-image-api-model-override

Conversation

@y0shua1ee

@y0shua1ee y0shua1ee commented May 2, 2026

Copy link
Copy Markdown
Contributor

Problem

The OpenAI image gen plugin hardcodes API_MODEL = "gpt-image-2" in the API request payload, making it impossible to use third-party OpenAI-compatible backends (e.g., Krill AI, OpenRouter image endpoints) that expect different model identifiers.

# Before this PR — always sends "gpt-image-2" regardless of config
payload = {"model": API_MODEL, ...}
client = openai.OpenAI()  # no base_url support from config

Closes #18793

Solution

Three new escape hatches, fully backward-compatible — defaults unchanged:

API model override

Priority Source
1 OPENAI_IMAGE_API_MODEL env var
2 image_gen.openai.api_model config key
3 API_MODEL (default: gpt-image-2)

Base URL

Priority Source
1 OPENAI_BASE_URL env var (standard SDK behaviour)
2 image_gen.openai.base_url config key
3 None → api.openai.com

Usage example (Krill AI)

# config.yaml
image_gen:
  provider: openai
  openai:
    api_model: gpt-image-2-1k-medium
    base_url: https://api.krill-ai.com/v1

Or via env var:

export OPENAI_IMAGE_API_MODEL=gpt-image-2-1k-medium

Changes

  • Added _resolve_api_model() — resolves the model name for the API request
  • Added _resolve_base_url() — resolves the base URL for the OpenAI client
  • Updated generate() to use resolved values instead of hardcoded constants
  • Updated module docstring with new precedence rules

No breaking changes. Default behaviour unchanged.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins tool/vision Vision analysis and image generation labels May 2, 2026
@y0shua1ee
y0shua1ee force-pushed the fix/openai-image-api-model-override branch from 5992c0b to 815f9f2 Compare May 9, 2026 05:03
@y0shua1ee

y0shua1ee commented May 9, 2026

Copy link
Copy Markdown
Contributor Author

Rebased and updated this PR against the current main after #21273 landed.

What changed in this revision:

  • Keeps the existing virtual tier IDs (gpt-image-2-low, gpt-image-2-medium, gpt-image-2-high) as quality selectors.
  • Separates those tier IDs from the actual model field sent to images.generate.
  • Honors feat(image-gen): honor image_gen.model from config.yaml (salvage #19376) #21273's dispatcher-provided model kwarg:
    • if it is one of the virtual tier IDs, it selects the quality tier;
    • if it is not a virtual tier ID, it is treated as the actual API model for OpenAI-compatible backends.
  • Keeps explicit escape hatches with higher precedence:
    • OPENAI_IMAGE_API_MODEL
    • image_gen.openai.api_model
  • Adds base URL support through:
    • OPENAI_BASE_URL
    • image_gen.openai.base_url
  • Adds tests for API model resolution, base URL resolution, provider.generate(...), and dispatcher model forwarding.

Local verification:

python -m pytest -o addopts= tests/plugins/image_gen/test_openai_provider.py -q
# 37 passed

python -m pytest -o addopts= tests/plugins/image_gen -q
# 74 passed

python -m pytest -o addopts= tests/tools -k image -q
# 187 passed, 7 skipped

python -m ruff check plugins/image_gen/openai/__init__.py tests/plugins/image_gen/test_openai_provider.py tests/tools/test_image_generation_plugin_dispatch.py
# All checks passed

This should make the PR complementary to #21273 rather than conflicting with it: #21273 forwards image_gen.model; this plugin now actually interprets that forwarded value correctly while still supporting provider-specific api_model / base_url config for third-party OpenAI-compatible image endpoints.

CI note after the latest push:

  • ruff enforcement (blocking) passed.
  • nix ubuntu/macOS, e2e, supply-chain scan, Windows footguns, attribution, and Docker build jobs passed.
  • ruff + ty diff failed only while trying to post its PR comment (Resource not accessible by integration on issues/.../comments); the generated summary reported no new ruff issues and labels ty diagnostics as warnings.
  • test failed in two unchanged areas outside this PR's diff:
    • tests/gateway/test_restart_drain.py::test_restart_command_while_busy_requests_drain_without_interrupt
    • tests/hermes_cli/test_tencent_tokenhub_provider.py::TestTencentTokenhubContextLength::test_hy3_preview_context_length

This PR's changed files remain limited to:

plugins/image_gen/openai/__init__.py
tests/plugins/image_gen/test_openai_provider.py
tests/tools/test_image_generation_plugin_dispatch.py

@y0shua1ee

Copy link
Copy Markdown
Contributor Author

Rebased this PR and refreshed CI.

Status after the refresh:

  • Head: 87889255b
  • Merge state: CLEAN
  • Checks: 18 success, 2 skipped

Local verification run before pushing:

uv run --with ruff ruff check plugins/image_gen/openai/__init__.py tests/plugins/image_gen/test_openai_provider.py tests/tools/test_image_generation_plugin_dispatch.py
uv run --extra dev --with pytest-xdist python -m pytest tests/plugins/image_gen/test_openai_provider.py tests/tools/test_image_generation_plugin_dispatch.py tests/gateway/test_restart_drain.py::test_restart_command_while_busy_requests_drain_without_interrupt tests/hermes_cli/test_tencent_tokenhub_provider.py::TestTencentTokenhubContextLength::test_hy3_preview_has_registered_context_length -q --tb=short -n 0

The previous red checks were from the stale branch state; the fork-comment permission issue in the lint summary path is no longer present on the refreshed run.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for separating the virtual quality-tier IDs from the API model; the underlying hard-coded-model problem is still present on current main.

Problems

  • Blocking: this branch predates the image-to-image refactor in c02192ff6 (#48705). The submitted change configures only images.generate at plugins/image_gen/openai/__init__.py:308-329 (PR head 87889255b). Current main also routes image inputs through images.edit at plugins/image_gen/openai/__init__.py:298-305, where it still uses API_MODEL, while its shared client at line 273 has no configured base_url. A salvage of the current patch would therefore leave edit requests on the hard-coded model/default endpoint.

Suggested changes

  • Apply the API-model and base-URL resolution before the current text/edit branch, instantiate one configured client, and use the resolved model for both images.generate and images.edit.
  • Add an edit-path regression covering a configured API model and base URL.

Automated hermes-sweeper review.

client_kwargs: Dict[str, Any] = {}
if base_url:
client_kwargs["base_url"] = base_url
client = openai.OpenAI(**client_kwargs)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking for salvage onto current main: #48705 added a shared client plus an images.edit() branch after this commit's parent. Apply this configured client before that branch and use the resolved API model for images.edit too; otherwise image-edit requests still use the hard-coded model and default endpoint.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 12, 2026
@y0shua1ee
y0shua1ee force-pushed the fix/openai-image-api-model-override branch from 8788925 to 6f77ad9 Compare July 13, 2026 14:45

Copy link
Copy Markdown
Contributor Author

@teknium1 — I rebased this PR onto current main and addressed the current image-edit blocker in 6f77ad96f.

What changed:

  • Resolve the quality tier, actual API model, and base URL before the generate/edit branch.
  • Build one configured OpenAI client and reuse it for both images.generate() and images.edit().
  • Use the resolved API model in both paths while keeping Hermes’ virtual low/medium/high tier IDs as quality selectors.
  • Added an edit-path regression using a real temporary PNG, configured api_model and base_url, and verified that edit is called instead of generate.
  • Isolated the new OpenAI override environment variables in the test fixture to avoid host-environment flakes.

Validation:

  • Wider image-generation suite: 226 passed, 0 failed
  • After the final rebase: provider + dispatcher files: 48 passed, 0 failed
  • Ruff and git diff --check: passed

CI is now running on the refreshed branch. Could you take another look when it is ready?

Split the OpenAI image plugin's virtual quality-tier IDs from the actual API model sent in images.generate requests.

This keeps gpt-image-2-low/medium/high as UI/config tier selectors while allowing OpenAI-compatible backends to set the real payload model via OPENAI_IMAGE_API_MODEL, image_gen.openai.api_model, or the dispatcher-provided image_gen.model from NousResearch#21273 when it is not one of the virtual tier IDs.

Also pass OPENAI_BASE_URL or image_gen.openai.base_url into openai.OpenAI(), and cover the resolver, provider.generate, and dispatcher model forwarding paths with tests.

Closes: NousResearch#18793
@y0shua1ee
y0shua1ee force-pushed the fix/openai-image-api-model-override branch from 6f77ad9 to 6502312 Compare July 15, 2026 02:31
@y0shua1ee

Copy link
Copy Markdown
Contributor Author

@teknium1 — follow-up refresh complete on head 6502312d7.

  • Rebased onto current main while keeping the diff limited to the OpenAI image provider and its provider/dispatcher regressions.
  • Both images.generate() and images.edit() use the same resolved API model, configured base URL, and shared client.
  • Local image-generation validation: 281 passed, 0 failed; focused post-rebase validation: 51 passed, 0 failed; Ruff and git diff --check passed.
  • GitHub CI is fully green, including All required checks pass.

This is also the prerequisite contract for the image-provider fallback work in #32320. Could you take another look when convenient?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

image_gen openai plugin: API_MODEL hardcoded, breaks third-party OpenAI-compatible backends

3 participants