Skip to content

feat(image-gen): add OpenAI image edit provider - #26969

Closed
BeliefanX wants to merge 3 commits into
NousResearch:mainfrom
BeliefanX:feat/openai-image-edit-api
Closed

feat(image-gen): add OpenAI image edit provider#26969
BeliefanX wants to merge 3 commits into
NousResearch:mainfrom
BeliefanX:feat/openai-image-edit-api

Conversation

@BeliefanX

@BeliefanX BeliefanX commented May 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Relationship to #26967 / #26968

This PR is intentionally stacked on the clean image-edit foundation series:

Recommended merge order: #26967#26968#26969#21765. Please merge this PR after #26967 and #26968; after they land, I will rebase/refresh this branch onto main so the diff only shows the OpenAI provider delta.

  1. feat(image-gen): add provider edit contract #26967: provider-level edit capability contract
  2. feat(image-gen): validate image references #26968: reusable image reference validation helper
  3. this PR: OpenAI provider implementation for image editing

Until the earlier PRs merge, GitHub will show their commits in this PR too. The provider-only delta in this PR is limited to:

  • plugins/image_gen/openai/__init__.py
  • tests/plugins/image_gen/test_openai_provider.py

Implementation

  • adds OpenAIImageProvider.supports_edit() -> True
  • adds OpenAIImageProvider.edit(...) using client.images.edit(...)
  • validates source images through agent.image_reference.validate_image_reference(...)
  • supports local/cache files and data URLs as SDK file inputs
  • rejects HTTP(S) source references for edit requests because the OpenAI SDK expects file-like image inputs
  • supports optional mask images, multi-image input lists, size/aspect-ratio resolution, quality tier/model resolution, and local caching of returned images

Testing

  • python3 -m pytest tests/plugins/image_gen/test_openai_provider.py tests/agent/test_image_gen_provider_edit_contract.py tests/agent/test_image_reference.py -q -o 'addopts='
  • python3 -m py_compile plugins/image_gen/openai/__init__.py tests/plugins/image_gen/test_openai_provider.py agent/image_gen_provider.py agent/image_reference.py
  • git diff --check HEAD~1..HEAD

No live OpenAI API calls used; tests mock SDK request behavior.

@cardtest15-coder

This comment was marked as spam.

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins provider/openai OpenAI / Codex Responses API labels May 16, 2026
@cardtest15-coder

This comment was marked as spam.

@cardtest15-coder

This comment was marked as spam.

@BeliefanX
BeliefanX force-pushed the feat/openai-image-edit-api branch from 05c04e5 to f91189d Compare May 16, 2026 15:22
@BeliefanX

Copy link
Copy Markdown
Contributor Author

Addressed review feedback in the updated stack:

  • Inherited the hardened PR2 reference validation changes.
  • Cleaned up edit quality-tier resolution and added OpenAI edit error-path tests for missing dependency, API error, empty response data, and empty response item.
  • The HTTP(S) reference rejection before the OpenAI call is intentional and remains covered by test; this avoids silently fetching remote URLs in the provider before there is an explicit SSRF/download policy.
  • resolve_aspect_ratio already clamps invalid/empty aspect values via the base contract coverage.

@BeliefanX
BeliefanX force-pushed the feat/openai-image-edit-api branch from f91189d to a259cd6 Compare May 16, 2026 15:39
@BeliefanX
BeliefanX marked this pull request as ready for review May 16, 2026 15:55
@BeliefanX

Copy link
Copy Markdown
Contributor Author

Refreshed the PR branch with a small OpenAI-Codex image stream fallback fix: if the Responses SDK final-response parser raises TypeError("'NoneType' object is not iterable") after streamed image events, the provider now preserves the image already collected from the stream instead of returning an API error. Added a regression test for that path.\n\nLocal checks run:\n- pytest -q tests/plugins/image_gen/test_openai_codex_provider.py -q\n- python -m py_compile plugins/image_gen/openai-codex/__init__.py tests/plugins/image_gen/test_openai_codex_provider.py

@BeliefanX

Copy link
Copy Markdown
Contributor Author

Updated with an additional Codex image stream fallback: if the Responses stream iterator raises TypeError after yielding an image event, the provider now preserves that streamed image; iterator TypeError before any image still fails.\n\nValidation run locally:\n- pytest tests/plugins/image_gen/test_openai_codex_provider.py -q\n- python -m py_compile plugins/image_gen/openai-codex/init.py tests/plugins/image_gen/test_openai_codex_provider.py\n- git diff --check

@BeliefanX

Copy link
Copy Markdown
Contributor Author

Closing this cumulative OpenAI API edit-provider branch in favor of the clean provider replacement #40491. The replacement keeps the provider work isolated from the contract/validation/tool layers and targets the current OpenAI Codex image-edit path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants