Skip to content

feat(image_gen): add per-call Codex output controls - #14819

Closed
lumenradley wants to merge 1 commit into
NousResearch:mainfrom
lumenradley:feat/codex-image-output-controls
Closed

lumenradley wants to merge 1 commit into
NousResearch:mainfrom
lumenradley:feat/codex-image-output-controls

Conversation

@lumenradley

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds per-call output controls for the bundled openai-codex image generation backend. This lets callers keep openai-codex selected globally while overriding safe output options for a single generation:

  • quality: auto, low, medium, high
  • size: auto or a validated custom WIDTHxHEIGHT for gpt-image-2
  • output_format: png, jpeg, webp
  • output_compression: 0-100 for jpeg/webp

The provider keeps unsafe or currently-unhelpful knobs fixed:

  • background: remains opaque because gpt-image-2 does not support transparent backgrounds
  • partial_images: remains 0 to avoid extra image output token cost when Hermes is not surfacing previews
  • model selection remains provider-owned (gpt-image-2)

The default Codex image tier is now gpt-image-2-auto, matching OpenAI's provider-selected quality behavior while still allowing explicit low/medium/high overrides.

Related Issue

Follow-up to #14317. No separate issue found for per-call openai-codex output controls.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • tools/image_generation_tool.py
    • Exposes optional quality, size, output_format, and output_compression arguments in the image_generate schema.
    • Forwards those controls to plugin image providers.
  • plugins/image_gen/openai-codex/__init__.py
    • Adds gpt-image-2-auto as the default selectable tier.
    • Resolves per-call quality overrides ahead of env/config defaults.
    • Validates custom gpt-image-2 sizes against OpenAI constraints.
    • Validates output format and JPEG/WebP compression combinations.
    • Saves generated files with an extension matching the selected output format.
    • Keeps partial_images=0 to avoid extra preview-token cost.
  • Tests
    • Adds coverage for per-call quality, custom size, output format/compression, invalid sizes, and plugin-dispatch forwarding.

How to Test

  1. Select the Codex image backend:
    hermes config set image_gen.provider openai-codex
    hermes config set image_gen.model gpt-image-2-auto
  2. Generate normally, or call image_generate with overrides such as:
    • quality="high"
    • size="1152x2496"
    • output_format="webp"
    • output_compression=85
  3. Run tests:
    scripts/run_tests.sh tests/hermes_cli/test_image_gen_picker.py tests/tools/test_image_generation*.py tests/plugins/image_gen/ -q

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS, Python 3.13.13

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

N/A — this PR does not add a skill.

Screenshots / Logs

Targeted test suite:

scripts/run_tests.sh tests/hermes_cli/test_image_gen_picker.py tests/tools/test_image_generation*.py tests/plugins/image_gen/ -q

152 passed in 1.49s

Full test suite was attempted using the canonical runner with a clean temporary HERMES_HOME:

TMP_HERMES_HOME=$(mktemp -d)
ulimit -n 8192
HERMES_HOME="$TMP_HERMES_HOME" scripts/run_tests.sh -q

Result in this local macOS environment:

39 failed, 14916 passed, 40 skipped, 149 warnings in 247.94s

The failures appear unrelated to this PR (provider switching, WSL/systemd service tests, file write-deny temp paths, transcription local-command detection, process cleanup timing, etc.). The image generation tests touched by this PR pass.

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins tool/vision Vision analysis and image generation provider/openai OpenAI / Codex Responses API labels Apr 24, 2026
@lumenradley
lumenradley force-pushed the feat/codex-image-output-controls branch from 89300b2 to 17b2cef Compare April 24, 2026 00:21
@lumenradley

Copy link
Copy Markdown
Contributor Author

Update:

  • Rebased the branch onto latest upstream/main.
  • Added a scoped suppression for the noisy OpenAI SDK/Pydantic serializer warning that can appear for valid custom gpt-image-2 sizes such as 1152x2496. The warning is from the SDK's stale generated schema, not from the API rejecting the request.
  • Added regression coverage that simulates the stale-schema warning and verifies it does not leak to callers.

Targeted verification after the update:

scripts/run_tests.sh tests/plugins/image_gen/test_openai_codex_provider.py -q
37 passed

scripts/run_tests.sh tests/hermes_cli/test_image_gen_picker.py tests/tools/test_image_generation*.py tests/plugins/image_gen/ -q
153 passed

Note on the red CI Tests job: the same workflow is currently red on main as well, with unrelated failures outside this image-generation PR. This branch's targeted image-generation coverage is passing locally.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for the careful work here, @lumenradley — the validation logic (gpt-image-2 pixel/aspect/multiple-of-16 bounds, scoped Pydantic warning suppression, PNG-rejects-compression) is well done and the tests are thorough.

Unfortunately we can't land this as-is: exposing quality/size/output_format/output_compression on the global image_generate schema means every provider sees them in the agent's tool list, and Nous Portal (the default-managed provider for the majority of our users) doesn't support per-call output controls for image generation. Adding agent-facing params that aren't universally honored across the provider matrix would leave Nous users with a tool schema advertising knobs that silently no-op or error on their backend.

Closing for that reason rather than any issue with the implementation. If you want to revisit this as a Codex-only path that doesn't touch the global schema (e.g. via env var / config.yaml overrides resolved inside the openai-codex plugin only), that would be mergeable. Sorry for the wasted cycles, and thanks again for the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API tool/vision Vision analysis and image generation type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants