feat: add image generation fallback controls - #32320
Conversation
f07ca9e to
2509142
Compare
|
Rebased this PR onto current Current status after the refresh:
|
2509142 to
c59487f
Compare
Add image_gen.fallback_providers dispatch with retryable-failure gating, provider/model override propagation, and /image_model switching for CLI and gateway. Covers fallback routing, provider model resolution, command handling, managed config behavior, and active-session bypass tests.
c59487f to
afe3777
Compare
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the focused fallback routing and retry-gating work. The feature premise still holds on current main: tools/image_generation_tool.py:1295-1401 dispatches to one configured image provider and returns that result without an image-provider fallback chain.
Problems
- The stale dispatcher replacement loses the source-image parameters added by
c02192ff6(tools/image_generation_tool.py:1274-1279,1336-1349). In this PR,tools/image_generation_tool.py:1116accepts only prompt/aspect ratio and line 1198 sends only those plus model, so image-to-image calls cannot reach any fallback provider. - The same regression exists in the handler: PR
tools/image_generation_tool.py:1269-1283does not read or forwardimage_url/reference_image_urls, unlike current main attools/image_generation_tool.py:1518-1551. The added fallback tests exercise only text-to-image calls (tests/tools/test_image_generation_plugin_dispatch.py:154).
Suggested changes
- Salvage the fallback loop onto the current image-to-image dispatcher and carry source-image inputs through every attempt; add primary-failure/fallback-success coverage with both image input fields.
- Reconcile command wiring with the current gateway command surface and preserve the existing
hermes toolspicker.
Automated hermes-sweeper review.
| return any(marker in error for marker in _RETRYABLE_IMAGE_ERROR_MARKERS) | ||
|
|
||
|
|
||
| def _dispatch_to_plugin_provider(prompt: str, aspect_ratio: str): |
There was a problem hiding this comment.
Current main's dispatcher accepts and forwards image_url and reference_image_urls for image-to-image calls (tools/image_generation_tool.py:1274-1279, 1336-1349). Please retain those parameters here and pass them to every fallback attempt; otherwise this stale replacement drops source images before either the primary or fallback provider sees them.
| # Route to a plugin-registered provider if one is active (and it's | ||
| # not the in-tree FAL path). | ||
| # Route to a plugin-registered provider if one is active. | ||
| dispatched = _dispatch_to_plugin_provider(prompt, aspect_ratio) |
There was a problem hiding this comment.
This handler now calls the dispatcher with only prompt and aspect ratio. Current main reads image_url and reference_image_urls and forwards both through plugin, managed-Krea, and FAL routes (tools/image_generation_tool.py:1518-1551). Preserve that forwarding and add a fallback test that asserts the backup provider receives the source-image inputs.
What does this PR do?
Adds first-class image generation fallback controls:
image_gen.fallback_providerssupport for image provider fallback./image_modelCLI/gateway command for switching image provider/model.This implements the image generation provider-fallback slice of the broader tool-level fallback request. The wider issue also discusses vision fallback, script fallback, and other tool categories, so this PR links it as a partial implementation.
Closest related work checked:
image_gen, vision, and other non-LLM services.Related Issue
Partially addresses #30455 for
image_genprovider fallback and image model switching.Type of Change
Changes Made
tools/image_generation_tool.pyhermes_cli/image_model_switch.pycli.py,gateway/run.py,hermes_cli/commands.py/image_modelinto command dispatch.plugins/image_gen/fal/__init__.pyplugins/image_gen/openai-codex/__init__.pyplugins/image_gen/openai/__init__.pyplugins/image_gen/xai/__init__.pytests/hermes_cli/test_image_model_switch.pytests/gateway/test_image_model_command.pytests/tools/test_image_generation.pytests/tools/test_image_generation_plugin_dispatch.pytests/plugins/image_gen/test_fal_provider.pytests/plugins/image_gen/test_openai_provider.pytests/plugins/image_gen/test_xai_provider.pytests/plugins/image_gen/test_openai_codex_provider.pyHow to Test
uv run pytest -q tests/hermes_cli/test_image_model_switch.py \ tests/gateway/test_image_model_command.py \ tests/tools/test_image_generation.py \ tests/tools/test_image_generation_plugin_dispatch.py \ tests/plugins/image_gen/test_fal_provider.py \ tests/plugins/image_gen/test_openai_provider.py \ tests/plugins/image_gen/test_xai_provider.py \ tests/plugins/image_gen/test_openai_codex_provider.py \ tests/gateway/test_command_bypass_active_session.pyExpected result from the branch:
uv run ruff check cli.py gateway/run.py hermes_cli/commands.py \ hermes_cli/image_model_switch.py tools/image_generation_tool.py \ plugins/image_gen/fal/__init__.py \ plugins/image_gen/openai/__init__.py \ plugins/image_gen/xai/__init__.py \ plugins/image_gen/openai-codex/__init__.py \ tests/gateway/test_image_model_command.py \ tests/hermes_cli/test_image_model_switch.py \ tests/tools/test_image_generation.py \ tests/tools/test_image_generation_plugin_dispatch.py \ tests/plugins/image_gen/test_fal_provider.py \ tests/plugins/image_gen/test_openai_provider.py \ tests/plugins/image_gen/test_xai_provider.py \ tests/plugins/image_gen/test_openai_codex_provider.pyExpected result:
Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/AScreenshots / Logs
Targeted tests and lint: