feat(image_gen): support array prompt for parallel multi-image generation - #71345
LAN-TINA-WS wants to merge 5 commits into
Conversation
…tion - prompt schema: string → [string, array] with items - _handle_image_generate: array mode uses ThreadPoolExecutor - _read_max_parallel() reads image_gen.max_parallel config (default 5) - re-register with override=True to bump registry _generation - backward compatible: single string prompt unchanged
- generatedImageFromResult returns string[] (was string|null) - GeneratedImage maps over array, renders each as SingleImage - generatedImageEchoSources also reads images array for dedup - backward compatible: single image returns [singlePath]
- schema accepts [string, array] type - empty array rejects with prompt required - array dispatches via thread pool (3 items → 3 calls) - partial failures preserve successful images - backward compatible: single string unchanged
alt-glitch
added
type/feature
Related: #33971 parallelizes separate image_generate calls in the agent executor; this PR adds an array-prompt batch contract and Desktop rendering. They use distinct mechanisms. |
…ve dynamic_schema_overrides
teknium1
left a comment
teknium1
left a comment
There was a problem hiding this comment.
Thanks for adding a concrete batch contract and Desktop rendering path. The feature is still absent from current main (tools/image_generation_tool.py:1181-1188, :1501-1540), but the batch result contract needs a few corrections.
Problems
apps/desktop/src/lib/generated-images.ts:64-66discards theimagesarray when top-levelsuccessis false. The new backend intentionally emitssuccess: falsewith successful child entries for a partial failure (tools/image_generation_tool.py:1599-1613in the PR), so those images render as an indefinite placeholder.tools/image_generation_tool.py:1568-1598silently skips blank/non-string elements. An all-invalid non-empty array consequently returnssuccess: truewith no image.- The new nested
imagesshape is not wired into gateway delivery. Currentgateway/run.py:1473-1480reads top-level image fields and stops after the first path, so messaging surfaces would deliver only one batch image.
Suggested changes
- Add partial-success rendering/deduplication coverage, strict per-item validation, and gateway delivery/history coverage for every nested image.
- Keep the unrelated
API_MODELrename inplugins/image_gen/openai/__init__.py:52out of this feature PR unless separately validated.
Automated hermes-sweeper review.
|
|
||
| if (!record || record.success === false) { | ||
| return null | ||
| return [] |
There was a problem hiding this comment.
The batch handler returns success: false while retaining successful entries in images on a partial failure. Returning here prevents the new batch scan below, so Desktop shows a pending placeholder instead of the successful images. Handle successful child entries before treating the whole result as unusable, and add a partial-success renderer test.
| with ThreadPoolExecutor(max_workers=max_workers) as executor: | ||
| future_to_idx = {} | ||
| for idx, p in enumerate(prompt): | ||
| if not isinstance(p, str) or not p.strip(): |
There was a problem hiding this comment.
Skipping invalid entries leaves all_ok true. For example, [""] submits no futures and returns {"success": true, "image": null, "images": []}. Validate every item up front and reject a blank or non-string entry (with coverage for an all-invalid non-empty array).
| return json.dumps({ | ||
| "success": all_ok, | ||
| "image": first_img, | ||
| "images": images_list, |
There was a problem hiding this comment.
This nested result shape also needs gateway delivery support. Current gateway/run.py:1473-1480 only reads top-level image fields and breaks after the first path, so messaging platforms will automatically receive only image, not the remaining entries here. Traverse images in auto-append and history-dedup paths with gateway coverage.
teknium1
added
sweeper:risk-message-delivery
3 participants
Summary
image_generatenow acceptspromptas either a string (unchanged) or an array of strings. Array prompts dispatch parallel generation viaThreadPoolExecutor, with concurrency controlled byimage_gen.max_parallel(default 5).Changes
Backend (
tools/image_generation_tool.py):promptschema:"string"→["string", "array"]ThreadPoolExecutorwith_read_max_parallel()image_gen.max_parallelin config.yaml (default 5, max 20)Frontend (
apps/desktop/):generatedImageFromResult()returnsstring[]scanningimagesarrayGeneratedImagecomponent maps over array, renders each imagegeneratedImageEchoSourcesalso readsimagesfor dedupTests: 5 new tests covering schema, empty array rejection, parallel dispatch (monkeypatched), and partial failure handling. 69/69 pass.
Usage