Skip to content

feat(image): parallelize image_generate batches + serialize FileSyncManager cycles (salvage #33971) - #77596

Merged
kshitijk4poor merged 3 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/33971-image-parallel
Aug 3, 2026
Merged

feat(image): parallelize image_generate batches + serialize FileSyncManager cycles (salvage #33971)#77596
kshitijk4poor merged 3 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/33971-image-parallel

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Salvage of #33971 by @EndeavorYen — both commits cherry-picked verbatim (authorship preserved); the only conflict was test-file context drift (main deleted 3 stale postprocess tests that sat in the hunk's context; resolution keeps only the PR's new test).

What this does for users

  1. image_generate joins _PARALLEL_SAFE_TOOLS: when the model requests several images in one turn, they generate CONCURRENTLY instead of one-after-another. Generation latency is network-bound (seconds per image), so a 4-image batch completes in ~1× single-image latency instead of ~4×.
  2. Worker cap is configurable via image_gen.max_parallel_requests in config.yaml (default 4, clamped 1..8) — follows the documented convention that image_gen is an intentionally-absent DEFAULT_CONFIG root; documented in the website image-generation docs.
  3. Fixes a REAL race the parallelism would otherwise widen: FileSyncManager.sync()/sync_back() had no in-process serialization (the existing fcntl.flock is cross-process, sync_back-only, and a no-op on Windows). Concurrent artifact syncs could drop files from _synced_files (lost update). Now serialized by a threading.Lock.

Verification

  • 21 passed (test_file_sync.py + test_image_generation_artifacts.py + new test_image_generate_parallel.py)
  • MUTATION-VERIFIED: reverting only the file_sync lock makes both new concurrency tests fail with the real lost-update (second.png dropped from _synced_files) — the tests bind the race, they are not change-detectors
  • Parallel-safety of the handler traced end-to-end: per-task-env FileSyncManager (covered by the new lock), managed FAL client (already lock-guarded, httpx thread-safe), filename collisions impossible (<prefix>_<ts>_<uuid4[:8]>), config read is mtime-cached (stat + deepcopy per batch, no YAML parse)
  • Routing verified: 2+ image_generate calls form one ("parallel", …) segment → _execute_tool_calls_concurrent; mixed batches take the same executor per segment

Honest caveat on the cap: a mixed batch (images + fast reads) shares one worker pool per parallel segment, so max_parallel_requests: 1 serializes unrelated tools in that segment — blunt but user-opted-in and clamped ≥1; reads drain in milliseconds through remaining slots at the default of 4.

Closes #33971.

@kshitijk4poor
kshitijk4poor enabled auto-merge (rebase) August 3, 2026 11:20
@alt-glitch alt-glitch added type/perf Performance improvement or optimization P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint backend/file-sync File sync across remote backends needs-decision Awaiting maintainer decision before any implementation labels Aug 3, 2026
@kshitijk4poor
kshitijk4poor force-pushed the salvage/33971-image-parallel branch from deb11db to 2a05b54 Compare August 3, 2026 13:46
@kshitijk4poor
kshitijk4poor force-pushed the salvage/33971-image-parallel branch from 2a05b54 to 828f67c Compare August 3, 2026 17:16
@kshitijk4poor
kshitijk4poor merged commit a7ad713 into NousResearch:main Aug 3, 2026
35 checks passed
@kshitijk4poor
kshitijk4poor deleted the salvage/33971-image-parallel branch August 5, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend/file-sync File sync across remote backends comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint needs-decision Awaiting maintainer decision before any implementation P3 Low — cosmetic, nice to have type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants