Repository navigation
[diffusion] comfyui: key cached conditioning on full content - #43109
Closed
OnePunchMonk wants to merge 2 commits into
Closed
OnePunchMonk wants to merge 2 commits into
OnePunchMonk wants to merge 2 commits into
Conversation
_cond_key used the shape plus the first and last scalar of prompt_embeds[0]. For Flux that is the pooled CLIP-L vector, so prompts that only differ in T5 got the same key and later steps reused the first prompt's T5 context. Hash every field drop_cached_fields removes instead. Fixes sgl-project#43107
OnePunchMonk
requested review from
AgainstEntropy,
BBuf,
HaiShaw,
kevin-mii,
mickqian,
niehen6174,
ping1jing2 and
yichiche
as code owners
October 8, 2026 15:36
Adapters now declare cached_extra_keys; drop_cached_fields and the cond key both read it, so H3's h3_* fields and sigmas are part of the key. The worker stored packed extras in one shared slot, so on a cache hit the positive cond could get the negative's h3_context/payload back. Store them per comfyui_cond_key instead. Hash nested dicts/lists by content with framing instead of repr().
This was referenced Oct 8, 2026
niehen6174
reviewed
Oct 9, 2026
Contributor
Author
|
Superseded by #43492, which consolidates this fix with 3 related executor conditioning-cache fixes per maintainer request to reduce PR overhead. Closing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fixes #43107.
SGLDiffusionExecutor._cond_keykeyed cached conditioning on the shape plus the first and last scalar ofprompt_embeds[0]. On a key hit the executor drops the embeddings and the worker restores whatever is cached under that key. For Flux,prompt_embeds[0]is the pooled CLIP-Ly, which ComfyUI takes from the first 77-token chunk only, so two prompts that differ only past that point (orCLIPTextEncodeFluxwith the sameclip_ltext) get the same key and the second prompt runs with the first one's T5 context. Qwen-Image-Edit had the same problem withimage_latent, which is dropped but was not part of the key.Modifications
_cond_keyreturns a blake2b digest overprompt_embeds,pooled_embeds,prompt_seq_lensand the adapter's cachedextra_reqfields. Nested dicts/lists are hashed by content with length framing (norepr()of tensors, which abbreviates large ones).cached_extra_keys.drop_cached_fieldsand the key both read it, so they can't drift apart. For MiniMax-H3 this puts theh3_*fields andsigmasin the key (previously only the text was keyed).bind_comfyui_sessionstores packed extras percomfyui_cond_keyinstead of in one shared slot. Before, on an H3 cache hit the positive cond could get the negative'sh3_context/ payload back (last write won). Thecomfyui_cache_fpcheck is unchanged.test_comfyui_session.pyrun the real executor bookkeeping plusbind_comfyui_session: Flux sameywith different T5, same first/last scalar, differentimage_latent, H3 pos/neg extras on a cache hit, H3 same text with a different payload, large tensors inside a payload list, per-key extras on the worker, and a repeat that must still use the cache.Accuracy Tests
No model forward changes. Repro in a real ComfyUI KSampler run (ComfyUI master 58b176f, real CLIP-L weights, plugin
FluxExecutor+SGLDModelPatcherwired likecore/generator.py, scheduler round-trip replaced by the realbind_comfyui_session), prompt A for 0 to 45%, prompt B from 50%:Unit tests (CPU):
Speed Tests and Profiling
The key now copies the embeddings to CPU and hashes them once per step (the old key already synced via
.item()). Hashing measured on CPU: about 2.5 ms for a 512x4096 bf16 T5 context, 1.3 ms for a Qwen-Image context. For H3 the payload's cond latents are hashed too. Small next to a DiT step, but I did not measure it on GPU.Checklist
CI States
Latest PR Test (Base): ❌ Run #37803977549
Latest PR Test (Extra): ❌ Run #37803977308
Latest PR Test (AMD ROCm 10): ❌ Run #37803977583