-
Notifications
You must be signed in to change notification settings - Fork 929
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix for sdxl_gen_img.py unable to load in Google Colab Env and Distributed Inference Version (accel_sdxl_gen_img.py) #1901
Open
DKnight54
wants to merge
222
commits into
kohya-ss:dev
Choose a base branch
from
DKnight54:test_colab
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Try force stay FP16 on text encoders
draft accel image gen
testing logger
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For the Colab enviroment loading issue, it was pretty straight forward by implementing --full_fp16 and --full_bf16 when loading the model.
#1887
Basically modifying the linked lines.
For the distributed version accel_sdxl_gen_img.py, there are some major changes.
a. It now collects all prompts (prompts x number of images per prompt) in a list.
b. The prompt preparation loop runs only on the main process to prevent messy double printing of prompts, especially when dynamic prompts are used.
c. The list of prompts are then split into a list of batches of prompts.
d. The batches are then distributed onto available GPUs.
a. Using similar workaround,
global_step
is used as image generation index.b. An integer variable
global_count
is added to the class BatchDatac. It is used during the inference to add an index to the filenames.
a. I have a question about the logic though, because it appears that if the arguements are added to any prompts in a list of prompts, it would override and affect all prompts in that batch and any other batches after that.
b. Is this an intended behaviour? Especially since I don't see a way to unset it.
c. Okay, maybe I do have a way to update all processes. Maybe by using
gather_object()
to get a list of unet or pipes and iterate through them to update. Needs testing to see if it works., but since I don't understand fully Deep Shrink and Gradual Latent, not sure if I can correctly evaluate.I've also left in some testing and troubleshooting log outputs as it's pretty useful to see what's going on.
I get that the distributed inference is probably not critical, but I have found it useful in jupyter notebooks enviroments with more than one GPU.