Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for sdxl_gen_img.py unable to load in Google Colab Env and Distributed Inference Version (accel_sdxl_gen_img.py) #1901

Open
wants to merge 222 commits into
base: dev
Choose a base branch
from

Conversation

DKnight54
Copy link
Contributor

For the Colab enviroment loading issue, it was pretty straight forward by implementing --full_fp16 and --full_bf16 when loading the model.
#1887
Basically modifying the linked lines.

For the distributed version accel_sdxl_gen_img.py, there are some major changes.

  1. Chances are, interactive mode may be broken, but unable to test without a powerful enough GPU.
  2. Instead of processing batches one at a time as it runs through the loop, the code now does the following:
    a. It now collects all prompts (prompts x number of images per prompt) in a list.
    b. The prompt preparation loop runs only on the main process to prevent messy double printing of prompts, especially when dynamic prompts are used.
    c. The list of prompts are then split into a list of batches of prompts.
    d. The batches are then distributed onto available GPUs.
  3. Simliar to the previous distributed sample generation, image generation order is mixed up.
    a. Using similar workaround, global_step is used as image generation index.
    b. An integer variable global_count is added to the class BatchData
    c. It is used during the inference to add an index to the filenames.
  4. Something else that is likely to be broken now is the section that overrides Deep Shrink and Gradual Latent as it would only be updated in the main process and I can't think of a way to push it to any sub processes.
    a. I have a question about the logic though, because it appears that if the arguements are added to any prompts in a list of prompts, it would override and affect all prompts in that batch and any other batches after that.
    b. Is this an intended behaviour? Especially since I don't see a way to unset it.
    c. Okay, maybe I do have a way to update all processes. Maybe by using gather_object() to get a list of unet or pipes and iterate through them to update. Needs testing to see if it works., but since I don't understand fully Deep Shrink and Gradual Latent, not sure if I can correctly evaluate.

I've also left in some testing and troubleshooting log outputs as it's pretty useful to see what's going on.

I get that the distributed inference is probably not critical, but I have found it useful in jupyter notebooks enviroments with more than one GPU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants