Skip to content

num_in_channels has no effect when call StableDiffusionXLInpaintPipeline.from_single_file  #6610

@Sanster

Description

@Sanster

Describe the bug

num_in_channels has no effect when call StableDiffusionXLInpaintPipeline.from_single_file and StableDiffusionXLPipeline.from_single_file

Reproduction

Running following code can produce the issue:

from diffusers import StableDiffusionXLInpaintPipeline, StableDiffusionXLPipeline

# load normal sdxl model
pipe = StableDiffusionXLPipeline.from_single_file("sd_xl_base_1.0.safetensors", local_files_only=False, load_safety_checker=False)
print("load normal sdxl model, unet in_channels: ", pipe.unet.config.in_channels)

# load normal sdxl model, by setting num_in_channels=9, it should raise error
pipe = StableDiffusionXLPipeline.from_single_file("sd_xl_base_1.0.safetensors", local_files_only=False, load_safety_checker=False, num_in_channels=9)
print("load normal sdxl model with num_in_channels=9, should raise error, but unet in_channels is: ", pipe.unet.config.in_channels)


print("""This should load ok, but will raise error: ValueError: Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.""")
pipe = StableDiffusionXLInpaintPipeline.from_single_file("sd_xl_base_1.0_inpainting_0.1.safetensors", local_files_only=False, load_safety_checker=False, num_in_channels=9)

The original configuration file used by SDXL is using network_config instead of unet_config, which suggests that there may be an issue with this line of code:

if "unet_config" in original_config["model"]["params"]:

Logs

No response

System Info

  • diffusers version: 0.25.0
  • Platform: macOS-13.5-arm64-arm-64bit
  • Python version: 3.8.10
  • PyTorch version (GPU?): 2.1.0 (False)
  • Huggingface_hub version: 0.20.2
  • Transformers version: 4.36.2
  • Accelerate version: 0.24.1
  • xFormers version: not installed
  • Using GPU in script?:
  • Using distributed or parallel set-up in script?: <fill in

Who can help?

@yiyixuxu @sayakpaul

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions