The Generated Images Contain a Lot of Noise

[Shi5013]

Hello, @nickk124 . Thank you for your exciting work!
I ran your code on my own dataset, but the images output in the ./ddim-{dataset}-256-segguided/samples folder contain a lot of noise. I would greatly appreciate any advice you could offer. Here are the details of my training:

CUDA_VISIBLE_DEVICES=0,1 python3 main.py \
    --mode train \
    --model_type DDIM \
    --img_size 256 \
    --num_img_channels 1 \
    --dataset Liver_4DCT \
    --img_dir ./data \
    --seg_dir ./seg_data \
    --segmentation_guided \
    --num_segmentation_classes 2 \
    --train_batch_size 8 \
    --eval_batch_size 4 \
    --num_epochs 200

By the way, during the first training, I encountered the following error when saving the safetensor file at the 30th epoch:

File "/media/user_gou/Elements/Shi/diffusion/main.py", line 418, in <module>
    main(
  File "/media/user_gou/Elements/Shi/diffusion/main.py", line 341, in main
    train_loop(
  File "/media/user_gou/Elements/Shi/diffusion/training.py", line 203, in train_loop
    pipeline.save_pretrained(config.output_dir)
  File "/home/user_gou/anaconda3/envs/voxelmorph/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 661, in save_pretrained
    save_method(os.path.join(save_directory, pipeline_component_name), **save_kwargs)
  File "/home/user_gou/anaconda3/envs/voxelmorph/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 361, in save_pretrained
    safetensors.torch.save_file(
AttributeError: module 'safetensors' has no attribute 'torch'

However, I have since modified the code and re-ran it, and this error no longer occurs.

[nickk124]

Hi!

For your first question, how many epochs did you train for? I've noticed that for some datasets, it can take 100, 200 or even more epochs until the noisyness in the images goes away.

For your second question, my apologies for the bug! Can I ask, what fix you made?

Best,
Nick

[Shi5013]

Thank you for your answer！
Regarding the first issue, I initially trained for only 30 epochs. I tried again and after training for 200 epochs, the noise in the images almost disappeared. It was my mistake.

Regarding the second issue, I don’t think it’s your mistake. After searching on Google, I found this:
safetensors_issues

Following their suggestion, I changed import safetensors to from safetensors.torch import load_file, save_file, and this resolved the issue. I’m not sure why it worked either.

[nickk124]

Hi, where exactly did you change import safetensors to from safetensors.torch import load_file, save_file? I'd like to post about this potential bug on the README.

Otherwise, I think your issues are resolved so I'll go ahead and close this.
Thanks!

[Shi5013]

The changes were made in the site-packages of the diffusers library within the Anaconda environment, specifically at the location indicated by the error message.

File "/home/user_gou/anaconda3/envs/voxelmorph/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 361, in save_pretrained
    safetensors.torch.save_file(

In the import section of this file, change import safetensors to from safetensors.torch import load_file, save_file. Additionally, modify the calls to the two functions. Maybe on lines 109 and 361? (I'm not not sure the actually line because I have already modified.)

[nickk124]

Ok, thanks so much for the help. I'll add it to the README!

[ssqrots]

"Hello, could you please tell me specifically what changes should be made to line 109 and line 361?"

[Shi5013]

Simply changed the way of function calling. In the previous file, because it was import safetensors, the usage on lines 109 and 361 was safetensors.torch.save_file(.

However, later I changed import safetensors to from safetensors.torch import load_file, save_file, so the prefix in the previous usage should be removed, and the function should be called directly as save_file(.