[Bug]: Pix2Pix not working as expected

[rabbitflyer]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

Pix2Pix models don't seem to be working properly.

Specifically, the results I get are unrelated to the source image unless I reduce the denoising strength, i.e. it's behaving a lot like i'd expect an inpainting model to behave. Also, the slider for image CFG scale no longer has any effect on the result.

Steps to reproduce the problem

1. Download and load the default instruct_pix2pix model
2. Set denoising strength to 1, and leave image CFG scale at 1.5
3. Attempting to use the model should produce a result unrelated to the original image.
4. Change image CFG scale to any other value.
5. Re-running will produce the same image as in step 3.

What should have happened?

The output should be related to the input image, and the image CFG slider should affect the similarity of the input and output images.

Sysinfo

https://pastebin.com/XsmgD15K

What browsers do you use to access the UI ?

Mozilla Firefox

Console logs

################################################################
Launching launch.py...
################################################################
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --medvram --xformers --opt-channelslast --api --cors-allow-origins=http://localhost:7860
[T+00:00:00.000000]: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
[T+00:00:00.154548]: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
[T+00:00:00.781084]: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
*** "Disable all extensions" option was set, will only load built-in extensions ***
Loading weights [a51303fb03] from [REDACTED]/stable-diffusion-webui/models/Stable-diffusion/instruct-pix2pix-00-22000.safetensors
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 7.5s (prepare environment: 1.5s, import torch: 3.3s, import gradio: 0.6s, setup paths: 0.7s, other imports: 0.5s, load scripts: 0.4s, create ui: 0.3s).
Creating model from config: [REDACTED]/stable-diffusion-webui/configs/instruct-pix2pix.yaml
LatentDiffusion: Running in eps-prediction mode
Applying attention optimization: sdp-no-mem... done.
Model loaded in 1.7s (load weights from disk: 0.4s, create model: 0.5s, apply weights to model: 0.4s, calculate empty prompt: 0.2s).
100%|███████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00,  2.88it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00,  2.88it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00,  3.07it/s]

Additional information

No response

[w-e-w]

I have tested this earlier and this seems to be indeed an issue
#12891

[rabbitflyer]

Thanks! Not sure how I didn't see that ticket considering it has almost the exact same name 🤦.
I'll close this and add some extra info to #12891.

[rabbitflyer]

Sorry, reopening since #12891 is an FAQ question and not a bug report. Sorry for the clutter.

[rabbitflyer]

Update: Based on a bisect I just did, the first bad commit to master is 75336df.

[w-e-w]

verified that Instruct Pix2Pix is broken after #12311 is merged

[Eliza-and-black]

verified that Instruct Pix2Pix is broken after #12311 is merged

@w-e-w Do you have plan to fix it? Thx.

[w-e-w]

@Eliza-and-black irrc Auto commit some fixes related to it last week in the dev branch
5e80d9e