Flax SDXL img2img pipeline

[pcuenca]

WIP. Inference works but results don't seem correct.

Continuation of #4254

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[PawKanarek]

Hi @pcuenca, I know this is still a work in progress, but nevertheless I wanted to check how the Flax Refiner for SDXL is doing and I made interference on google tpu v3-8 using yours FlaxStableDiffusionXLImg2ImgPipeline from this commit a7a15aa on branch sdxl-img2img-flax and here are results for comparison:

1. base vs refiner
2. base vs refiner
3. base vs refiner
4. base vs refiner
5. base vs refiner
6. base vs refiner
7. base vs refiner
8. base vs refiner

Code for generation
Model parameters:

PROMPT = "A dinosaur in the forest with a backpack blows the candles on a birthday cake"
NEGATIVE_PROMPT = "blurry, low res, low quality"
STEPS = 40
SEED = 420
GUIDANCE_SCALE = 5.0

I'm wondering if this weird refiner effect is happening because there is no denosing_start and denoising_end parameters in FlaxStableDiffusionXLImg2ImgPipeline and FlaxStableDiffusionXLPipeline?

Thanks for all hard work!

[moraviamoti]

Hi @pcuenca, I would be happy to learn what is this PR's status? Will you finish building the flax pipeline for sdxl img2img? And are you going to do sdxl inpaint flax pipeline as well?
Thank you for your amazing work!