Fix dtype error for StableDiffusionXL#9217

sayakpaul merged 2 commits intohuggingface:mainhuggingface/diffusers:mainfrom

leisuzz:mainleisuzz/diffusers:mainCopy head branch name to clipboard