SD Implementation - PyTorch Exception

[sombraguerrero]

In a setup that was previously working under Mini for WSL2 (Ubuntu 20.04), the generate endpoint throws a 500 due to a runtime error from PyTorch as follows:

ERROR:app:Exception on /generate [POST]
Traceback (most recent call last):
  File "/home/rsetter/.local/lib/python3.8/site-packages/flask/app.py", line 2077, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/rsetter/.local/lib/python3.8/site-packages/flask/app.py", line 1525, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/rsetter/.local/lib/python3.8/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/rsetter/.local/lib/python3.8/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/rsetter/.local/lib/python3.8/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/rsetter/.local/lib/python3.8/site-packages/flask_cors/decorator.py", line 128, in wrapped_function
    resp = make_response(f(*args, **kwargs))
  File "app.py", line 33, in generate_images_api
    generated_imgs = stable_diff_model.generate_images(text_prompt, num_images)
  File "/home/rsetter/source/dalle-playground/backend/stable_diffusion_wrapper.py", line 19, in generate_images
    images = self.pipe(prompt, num_inference_steps=10).images
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 34, in decorate_context
    return func(*args, **kwargs)
  File "/home/rsetter/.local/lib/python3.8/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 519, in __call__
    text_embeddings = self._encode_prompt(
  File "/home/rsetter/.local/lib/python3.8/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 299, in _encode_prompt
    text_embeddings = self.text_encoder(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1480, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rsetter/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 726, in forward
    return self.text_model(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1480, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rsetter/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 647, in forward
    encoder_outputs = self.encoder(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1480, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rsetter/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 578, in forward
    layer_outputs = encoder_layer(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1480, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rsetter/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 321, in forward
    hidden_states, attn_weights = self.self_attn(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1480, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rsetter/.local/lib/python3.8/site-packages/transformers/models/clip/modeling_clip.py", line 260, in forward
    attn_output = torch.bmm(attn_probs, value_states)
RuntimeError: expected scalar type Half but found Float