In this 3-part demonstration we will learn how to use the AWS SDK for Python (Boto3) to create Generative AI images and simple video animations using Amazon Bedrock and Stability.ai Stable Diffusion XL 1.0 model.
- Text-to-image
- Image-to-image
- Video-to-video (video-frame-frame-video)
All instructions for this project can be found in the Jupyter Notebook and on the accompanying YouTube video.
There is second video containing several sample animations available on YouTube.
- Amazon Bedrock: Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon via a single API
- Stable Diffusion XL Model: Stable Diffusion XL or SDXL is the latest image generation model that is tailored towards more photorealistic outputs with more detailed imagery and composition compared to previous SD models, including SD 2.1. Stable Diffusion XL is a significant advancement in image generation capabilities, offering enhanced image composition and face generation that results in stunning visuals and realistic aesthetics.
- Sample generated text-to-image images
- Sample generated image-to-image images
- Sample generated video frames
- Sample generated animated GIFs
|
|
|
|
|
|
 Original Image |
 Variation #1 |
 Variation #2 |
 Variation #3 |
Original MP4 video files, created with FFPmpeg, converted to scaled down animated GIF files.
 Selfie 1 v1 |
 Selfie 1 v2 |
 Selfie 3 v1 |
 Male 2 |
 Female 1 v1 |
 Female 2 v1 |
 Female 3 v2 |
 Owl |
 Male 1 |
 Squirrel |
 Dancer |
 Snowy Day |
This demonstration was very basic. If you are interested in state-of-the-art Generative AI image and video generation techniques, I would suggest looking at a combination of the AnimateDiff, ControlNet, IP-Adapter, and A1111 or ComfyUI, along with the use of custom fine-tuned SDXL models. Most high-quality animations are created with multiple fine-tuned SDXL LoRAs (Low-Rank Adaptation of Large Language Models). These tools can all be used on AWS using a GPU-based Amazon WorkSpaces environment.
- Automatic1111 (aka A1111) Stable Diffusion web UI
- ComfyUI Stable Diffusion web UI
- AnimateDiff
- ControlNet
- IP-Adapter
- AnimateDiff for Stable Diffusion WebU (A1111)
- ControlNet for Stable Diffusion WebUI (A1111)
- Fine-tuned Stable Diffusion XL LoRAs
- Hugging Face: LoRA the Explorer
If you are looking for examples of state-of-the-art Generative AI-based animations, check out Civitai. WARNING! EXTREMELY INAPPROPRIATE CONTENT!
|
These are simple examples of ComfyUI workflows from Reddit's ComfyUI and Stable Diffusion communities, used to create high-quality generative images. Many workflows are even more complex than the two shown below.
 ComfyUI Example 1 |
 ComfyUI Example 2 |