Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OnnxRuntime Support for Text2Video and Img2Video Pipelines #2168

Open
jdp8 opened this issue Jan 25, 2025 · 1 comment
Open

OnnxRuntime Support for Text2Video and Img2Video Pipelines #2168

jdp8 opened this issue Jan 25, 2025 · 1 comment
Labels
diffusers onnxruntime Related to ONNX Runtime

Comments

@jdp8
Copy link

jdp8 commented Jan 25, 2025

Feature request

There are OnnxRuntime pipelines for various models such as ORTStableDiffusionInpaintPipeline,ORTStableDiffusionPipeline, etc. but there are no pipelines that I know of for Text-To-Video and Image-To-Video. It would be nice if these pipelines were implemented so that they can be ran in OnnxRuntime and the models themselves converted to ONNX.

Motivation

I have Text-To-Video and Image-To-Video models converted to ONNX and would like to use them in OnnxRuntime the same way that Text-To-Image, Inpainting and other models are used.

Your contribution

I can submit a PR or help in any way with the proper guidance.

@IlyasMoutawwakil
Copy link
Member

I would love to review a PR adding support for text to video diffusion models.
Here's an example of adding flux and sd3 to ORTPipeline https://github.com/huggingface/optimum/pull/2073/files,

@IlyasMoutawwakil IlyasMoutawwakil added onnxruntime Related to ONNX Runtime diffusers labels Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
diffusers onnxruntime Related to ONNX Runtime
Projects
None yet
Development

No branches or pull requests

2 participants