is there an option to set it to run in the gpu? #41
Labels
question
Further information is requested
Comments
|
The models are loaded via
This is currently a working item for them for 2024. It is still in development. They did add some early code but it isn't fully deployed yet. Also, they don't support models large than 2.0GB on Once they add GPU support, there will be GPU support added, both for Whisper recognition and possibly for synthesis with the VITS models, and maybe for other features like speech language recognition. Same for large model support. If you are using Echogarden mostly for speech recognition, just know that it isn't actually its strongest area (strongest is likely alignment). There are faster implementations of OpenAI Whisper models, like whisper.cpp, that support NVIDIA GPUs and are otherwise significantly faster on CPU due to quantization and other optimizations they use. |
|
I got it. Thanks for your detailded explanation |
|
GPU support (DirectML and CUDA ONNX providers, and GPU build support for |
|
@rotemdan i'm following the docs here: I have successfully build the lib for usage with dml: But when I try to use it dml: it gives a warning: Running with InferenceSession.create with verbose return: Looks like DmlExecutionProvider was enable but the lib fallback to CPU execution provider What could be wrong here? |
|
The build I use in
I haven't actually tried to make custom builds of I don't know which model you're trying to load, and whether you're doing it with Echogarden. Anyway, the info message mentions "Having memory pattern enabled is not supported while using DmlExecutionProvider. So disabling it for this session since it uses DmlExecutionProvider." It doesn't seem to say it falls back to CPU (I'm not sure). I don't know exactly what "memory pattern" means here. In Echogarden I suppress these messages by setting a higher |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
when I run audio-to-txt using api, it always run on my CPU and my gpu is free, I want to set it run on my gpu to improve running speed.
The text was updated successfully, but these errors were encountered: