Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Get CUDA available memory at runtime and skip loading models in order of priority #31

Open
kristiankielhofner opened this issue Apr 6, 2023 · 0 comments
Open

Get CUDA available memory at runtime and skip loading models in order of priority #31

kristiankielhofner opened this issue Apr 6, 2023 · 0 comments

Comments

@kristiankielhofner
Copy link
Contributor

For cards with insufficient memory we should probably have some basic heuristics to skip loading models that don't fit or whatever, possibly by user configured priority. Just skipping TTS would allow all Whisper models on 3GB cards, which should really be the minimum (GTX 1060 3GB).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kristiankielhofner