Library libcublas.so.12 is not found

[Xiaomingpapapa]

upgrade faster-whisper version 0.10.0 -> 1.0.0
and meet the error: Library libcublas.so.12 is not found or cannot be loaded

operation system version: centos_7_9_x64
cuda version: 12.2

anyone else had the same problem?

[tuotuoshao]

I meet the same error. But my cuda version is 11.8 via command nvcc -V. So why faster-whisper use cuda12? I can also get cuda version 12.2 via nvidia-smi, but as i known it's the driver version which means the latest cuda runtime version support.

[jhj0517]

Yea it seems that the latest version of faster-whisper (version 1.0.0) is not compatible with some CUDA versions. I'm using CUDA 11.8 and faced the same problem.
As a temporary fix, you can downgrade faster-whisper to 0.10.1:

pip uninstall faster-whisper
pip install faster-whisper==0.10.1

[minhthuc2502]

From faster-whisper 1.0.0, it requires CUDA 12 as the description here . You need to install CUDA runtime version 12.* . It is no longer compatible with CUDA 11.* .

[Oheed911]

`
#Use an official Python runtime as a parent image
FROM python:3.8

#Add the NVIDIA key for CUDA 12 repositories
RUN curl "https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb" -o cuda.deb &&
dpkg -i cuda.deb && rm cuda.deb

#Install CUDA 12 runtime and development packages along with libcudnn and libcublas
RUN apt-get update -y &&
apt-get install -y --no-install-recommends
cuda-cudart-12-0 cuda-nvcc-12-0 cuda-nvrtc-12-0
libcudnn8 libcublas-12-0 &&
rm -rf /var/lib/apt/lists/*

#Install python3, pip, and ffmpeg
RUN apt-get update && apt-get install -y python3 python3-pip libsndfile1 ffmpeg

#Set the working directory in the container
WORKDIR /app

#Copy the requirements file and the script to the container
COPY ./src /app

#Install Python dependencies
RUN pip3 install --no-cache-dir -r requirements.txt
`

above is my docker file, that resolved this issue.

[grzegorz700]

@minhthuc2502 Please refer to the right place where it mentions the requirements of cuda 12, if it exists. I understand that the release information only added support for Cuda 12. And still, in the README, there is nothing about stopping support for CUDA 11.*

So this version (1.0.x) must be fixed to properly use cuda 11.8...

[minhthuc2502]

You can see it in this release note in Ctranslate2. It needs to be more clear in the notes but it was emphasized that this is the major version of Ctranslate2 because of the breaking change when going up to CUDA 12.*.

Thank you for your remark.

[aryn832]

Update the silly README:

`GPU
GPU execution requires the following NVIDIA libraries to be installed:

cuBLAS for CUDA 11
cuDNN 8 for CUDA 11`

[SeaDude]

To install CUDA 12 and use the latest faster-whisper, is it as simple as pip install nvidia-cublas-cu12 nvidia-cudnn-cu12?

[aryn832]

To install CUDA 12 and use the latest faster-whisper, is it as simple as pip install nvidia-cublas-cu12 nvidia-cudnn-cu12?

it's actually insanely broken because it still requires cuDNN 11 but cuBLAS 12, just do a downgrade as @jhj0517 recommended.

[SeaDude]

I've been able to use the -cu12 suffixes without issue.

[fedirz]

Here's a workaround, which might help for those trying to make it work in Docker.

#516 (comment)

[MalikHaroonKhokhar]

install cuda 12 to resolve this issue : https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Debian&target_version=12&target_type=deb_local