Ruby #507

ggerganov · 2023-02-15T17:48:41Z

ggerganov
Feb 15, 2023
Maintainer

General discussion about the ruby bindings for whisper.cpp

Initial merge: #500
CC: @taf2

taf2 · 2023-02-16T15:50:03Z

taf2
Feb 16, 2023

Working on getting a gem published to rubygems so people can just

gem install whispercpp

4 replies

taf2 Feb 25, 2023

Looks like someone already published a gem named whisper... So I think if we go with whispercpp - that would make sense and is available. Will work on pushing a gem and pull request.

KitaitiMakoto Oct 28, 2024

@taf2 Hi, thank you for publishing the gem. I was excited whisper.cpp bindings already exist.

Are you gem owner of whispercpp gem? Could you check updates(#2484, #2506, #2516, #2518) and publish the new release?

taf2 Oct 28, 2024

hey @KitaitiMakoto i believe i did publish the original gem... happy to publish the updates - i should be able to do this this week. Let me pull down the changes and confirm if that's cool and should not be a problem

KitaitiMakoto Oct 28, 2024

Sure! I don't hurry and don't mean to rush you. Thanks for the response.

taf2 · 2023-02-27T14:31:29Z

taf2
Feb 27, 2023

@ggerganov I'm looking for good solution to interrupt whisper_full_parallel. In ruby we can wrap the call to this in an unlock method, since ruby has a global interpreter lock it is ideal to unlock while the c++ code runs allowing ruby to run other threads and even be interrupted via ctrl+c and other interrupt signals. It looks whisper_full_parallel calls whisper_full in each thread or just calls whisper_full, if there was a maybe volatile stop variable we added to the context then we could toggle that to false/0 and cause the while loop to break. thoughts?

3 replies

ggerganov Feb 27, 2023
Maintainer Author

There is currently an existing abort mechanism via the "encoder_begin" callback:

whisper.cpp/whisper.h

Lines 240 to 245 in f420de1

    
           // Encoder begin callback 
        
           // If not NULL, called before the encoder starts 
        
           // If it returns false, the computation is aborted 
        
           typedef bool (*whisper_encoder_begin_callback)(struct whisper_context * ctx, void * user_data);

Would this work?

taf2 Feb 27, 2023

ah perfect - i'll dig into this and rework the ruby bindings to use that shortly.

ggerganov Feb 27, 2023
Maintainer Author

Usage of the abort mechanism is demonstrated here:

whisper.cpp/examples/main/main.cpp

Lines 603 to 616 in b4ebdb6

    
           // example for abort mechanism 
        
           // in this example, we do not abort the processing, but we could if the flag is set to true 
        
           // the callback is called before every encoder run - if it returns false, the processing is aborted 
        
           { 
        
               static bool is_aborted = false; // NOTE: this should be atomic to avoid data race 
        
               wparams.encoder_begin_callback = [](struct whisper_context * /*ctx*/, void * user_data) { 
        
                   bool is_aborted = *(bool*)user_data; 
        
                   return !is_aborted; 
        
               }; 
        
               wparams.encoder_begin_callback_user_data = &is_aborted; 
        
           }

brauliobo · 2024-09-26T22:33:16Z

brauliobo
Sep 26, 2024

am I missing something? It doesn't install here. I have whisper.cpp AUR package installed on the system

braulio @ bhavapower ➜  ruby-whisper.cpp git:(main) ✗  LANG=en gem install whispercpp
Building native extensions. This could take a while...
ERROR:  Error installing whispercpp:
        ERROR: Failed to build gem native extension.

    current directory: /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0/ext
/home/braulio/.rvm/rubies/ruby-3.1.2/bin/ruby -I /home/braulio/.rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0 -r ./siteconf20240926-2567174-n68uhi.rb extconf.rb
cp: cannot stat './../../../whisper.cpp': No such file or directory
cp: cannot stat './../../../whisper.h': No such file or directory
cp: cannot stat './../../../ggml.h': No such file or directory
cp: cannot stat './../../../ggml.c': No such file or directory
cp: cannot stat './../../../ggml-impl.h': No such file or directory
cp: cannot stat './../../../ggml-alloc.h': No such file or directory
cp: cannot stat './../../../ggml-alloc.c': No such file or directory
cp: cannot stat './../../../ggml-backend-impl.h': No such file or directory
cp: cannot stat './../../../ggml-backend.h': No such file or directory
cp: cannot stat './../../../ggml-backend.c': No such file or directory
cp: cannot stat './../../../ggml-common.h': No such file or directory
cp: cannot stat './../../../ggml-quants.h': No such file or directory
cp: cannot stat './../../../ggml-quants.c': No such file or directory
cp: cannot stat './../../../examples/dr_wav.h': No such file or directory
creating Makefile

current directory: /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0/ext
make DESTDIR\= clean

current directory: /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0/ext
make DESTDIR\=
compiling ggml.c
ggml.c:4:10: fatal error: ggml-impl.h: No such file or directory
    4 | #include "ggml-impl.h"
      |          ^~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:246: ggml.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0 for inspection.
Results logged to /home/braulio/.rvm/gems/ruby-3.1.2/extensions/x86_64-linux/3.1.0/whispercpp-1.3.0/gem_make.out

2 replies

oriolgual Oct 14, 2024

I am also getting this error, how did you fix it?

KitaitiMakoto Oct 25, 2024

This should be fixed by #2484

brauliobo · 2024-09-27T02:04:25Z

brauliobo
Sep 27, 2024

just pushed a new gem at https://rubygems.org/gems/whisper.cpp

1 reply

oriolgual Oct 14, 2024

Hi! Can this be installed without CUDA? I'm getting this error:

Cloning whisper.cpp repository...
Cloning into 'whisper.cpp'...
Building libwhispercpp.so with GGML_CUDA=1...
I ccache not found. Consider installing it for faster compilation.
expr: syntax error
I whisper.cpp build info:
I UNAME_S:   Darwin
I UNAME_P:   arm
I UNAME_M:   arm64
I CFLAGS:    -Iggml/include -Iggml/src -Iinclude -Isrc -Iexamples -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -DNDEBUG -DGGML_USE_ACCELERATE -DGGML_USE_BLAS -DACCELERATE_NEW_LAPACK -DACCELERATE_LAPACK_ILP64 -DGGML_USE_CUDA -I/usr/local/cuda/include -I/usr/local/cuda/targets/arm64-linux/include  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -pthread -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion
I CXXFLAGS:  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi -Iggml/include -Iggml/src -Iinclude -Isrc -Iexamples -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -DNDEBUG -DGGML_USE_ACCELERATE -DGGML_USE_BLAS -DACCELERATE_NEW_LAPACK -DACCELERATE_LAPACK_ILP64 -DGGML_USE_CUDA -I/usr/local/cuda/include -I/usr/local/cuda/targets/arm64-linux/include
I NVCCFLAGS: -std=c++11 -O3 -use_fast_math --forward-unknown-to-host-compiler -arch=native -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMV_Y=1 -DK_QUANTS_PER_ITERATION=2 -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128
I LDFLAGS:   -framework Accelerate -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/usr/lib64 -L/usr/local/cuda/targets/arm64-linux/lib -L/usr/local/cuda/lib64/stubs -L/usr/lib/wsl/lib
I CC:        Apple clang version 16.0.0 (clang-1600.0.26.3)
I CXX:       Apple clang version 16.0.0 (clang-1600.0.26.3)
/bin/sh: nvcc: command not found
I NVCC:
/bin/sh: nvcc: command not found
grep: invalid option -- P
usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]]
	[-e pattern] [-f file] [--binary-files=value] [--color=when]
	[--context[=num]] [--directories=action] [--label] [--line-buffered]
	[--null] [pattern] [file ...]
Makefile:855: *** I ERROR: For CUDA versions < 11.7 a target CUDA architecture must be explicitly provided via environment variable CUDA_DOCKER_ARCH, e.g. by running "export CUDA_DOCKER_ARCH=compute_XX" on Unix-like systems, where XX is the minimum compute capability that the code needs to run on. A list with compute capabilities can be found here: https://developer.nvidia.com/cuda-gpus .  Stop.
Failed to build libwhisper.a and libggml.a

extconf failed, exit code 1

brauliobo · 2024-10-14T18:53:30Z

brauliobo
Oct 14, 2024

@oriolgual unfortunately not, currently GGML_CUDA=1 is set on the extconf.rb. I'm not sure it is a good idea to allow the user to set this environment variable on the build.

2 replies

oriolgual Oct 15, 2024

And why is CUDA required?

brauliobo Oct 15, 2024

it is just my use case here. would be ideal if whisper.cpp could be compiled with everything and then cuda is selected on runtime

KitaitiMakoto · 2024-10-15T15:19:57Z

KitaitiMakoto
Oct 15, 2024

2 replies

ggerganov Oct 15, 2024
Maintainer Author

ggml-backend.c is old and should be removed. It was changed to C++ ggml-backend.cpp

KitaitiMakoto Oct 15, 2024

@ggerganov
Oops. I removed my comment because it should be on Issue. I filed my removed comment as an issue: #2483
I'm sorry for removing it.

BTW, thanks for your quick response.

KitaitiMakoto · 2024-10-18T03:37:09Z

KitaitiMakoto
Oct 18, 2024

I would like to make GPU work, especially on my Mac machine (Apple silicon, Metal). But, I need learn many things such as building C/C++, Make, assembly, GPU (CUDA, Metal, ...), GGML and so on. For instance, I cannot understand Makefile to build examples/main/main.cpp. Learning takes time. Could someone work with GPU? Or, show me brief direction of learning?

I found a document telling how to set some build flags: https://docs.ruby-lang.org/en/3.3/extension_rdoc.html#label-Prepare+extconf.rb

3 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ruby #507

{{title}}

Replies: 7 comments 17 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Ruby #507

ggerganov Feb 15, 2023 Maintainer

Replies: 7 comments · 17 replies

ggerganov Feb 27, 2023 Maintainer Author

ggerganov Feb 27, 2023 Maintainer Author

ggerganov Oct 15, 2024 Maintainer Author

ggerganov
Feb 15, 2023
Maintainer

Replies: 7 comments 17 replies

ggerganov Feb 27, 2023
Maintainer Author

ggerganov Feb 27, 2023
Maintainer Author

ggerganov Oct 15, 2024
Maintainer Author