Adding Distil-Whisper - distilled version of Whisper that is 6 times faster, 49% smaller #126
Labels
Comments
|
Coming soon with 1.5.0 |
sandrohanea
added a commit
that referenced
this issue
Nov 25, 2023
- Improved cancellation using CancellationToken provided => it's using the onabort handler in native library to cancel the processing faster
=> addressing concerns from #23
- Logging improvements: not logging to stderr by default anymore, but events can be registered (see [LogProvider](https://github.com/sandrohanea/whisper.net/tree/main/Whisper.net/Logger/LogProvider))
=> Added optional logging in simple samples + CoreML sample
=> Logging is now suppressed by default: #129
- Added `useGpu` on WhisperFactory creation so that GPU usage can be disabled on specific runtimes (e.g. Cublas, CoreML, etc)
- Upgraded dotnet targets to dotnet 8
- Added support for whisper large v3 (including downloader + coreml model)
=> Old model Large in downloader should be renamed to LargeV2 see [GgmlType](https://github.com/sandrohanea/whisper.net/blob/main/Whisper.net/Ggml/GgmlType.cs)
- Added linux-x64 support for Whisper.net.Runtime.Cublas
- Added support for distil-whisper #126
=> No downloader added, you'll need to load the model yourself
=> How to build the model: https://github.com/ggerganov/whisper.cpp/tree/master/models#distilled-models
- **Added automated builds for all runtimes using Github Actions**
- Unified runtime builds with make for linux, macos, ios, tvos, wasm and powershell scripts for windows
- Added Blazor with InteractivityAuto, InteractivityServer and InteractivityWebAssemly sample with Blazor updated in dotnet 8.
- Unified Whisper.net.Runtime.Wasm with Whisper.net.Runtime
=> If previously you were using Whisper.net.Runtime.Wasm, just replace it with Whisper.net.Runtime
- Added metal support for arm-based apple platforms (both Whisper.net.Runtime + Whisper.net.Runtime.CoreML)
Merged
sandrohanea
added a commit
that referenced
this issue
Nov 25, 2023
* - Upgraded to Whisper.cpp 1.5.1
- Improved cancellation using CancellationToken provided => it's using the onabort handler in native library to cancel the processing faster
=> addressing concerns from #23
- Logging improvements: not logging to stderr by default anymore, but events can be registered (see [LogProvider](https://github.com/sandrohanea/whisper.net/tree/main/Whisper.net/Logger/LogProvider))
=> Added optional logging in simple samples + CoreML sample
=> Logging is now suppressed by default: #129
- Added `useGpu` on WhisperFactory creation so that GPU usage can be disabled on specific runtimes (e.g. Cublas, CoreML, etc)
- Upgraded dotnet targets to dotnet 8
- Added support for whisper large v3 (including downloader + coreml model)
=> Old model Large in downloader should be renamed to LargeV2 see [GgmlType](https://github.com/sandrohanea/whisper.net/blob/main/Whisper.net/Ggml/GgmlType.cs)
- Added linux-x64 support for Whisper.net.Runtime.Cublas
- Added support for distil-whisper #126
=> No downloader added, you'll need to load the model yourself
=> How to build the model: https://github.com/ggerganov/whisper.cpp/tree/master/models#distilled-models
- **Added automated builds for all runtimes using Github Actions**
- Unified runtime builds with make for linux, macos, ios, tvos, wasm and powershell scripts for windows
- Added Blazor with InteractivityAuto, InteractivityServer and InteractivityWebAssemly sample with Blazor updated in dotnet 8.
- Unified Whisper.net.Runtime.Wasm with Whisper.net.Runtime
=> If previously you were using Whisper.net.Runtime.Wasm, just replace it with Whisper.net.Runtime
- Added metal support for arm-based apple platforms (both Whisper.net.Runtime + Whisper.net.Runtime.CoreML)
* Commented local_whisper nuget source + reset github actions to execute against PRs
* Fixed linux build + changed native github actions to work with PRs as well, not only branches
* Add native changes wasm
* Add native changes android
* Add native changes windows
* Add native changes macos
* Updated readme
* Add native changes linux
---------
Co-authored-by: sandrohanea <[email protected]>
|
Distil whisper models are supported now, follow: https://github.com/ggerganov/whisper.cpp/tree/master/models#distilled-models to convert them to ggml and use them with whisper.net :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Per the HN link, it would be great to get Distil-Whisper as an option in this repo!
The text was updated successfully, but these errors were encountered: