Prerequisites
Feature Description
Hi,
seeing https://github.com/ggerganov/llama.cpp/releases provides a variety of windows prebuilt binary variants with CUDA, SYCL, vulkan etc..
like the latest vulkan enabled one:
https://github.com/ggerganov/llama.cpp/releases/download/b4282/llama-b4282-bin-win-vulkan-x64.zip
others like AVX512:
llama-b4282-bin-win-avx512-x64.zip
sadly no variant for Ubuntu is provided only:
llama-b4282-bin-ubuntu-x64.zip
which I downloaded and shows:
./llama-cli --list-devices
Available devices:
Motivation
Vulkan seems for Linux right now is the most performant crossvendor variant specially with cooperative matrix support merged:
#10597
seems sad no easy testing and to have to build from source..
ideally could request more: CUDA, and HIP/RocM builds could be provided.. and even more than x64 buils also ubuntu arm64 builds..
Possible Implementation
No response
👍 React with 👍 2KhazAkar and SapoKR