Nomic Vulkan backend#4456

Merged

cebtenzzre merged 155 commits intomasterfrom

ceb/nomic-vulkan

Jan 29, 2024

Commits on Oct 5, 2023

Nomic vulkan backend licensed under the Software for Open Models License (SOM), version 1.0.

niansa
authored and
cebtenzzre
committed
Remove warning which fails on windows.

manyoso
authored and
cebtenzzre
committed
remove dynamic deps from kompute build

apage43
authored and
cebtenzzre
committed
Switch to a dynamic dispatch table instead of linking hard against libvulkan.

manyoso
authored and
cebtenzzre
committed
Completely revamp how we do object management with the vulkan backend and

manyoso
authored and
cebtenzzre
committed
Make kompute actually include external SDK headers when requested

apage43
authored and
cebtenzzre
committed
Throw an exception when allocation fails for vulkan.

manyoso
authored and
cebtenzzre
committed
vulkan: disambiguate gpus with the same name

apage43
authored and
cebtenzzre
committed
Don't try and install kompute artifacts.

manyoso
authored and
cebtenzzre
committed
Sync from device back to host at begin of new prompt.

manyoso
authored and
cebtenzzre
committed
Only use vulkan with known quant that work.

manyoso
authored and
cebtenzzre
committed
Set the singleton to nullptr here.

manyoso
authored and
cebtenzzre
committed
Don't crash on available devices if we can't even create an instance.

manyoso
authored and
cebtenzzre
committed

Commits on Nov 3, 2023

Support for gguf.

manyoso
authored and
cebtenzzre
committed
kompute : don't fail build because of -Warray-bounds
cebtenzzre
committed
Upload immediately to device.

manyoso
authored and
cebtenzzre
committed
Add a common boilerplate code via include and elim copy pasta

manyoso
authored and
cebtenzzre
committed
Consolidate code for mat x vec kernels and use subgroups more extensively.

manyoso
authored and
cebtenzzre
committed
Move the subgroups and printf into common.

manyoso
authored and
cebtenzzre
committed
Minor cleanup.

manyoso
authored and
cebtenzzre
committed
Refactor getrows to use common code and get ready for q6_k.

manyoso
authored and
cebtenzzre
committed
Add q6_k getrows and mul*vec kernel.

manyoso
authored and
cebtenzzre
committed
Fix offset into the qh and now we have working vulkan accelerated for gguff'd llama.

manyoso
authored and
cebtenzzre
committed
Fixes for norm.

manyoso
authored and
cebtenzzre
committed
Fixup the upstream CMakelists.txt so we can build just llama.cpp with our branch.

manyoso
authored and
cebtenzzre
committed
Change this back to be in agreement with metal and our previous softmax kernel.

manyoso
authored and
cebtenzzre
committed
Fixes for subgroup size to bring AMD and NVIDIA inline with eachother for all kernels.

manyoso
authored and
cebtenzzre
committed
kompute : only try to use Vulkan for LLaMA itself
cebtenzzre
committed
kompute : remove Q6_K from list of supported quant types
cebtenzzre
committed
f16 mv broadcasting fix (gqa fix)

apage43
authored and
cebtenzzre
committed
q8 mat*vec

apage43
authored and
cebtenzzre
committed
vulkan: implement neox mode for rope

apage43
authored and
cebtenzzre
committed
falcon h2d + reenable vulkan

apage43
authored and
cebtenzzre
committed
Delete TODO now that we have q8_0.

manyoso
authored and
cebtenzzre
committed
add mat*mat ops

apage43
authored and
cebtenzzre
committed
misc vulkan cleanup

apage43
authored and
cebtenzzre
committed
perf: use bigger threadgroups in mm

apage43
authored and
cebtenzzre
committed
use op param epsilon for norms

apage43
authored and
cebtenzzre
committed
q6k mm works

apage43
authored and
cebtenzzre
committed
rm commented dbg print

apage43
authored and
cebtenzzre
committed
q4_1 mat*mat

apage43
authored and
cebtenzzre
committed
clean up vulkan/cpu switch

apage43
authored and
cebtenzzre
committed
attempted speedups

apage43
authored and
cebtenzzre
committed
attempted speedups 2

apage43
authored and
cebtenzzre
committed
use mat*vec shaders for mat*mat

apage43
authored and
cebtenzzre
committed
kompute : enable kp_logger and make it static (#8)
cebtenzzre
committed
kompute : make scripts executable
cebtenzzre
committed
Don't try an allocation on a heap that is smaller than the size we require.

manyoso
authored and
cebtenzzre
committed
Remove unused push constant that was giving validation errors.

manyoso
authored and
cebtenzzre
committed
Lower the workgroup count for some shaders by providing a loop that processes

manyoso
authored and
cebtenzzre
committed
Fix synchronization problem for AMD Radeon with amdvlk driver or windows

manyoso
authored and
cebtenzzre
committed
vulkan : fix missing break in matmul selection (#9)
cebtenzzre
committed
llama : decide to disable Vulkan before loading tensors (#7)
cebtenzzre
committed
Scale the workgroup count down to allow correct generation for falcon with

manyoso
authored and
cebtenzzre
committed
Revert the prompt processing on gpu for now.

manyoso
authored and
cebtenzzre
committed
Remove this debug code.

manyoso
authored and
cebtenzzre
committed
llama : fix Vulkan whitelist (#11)
cebtenzzre
committed

Commits on Nov 8, 2023

Commits on Nov 23, 2023

Commits on Dec 14, 2023

ggml : restore 'static' specifiers
cebtenzzre
committed

Commits on Dec 15, 2023

Commits on Jan 8, 2024

Commits on Jan 10, 2024

Commits on Jan 24, 2024

Commits on Jan 25, 2024

Commits on Jan 26, 2024

Commits on Jan 27, 2024