Skip to content

vulkan: Pad Q3_K/Q6_K tensors out to 32-bit alignment #22951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
TheBlueMatt wants to merge 9 commits into
base: master
Choose a base branch
from
+345 −265
Draft

vulkan: Pad Q3_K/Q6_K tensors out to 32-bit alignment #22951

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8c9e3f2
vulkan: Use MMVQ for Q3_K/Q6_K
TheBlueMatt May 10, 2026
5fef012
vulkan: Block-load Q3_K/Q6_K block data and subtract on 32b ints
TheBlueMatt May 11, 2026
72ed4a2
vulkan: Swap `vk_tensor_offset() + view_offs` for a single function
TheBlueMatt May 10, 2026
36aa093
vulkan: Drop ggml_vk_buffer_write_nc_async
TheBlueMatt May 10, 2026
fd49560
vulkan: Switch to a vulkan-specific type size function
TheBlueMatt May 10, 2026
b6cf814
vulkan: Replace tensor size lookups with a vulkan-specific func
TheBlueMatt May 10, 2026
a9da6df
vulkan: support copying tensors with extra padding between blocks
TheBlueMatt May 11, 2026
96c097a
vulkan: Pad Q3_K and Q6_K to 4-byte-aligned
TheBlueMatt May 11, 2026
f717e22
vulkan: Rewrite MMVQ Q3_K/Q6_K block loads to load through 4xu32
TheBlueMatt May 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading