models : fix dflash wo_a reshape on load - #26577
Merged
Merged
Conversation
Nathanw1014
added a commit
to Nathanw1014/llama.cpp
that referenced
this pull request
Aug 9, 2026
TENSOR_ALLOW_RESHAPE (ggml-org#26531) recomputes nb from the caller's shape as nb[dim] = dim == 0 ? ggml_type_size(type) : ne[dim-1]*nb[dim-1] so nb[1] comes out as ne[0]*nb[0]. A row of a block-quantised tensor is ne[0]/blck_size blocks, not ne[0] elements, so for q8_0 with ne[0]=4096 the stride is 4096*34 = 139264 instead of 128*34 = 4352. F16/F32 are unaffected because their block size is 1, which is why this went unnoticed. The malformed meta is what buft_supported() probes the backend with, so the weight looks non-contiguous: ggml_vk_dim01_contiguous() is false, the Vulkan supports_op refuses the mul_mat (quantised src0 has no F32/F16/BF16 bypass), and the loader falls back to a CPU buffer for that weight. Reachable since ggml-org#26577 gave dflash and deepseek4 a 3d wo_a. On DeepSeek-V4-Flash UD-IQ3_XXS + DSpark, gfx1151/RADV, that put attn_wo_a on the CPU backend for all 43 attention layers, moved 1462 MiB of weights off the GPU (Vulkan0 model buffer 6624.99 -> 5162.99 MiB) and took decode splits from 88 to 174, which is a large host-side cost per token with the GPU idle. With this patch the load-time probe sees nb=[34,4352,...], the demotion is gone, splits are back to 88 (bs=1) and 2 for the draft context, and generation is unchanged. The same expression in the dummy-tensor path above is corrected too; it is latent there, not reached by this path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
satindergrewal
pushed a commit
to satindergrewal/llama.cpp
that referenced
this pull request
Aug 12, 2026
brittlewis12
pushed a commit
to brittlewis12/llama.cpp
that referenced
this pull request
Aug 17, 2026
thecodacus
pushed a commit
to thecodacus/llama.cpp
that referenced
this pull request
Sep 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
fix #26576
cont #26531
Missed a reshape in the dflash implementation.
Requirements