tests: initialise non-contiguous tensors row by row - #276
Merged
Conversation
init_tensor_uniform wrote quantized data with a single packed
ggml_backend_tensor_set. That function copies size bytes contiguously from
tensor->data and never strides by nb[1], so for a view whose rows are spaced
over a wider base the data lands at i*row_size instead of i*nb[1].
For the TQ4_1S k_v=1600 case that meant 245,760 bytes written into a view whose
logical extent is 255,960 with a 1000-byte stride, leaving the last rows holding
whatever the buffer previously contained. Observed directly:
[COPYDBG] tensor_set name=a ne=[1536,256] nb1=1000 offset=0
size=245760 nbytes=255960 view_src=yes
The reference then read those rows and produced NaN, which presented as the
CUDA backend failing because CPU is the reference and is skipped as a backend
under test.
Write row by row when the tensor is not contiguous. Credit to apollo-mg, whose
byte arithmetic in #268 predicted both the NaN count and the first affected row
from the geometry alone, which is what identified the packed write as the
culprit before it was observed.
Note this does not make the case pass: with correctly strided data the failure
reverses, CUDA now returning NaN against a finite reference. That is a separate
and genuine defect in the CUDA TQ4_1S path with non-contiguous src0, which this
bug was masking. Tracked in #268.
This was referenced Aug 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the harness half of #268.
init_tensor_uniformwrote quantized data with one packedggml_backend_tensor_set. That function copiessizebytes contiguously fromtensor->dataand never strides bynb[1], so for a view whose rows are spaced over a wider base the data lands ati*row_sizeinstead ofi*nb[1].Observed directly for the TQ4_1S
k_v=1600case:245,760 bytes into a 255,960-byte logical extent, leaving the final rows holding prior buffer contents. The reference then read them and produced NaN, which presented as CUDA failing because CPU is the reference and is skipped as a backend under test.
Credit to @apollo-mg, whose byte arithmetic in #268 predicted both the NaN population (2,816) and the first affected row (245) from the geometry alone, and whose observation that a generic
nbytes-based copy could not produce that signature is what ruled out the copy path and pointed at the initialiser.Effect
tq3_1satk_v=1600now reports OK; it previously had no meaningful coverage at this shape.tq4_1scase still fails, but the failure reverses: CPU is finite and CUDA now returns NaN, consistently across runs. That is a separate genuine defect in the CUDA TQ4_1S path with non-contiguoussrc0which this bug was masking. CUDA TQ4_1S MUL_MAT returns NaN with non-contiguous src0 (k_v > k) #268 stays open for it.Verified on GB10 (sm_121): MUL_MAT 1492/1493, no other case affected.