docs(tasks): the regression comment to post on ggml-org/llama.cpp#27044 - #120
Merged
Conversation
Moved out of a session scratchpad, which is not reachable from the working
tree. Ready to paste as a comment on the upstream PR.
Records the regression attribution: introduced by 6eddde06a4f2 ("CUDA: refactor
MMQ kernel configuration", #24127) at build b9992, which swapped a
compute-capability constant for get_J_max(..., ne11) in both allocation
branches. Correct for !ids where ne11 is the row count; wrong for ids where it
is ne12*n_expert_used.
Includes the runtime boundary measured on both sides -- b9888 (ollama 0.32.1)
clean in 3/3 cold runs, b10069 (ollama 0.32.2) faulting, b10353 faulting -- so
"first affected release" is a measurement rather than an inference from source.
Not posted. Per ggml-org's contribution guidelines the post has to be written or
at least owned by the person submitting it, so this is material to paste and
edit, not text to relay verbatim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The comment argued the defect from one machine. The table shows it is not architecture specific at all, which matters more for this report than any of the sm_120 evidence. get_J_max never reads cc before the "ret -= ret % 8" step, so ne11 == 1 reduces to zero padding on every architecture. Measured by compiling the real mmq.cuh and evaluating it on the host -- no GPU needed, since get_J_max and get_config are pure functions of (type, J, fallback, cc). Zero on all 14 parts from Pascal to Blackwell and Vega to CDNA3. The degenerate range reaches ne11 = 15 on RDNA3/3.5/4 and CDNA2/3 rather than 7, because no valid config exists at J = 8 there either. This is the part worth putting in front of maintainers: #24399 and #18331 were both closed as Blackwell codegen problems. A defect that reproduces identically on fourteen architectures, including every AMD part, is not codegen. Reachability is stated separately and marked as read rather than run -- should_use_mmq queries smpbo and needs a device. Everything traced reaches the branch except Volta, which drops off the MMQ path entirely at this batch size once fp16 MMA is available. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moved out of a session scratchpad, which isn't reachable from the working tree, so it can actually be opened and copied.
Records the regression attribution for ggml-org/llama.cpp#27044: introduced by
6eddde06a4f2("CUDA: refactor MMQ kernel configuration", #24127) at build b9992, which swapped a compute-capability constant forget_J_max(..., ne11)in both allocation branches — correct for!idswherene11is the row count, wrong foridswhere it'sne12*n_expert_used.Includes the runtime boundary measured on both sides — b9888 (ollama 0.32.1) clean in 3/3 cold runs, b10069 (ollama 0.32.2) faulting, b10353 faulting — so "first affected release" is a measurement rather than an inference.
And the per-architecture table, which is the part most likely to matter upstream:
get_J_maxreturns 0 forne11 = 1on all 14 architectures, Pascal→Blackwell and Vega→CDNA3, evaluated host-side with no GPU. #24399 and #18331 were both closed as Blackwell codegen problems; a defect that reproduces identically on fourteen architectures including every AMD part is not codegen. Reachability is stated separately and marked as read-not-run, sinceshould_use_mmqqueriessmpboand needs a device.Not posted. Per ggml-org's contribution guidelines the post has to be owned by the person submitting it — this is material to paste and edit, not text to relay verbatim.
Docs only.
🤖 Generated with Claude Code