model : load relevant arrays with n_layer_all - #28173
Conversation
|
@CISC re "there may be other breakages the other way", I went looking, here is what I found. A file can only carry a trunk-length array if llama.cpp's C++ saver wrote it while So the exposed set is narrow: NextN block, plus per-layer variation, plus C++ saver, all three at once. In tree it is empty. I generated the full set of 111 test models: ten carry at least one of the three arrays, and none of those ten has a NextN block. Nothing in tree regresses from widening the read. Separately, this PR is sufficient for glm5next on its own. I ran my branch with and without my mirror commit on top of it, and with neither applied the MTP draft dies as before, sizing its KV cache from |
|
The assert still blocks the all-nextn case and details are in #28159 |
|
This appears to fix Qwen + MTP loaded from GGUF for the 4B, 9B, and 27B on Vulkan (GFX1151) and CUDA (GA104). |
Overview
cont #28159
The array sizes were effectively correct after all, checked affected models.
Requirements