ggml : fix view init skipped after buft max_size split

[Yoshi4470]

When ggml_backend_alloc_ctx_tensors_from_buft splits allocation on buft max_size, a view-only tail at the end of the context could skip the final alloc_tensor_range. Persistent views (e.g. KV k_stream / v_stream) were then left without ggml_backend_view_init.
Allocate only parent tensors in alloc_tensor_range and initialize all views in a final pass over the context after all splits complete.

Overview

ggml_backend_alloc_ctx_tensors_from_buft can split a context across multiple buffers when a tensor would exceed the backend buffer-type max_size (for example Vulkan's default 1 GiB). If the remaining tail of the context contains only views, the final alloc_tensor_range may be skipped, so those views never get ggml_backend_view_init.
Persistent KV stream views (layer.k_stream / v_stream) then keep tensor->data == NULL while view_src is allocated. Host state save/restore later calls ggml_backend_tensor_get / set on those views and hits:
GGML_ASSERT(tensor->data != NULL && "tensor not allocated")
This change:

• allocates only parent tensors inside each split range
• initializes all uninitialized views in one final pass over the context
  Also adds test_view_init_after_max_size_split in tests/test-alloc.cpp
  to cover the oversized-parent + view-only-tail case.

Additional information

Fixes #19839
Fixes #23737
Fixes #21762

Related:

• kv-cache : fix crash in state save/restore #21576 (state-IO !tensor->data guards; complementary discussion)
  I reproduced the assert with Qwen3.6-27B (MTP) when a parent KV tensor exceeded buft max_size and the context ended with a view-only tail.
  Note on kv-cache : fix crash in state save/restore #21576: skipping tensors with !tensor->data avoids the assert, but can omit real KV still reachable via view_src and misalign the serialized state. This PR fixes the missing view init in the allocator instead.
  Prior investigation commit (same patch on older master; kept for reference):
  Yoshi4470@99655d5

Test plan

• test-alloc (includes new test_view_init_after_max_size_split)
• Vulkan A/B on the Qwen3.6-27B MTP speculative path (same command, 10 runs each):
  • before (e3546c794): 0/10 (assert every time)
  • after (this PR): 10/10 (no assert, completed normally)
• Earlier amdvlk64.dll crash after the assert site was resolved by an AMD driver update; treated as separate from this allocator fix
• N/A: equivalent-size CPU repro (not practical on this machine)

Command used for the A/B runs:

• Vulkan / AMDVLK, Windows, AMD Software 26.6.4
• AMD Ryzen AI Max+ 395 with Radeon 8060S (gfx1151)

llama-cli.exe -np 8 -c 2097152 --no-mmap -m models/Qwen3.6-27B-UD-Q4_K_XL.gguf \
-ctk q4_0 -ctv q4_0 -ctkd q4_0 -ctvd q4_0 \
--spec-type ngram-mod,draft-mtp --spec-draft-n-max 6 --spec-draft-p-min 0.75 -p "Hello!"

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - Cursor assisted with implementing the finalize-views pass and regression test after I identified the root cause in the alloc split path, and with local verification.

[ehotting]

Disclosure: I used Claude Code to run these experiments on my machine and to write about the technical results in a useful way. The builds, the runs and the numbers are real and reproducible on the hardware described; the analysis and the wording are AI assisted. Happy to re-run anything, vary any parameter, or share the raw logs.

Tested this on the hardware and model that #21576 was opened for.

TL;DR: the assert is gone and the saved state is complete again

Setup

• This PR (3f82bd2bb) against its own base e3546c794 (b9976). Both built from the same checkout, same flags, same day.
• Vulkan / RADV (Mesa 26.1.4), AMD Ryzen AI Max+ 395 with Radeon 8060S (gfx1151), 96 GB VRAM carveout
• gemma-4-26B-A4B-it Q8_0, sha256 7778df567283364f98693690c5b45ebba63eb5658e6e20f2546acce6e8b52e7c

llama-server -m gemma-4-26B-A4B-it-Q8_0.gguf \
  -c <ctx> -np 4 -ngl 999 --no-mmap --flash-attn on \
  --cache-type-k f16 --cache-type-v f16 --slot-save-path /tmp/kvslots

37-token prompt into slot 0, then POST /slots/0?action=save, then ?action=restore.

Results

build	-c	GGML_VK_SUBALLOCATION_BLOCK_SIZE	KV tensor split	save	bytes written
base e3546c794	262144	default (1 GiB)	no	ok	8,337,184
base e3546c794	1048576	default	yes	abort	–
base e3546c794	262144	256 MiB	yes	abort	–
#21576 (state-IO guards) *	1048576	default	yes	ok	8,185,608
this PR	262144	default	no	ok	8,337,184
this PR	1048576	default	yes	ok	8,337,184
this PR	262144	256 MiB	yes	ok	8,337,184

Both aborts are GGML_ASSERT(tensor->data != NULL && "tensor not allocated") at ggml/src/ggml-backend.cpp:348, reached from llama_kv_cache_iswa::state_write → state_write_data → ggml_backend_tensor_get.

* that row is from my earlier run reported in #21576, not rebuilt today; every other row comes from this one checkout.

The byte count: 8,337,184 is what the non-splitting path writes. This PR reproduces that number exactly in both splitting configurations, so whether the allocation splits no longer changes what ends up in the state. That is the invariant that was broken.

The guards in #21576 write 151,576 bytes less. That is exactly one layer's K plus V for 37 cells (2 × 37 × 1024 × 2 B, plus 24 B of header): the last layer is silently dropped, which is what @liminfei-amd predicted from fault injection. So this PR does not merely remove the assert, it restores the data the guards were throwing away.

Restore round-trips in every passing case: n_restored = 37, n_read = 8,337,184.

test-alloc passes, including the new test_view_init_after_max_size_split.

On the amdvlk64.dll crash

I see no crash after the save on RADV/Mesa on Linux. The server keeps serving and a subsequent restore works. That is a different driver stack from yours, so it is only a datapoint, but it does suggest the crash you still see afterwards is not part of this allocator bug.

Thanks for chasing this to the root. I will close #21576 once this lands.

[Yoshi4470]

Follow-up on local A/B testing after an AMD driver update cleared the earlier amdvlk64.dll crash (separate from this allocator fix).

Same command, 10 runs each on Vulkan:

• before (e3546c794): 0/10 — GGML_ASSERT(tensor->data != NULL) every time
• after (this PR): 10/10 — no assert, runs completed normally

llama-cli.exe -np 8 -c 2097152 --no-mmap -m models/Qwen3.6-27B-UD-Q4_K_XL.gguf \
-ctk q4_0 -ctv q4_0 -ctkd q4_0 -ctvd q4_0 \
--spec-type ngram-mod,draft-mtp --spec-draft-n-max 6 --spec-draft-p-min 0.75 -p "Hello!"

I also updated the PR Test plan accordingly. Thanks again to @ehotting for the Gemma / slot save-restore validation.

[Yoshi4470]

Additional verification for both #23737 and #19839 on the same hardware.

Setup

• Vulkan / AMDVLK, Windows, AMD Software 26.6.4
• AMD Ryzen AI Max+ 395 with Radeon 8060S (gfx1151)
• Builds: before = e3546c794 (PR base); after = this PR

---

1) #23737 (speculative MTP / llama-cli)

Command (issue-style):

llama-cli -m Qwen3.5-4B-Q4_K_M.gguf --spec-default --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00
-fa 1 --no-mmap --spec-type draft-mtp --spec-draft-n-max 5 -ctk q5_1 -ctv q5_1 -c 1048576
-n 16 -st -p "python function for celcius to fahrenheit"

Causal check via GGML_VK_SUBALLOCATION_BLOCK_SIZE (before build only, plus after under forced split)

build	BLOCK_SIZE	result
before	default (1 GiB)	ASSERT
before	16 GiB (avoid split)	OK
before	256 MiB (force split)	ASSERT
after	256 MiB (force split)	OK

---

2) #19839 (server multi-slot / prompt-cache path)

does not fit my machine spec: Vulkan ErrorOutOfDeviceMemory during context init.
Minimal repro that fits:

GGML_VK_SUBALLOCATION_BLOCK_SIZE=268435456 llama-server -m gpt-oss-120b-mxfp4-00001-of-00003.gguf
-c 1048576 --parallel 4 -ngl 999 --no-mmap
-sps 0 --cache-prompt --cache-idle-slots --cache-ram 8192
--host 127.0.0.1 --port 9091

Then sequential /v1/chat/completions with distinct prompts so LRU rotates slots (-sps 0).

build	result
before	req 1 OK; on req 2 (slot 3 -> 2) ASSERT
after	reqs 1–6 OK

---

Takeaway

Both issues hit the same assert class: host state IO over KV stream views left without ggml_backend_view_init after a buft max_size split. #23737 enters via speculative MTP checkpointing; #19839 via server multi-slot / idle-slot cache save. This PR fixes that missing view init in the allocator.

[Yoshi4470]

Additional verification for #21762 on the same hardware.

Setup

• Vulkan / AMDVLK, Windows, AMD Software 26.6.4
• AMD Ryzen AI Max+ 395 with Radeon 8060S (gfx1151)
• Builds: before = e3546c794 (PR base); after = this PR
• Model: NVIDIA-Nemotron3-Nano-4B-Q4_K_M.gguf

---

#21762 (server prompt-cache / 2nd request)

llama-server.exe --host 0.0.0.0 --port 9090 -m NVIDIA-Nemotron3-Nano-4B-Q4_K_M.gguf --no-mmap --cache-prompt --cache-ram 8192 -ngl 999

Then two independent /v1/chat/completions prompts (not a chat continuation):

1. "Hello request 1. Reply with one word: ok"
2. "Hello request 2. Reply with one word: ok"

build	2nd request	result
before	f_keep = 0.414	ASSERT
after	f_keep = 0.414	OK

Crash is GGML_ASSERT(tensor->data != NULL && "tensor not allocated") at ggml/src/ggml-backend.cpp:348, reached when f_keep < 0.5 so the server runs prompt_save / llama_state_seq_get_data_ext over KV stream views.

With f_keep >= 0.5, prompt-cache save is skipped, so the broken build does not hit this path.

---

Takeaway

Same missing view_init after buft max_size split; #21762 hits it via prompt-cache save on the 2nd request (f_keep < 0.5).

Fixes #21762

[Biggles10-claude]

Confirmed on b10352 / RADV gfx1151 — this PR is the right fix

Adopted this patch locally on 4dee52f82 (b10352). Same assert class as the PR description.

Hardware: AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), RADV, Mesa 25.2.8, Vulkan. Unified memory, GTT pool 124 GiB.

Model: Qwen3.8-27B UD-Q8_K_XL + F16 mmproj, arch qwen35 hybrid (64 layers, full_attention_interval=4 → 16 KV layers, n_kv=4, k/v=256, f16 KV). MTP draft on (--spec-type draft-mtp --spec-draft-n-max 2).

Reproduce (stock b10352, no patch) — llama-server only

No extra proxy in front. The abort is in llama-server itself.

llama-server \
  --ctx-size 819200 --parallel 4 \
  --cache-type-k f16 --cache-type-v f16 \
  --cache-prompt --cache-reuse 256 --cache-ram 40960 --slot-save-path <dir> \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --flash-attn on -ngl 999 -ub 64

Same flags with --ctx-size 655360 (4×163840) also abort. 4×131072 with the same cache flags serves.

shape	idle GTT	cache-on (stock b10352)
4×131072	67.4 / 124 GiB	serves
4×163840	—	ABORT first request, create_checkpoint → update_dft (MTP draft)
4×204800	86.5 / 124 GiB	ABORT second slot, prompt_save (also with MTP off)

Slot 0 can answer. The next host-copy of slot state kills the process in ~1 s (not DeviceLost, not the 60 s amdgpu watchdog):

ggml/src/ggml-backend.cpp
GGML_ASSERT(tensor->data != NULL && "tensor not allocated")

#3  ggml_backend_tensor_get
#4  llama_io_write_host::~llama_io_write_host
#5  … → server prompt_save
     or create_checkpoint → update_dft

--cache-ram 0 (and no --slot-save-path) avoids the path: prompt cache is never instantiated, so idle prompt_save never runs. 4×204800 then serves. That is a workaround, not a fix — it throws away cache.

This is not OOM. 4×204800 allocates and idles at 86.5 / 124 GiB. RADV maxMemoryAllocationSize / maxBufferSize are 4 GiB−4; per-layer K or V is 256–400 MiB. The trip is tensor->data == NULL, not an alloc-size error.

Why data == NULL

Not an unallocated hybrid layer. llama_memory_recurrent::state_write_data already skips r_l[il] == nullptr. The objects that fault are persistent KV stream views (k_stream / v_stream) whose parent is allocated.

In ggml/src/ggml-alloc.c, ggml_backend_alloc_ctx_tensors_from_buft_impl:

• Parent tensors contribute this_size; views contribute 0.
• Vulkan get_max_size is GGML_VK_SUBALLOCATION_BLOCK_SIZE (default 1 GiB).
• A parent with nbytes > max_size is allowed as the first tensor of a new buffer (cur_buf_size == 0), so cur_buf_size becomes greater than max_size.
• The next view has this_size == 0 and (cur_buf_size + 0) > max_size → split.
• The new range starts at that view with cur_buf_size = 0. After the loop, if (cur_buf_size > 0) is false, so alloc_tensor_range never runs on the tail.
• Those views keep buffer == NULL / data == NULL. Later llama_io_write_host calls ggml_backend_tensor_get and asserts.

The split uses >, not >=. For this layout (4 streams × n_kv=4 × 256 × f16) one parent K (or V) tensor is exactly 1024 MiB at 131072 and 1280 MiB at 163840. At exactly 1 GiB the views stay in the parent range and view_init runs. One cell past that, the view-only tail is skipped. That is the 131k / 160k boundary.

Draft MTP at 4×131072 is 2048 MiB = 2 × 1024 (K+V) — same exact-1-GiB fingerprint. At 4×163840 the draft parent is 2560 MiB and update_dft is the first stack we hit. At 4×204800 with MTP off, main prompt_save still dies the same way (1600 MiB parents).

We did not take #21576 (!tensor->data skip on write). That omits a real KV layer still reachable via view_src and misaligns saved state (ehotting’s 151,576 B hole on Gemma in this thread).

After this PR on 4dee52f82

• tests/test-alloc.cpp including test_view_init_after_max_size_split passed
• 4×204800 with the cache flags above: all four slots answered distinct pinned /completions; /props total_slots=4 n_ctx=204800; idle GTT 86.516 → 86.556 / 124; process stayed Sl; 0 asserts
• Slot save/restore: n_saved == n_restored, n_written == n_read == 159123820 (complete buffer, not a dropped layer); overwrite moved n_prompt_tokens 34→37, restore put it back to 34
• A 3-turn chat against llama-server with a shared prefix: cache_read_input_tokens 0 / 25 / 52

Please merge. We will keep this as a local patch on b10352 until it lands.

Patch applied (identical to this PR)

diff --git a/ggml/src/ggml-alloc.c b/ggml/src/ggml-alloc.c
index 3bda9abbe..d6165299d 100644
--- a/ggml/src/ggml-alloc.c
+++ b/ggml/src/ggml-alloc.c
@@ -1123,6 +1123,20 @@ static void free_buffers(ggml_backend_buffer_t ** buffers, const size_t * n_buff
     free(*buffers);
 }
 
+static bool alloc_ctx_tensors_finalize_views(struct ggml_context * ctx) {
+    for (struct ggml_tensor * t = ggml_get_first_tensor(ctx); t != NULL; t = ggml_get_next_tensor(ctx, t)) {
+        if (t->view_src != NULL && t->buffer == NULL) {
+            enum ggml_status status = ggml_backend_view_init(t);
+            if (status != GGML_STATUS_SUCCESS) {
+                GGML_LOG_ERROR("%s: failed to initialize view tensor %s\n", __func__, t->name);
+                return false;
+            }
+        }
+    }
+
+    return true;
+}
+
 static bool alloc_tensor_range(struct ggml_context * ctx,
         struct ggml_tensor * first, struct ggml_tensor * last,
         ggml_backend_buffer_type_t buft, size_t size,
@@ -1141,24 +1155,14 @@ static bool alloc_tensor_range(struct ggml_context * ctx,
     struct ggml_tallocr tallocr = ggml_tallocr_new(buffer);
 
     for (struct ggml_tensor * t = first; t != last; t = ggml_get_next_tensor(ctx, t)) {
-        enum ggml_status status = GGML_STATUS_SUCCESS;
-        if (t->data == NULL) {
-            if (t->view_src == NULL) {
-                status = ggml_tallocr_alloc(&tallocr, t);
-            } else if (t->buffer == NULL) {
-                status = ggml_backend_view_init(t);
-            }
-        } else {
-            if (t->view_src != NULL && t->buffer == NULL) {
-                // view of a pre-allocated tensor
-                status = ggml_backend_view_init(t);
+        if (t->view_src == NULL && t->data == NULL) {
+            enum ggml_status status = ggml_tallocr_alloc(&tallocr, t);
+            if (status != GGML_STATUS_SUCCESS) {
+                GGML_LOG_ERROR("%s: failed to allocate tensor %s\n", __func__, t->name);
+                free_buffers(buffers, n_buffers);
+                return false;
             }
         }
-        if (status != GGML_STATUS_SUCCESS) {
-            GGML_LOG_ERROR("%s: failed to initialize tensor %s\n", __func__, t->name);
-            free_buffers(buffers, n_buffers);
-            return false;
-        }
     }
 
     return true;
@@ -1216,6 +1220,11 @@ static ggml_backend_buffer_t ggml_backend_alloc_ctx_tensors_from_buft_impl(
         return NULL;
     }
 
+    if (!alloc_ctx_tensors_finalize_views(ctx)) {
+        free_buffers(&buffers, &n_buffers);
+        return NULL;
+    }
+
     ggml_backend_buffer_t buffer;
     if (n_buffers == 1) {
         buffer = buffers[0];
diff --git a/tests/test-alloc.cpp b/tests/test-alloc.cpp
index 6d5428493..65ae1cd51 100644
--- a/tests/test-alloc.cpp
+++ b/tests/test-alloc.cpp
@@ -583,6 +583,31 @@ static void test_reallocation() {
     }
 }
 
+// Parent nbytes > max_size leaves cur_buf_size > max_size. A following view
+// then triggers a split with this_size == 0, so the view-only residual range
+// skips alloc_tensor_range unless views are finalized after all splits.
+static void test_view_init_after_max_size_split() {
+    const size_t max_size = 16;
+    dummy_backend backend = dummy_backend_init(max_size);
+    auto [ctx, graph, ctx_ptr] = make_context();
+    (void) graph;
+
+    ggml_tensor * parent = make_input_with_size(ctx, 24); // 6 x f32, > max_size
+    ggml_tensor * view0  = ggml_view_1d(ctx, parent, 2, 0);
+    ggml_tensor * view1  = ggml_view_1d(ctx, parent, 2, 2 * sizeof(float));
+    assign_names(ctx);
+
+    ggml_backend_buffer_ptr buf(ggml_backend_alloc_ctx_tensors_from_buft(ctx, &backend.buffer_type));
+    GGML_ASSERT(buf);
+
+    for (ggml_tensor * t = ggml_get_first_tensor(ctx); t; t = ggml_get_next_tensor(ctx, t)) {
+        GGML_ASSERT(t->buffer != nullptr);
+        GGML_ASSERT(t->data != nullptr);
+    }
+    GGML_ASSERT(view0->view_src == parent);
+    GGML_ASSERT(view1->view_src == parent);
+}
+
 static void run(const char * name, void (*f)()) {
     printf("%s ", name);
     fflush(stdout);
@@ -604,5 +629,6 @@ int main() {
     run("test_multiple_buffer_types", test_multiple_buffer_types);
     run("test_buffer_size_zero", test_buffer_size_zero);
     run("test_reallocation", test_reallocation);
+    run("test_view_init_after_max_size_split", test_view_init_after_max_size_split);
     return 0;
 }

[aarononeal]

The PR fixed the GGML_ASSERT(tensor->data != NULL && "tensor not allocated") failed issue for me. Recommend review.

• Vulkan / Strix Halo / AMD Ryzen AI Max+ 395 with Radeon 8060S (gfx1151)
• unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL

  --chat-template-kwargs {"preserve_thinking": "true"}
  --no-context-shift
  --host 127.0.0.1
  --metrics
  --min-p 0.0
  --presence-penalty 0.0
  --reasoning-preserve
  --repeat-penalty 1.0
  --spec-draft-n-max 2
  --cache-type-k-draft f16
  --cache-type-v-draft f16
  --spec-type draft-mtp
  --temperature 1.0
  --top-k 20
  --top-p 0.95
  --alias qwen
  --ctx-size 1048576
  --cache-type-k q8_0
  --cache-type-v q8_0
  --flash-attn 1
  --fit off
  --hf-repo unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL
  --no-kv-unified
  --n-gpu-layers 999
  --parallel 4

Original failure trace:

[60225] 2.51.532.931 I slot print_timing: id  3 | task 0 | prompt processing, n_tokens =  34163, progress = 0.99, t = 156.67 s / 218.05 tokens per second
[60225] /work/llama.cpp/ggml/src/ggml-backend.cpp:348: GGML_ASSERT(tensor->data != NULL && "tensor not allocated") failed
[60225] /llama/libggml-base.so.0(+0x1c20b)[0x7ba75897420b]
[60225] /llama/libggml-base.so.0(ggml_print_backtrace+0x21c)[0x7ba75897468c]
[60225] /llama/libggml-base.so.0(ggml_abort+0x15b)[0x7ba75897486b]
[60225] /llama/libggml-base.so.0(+0x3492a)[0x7ba75898c92a]
[60225] /llama/libllama.so.0(_ZN19llama_io_write_hostD0Ev+0x49)[0x7ba758b3c739]
[60225] /llama/libllama.so.0(_ZN13llama_context18state_seq_get_dataEiPhmj+0x125)[0x7ba758b2fe85]
[60225] /llama/libllama-common.so.0(_ZN24common_prompt_checkpoint10update_dftEP13llama_contextij+0x84)[0x7ba7591bb164]
[60225] /llama/libllama-server-impl.so(_ZN19server_context_impl17create_checkpointER11server_slotlii+0x28f)[0x7ba759aaf3ff]
[60225] /llama/libllama-server-impl.so(_ZZN19server_context_impl10pre_decodeEvENKUlR11server_slotE3_clES1_+0x14e6)[0x7ba759ac58d6]
[60225] /llama/libllama-server-impl.so(_ZN19server_context_impl7iterateERSt6vectorI11server_slotSaIS1_EESt8functionIFvRS1_EE+0x57)[0x7ba759abafa7]
[60225] /llama/libllama-server-impl.so(_ZN19server_context_impl10pre_decodeEv+0x497)[0x7ba759abb5f7]
[60225] /llama/libllama-server-impl.so(_ZN19server_context_impl12update_slotsEv+0xbc)[0x7ba759abe45c]
[60225] /llama/libllama-server-impl.so(_ZN12server_queue10start_loopEl+0x125)[0x7ba759a64ab5]
[60225] /llama/libllama-server-impl.so(_Z12llama_serverR13common_paramsiPPc+0x3b03)[0x7ba759a018b3]
[60225] /llama/libllama-server-impl.so(_Z12llama_serveriPPc+0xd6)[0x7ba759a028d6]
[60225] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7ba75946c1ca]
[60225] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7ba75946c28b]
[60225] /llama/llama-server(+0x12a5)[0x5f1fce1402a5]

[Yoshi4470]

Rebased after #27644 updated the base files. Still waiting for review. Thanks.