kv-cache : initialize per-stream views after buffer allocation (#23737)

[ByungHyun21]

Overview

llama-server / llama-cli with --spec-type draft-mtp aborts on the Vulkan backend (#23737, open since May 26):

ggml-backend.cpp:348: GGML_ASSERT(tensor->data != NULL && "tensor not allocated") failed

backtrace: common_prompt_checkpoint::update_dft → server_context_impl::create_checkpoint → pre_decode. First bad commit: 6c4cbdc (#23646).

The k_stream/v_stream views are created before the k/v tensors are allocated. On allocation paths that skip ggml_backend_view_init (observed: the draft-MTP KV on Vulkan — debugged down to cache_k_l64 (view) / cache_v_l64 (view), layer 64 being the nextn/draft layer), they keep tensor->data == NULL forever while their storage lives on view_src. Direct IO — the draft-mtp checkpoint serialization and stream-to-stream copies (llama-kv-cache.cpp:846) — then feeds those views to ggml_backend_tensor_get/set, which require an initialized tensor on every backend that addresses via tensor->data (CPU memcpy; CUDA/HIP analogous).

Fix: after the KV buffers are allocated, initialize any still-uninitialized stream view via ggml_backend_view_init. 20 lines, guarded to !hparams.no_alloc, and only touches views that are actually uninitialized (data == NULL, backing allocated).

The root-cause analysis originates with @Yoshi4470 (issue thread, June 6, fork commit 09c1af8). This PR fixes the same bug at the source rather than relaxing the ggml-backend asserts: we verified that assert relaxation alone leaves the CPU backend reading through a NULL tensor->data (segfault) — only backends whose get/set address via view_src (Vulkan) survive it.

Reproduction / verification (gfx1151 / RADV 25.2.8, ROCm 7.2.2 for the HIP control, Qwen3.8-27B-UD-Q4_K_XL + mmproj)

Before (stock fc35562): abort in 14 s at first inference.

./llama-cli -m Qwen3.8-27B-UD-Q4_K_XL.gguf --spec-type draft-mtp --spec-draft-n-max 4 \
  -fa on -ctk q5_1 -ctv q5_1 -c 1048576 --no-mmap \
  -p "Say hello in one sentence." -n 32 --temp 0 --single-turn

After (this branch @ 67e1862, rebuilt from the pushed commit):

check	before	after
CLI repro above	abort in 14 s	completes, coherent output, 31.8 t/s
server -np 4 -c 1048576 --spec-type draft-mtp	crash	8 sequential requests OK, 0 asserts
/slots save/restore (same IO family)	crash path	save + restore + correct generation after
tool calling / JSON schema / vision (mmproj) + draft-mtp	—	all pass
HIP build, same tests	unaffected	unaffected

A unit test would need a full llama_kv_cache construction (model + device context); the CLI command above is the reproducible regression check, matching the issue's reporter environment.

Additional information

The issue has four independent reporters (Strix Halo / RADV, Radeon AI PRO r9700); the most recent crash report in the thread is from July 20. cc @cdanis @artisticMink @Andy4OS — verification on your machines welcome.

Performance (cross-backend data taken while verifying this fix)

Measured on a tree combining #27311 (the scheduler input-ring fix — required for valid HIP numbers under --parallel, see #27572) and this fix; both backends built from the same source. Qwen3.8-27B-UD-Q4_K_XL, -fa on -ctk/-ctv q4_0 -b 1024 -ub 512, per-slot ctx 262144, realistic corpus, temp 0, 3-run medians; server-process CPU from /proc deltas.

Server tg t/s (draft-mtp n4 / plain), with draft acceptance medians:

prompt	HIP MTP	HIP acc	VK MTP	VK acc	HIP plain	VK plain
616	21.1	0.64	19.5	0.53	11.5	7.3
3796	18.0	0.52	18.1	0.55	9.6	6.8
7404	16.1	0.47	14.4	0.44	7.6	6.7
15681	14.8	0.45	16.0	0.50	6.4	6.5

VK MTP tg tracks its acceptance, not length: the 7404 dip is a less draft-predictable corpus chunk (acc 0.44 vs ~0.50 elsewhere), while VK plain decode is position-flat — HIP MTP declines on both counts (position + acceptance).

Server pp t/s: HIP 266–315, VK 127–233 across the same sizes.

Position scaling (llama-bench tg32, depth fill, r1, no-warmup):

depth	HIP	VK
0	11.35	9.71
16384	9.75	9.20
32768	8.31	9.43
65536	6.41	10.28
98304	4.84	10.03

Vulkan decode is position-invariant (~9.2–10.3 flat) while HIP degrades linearly (-57% over 98K); crossover ~20–30K tokens. This is independent of this fix but is the reason it matters: long-context draft-mtp on Vulkan becomes reachable for the first time on this hardware class, and it is the faster backend there.

Server-process CPU: VK 0.6–1.0% avg (p95 2–3%) vs HIP 3.8–5.1% (p95 12–18%).

Vision (mmproj) image encode, 640×480: HIP 1.64 s, VK 3.60 s.

No measurable delta from this fix itself: the change only initializes views that were previously uninitialized (and aborting) — it adds no work to any steady-state path.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES — the change and verification commands were developed with AI assistance (root-cause debugging, patch, benchmark harness); the root cause was first identified by @Yoshi4470 in the issue thread. Reviewed and verified on hardware by the PR author.

[ggml-gh-bot]

Hi @ByungHyun21, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

• AI-generated content: While code is allowed to be generated by AI, please write the PR description and commit messages on your own without the help of AI.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[ByungHyun21]

Closing in favor of #25584.

After re-checking the cross-references on #23737 I found @Yoshi4470's open PR, which fixes this at the allocator level — a view-only tail skipping the final alloc_tensor_range after a buft max_size split — rather than patching at the KV-cache level like this one does. It also includes a unit test and covers #19839 and #21762 in addition to #23737. That is the better fix, it predates this PR by six weeks, and it already has multiple independent verifications on the same hardware class. My earlier search actually surfaced it and I failed to follow through — that's on me.

Thanks @Yoshi4470 for the root-cause work, and apologies for the duplicate noise.