[DSv4 fix proposal — pending upstream support] deepseek4

proposals/dsv4-fixes/fix-deepseek4-swa-multislot-views.md

@@ -0,0 +1,45 @@
+# PR #2: deepseek4: handle multi-slot SWA cache 4D views
+
+> Branch suggestion: `fix/deepseek4-swa-multislot-views`
+> Target: ggml-org/llama.cpp master (**same caveat as PR #1 — `src/models/deepseek4.cpp` doesn't exist upstream yet; open as Draft or target DSv4 fork**)
+> Commit: `43b226148` (14 lines, single file)
+
+## Description
+
+Follow-up to DeepSeek V4 Flash support work (nisparks PR #22378) and companion to `fix/deepseek4-fa-reserve-kv-cast`. With `-np >= 2`, the SWA (sliding window attention) cache produces a 4D view that the legacy reshape-to-3D codepath can't accept. Crash is a 4-vs-3 dimension mismatch the moment the second slot is exercised, after the FA reserve fix in the companion PR.
+
+Patch updates the SWA cache handling in `src/models/deepseek4.cpp` to retain the 4D view rather than collapse it to 3D when `n_seq_max > 1`.
+
+## Reproduction (before fix)
+
+After applying the F32-cast fix from the companion PR but before this one:
+
+```bash
+./llama-server \
+  -m DeepSeek-V4-Flash-IQ2XXS-...-chat-v2.gguf \
+  -ngl all -fa on \
+  -c 524288 -np 2 \
+  -b 2048 -ub 512 \
+  --jinja
+```
+
+Server now passes `graph_reserve`, but `np=2` decode trips a 4D-vs-3D dimension assertion the moment a second concurrent request is scheduled (i.e. only when both slots are active simultaneously and the SWA cache materializes a per-slot view).
+
+## After fix
+
+Same command, both slots can be active concurrently. `np=2` parallel-load smoke test fires 2 simultaneous requests and both complete correctly. With Phase 5 stream-concurrency batching (separate work) layered on top, this drops the N=2 wall/baseline ratio from 1.948 to 1.283 and lifts aggregate decode throughput from 50.3 to 73.2 tok/s. This PR is the necessary stability prerequisite for that optimization to be possible.
+
+## Hardware tested
+
+- NVIDIA RTX PRO 6000 Blackwell Workstation Edition (sm_120, 96 GB VRAM)
+- CUDA 13.0, driver 595.71.05
+- Build: `cmake -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DGGML_CUDA_FA=ON`
+
+## Why upstream
+
+Independent of the broader batched-prefill chain in nisparks PR #22378. Without this fix, multi-slot DSv4 simply cannot serve concurrent requests on the SWA path. Should apply cleanly to whichever branch eventually upstreams DSv4 support, alongside the F32-cast fix.
+
+## Related
+
+- nisparks PR #22378 (closed)
+- Companion PR: `fix/deepseek4-fa-reserve-kv-cast` (separate F32 cast issue on the FA reserve path)

proposals/dsv4-fixes/fix-deepseek4-swa-multislot-views.patch

@@ -0,0 +1,51 @@
+From 43b22614818537fab0a727fac1045af52101090c Mon Sep 17 00:00:00 2001
+From: Codex <codex@local>
+Date: Sun, 10 May 2026 11:11:57 +0000
+Subject: [PATCH] deepseek4: handle multi-slot swa cache views
+
+---
+ src/models/deepseek4.cpp | 14 ++++++++++++--
+ 1 file changed, 12 insertions(+), 2 deletions(-)
+
+diff --git a/src/models/deepseek4.cpp b/src/models/deepseek4.cpp
+index 317c3abd0..e00453171 100644
+--- a/src/models/deepseek4.cpp
++++ b/src/models/deepseek4.cpp
+@@ -206,6 +206,16 @@ static ggml_tensor * dsv4_new_filled_3d(ggml_context * ctx, int64_t n0, int64_t
+     return ggml_fill(ctx, ggml_new_tensor_3d(ctx, GGML_TYPE_F32, n0, n1, n2), value);
+ }
+
++static ggml_tensor * dsv4_swa_cache_3d(ggml_context * ctx, ggml_tensor * cache, int64_t head_dim) {
++    if (cache->ne[3] > 1) {
++        return ggml_view_3d(ctx, cache,
++                head_dim, 1, cache->ne[2],
++                cache->nb[1], cache->nb[2], 0);
++    }
++
++    return ggml_reshape_3d(ctx, cache, head_dim, 1, cache->ne[2]);
++}
++
+ static dsv4_state_layout dsv4_make_state_layout(int64_t compress_ratio, int64_t head_dim) {
+     const int64_t coff = compress_ratio == 4 ? 2 : 1;
+     const int64_t width = coff * head_dim;
+@@ -1041,7 +1051,7 @@ llm_build_deepseek4::llm_build_deepseek4(const llama_model & model, const llm_gr
+
+         if (compress_ratio == 0) {
+             ggml_tensor * k_cache = mctx_swa->get_k(ctx0, il);
+-            k_cache = ggml_reshape_3d(ctx0, k_cache, n_embd_head_k, 1, k_cache->ne[2]);
++            k_cache = dsv4_swa_cache_3d(ctx0, k_cache, n_embd_head_k);
+             cur = build_attn_mha(q, k_cache, k_cache, nullptr, inp_attn->get_kq_mask_swa(),
+                     layer.attn_sinks, nullptr, kq_scale, il);
+             cb(cur, "kqv_out", il);
+@@ -1224,7 +1234,7 @@ llm_build_deepseek4::llm_build_deepseek4(const llama_model & model, const llm_gr
+                 }
+
+                 ggml_tensor * k_raw = mctx_swa->get_k(ctx0, il);
+-                k_raw = ggml_reshape_3d(ctx0, k_raw, n_embd_head_k, 1, k_raw->ne[2]);
++                k_raw = dsv4_swa_cache_3d(ctx0, k_raw, n_embd_head_k);
+                 k_all = k_raw;
+                 v_all = k_raw;
+                 attn_mask = inp_attn->self_kq_mask_swa;
+-- 
+2.52.0
+