[DSv4 fix proposal — pending upstream support] deepseek4

[leon7609]

PR #1: deepseek4: cast padded KV to f32 on FA reserve path

Branch suggestion: fix/deepseek4-fa-reserve-kv-cast
Target: ggml-org/llama.cpp master (file src/models/deepseek4.cpp does not exist upstream yet — this PR depends on DSv4 architecture support landing first; recommend opening as Draft PR with Closes after DSv4 support note, OR target Fringe210/nisparks DSv4 branch instead)
Commit: 22452315e (10 lines, single file)

Description

Follow-up to the DeepSeek V4 Flash support work (originally PR ggml-org#22378). When the FA path emits kq_mask and pads KV to 256 (introduced in commit c77f14368 of that work), the padded KV tensor is kept in F16 while the downstream FA op expects F32. This causes a GGML_ASSERT deep in the FA path during graph_reserve, only surfaced once -np >= 2 is attempted on antirez's IQ2XXS DSv4 GGUF.

Single-file delta in src/models/deepseek4.cpp to insert a ggml_cast(..., GGML_TYPE_F32) on the padded KV path before it is consumed by the FA reserve graph.

Reproduction (before fix)

./llama-server \
  -m DeepSeek-V4-Flash-IQ2XXS-...-chat-v2.gguf \
  -ngl all -fa on \
  -c 524288 -np 2 \
  -b 2048 -ub 512 \
  --jinja --chat-template-file models/templates/deepseek-ai-DeepSeek-V4.jinja

Without the F32 cast, server crashes during sched_reserve with a GGML_ASSERT walking the worst-case graph for the second slot.

After fix

Same command, server boots in <60 s. np=2 × c=524288 (256 K per slot) reaches steady state. End-to-end probes (smoke / latency / tps / json_obj / json_schema strict / tools / long_ctx_12k / long_ctx_32k / chinese_prose / thinking) all pass 10/10.

Hardware tested

• NVIDIA RTX PRO 6000 Blackwell Workstation Edition (sm_120, 96 GB VRAM)
• CUDA 13.0, driver 595.71.05
• Ubuntu 24.04
• Build: cmake -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DGGML_CUDA_FA=ON

Why upstream

Independent of the broader batched-prefill chain in nisparks PR ggml-org#22378 (which is closed and marked reference-only). This single fix is required for any DSv4 multi-slot path that relies on c77f14368's KV padding. Should apply cleanly to whichever branch eventually upstreams DSv4 support.

Related

• nisparks PR Wip/deepseek v4 support ggml-org/llama.cpp#22378 (closed, "purely for reference")
• antirez/deepseek-v4-gguf (the IQ2XXS variant exposes this bug; other quants may not pad enough to trigger it)
• Companion PR: fix/deepseek4-swa-multislot-views (handles a separate 4D-cache-view multi-slot issue)