IK's experiments with DS4 - #2147

Closed

ikawrakow wants to merge 38 commits into

mainfrom

Commits on Jul 2, 2026

initial map to load deepseek 4 arch
SamuelOliveirads
committed
8b9ce35

Commits on Jul 3, 2026

wip
SamuelOliveirads
committed
b3f2a8b

Commits on Jul 4, 2026

wip: match graph build and attn logic for dpv4
SamuelOliveirads
committed
40eb9f1

Commits on Jul 5, 2026

wip: Enhance DeepSeek-V4 architecture with new tensor types and sqrtsoftplus gating function
SamuelOliveirads
committed
3de2b0a

Commits on Jul 6, 2026

Update DeepSeek-V4 to support raw key indexing with read/write indices
SamuelOliveirads
committed
1cffe9b

Commits on Jul 8, 2026

fix mismatch in attn_raw
SamuelOliveirads
committed
3599588

Commits on Jul 10, 2026

Enable FA with CSA/HCA
SamuelOliveirads
committed
129e2f7
Fix logit mismatch with FA path
SamuelOliveirads
committed
6bfe51f

Commits on Jul 11, 2026

Clean traces and logs for debug
SamuelOliveirads
committed
88f5247

Commits on Jul 12, 2026

Commits on Jul 13, 2026

Commits on Jul 14, 2026

dsv4: document dependency ordering and reset state
SamuelOliveirads
committed
724ebba
Remove DSV4 zero-dependency graph shim
SamuelOliveirads
committed
050021d
Fix DSV4 packed stream execution
SamuelOliveirads
committed
feb77a9
Remove DSV4 l_out backend override
SamuelOliveirads
committed
78a5226
Enable DSV4 quantized K-only cache
SamuelOliveirads
committed
04f9b42
Revert "Enable DSV4 quantized K-only cache"
SamuelOliveirads
committed
74fcc71
Fix DSV4 quantized cache accounting
SamuelOliveirads
committed
e2b0340

Commits on Jul 15, 2026

Fail closed on unsupported DSV4 cache lifecycle operations
SamuelOliveirads
committed
3d6acd3

Commits on Jul 17, 2026

Various optimizations
ikawrakow
committed
ef7d492
llama: fix GGML_METAL=ON build - missing ggml-metal.h include in llama-dflash.cpp (#2134)

hchengit
authored and
ikawrakow
committed
6470754
New op: ggml_sum_rows_ext (#2132)
ikawrakow
committed
0508580
Reuse shared inverse RoPE operation for DSV4

SamuelOliveirads
authored and
ikawrakow
committed
17114aa
Reuse maintainer CUDA concat implementation

SamuelOliveirads
authored and
ikawrakow
committed
a2d0415
Merge remote-tracking branch 'iwan/main' into ik/ds4_play1
ikawrakow
committed
80d6807

Commits on Jul 18, 2026

WIP
ikawrakow
committed
5ca6285
hc_pre
ikawrakow
committed
5e0694b
hc_post
ikawrakow
committed
7c3fc27
Remove unnecessary mask manipulations
ikawrakow
committed
a3cbb9b
WIP
ikawrakow
committed
8a72e5a
Take into account swiglu limits
ikawrakow
committed
fb26910