Reduce memory required for DSA - #2058

Closed

ikawrakow wants to merge 11 commits into

mainfrom

Commits on Jun 26, 2026

Add GLM-5.2/DeepSeek-V3.2 DSA lightning indexer (batch-local, single-seq prefill)

mb8565
and
claude
committed
70a7c8c
GLM-5.2 DSA indexer: decode-correct via persistent indexer-K cache

mb8565
and
claude
committed
f1d2c3a
GLM-5.2 DSA indexer: wire sparse mask into the flash-attention path (-fa 1)

mb8565
and
claude
committed
476b4ee
GLM-5.2 DSA indexer: UPDATE 4 — MLA-FA fix merged, FA path validated, multi-seq characterized

mb8565
and
claude
committed
8eb3a86
GLM-5.2 DSA indexer: per-sequence attention sink — fix multi-seq (n_seq>1)

mb8565
and
claude
committed
e0c7490
GLM-5.2 DSA indexer: UPDATE 6 — serving-correctness (kr_l maintained across shift/defrag/seq-ops; per-seq sink on first-present pos)

mb8565
and
claude
committed
c6a97cd

Commits on Jun 27, 2026

GLM-5.2 DSA indexer: UPDATE 7 — FIX latent graph-reuse cache-fixup omission for the kr_l indexer cache

mb8565
and
claude
committed
55a16cc
GLM-DSA: convert sparse-attention control from env vars to CLI args (off by default)

mb8565
and
claude
committed
a4e5f7c
GLM-DSA: warn that --dsa is inactive under -sm graph/attn (TP path runs dense MLA)

mb8565
and
claude
committed
71d81ce
GLM-DSA: drop in-tree dev reference docs from the PR branch

mb8565
and
claude
committed
03391cb

Commits on Jun 29, 2026

Perform indexer score calculation in a loop over attention heads
ikawrakow
committed
d536359