DeepseekV4: reduce graph splits - #25702
Merged
Merged
Conversation
fairydreaming
approved these changes
Jul 15, 2026
ggerganov
approved these changes
Jul 15, 2026
zengde
pushed a commit
to zengde/llama.cpp
that referenced
this pull request
Jul 16, 2026
TrevorS
added a commit
to TrevorS/llama.cpp
that referenced
this pull request
Jul 17, 2026
Squash-rebase of the ds4-flash-experiments branch (post-cleanup: 8 dead flags + MOE_TILE/FP4_RT ops removed, LID_CACHE_MXFP4 default-on) onto current upstream, which had independently evolved DeepSeek-V4 (fused HC ops ggml-org#25585, kv_stream cache refactor ggml-org#25702, seq_rm fix ggml-org#25588, lightning-indexer ggml-org#24231). Reconciliation (per Teej's calls — keep our tuned versions, adopt upstream only where cleanly additive): - KV cache: adopted upstream's kv_stream per-stream views; kept our kv_stash MTP frontier-rewind AND our frontier-aware seq_rm (upstream's ggml-org#25588 seq_rm rejects the 1-token-tail eviction our server/MTP post_decode relies on -> aborts; caught in re-validation, reverted to ours). - HC fusion: kept OUR validated bandwidth-minimal HC op; upstream's HC_PRE/COMB/ POST sit dormant. Renamed our colliding ggml_dsv4_hc_post -> _hc_fused_post (+ test struct); fixed models.h class decl to match our deepseek4 impl. - Server: adapted our --cache-disk L2 tier to upstream's split prompt-cache (server_prompt vs server_prompt_cache_state / server_prompt_data). - Kept all unique work: LID indexer CUDA kernels, CSA_TILE, FA_SPLIT/FA_MERGE, fp4-mma + packed MXFP4 container, radix/int8/dec, power governor, MTP, CVEC. - experiments/ untracked (repo-local ignore) — branch delta is code only. Verified on the rebased tree: - build clean (cli + server + tests) on upstream 86d86ed - DSV4 backend-ops 2/2 all kept ops (LID_TOPK/UNION/MEMB, HC_FUSED, QAT_SET_ROWS, FA_MERGE) - llama-cli shallow smoke c8192: coherent, byte-identical greedy to pre-rebase - llama-server c32768 + MTP + --cache-disk (LAN): MTP 21.5 t/s (draft 47/95 accepted), 3 requests no crash, --cache-disk spill + 660-tok disk restore Not yet run: deep-context (>=131k) serving — defer to an attended run (wedge risk).
CowboyTim
pushed a commit
to aardbeiplantje/llama.cpp
that referenced
this pull request
Jul 21, 2026
smalinin
pushed a commit
to smalinin/llama.cpp
that referenced
this pull request
Aug 4, 2026
satindergrewal
pushed a commit
to satindergrewal/llama.cpp
that referenced
this pull request
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Reduce the number of splits from 5->2 by avoiding the view
Additional information
Requirements