kv-cache : separate recurrent vs non-recurrent impl#12799

Merged

ggerganov merged 29 commits intomasterfrom

gg/llama-kv-cache-v6

May 2, 2025

Commits on May 2, 2025

kv-cache : serparate recurrent vs non-recurrent impl (wip)
ggerganov
committed
kv-cache : init -> contructor + add llama_memory_params
ggerganov
committed
kv-cache : fix callback reference
ggerganov
committed
context : llama_kv_cache -> llama_memory_i
ggerganov
committed
context : move memory creation logic to model
ggerganov
committed
llama : remove reference of memory during encode
ggerganov
committed
kv-cache : hide padding details in the implementation
ggerganov
committed
kv-cache : add ubatch_next()
ggerganov
committed
context : simplify sbatch logic
ggerganov
committed
kv-cache : hide defrag logic in the implementation
ggerganov
committed
context : hide kv cache details in implementation
ggerganov
committed
build : fix
ggerganov
committed
cont : another fix
ggerganov
committed
kv-cache : simplify interface (wip)
ggerganov
committed
kv-cache : use separate KV cell structs for unified/recurrent
ggerganov
committed
kv-cache : clean-up
ggerganov
committed
model : better llama_model::create_model() signature
ggerganov
committed
kv-cache : fix recurrent seq_rm()
ggerganov
committed
kv-cache : replace `struct callbacks` with `llama_model &`
ggerganov
committed
kv-cache : replace `struct graph_params` with `llama_context &`
ggerganov
committed
kv-cache : fix offload check
ggerganov
committed
context : avoid passing unique_ptr
ggerganov
committed
kv-cache : avoid using the backends from the llama_context
ggerganov
committed
kv-cache : more consistent debug logs [no ci]
ggerganov
committed
kv-cache : do not pass the full llama_context for kv graphs
ggerganov
committed
kv-cache : remove comment
ggerganov
committed
kv-cache : ggml_rope_ext_inplace -> ggml_rope_ext
ggerganov
committed
kv-cache : fix recurrent multi-user case
ggerganov
committed
memory : remove comments [no ci]
ggerganov
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

kv-cache : separate recurrent vs non-recurrent impl#12799

kv-cache : separate recurrent vs non-recurrent impl#12799
ggerganov merged 29 commits intomasterfrom
gg/llama-kv-cache-v6

Commits on May 2, 2025

kv-cache : serparate recurrent vs non-recurrent impl (wip)

kv-cache : init -> contructor + add llama_memory_params

kv-cache : fix callback reference

context : llama_kv_cache -> llama_memory_i

context : move memory creation logic to model

llama : remove reference of memory during encode

kv-cache : hide padding details in the implementation

kv-cache : add ubatch_next()

context : simplify sbatch logic

kv-cache : hide defrag logic in the implementation

context : hide kv cache details in implementation

build : fix

cont : another fix

kv-cache : simplify interface (wip)

kv-cache : use separate KV cell structs for unified/recurrent

kv-cache : clean-up

model : better llama_model::create_model() signature

kv-cache : fix recurrent seq_rm()

kv-cache : replace `struct callbacks` with `llama_model &`

kv-cache : replace `struct graph_params` with `llama_context &`

kv-cache : fix offload check

context : avoid passing unique_ptr

kv-cache : avoid using the backends from the llama_context

kv-cache : more consistent debug logs [no ci]

kv-cache : do not pass the full llama_context for kv graphs

kv-cache : remove comment

kv-cache : ggml_rope_ext_inplace -> ggml_rope_ext

kv-cache : fix recurrent multi-user case

memory : remove comments [no ci]

kv-cache : separate recurrent vs non-recurrent impl#12799ggerganov merged 29 commits intomasterggml-org/llama.cpp:masterfrom gg/llama-kv-cache-v6ggml-org/llama.cpp:gg/llama-kv-cache-v6Copy head branch name to clipboard

Commits

Commits on May 2, 2025

kv-cache : separate recurrent vs non-recurrent impl#12799
ggerganov merged 29 commits intomasterfrom
gg/llama-kv-cache-v6