llama : fix defrag bugs + add parameter

[ggerganov]

fix #3380

KV cache defragmentation can be done in 2 ways:

• on demand by the user code via llama_kv_cache_defrag()
• automatically when a fragmentation threshold is exceeded

llama.cpp/llama.h

Line 246 in 547ddae

float defrag_thold; // defragment the KV cache if holes/size > thold, < 0 disabled (default)

Examples:

# parallel without defragmentation enabled
./parallel -m ./models/llama-7b-v2/ggml-model-f16.gguf -n 128 -ngl 99 -c 2048 -s 1 -np 8 -ns 128 -cb

# with defragmentation enabled (thold = 10%)
./parallel -m ./models/llama-7b-v2/ggml-model-f16.gguf -n 128 -ngl 99 -c 2048 -s 1 -np 8 -ns 128 -cb -dt 0.1