common: add system-level config file

[jcmdln]

Overview

Currently running llama-cpp tools (llama-cli, llama-bench, llama-embedding, etc) require passing many CLI flags. In some cases, these commands require the same config options, such as defining multi-gpu scenarios, threads, parallel, api-key, etc.

The existing mechanism is models-preset but this only works for llama-server and is a portable, shareable config format. Administrators of multiple machines are missing the ability to define per-node configs independent of models-preset, instead needing to include this in init system configs, shell aliases or some other mechanism.

Instead of duplicating these values or smuggling them into other systems, let's add config precedence re-using the same ini config libraries that models-preset uses so that:

1. CLI flags have the highest precedence

2. ENV vars have the second-highest precedence

3. System and User configs have the lowest precedence

   • Linux/BSD/Mac
     • /etc/llama.cpp/config.ini < ${XDG_CONFIG_HOME:-~/.config}/llama.cpp/config.ini
   • Windows
     • %PROGRAMDATA%\llama.cpp\config.ini < %APPDATA%\llama.cpp\config.ini

Here is an example of what flags an operator might want to set for a particular node:

# ~/.config/llama.cpp/config.ini

# Disabled
ui = off
warmup = off

# API
host = 0.0.0.0
port = 8080
api-key = foobar

# Host-specific throughput tuning
batch-size = 2048
ubatch-size = 2048
flash-attn = on
cache-type-k = q8_0
cache-type-v = q8_0
mmap = off

# Host-specific hardware details
device = Vulkan0,Vulkan1
main-gpu = 1
parallel = 2
threads = 16

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES

[ngxson]

also missing docs about this feature

[jcmdln]

I believe I have completed the requested changes. Anything else to improve or is this looking better?

[ngxson]

sorry for the delay, I'll push my changes directly to this PR

[ngxson]

@jcmdln I'm taking over this PR and please do not push any further commits

here are quite many correctness issues that it would be better if I resolve it myself

[ngxson]

all UBs are fixed and behaviors are well-documented in ‎docs/preset.md