Skip to content

Bump llama.cpp to ff067f76d (b10133) for v0.8.38 - #75

Merged
nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-ff067f76d
Jul 26, 2026
Merged

Bump llama.cpp to ff067f76d (b10133) for v0.8.38#75
nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-ff067f76d

Conversation

@nyo16

@nyo16 nyo16 commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Breaking upstream API change

Unlike recent bumps, this range breaks the upstream C API and required a NIF change.

llama_model_params lost its use_mmap, use_direct_io, and use_mlock booleans; they
were replaced by a single enum llama_load_mode load_mode with values
LLAMA_LOAD_MODE_NONE / _MMAP / _MLOCK / _DIRECT_IO (upstream ggml-org/llama.cpp#20834).
Note that LLAMA_LOAD_MODE_MLOCK now means "mmap and mlock" — the two are no longer
independent.

model_load/10 maps the existing :use_mmap / :use_mlock / :use_direct_io options onto
that enum, preserving the precedence the docs already promised:

Options Resolved mode
use_direct_io: true dio
use_mlock: true mlock
use_mmap: true (default) mmap
all false none

The public Elixir API and its defaults are unchanged — no caller updates are needed, and
no capability is lost, since the three booleans still reach all four modes.

Also in this range: common_chat_params::thinking_end_tag became thinking_end_tags
(now a std::vector<std::string>, ggml-org/llama.cpp#25544) so the reasoning-budget sampler can
accept multiple end sequences. The binding never read that field, so no change was needed.
common/common.h changes don't apply either (the binding doesn't use common_params), and
json-schema-to-grammar.h, speculative.h, sampling.h, plus all of ggml/include/ are
untouched.

Behaviour change

One user-visible edge case: use_mlock: true, use_mmap: false now memory-maps the file
rather than reading it into anonymous memory, because upstream's MLOCK implies mmap. This is
inherent to the upstream collapse and is documented in Model.load/2 and docs/performance.md.

Verification

NIF rebuilt in both dev and test envs (deleting the obj/llama_build/.built stamps, which
a submodule content bump alone doesn't invalidate).

Full suite with real GGUF models, --include smoke --include slow:
248/249 passed, 0 failures, 1 skipped (the skip is the pre-existing @tag :skip on
server_smoke_test.exs:179).

No test covers the load flags, so that path was verified directly against a real model load —
all five combinations resolve correctly per llama.cpp's own load_mode = log line
(mmap, none, mlock, dio, and dio taking precedence over mlock), with generation
working in each mode.

Not covered: the MTP speculative-decoding tests, which need LLAMA_MTP_MODEL_PATH pointed
at a GGUF with MTP head layers. No such model was available locally at the time of this run.

Notes

  • CHANGELOG.md has a full categorized entry for the 65 commits; @version bumped to 0.8.38.
  • checksum.exs intentionally untouched — CI regenerates it after the release tag.

@nyo16
nyo16 force-pushed the bump-llama-cpp-ff067f76d branch from 2170820 to 35f6888 Compare July 26, 2026 13:37
Updates vendor/llama.cpp from 76f46ad29 (b10075, released in v0.8.38) to
ff067f76d (b10133), 58 commits.

Unlike recent bumps this range breaks the upstream C API:
llama_model_params loses the use_mmap/use_direct_io/use_mlock booleans in
favour of a single `enum llama_load_mode load_mode` field (upstream
#20834). model_load/10 now maps the existing :use_mmap / :use_mlock /
:use_direct_io options onto that enum, preserving the documented
precedence (dio > mlock > mmap > none). The public Elixir API and its
defaults are unchanged, so callers need no updates; all four resolved
modes were verified against a real model load.

common/chat.h also renamed common_chat_params::thinking_end_tag to
thinking_end_tags (now a vector, #25544), but the binding never read
that field.

Full suite against the rebuilt NIF with real GGUF models, including the
smoke, slow and MTP speculative-decoding tests: 252 passed, 0 failures.
MTP was exercised against a Qwen3.5-9B-MTP GGUF (arch qwen35, nextn
tensors present).
@nyo16
nyo16 force-pushed the bump-llama-cpp-ff067f76d branch from 35f6888 to abebe1e Compare July 26, 2026 17:42
@nyo16
nyo16 merged commit 078f74e into master Jul 26, 2026
2 of 4 checks passed
@nyo16
nyo16 deleted the bump-llama-cpp-ff067f76d branch July 26, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant