Bump llama.cpp to ff067f76d (b10133) for v0.8.38 - #75
Merged
Conversation
nyo16
force-pushed
the
bump-llama-cpp-ff067f76d
branch
from
July 26, 2026 13:37
2170820 to
35f6888
Compare
Updates vendor/llama.cpp from 76f46ad29 (b10075, released in v0.8.38) to ff067f76d (b10133), 58 commits. Unlike recent bumps this range breaks the upstream C API: llama_model_params loses the use_mmap/use_direct_io/use_mlock booleans in favour of a single `enum llama_load_mode load_mode` field (upstream #20834). model_load/10 now maps the existing :use_mmap / :use_mlock / :use_direct_io options onto that enum, preserving the documented precedence (dio > mlock > mmap > none). The public Elixir API and its defaults are unchanged, so callers need no updates; all four resolved modes were verified against a real model load. common/chat.h also renamed common_chat_params::thinking_end_tag to thinking_end_tags (now a vector, #25544), but the binding never read that field. Full suite against the rebuilt NIF with real GGUF models, including the smoke, slow and MTP speculative-decoding tests: 252 passed, 0 failures. MTP was exercised against a Qwen3.5-9B-MTP GGUF (arch qwen35, nextn tensors present).
nyo16
force-pushed
the
bump-llama-cpp-ff067f76d
branch
from
July 26, 2026 17:42
35f6888 to
abebe1e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Breaking upstream API change
Unlike recent bumps, this range breaks the upstream C API and required a NIF change.
llama_model_paramslost itsuse_mmap,use_direct_io, anduse_mlockbooleans; theywere replaced by a single
enum llama_load_mode load_modewith valuesLLAMA_LOAD_MODE_NONE/_MMAP/_MLOCK/_DIRECT_IO(upstream ggml-org/llama.cpp#20834).Note that
LLAMA_LOAD_MODE_MLOCKnow means "mmap and mlock" — the two are no longerindependent.
model_load/10maps the existing:use_mmap/:use_mlock/:use_direct_iooptions ontothat enum, preserving the precedence the docs already promised:
use_direct_io: truediouse_mlock: truemlockuse_mmap: true(default)mmapnoneThe public Elixir API and its defaults are unchanged — no caller updates are needed, and
no capability is lost, since the three booleans still reach all four modes.
Also in this range:
common_chat_params::thinking_end_tagbecamethinking_end_tags(now a
std::vector<std::string>, ggml-org/llama.cpp#25544) so the reasoning-budget sampler canaccept multiple end sequences. The binding never read that field, so no change was needed.
common/common.hchanges don't apply either (the binding doesn't usecommon_params), andjson-schema-to-grammar.h,speculative.h,sampling.h, plus all ofggml/include/areuntouched.
Behaviour change
One user-visible edge case:
use_mlock: true, use_mmap: falsenow memory-maps the filerather than reading it into anonymous memory, because upstream's
MLOCKimplies mmap. This isinherent to the upstream collapse and is documented in
Model.load/2anddocs/performance.md.Verification
NIF rebuilt in both
devandtestenvs (deleting theobj/llama_build/.builtstamps, whicha submodule content bump alone doesn't invalidate).
Full suite with real GGUF models,
--include smoke --include slow:248/249 passed, 0 failures, 1 skipped (the skip is the pre-existing
@tag :skiponserver_smoke_test.exs:179).No test covers the load flags, so that path was verified directly against a real model load —
all five combinations resolve correctly per llama.cpp's own
load_mode =log line(
mmap,none,mlock,dio, anddiotaking precedence overmlock), with generationworking in each mode.
Not covered: the MTP speculative-decoding tests, which need
LLAMA_MTP_MODEL_PATHpointedat a GGUF with MTP head layers. No such model was available locally at the time of this run.
Notes
CHANGELOG.mdhas a full categorized entry for the 65 commits;@versionbumped to0.8.38.checksum.exsintentionally untouched — CI regenerates it after the release tag.