chore: bump llama.cpp to b10448 - #33
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Updates the pinned upstream ggml-org/llama.cpp version used by this repository’s WebGPU/WASM bridge builds, moving the dependency baseline forward to the b10423 release.
Changes:
- Bump
llama_cpp.versionfromb10375tob10423.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
leehack
changed the title
leehack
force-pushed
the
automation/bump-llama-cpp
branch
from
August 15, 2026 03:58
a6feda5 to
d6435b0
Compare
leehack
changed the title
leehack
force-pushed
the
automation/bump-llama-cpp
branch
from
August 16, 2026 04:06
d6435b0 to
6396f98
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
llama.cpp update
b10375b10448Upstream changelog
Release notes for b10448
Details
model: add Kimi-K3 text model (#26185)
Hybrid KDA (linear) + MLA (full) attention as in Kimi-Linear-48B, plus five
things that architecture does not have:
K3's text_config reports KimiLinearForCausalLM - the older 48B architecture -
so get_model_architecture routes on the top-level name instead.
The KDA decay gate has two forms, selected by linear_attn_config's
gate_lower_bound. It is not a clamp: when set it swaps the activation entirely
(fla/ops/kda/gate.py), from -exp(A_log)softplus(x) to
lower_boundsigmoid(exp(A_log)*x). K3 sets it to -5.0; kimi-linear leaves it
unset, so that path is unchanged.
Cross-layer residuals reuse ggml_dsv4_hc_pre for the weighted sum. That op is
CPU + CUDA only, so Metal/Vulkan will fall back per-node until those kernels
exist.
The routed experts ship as compressed-tensors "mxfp4-pack-quantized". That is
bit-compatible with ggml's MXFP4 - same E2M1 code assignment, same E8M0 scale
byte, only the nibble positions within a block differ - so they are repacked
rather than dequantized, losslessly and without a ~5.5 TB bf16 round-trip.
The repack is built lazily because gguf_writer holds every added tensor until
the final write. DeepSeek-V4 was already doing the identical bit-shuffling, so
it now shares the helper.
Verified against Moonshot's own code path (transformers + fla's Triton KDA
kernels) on a tiny model exercising every K3-specific feature. Final-position
logits vs the fp32 reference: 6.7e-05 rel / corr 1.00000000 for both the
chunked and the recurrent delta-net path. MXFP4 blocks dequantize to the source
weights with 0.0e+00 error.
Assisted-By: Claude Opus 5 (1M context) noreply@anthropic.com
_res_partsbuffers (kind, tensor) pairs, not bare tensorsget_tensorsmust return an Iterator, matching ModelBasefunctakes one argument, so pass the expert loaders throughargsinstead of the closuredeliberate and safe, but not expressible in the signature
No behaviour change: the MXFP4 repack still dequantizes to the source weights
with 0.0e+00 error and end-to-end logits are unchanged (8.386e-03 rel,
corr 0.99996630).
Assisted-By: Claude Opus 5 (1M context) noreply@anthropic.com
Co-authored-by: Boris Dvorkin b_dvorkin@niuitmo.ru
Increase LLAMA_MAX_EXPERTS from 512 to 1024
tests : support for Kimi K3 in archs test
chat : add Kimi K3 chat format (reasoning, content, typed tool calls)
K3's assistant output is an XTML-ish tagged format built by the template's
open_tag/close_tag macros. Two properties break generic parsing:
starts inside the think section with no opening marker in the output
(thinking_forced_open).
names ("think", "response", "message") are ordinary text tokens.
Adds common_chat_params_init_kimi_k3 (PEG_NATIVE) with detection on the
marker trio, reasoning extraction, response unwrapping, and tool-call
parsing of the tools/call/argument tag structure with argument types
taken from the tool schema. Includes the K3 chat template fixture and 9
test-chat cases derived from real generations of the full 2.8T model.
Verified end-to-end against Kimi-K3-Q2_K (GrEarl/Kimi-K3-GGUF) on 8x B200:
content, reasoning_content, streaming deltas, and tool_calls all correct;
finish_reason stop/tool_calls as appropriate.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Per-role message-start markers for token-level span splitting. User and
assistant messages carry only the role attribute, so their full opener
(through <|sep|>) is used; system and tool messages continue with more
attributes (type=/tool=/index=), so those delimiters stop after the
role's closing quote. Verified against the K3 tiktoken vocabulary that
the closing quote is always a standalone token across all attribute
variants, so the token-level prefix match stays exact.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
fix: apply nits from @ngxson and text fixes from @danielhanchen
tests : added missing hyperparameters and tensors for Kimi K3 in test-llama-archs
chore : move overly verbose header file comments to Kimi K3 source file
tests : re-enabled KIMI_K3 in test-llama-archs for WebGPU backend
model-saver : emit kda_gate_lower_bound for Kimi K3
Quick fix. The Kimi K3 loader reads kda_gate_lower_bound and gates a graph branch on it (it scales the KDA gate when the bound is above -INFINITY), but the model
saver never wrote the key, so a save->load roundtrip silently dropped it back to the -INFINITY default and changed the model's output. The real K3 config sets gate_lower_bound = -5.0.
I propose to emit it from the saver, and set it to -5.0 in the test-llama-archs K3 case so the roundtrip check exercises it (the roundtrip fails without the saver line).
Refactor conditional for model architecture check
tests : re-enabled (again) KIMI_K3 and MINIMAX_M3 in test-llama-archs for WebGPU backend
fix code comments
add template on conversion
move repack_mxfp4_blocks to model base
nits
add_value_length
optimize res_stack construction
nits
Co-authored-by: Boris Dvorkin b_dvorkin@niuitmo.ru
Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com
Co-authored-by: Deepankar Singh singh.deepankar39@gmail.com
Co-authored-by: Claude Fable 5 noreply@anthropic.com
Co-authored-by: Caleb DeLeeuw caleb.deleeuw@gmail.com
Co-authored-by: Xuan Son Nguyen son@huggingface.co
Website:
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
Commit range
Commits from b10375 to b10448 (first 80)
f785fc9)5988633)ebb546b)7b13a84)0b1bad1)cb27fe9)6eff593)55f453b)a7cd2f0)a4a4c51)89e0aa6)4dd1275)5d9e5ac)13fd0bb)af05a42)ece98b8)132753b)d8a8bea)680a9ae)7a9ff95)9558fa4)84e908c)8e7f22b)e21152d)1f368f3)a6040c9)094e53d)styles/to$libscope (#26950) (f2efd64)d86c7d6)e79e4bf)decaf50)d415e65)8efbf65)1ee1cd9)154d57a)eeae28b)0d0bfcd)f65e568)4a84b0a)1d2869c)981184e)2606220)a97123e)aee56b3)9c5531e)fa4ec45)bdffafa)a94d563)2bacf9e)3d93885)c6f6a92)6509138)885c5bb)77918ca)4c1a0af)1692f9e)06ae232)9b05354)7e4c0a9)9e40df6)6fed9f6)16d222f)9d57ce4)7b38cb7)6b4344e)0177dcc)9b0a2ce)27df919)5f754ea)77140d2)adb55e5)22b8e31)ad1de39)Web bridge review focus
Please pay extra attention to upstream changes touching:
Validation
Automation behavior
This PR is managed from the stable branch
automation/bump-llama-cpp. If another llama.cpp release appears before merge, the scheduled workflow updates this same PR instead of opening a duplicate. The workflow skips if a non-automation PR already changesllama_cpp.version.