Sync master with upstream release b5798#142

Merged

Minh141120 merged 192 commits intodevfrom

update-dev-from-master-2025-07-02-06-38

Jul 2, 2025

Commits on Jul 2, 2025

CANN: Enable labeler for Ascend NPU (#13914)

shink
authored and
qnixsynapse
committed
add geglu activation function (#14074)

authored and
qnixsynapse
committed
sycl: Add reorder to Q6_K mmvq implementation (#13885)

s-Nick
authored and
qnixsynapse
committed
server : fix LRU check (#14079)

ggerganov
authored and
qnixsynapse
committed
webui: fix sidebar being covered by main content (#14082)

yeahdongcn
authored and
qnixsynapse
committed
CANN: Simplify the environment variable setting(#13104)

bachelor-dou
authored and
qnixsynapse
committed
graph : fix geglu (#14077)

ggerganov
authored and
qnixsynapse
committed
cuda : fix device sync on buffer clear (#14033)

slaren
authored and
qnixsynapse
committed
ggml-cpu : split arch-specific implementations (#13892)

authored and
qnixsynapse
committed
llama : allow building all tests on windows when not using shared libs (#13980)

authored and
qnixsynapse
committed
kv-cache : fix shift and defrag logic (#14081)

ggerganov
authored and
qnixsynapse
committed
metal : use less stack memory in FA kernel (#14088)

ggerganov
authored and
qnixsynapse
committed
Add in-build ggml::ggml ALIAS library (ggml/1260)

dg0yt
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
rpc : nicer error messages for RPC server crash (#14076)

isaac-mcfadyen
authored and
qnixsynapse
committed
Vulkan: Don't default to CPU device (like llvmpipe), even if no other device is available, to allow fallback to CPU backend (#14099)

0cc4m
authored and
qnixsynapse
committed
ggml : fix weak alias win32 (whisper/0)

ggerganov
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
Fixed spec timings to: accepted/tested instead of accepted/drafted (#14104)

jukofyork
authored and
qnixsynapse
committed
vulkan: force device 0 in CI (#14106)

jeffbolznv
authored and
qnixsynapse
committed
llama : support GEGLU for jina-bert-v2 (#14090)

CISC
authored and
qnixsynapse
committed
convert : fix duplicate key DeepSeek-R1 conversion error (#14103)

CISC
authored and
qnixsynapse
committed
kv-cache : avoid modifying recurrent cells when setting inputs (#13834)

compilade
authored and
qnixsynapse
committed
opencl: add `mul_mv_id_q4_0_f32_8x_flat` (#14003)

lhez
authored and
qnixsynapse
committed
vulkan: Track descriptor pools/sets per-context (#14109)

jeffbolznv
authored and
qnixsynapse
committed
kv-cache : add LLAMA_KV_CACHE_DEBUG environment variable (#14121)

ggerganov
authored and
qnixsynapse
committed
server : pass default --keep argument (#14120)

MightyAlex200
authored and
qnixsynapse
committed
kv-cache : relax SWA masking condition (#14119)

ggerganov
authored and
qnixsynapse
committed
webui: Wrap long numbers instead of infinite horizontal scroll (#14062)

am17an
authored and
qnixsynapse
committed
vulkan: Better thread-safety for command pools/buffers (#14116)

jeffbolznv
authored and
qnixsynapse
committed
tests : add test-tokenizers-repo (#14017)

CISC
authored and
qnixsynapse
committed
chore : clean up relative source dir paths (#14128)

CISC
authored and
qnixsynapse
committed
Implement GGML_CPU_ALL_VARIANTS for ARM (#14080)

ckastner
authored and
qnixsynapse
committed
common: fix issue with regex_escape routine on windows (#14133)

bandoti
authored and
qnixsynapse
committed
context : round n_tokens to next multiple of n_seqs when reserving (#14140)

compilade
authored and
qnixsynapse
committed
kv-cache : fix split_equal handling in unified implementation (#14130)

ggerganov
authored and
qnixsynapse
committed
cmake : handle whitepsaces in path during metal build (#14126)

authored and
qnixsynapse
committed
batch : remove logits_all flag (#14141)

ggerganov
authored and
qnixsynapse
committed
context : simplify output counting logic during decode (#14142)

ggerganov
authored and
qnixsynapse
committed
server : re-enable SWA speculative decoding (#14131)

ggerganov
authored and
qnixsynapse
committed
readme : remove project status link (#14149)

ggerganov
authored and
qnixsynapse
committed
sycl: Remove not needed copy f16->f32 for dnnl mul mat (#14125)

ShanoToni
authored and
qnixsynapse
committed
vocab : prevent heap overflow when vocab is too small (#14145)

ggerganov
authored and
qnixsynapse
committed
cmake : Improve build-info.cpp generation (#14156)

ckastner
authored and
qnixsynapse
committed
SYCL: Bump oneMath commit (#14152)

Ewan Crawford
authored and
qnixsynapse
committed
sycl: Adding additional cpy dbg print output (#14034)

ShanoToni
authored and
qnixsynapse
committed
server : fix SWA condition for full context reprocess (#14163)

ggerganov
authored and
qnixsynapse
committed
pooling : make cls_b and cls_out_b optional (#14165)

authored and
qnixsynapse
committed
cmake: Add ability to pass in LLAMA_BUILD_NUMBER/COMMIT (#14167)

ckastner
authored and
qnixsynapse
committed
readme : remove survey link (#14168)

ggerganov
authored and
qnixsynapse
committed
batch : rework llama_batch_allocr (#14153)

ggerganov
authored and
qnixsynapse
committed
docs : Update multimodal.md (#14122)

ddpasa
authored and
qnixsynapse
committed
batch : add LLAMA_BATCH_DEBUG environment variable (#14172)

ggerganov
authored and
qnixsynapse
committed
Merge commit from fork

authored and
qnixsynapse
committed
sycl: fix docker image (#14144)

sgeor255
authored and
qnixsynapse
committed
vocab : fix build (#14175)

ggerganov
authored and
qnixsynapse
committed
compare-llama-bench: add option to plot (#14169)

am17an
authored and
qnixsynapse
committed
llama-chat : Do not throw when tool parsing fails (#14012)

p1-0tr
authored and
qnixsynapse
committed
docs : remove WIP since PR has been merged (#13912)

pepijndevos
authored and
qnixsynapse
committed
batch : auto-gen positions + verify multi-sequence input (#14177)

ggerganov
authored and
qnixsynapse
committed
cparams : rename LLAMA_MAX_PARALLEL_SEQUENCES to LLAMA_MAX_SEQ (#14188)

ggerganov
authored and
qnixsynapse
committed
model : add dots.llm1 architecture support (#14044) (#14118)

Noeda
authored and
qnixsynapse
committed
kv-cache : fix use-after-move of defrag info (#14189)

ggerganov
authored and
qnixsynapse
committed
HIP: Replace usage of depricated preprocessor macro __AMDGCN_WAVEFRONT_SIZE__ (#14183)

IMbackK
authored and
qnixsynapse
committed
CUDA/HIP: fix ssm_scan on devices where warp size is not 32 (#14196)

IMbackK
authored and
qnixsynapse
committed
quantize : change int to unsigned int for KV overrides (#14197)

EAddario
authored and
qnixsynapse
committed
server : When listening on a unix domain socket don't print http:// and port (#14180)

ericcurtin
authored and
qnixsynapse
committed
model : Add support for Arcee AI's upcoming AFM model (#14185)

authored and
qnixsynapse
committed
ggml-cpu : rework weak alias on apple targets (#14146)

xctan
authored and
qnixsynapse
committed
vulkan: mutex around vkQueueSubmit (#14127)

jeffbolznv
authored and
qnixsynapse
committed
gguf-py : allow key override when adding value to GGUFWriter (#14194)

authored and
qnixsynapse
committed
convert : remove arcee change in convert_hf_to_gguf_update.py (#14207)

bartowski1182
authored and
qnixsynapse
committed
ggml: Add Android support for GGML_CPU_ALL_VARIANTS (#14206)

chaxu01
authored and
qnixsynapse
committed
llama : rework embeddings logic (#14208)

ggerganov
authored and
qnixsynapse
committed
HIP: disable rocwmma on gfx12 by default until rocm 7.0 (#14202)

IMbackK
authored and
qnixsynapse
committed
model : add NeoBERT (#14164)

authored and
qnixsynapse
committed
cmake: clean up external project logic for vulkan-shaders-gen (#14179)

bandoti
authored and
qnixsynapse
committed
llama : add thread safety test (#14035)

authored and
qnixsynapse
committed
server : fix incorrect usage of llama_get_embeddings() (#14225)

ggerganov
authored and
qnixsynapse
committed
common : suggest --jinja when autodetection fails (#14222)

CISC
authored and
qnixsynapse
committed
musa: fix build warning (unused variable) (#14231)

yeahdongcn
authored and
qnixsynapse
committed
ggml-cpu : remove the weak alias trick (#14221)

xctan
authored and
qnixsynapse
committed
cmake: remove shader-gen step-targets from ggml-vulkan (#14226)

bandoti
authored and
qnixsynapse
committed
examples : include examples in msvc disable warn (ggml/1270)

danbev
authored and
qnixsynapse
committed
ggml : remove unused ggml_context_container (ggml/1272)

danbev
authored and
qnixsynapse
committed
ggml : disable warnings for tests when using MSVC (ggml/1273)

danbev
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
convert : fix null head_dim AutoConfig regression (#14248)

CISC
authored and
qnixsynapse
committed
llama-chat : fix multiple system message for gemma, orion (#14246)

ngxson
authored and
qnixsynapse
committed
mtmd : refactor llava-uhd preprocessing logic (#14247)

ngxson
authored and
qnixsynapse
committed
ggml: Add Apple support for GGML_CPU_ALL_VARIANTS (#14258)

chaxu01
authored and
qnixsynapse
committed
ggml-cpu: fix uncaught underscore terminators (#14023)

taronaeo
authored and
qnixsynapse
committed
ggml-cpu: reduce asm calls for hsum (#14037)

taronaeo
authored and
qnixsynapse
committed
docs: add s390x build documentation (#14264)

taronaeo
authored and
qnixsynapse
committed
metal : add mean kernel (#14267)

ggerganov
authored and
qnixsynapse
committed
memory : Hybrid recurrent cache (#13979)

authored and
qnixsynapse
committed
Vulkan: Set device max size for host memory to avoid OOM warning and fallback to CPU buffer (#14249)

0cc4m
authored and
qnixsynapse
committed
llamafile : support s390x SIMD instruction set (#14273)

taronaeo
authored and
qnixsynapse
committed
convert : fix remote option in Windows (#14100)

pqnet
authored and
qnixsynapse
committed
llama-bench : add --no-warmup flag (#14224) (#14270)

s2010
authored and
qnixsynapse
committed
sycl: Cleanup codepaths in Get Rows in sycl backend (#14215)

ShanoToni
authored and
qnixsynapse
committed
build : suppress gcc15 compile warnings (#14261)

fanyang89
authored and
qnixsynapse
committed
server : add server parameters for draft model cache type (#13782)

aa956
authored and
qnixsynapse
committed
gguf-py : make sentencepiece optional (#14200)

authored and
qnixsynapse
committed
ggml-cpu : remove unnecesary arm feature detection (#14281)

slaren
authored and
qnixsynapse
committed
CUDA: add conv_2d_dw (#14265)

am17an
authored and
qnixsynapse
committed
ubatch : new splitting logic (#14217)

ggerganov
authored and
qnixsynapse
committed
model : more uniform output id handling (#14275)

ggerganov
authored and
qnixsynapse
committed
ggml: Update KleidiAI to v1.9.0 (#14277)

chaxu01
authored and
qnixsynapse
committed
ggml : fix repack work size for mul_mat_id (#14292)

ggerganov
authored and
qnixsynapse
committed
cuda : synchronize graph capture and cublas handle destruction (#14288)

slaren
authored and
qnixsynapse
committed
llama : improve sep token handling (#14272)

CISC
authored and
qnixsynapse
committed
Implement GGML_CPU_ALL_VARIANTS for PowerPC (#14286)

authored and
qnixsynapse
committed
sycl: add usage of enqueue_functions extension (#14244)

s-Nick
authored and
qnixsynapse
committed
vocab : prevent tokenizer overflow (#14301)

retr0reg
authored and
qnixsynapse
committed
lint : remove trailing whitepace (#14304)

CISC
authored and
qnixsynapse
committed
CUDA: add conv_2d_transpose (#14287)

am17an
authored and
qnixsynapse
committed
docs : fix the link to llama.h (#14293)

david20571015
authored and
qnixsynapse
committed
Add `ggml_roll` (ggml/1274)

Acly
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
convert : fix Llama 4 conversion (#14311)

danielhanchen
authored and
qnixsynapse
committed
memory : rename interface to llama_memory_context_i (#14296)

ggerganov
authored and
qnixsynapse
committed
metal : fix thread-safety (#14300)

ggerganov
authored and
qnixsynapse
committed
gguf-py : fix TemplateProcessing pair when bos/eos is missing (#14312)

CISC
authored and
qnixsynapse
committed
Add support for VK_EXT_debug_utils to add labels to Vulkan objects. (#13792)

mtavenrath
authored and
qnixsynapse
committed
gguf-py : fix Qwen3-Embedding eos token (#14314)

CISC
authored and
qnixsynapse
committed
CUDA: add mean operation (#14313)

am17an
authored and
qnixsynapse
committed
common : use std::string_view now that we target c++17 (#14319)

CISC
authored and
qnixsynapse
committed
mtmd : fix Pixtral OOM with large images by capping image_size to 1024 (#14326)

yuiseki
authored and
qnixsynapse
committed
HIP: enable vec fattn on RDNA4 (#14323)

IMbackK
authored and
qnixsynapse
committed
examples : fix is_first logic for tokenization (#14329)

ggerganov
authored and
qnixsynapse
committed
run : avoid double tokenization (#14327)

retr0reg
authored and
qnixsynapse
committed
gguf-py : fix SpecialVocab parsing when post_processor is null (#14330)

CISC
authored and
qnixsynapse
committed
quantize : handle user-defined pruning of whole layers (blocks) (#13037)

EAddario
authored and
qnixsynapse
committed
vulkan: update windows SDK in CI (#14334)

jeffbolznv
authored and
qnixsynapse
committed
kv-cells : fix tracking of seq_pos (#14339)

ggerganov
authored and
qnixsynapse
committed
CUDA: mul_mat_v support for batch sizes > 1 (#14262)

JohannesGaessler
authored and
qnixsynapse
committed
llama : better rwkv chat template and add missing `inputs.use_jinja` setting (#14336)

MollySophia
authored and
qnixsynapse
committed
vulkan: update windows SDK in release.yml (#14344)

jeffbolznv
authored and
qnixsynapse
committed
ci: add workflow for relocatable cmake package (#14346)

bandoti
authored and
qnixsynapse
committed
CUDA/HIP: optimize mmv paths taken for HIP devices (#14324)

authored and
qnixsynapse
committed
jinja : Add Mistral-Small-3.2-24B-Instruct-2506.jinja (#14349)

bartowski1182
authored and
qnixsynapse
committed
main : honor --verbose-prompt on interactive prompts (#14350)

CISC
authored and
qnixsynapse
committed
server : move no API key doc to /health (#14352)

pnb
authored and
qnixsynapse
committed
cmake : use LLAMA_BUILD_NUMBER when defining LLAMA_INSTALL_VERSION (#14362)

mbaudier
authored and
qnixsynapse
committed
batch : fix check for empty sequences in memory (#14364)

ggerganov
authored and
qnixsynapse
committed
opencl: ref count `ggml_backend_opencl_context` and refactor profiling (#14254)

lhez
authored and
qnixsynapse
committed
sycl: GGML_SYCL_DISABLE_OPT on by default for all Intel Devices (#13973)

ShanoToni
authored and
qnixsynapse
committed
ggml : do not output unprintable characters on GGUF load failure (#14381)

CISC
authored and
qnixsynapse
committed
ggml-cpu: enable IBM NNPA Vector Intrinsics (#14317)

authored and
qnixsynapse
committed
musa: enable fp16 mma (all) and cublas on qy2 (#13842)

authored and
qnixsynapse
committed
docs: update s390x documentation + add faq (#14389)

taronaeo
authored and
qnixsynapse
committed
metal : batch rows copy in a single threadgroup (#14384)

ggerganov
authored and
qnixsynapse
committed
metal : add special-case mat-vec mul for ne00 == 4 (#14385)

ggerganov
authored and
qnixsynapse
committed
llama : return mistral-v7-tekken as default template only (#14390)

CISC
authored and
qnixsynapse
committed
cmake: regen vulkan shaders when shaders-gen sources change (#14398)

bandoti
authored and
qnixsynapse
committed
model : gemma3n text-only (#14400)

ngxson
authored and
qnixsynapse
committed
convert : fix broken sentencepiece vocab (#14416)

CISC
authored and
qnixsynapse
committed
ggml : add ggml_set_rows (#14274)

authored and
qnixsynapse
committed
recurrent : call balloc split_reset() in init_batch() (#14414)

ggerganov
authored and
qnixsynapse
committed
graph : make llm_graph_context destructor virtual (#14410)

ggerganov
authored and
qnixsynapse
committed
vulkan: Fix GGML_VULKAN_SHADER_DEBUG_INFO (#14427)

jeffbolznv
authored and
qnixsynapse
committed
ci : fix windows build and release (#14431)

CISC
authored and
qnixsynapse
committed
fix async_mode bug (#14432)

bachelor-dou
authored and
qnixsynapse
committed
model : add support for ERNIE 4.5 0.3B model (#14408)

ownia
authored and
qnixsynapse
committed
vulkan: lock accesses of pinned_memory vector (#14333)

jeffbolznv
authored and
qnixsynapse
committed
vulkan: handle noncontig in the final case of ggml_vk_get_cpy_pipeline (#14378)

jeffbolznv
authored and
qnixsynapse
committed
CUDA: add bf16 and f32 support to cublas_mul_mat_batched (#14361)

am17an
authored and
qnixsynapse
committed
vulkan: Add fusion support for RMS_NORM+MUL (#14366)

authored and
qnixsynapse
committed
ggml : implement REGLU/GEGLU/SWIGLU ops (#14158)

committed
ggml : fix unmerged GGML_FPxx_TO_FPxx refactoring (#14443)

CISC
authored and
qnixsynapse
committed
SYCL: disable faulty fp16 exp kernel (#14395)
qnixsynapse
committed
server : fix appearance of the chats list context menu for Safari (#14322)

rntk
authored and
qnixsynapse
committed
server : support jinja extra template kwargs (Qwen3 enable_thinking feature), from command line and from client (#13196)

authored and
qnixsynapse
committed
scripts : make the shell scripts cross-platform (#14341)

vedranmiletic
authored and
qnixsynapse
committed
cmake : Remove redundant include path in CMakeLists.txt (#14452)

xiaobing318
authored and
qnixsynapse
committed
test-backend-ops : disable llama test (#14461)

slaren
authored and
qnixsynapse
committed
ggml-cpu: sycl: Re-enable exp f16 (#14462)

Rbiessy
authored and
qnixsynapse
committed
metal : disable fast-math for some cpy kernels (#14460)

ggerganov
authored and
qnixsynapse
committed
memory : correctly handle failure in apply() (#14438)

ggerganov
authored and
qnixsynapse
committed
Add Conv2d for CPU (#14388)

am17an
authored and
qnixsynapse
committed
opencl : add GEGLU, REGLU, SWIGLU (#14456)

lhez
authored and
qnixsynapse
committed
ggml-quants : rename best_mad to best_error (ggml/1283)

danbev
authored and
qnixsynapse
committed
ggml-cpu : "align corners" for bilinear upscale/downscale (ggml/1285)

Acly
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
ggml : remove trailing whitespace (#0)

ggerganov
authored and
qnixsynapse
committed
add GELU_ERF (#14455)

CISC
authored and
qnixsynapse
committed
vulkan: Split large mul_mat_id to fit in shared memory (#14451)

jeffbolznv
authored and
qnixsynapse
committed
CANN: update aclnnGroupedMatmulV2 to aclnnGroupedMatmulV3 (#14411)

noemotiovon
authored and
qnixsynapse
committed
Add Vulkan images to docker.md (#14472)

xek
authored and
qnixsynapse
committed
ci : disable fast-math for Metal GHA CI (#14478)

ggerganov
authored and
qnixsynapse
committed
ggml : Callback before abort (#14481)

authored and
qnixsynapse
committed

Sync master with upstream release b5798#142Minh141120 merged 192 commits intodevjanhq/llama.cpp:devfrom update-dev-from-master-2025-07-02-06-38janhq/llama.cpp:update-dev-from-master-2025-07-02-06-38Copy head branch name to clipboard

Commits

Commits on Jul 2, 2025

Sync master with upstream release b5798#142
Minh141120 merged 192 commits intodevfrom
update-dev-from-master-2025-07-02-06-38