UPSTREAM PR #19204: ggml-virtgpu: make the code thread safe#1094
UPSTREAM PR #19204: ggml-virtgpu: make the code thread safe#1094
Conversation
not necessary
The static init isn't thread safe.
|
No meaningful performance changes were detected across 115126 analyzed functions in the following binaries: build.bin.llama-tts, build.bin.llama-cvector-generator, build.bin.libllama.so, build.bin.libmtmd.so, build.bin.llama-tokenize, build.bin.llama-quantize, build.bin.llama-qwen2vl-cli, build.bin.libggml-base.so, build.bin.libggml-cpu.so, build.bin.libggml.so, build.bin.llama-bench, build.bin.llama-gemma3-cli, build.bin.llama-gguf-split, build.bin.llama-llava-cli, build.bin.llama-minicpmv-cli. 🔎 Full breakdown: Loci Inspector. |
6fab7f9 to
bbab1c9
Compare
4d805ce to
7ff3e7f
Compare
|
No meaningful performance changes were detected across 115126 analyzed functions in the following binaries: build.bin.llama-tts, build.bin.llama-cvector-generator, build.bin.libllama.so, build.bin.libmtmd.so, build.bin.llama-gguf-split, build.bin.llama-llava-cli, build.bin.llama-minicpmv-cli, build.bin.llama-quantize, build.bin.llama-gemma3-cli, build.bin.llama-tokenize, build.bin.llama-qwen2vl-cli, build.bin.llama-bench, build.bin.libggml-base.so, build.bin.libggml-cpu.so, build.bin.libggml.so. 🔎 Full breakdown: Loci Inspector. |
Note
Source pull request: ggml-org/llama.cpp#19204
This PR improves the code of the ggml-virtgpu backend to make it thread safe, by using mutex for accessing the host<>guest shared memory buffers, and by pre-caching, during the initialization, the constant values queried from the backend.
The unused
buffer_type_is_hostmethod is also deprecated.