Skip to content

UPSTREAM PR #19204: ggml-virtgpu: make the code thread safe#1094

Open
loci-dev wants to merge 11 commits intomainfrom
loci/pr-19204-leaks
Open

UPSTREAM PR #19204: ggml-virtgpu: make the code thread safe#1094
loci-dev wants to merge 11 commits intomainfrom
loci/pr-19204-leaks

Conversation

@loci-dev
Copy link

Note

Source pull request: ggml-org/llama.cpp#19204

This PR improves the code of the ggml-virtgpu backend to make it thread safe, by using mutex for accessing the host<>guest shared memory buffers, and by pre-caching, during the initialization, the constant values queried from the backend.

The unused buffer_type_is_host method is also deprecated.

@loci-review
Copy link

loci-review bot commented Jan 31, 2026

No meaningful performance changes were detected across 115126 analyzed functions in the following binaries: build.bin.llama-tts, build.bin.llama-cvector-generator, build.bin.libllama.so, build.bin.libmtmd.so, build.bin.llama-tokenize, build.bin.llama-quantize, build.bin.llama-qwen2vl-cli, build.bin.libggml-base.so, build.bin.libggml-cpu.so, build.bin.libggml.so, build.bin.llama-bench, build.bin.llama-gemma3-cli, build.bin.llama-gguf-split, build.bin.llama-llava-cli, build.bin.llama-minicpmv-cli.

🔎 Full breakdown: Loci Inspector.
💬 Questions? Tag @loci-dev.

@loci-dev loci-dev force-pushed the main branch 17 times, most recently from 6fab7f9 to bbab1c9 Compare January 31, 2026 22:08
@loci-dev loci-dev force-pushed the main branch 28 times, most recently from 4d805ce to 7ff3e7f Compare February 3, 2026 08:18
@loci-review
Copy link

loci-review bot commented Feb 3, 2026

No meaningful performance changes were detected across 115126 analyzed functions in the following binaries: build.bin.llama-tts, build.bin.llama-cvector-generator, build.bin.libllama.so, build.bin.libmtmd.so, build.bin.llama-gguf-split, build.bin.llama-llava-cli, build.bin.llama-minicpmv-cli, build.bin.llama-quantize, build.bin.llama-gemma3-cli, build.bin.llama-tokenize, build.bin.llama-qwen2vl-cli, build.bin.llama-bench, build.bin.libggml-base.so, build.bin.libggml-cpu.so, build.bin.libggml.so.

🔎 Full breakdown: Loci Inspector.
💬 Questions? Tag @loci-dev.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants