Skip to content

Optimize Vulkan buffer transfers on UMA (Unified Memory Architecture) devices #22462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
winstonma wants to merge 27 commits into from
Closed

Optimize Vulkan buffer transfers on UMA (Unified Memory Architecture) devices #22462

Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
80e793a
vulkan: avoid preferring transfer queue on AMD UMA devices
winstonma Apr 28, 2026
da5e315
Optimize Vulkan buffer transfers on UMA (Unified Memory Architecture)…
winstonma Apr 28, 2026
fe1eb03
vulkan: centralize UMA read heuristic and configurable threshold
winstonma Apr 28, 2026
bd5db36
fix incorrect async/event ordering on Vulkan, where a host read could…
winstonma May 1, 2026
91176d3
implement UMA write threshold to avoid non-cached memory penalty
winstonma May 2, 2026
dfcc950
implement an automatic calibration system for UMA (Unified Memory Arc…
winstonma May 2, 2026
8819a4f
refactoring and cleanup pass for this PR
winstonma May 2, 2026
f1c0532
reverses the decision criteria for using direct memory access when th…
winstonma May 2, 2026
4ecad4f
Merge commit 'refs/pull/22455/head' of https://github.com/ggml-org/ll…
winstonma May 2, 2026
3139dcf
fixes measurement bias that was causing suboptimal transfer strategy …
winstonma May 4, 2026
6133345
removes dead code structure in ggml_vk_buffer_read_2d_asyn
winstonma May 4, 2026
9309d72
Fixed the indentation inconsistency
winstonma May 4, 2026
bd7701b
added UMA write thresholding for ggml_vk_buffer_memset_async and ggml…
winstonma May 4, 2026
6f85dc0
revert prefers_transfer_queue definition and comments
winstonma May 4, 2026
8f2fb72
use read/write barrier to address potential race conditions
winstonma May 4, 2026
d38def3
cleanup and deduplication
winstonma May 4, 2026
e7db9e1
cleanup and deduplication #2
winstonma May 4, 2026
630716e
follows best practices for handling platform-specific size differences
winstonma May 4, 2026
5c78cdd
added a two-line comment explaining the contract
winstonma May 4, 2026
ab18b5a
Revert "fix incorrect async/event ordering on Vulkan"
winstonma May 4, 2026
b58976a
making the barrier a silent no-op
winstonma May 4, 2026
bd0a0ff
removed the premature UMA direct transfer check in ggml_vk_buffer_rea…
winstonma May 4, 2026
2aad038
fixing slow read speed
winstonma May 4, 2026
c52584d
fix calibration
winstonma May 5, 2026
e176a81
remove flush cache
winstonma May 5, 2026
8c67e77
refactor and optimize the calibration process
winstonma May 5, 2026
0ae5d6a
adjusted the read/write logic
winstonma May 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading