Repository navigation
TOOLS/PERF: Add cuda-async memory allocator - #11566
Conversation
| char send_mem_alloc_name[UCX_PERF_ALLOC_NAME_MAX]; /* Send memory allocator | ||
| name */ | ||
| char recv_mem_alloc_name[UCX_PERF_ALLOC_NAME_MAX]; /* Recv memory allocator | ||
| name */ |
There was a problem hiding this comment.
Alignment/long lines
| return UCS_OK; | ||
| } | ||
|
|
||
| static ucs_status_t uct_perf_cuda_alloc_reg_mem( |
There was a problem hiding this comment.
Here and in many other places you use uct_perf prefix, shouldn't it be ucx_perf?
| return status; | ||
| } | ||
|
|
||
| status = uct_perf_cuda_reg_mem(perf, length, UCS_MEMORY_TYPE_CUDA_MANAGED, |
There was a problem hiding this comment.
can it use the ASYNC_MEM_TYPE
There was a problem hiding this comment.
i would keep it for later pr
| #if CUDART_VERSION >= 11020 | ||
| static ucx_perf_allocator_t cuda_async_allocator = { | ||
| .name = "cuda-async", | ||
| .mem_type = UCS_MEMORY_TYPE_CUDA_MANAGED, |
There was a problem hiding this comment.
can it use the ASYNC_MEM_TYPE
There was a problem hiding this comment.
i would keep it for later pr
| ucs_memory_type_t send_mem_type; /* Send memory type */ | ||
| ucs_memory_type_t recv_mem_type; /* Recv memory type */ | ||
| /* Send memory allocator name */ | ||
| char send_mem_alloc_name[UCX_PERF_ALLOC_NAME_MAX]; |
There was a problem hiding this comment.
Hmm, that breaks the 1:1 relationship between memory type and allocator. And this change is quite intrusive therefore.. Maybe it's enough just to add an environment variable to force async allocayion?
There was a problem hiding this comment.
not necessarily, it just decouples memory type understood in UCX with allocation api. currently it is cuda-async, later there might be others, where memory type is still CUDA. so now we index by arg string to find allocator.
| } | ||
|
|
||
| status = ucx_perf_cuda_uct_reg_mem(perf, length, | ||
| UCS_MEMORY_TYPE_CUDA_MANAGED, flags, |
There was a problem hiding this comment.
why managed? Can we take it from alloc_mem? on grace we detect it as cuda
| #if CUDART_VERSION >= 11020 | ||
| static ucx_perf_allocator_t cuda_async_allocator = { | ||
| .name = "cuda-async", | ||
| .mem_type = UCS_MEMORY_TYPE_CUDA_MANAGED, |
|
also added |
Honor UCX_CUDA_COPY_ASYNC_MEM_TYPE when selecting -m cuda-async instead of hardcoding cuda-managed.
Route remaining default_mem_type call sites through the allocator resolve callback.
What?
Add a named perftest memory allocator registry and a new
cuda-asyncallocator selected with-m cuda-asyncand asymmetric case-m cuda-async,cuda.Why?
Perftest currently maps allocators by memory type, which prevents multiple allocation methods from sharing the same UCX memory type.
cudaMallocAsyncneeds a separate allocator name while still using CUDA memory type metadata.How?
cudaMallocAsyncandcudaFreeAsync