Skip to content

TOOLS/PERF: Add cuda-async memory allocator #11566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tvegas1 merged 10 commits into from
Aug 27, 2026
Merged

TOOLS/PERF: Add cuda-async memory allocator #11566

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
e3bfc5e
TOOLS/PERF: Allocate cuda-async memory
tvegas1 Jun 17, 2026
ab1968d
TOOLS/PERF: Allocate cuda-async memory
tvegas1 Jun 22, 2026
cf97549
TOOLS/PERF: Allocate cuda-async memory
tvegas1 Jun 22, 2026
886bfb2
TOOLS/PERF: Allocate cuda-async memory
tvegas1 Jun 22, 2026
8d25838
TOOLS/PERF: Allocate cuda-async memory
tvegas1 Jun 23, 2026
b05832a
TOOLS/PERF: Address review comments
tvegas1 Jul 3, 2026
1922402
TOOLS/PERF: Resolve cuda-async mem type from config
tvegas1 Aug 10, 2026
f7f7ce1
TOOLS/PERF: Allocate cuda-async memory
tvegas1 Aug 10, 2026
9f1b0b3
TOOLS/PERF: Use resolve_mem_type for allocator mem type
tvegas1 Aug 10, 2026
20234df
Merge branch 'master' into perftest_cuda_async
tvegas1 Aug 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions contrib/test_jenkins.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -690,6 +690,17 @@ run_ucx_perftest() {

echo "==== Running ucx_perf with cuda memory ===="

if $ucx_perftest -h 2>&1 | grep -q "cuda-async"
then
echo "==== Running ucx_perf with cuda-async memory ===="
cuda_async_test_args="-t tag_lat -D contig,contig"
cuda_async_test_args+=" -m cuda-async,cuda-async -s 8 -n 10 -w 1"
run_client_server_app "$ucx_perftest" "$cuda_async_test_args" \
"$(hostname)" 0 0
else
echo "==== cuda-async memory is not supported, skipping ===="
fi

for memtype_cache in y n
do
for gdr in $gdr_options
Expand Down
11 changes: 7 additions & 4 deletions src/tools/perf/api/libperf.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ typedef enum {
} ucx_perf_channel_mode_t;


#define UCX_PERF_ALLOC_NAME_MAX 32 /* Max memory allocator name length */


enum ucx_perf_test_flags {
UCX_PERF_TEST_FLAG_VALIDATE = UCS_BIT(1), /* Validate data. Affects performance. */
UCX_PERF_TEST_FLAG_ONE_SIDED = UCS_BIT(2), /* For tests which involves only one side,
Expand Down Expand Up @@ -272,6 +275,10 @@ typedef struct ucx_perf_params {
ucx_perf_wait_mode_t wait_mode; /* How to wait */
ucs_memory_type_t send_mem_type; /* Send memory type */
ucs_memory_type_t recv_mem_type; /* Recv memory type */
/* Send memory allocator name */
char send_mem_alloc_name[UCX_PERF_ALLOC_NAME_MAX];

@iyastreb iyastreb Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that breaks the 1:1 relationship between memory type and allocator. And this change is quite intrusive therefore.. Maybe it's enough just to add an environment variable to force async allocayion?

@tvegas1 tvegas1 Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessarily, it just decouples memory type understood in UCX with allocation api. currently it is cuda-async, later there might be others, where memory type is still CUDA. so now we index by arg string to find allocator.

/* Recv memory allocator name */
char recv_mem_alloc_name[UCX_PERF_ALLOC_NAME_MAX];
ucx_perf_accel_dev_t send_device; /* Send memory device */
ucx_perf_accel_dev_t recv_device; /* Recv memory device */
ucs_device_level_t device_level; /* Device level */
Expand Down Expand Up @@ -338,10 +345,6 @@ typedef struct {
} ucp_perf_daemon_req_t;


/* Allocators for each memory type */
typedef struct ucx_perf_allocator ucx_perf_allocator_t;
extern const ucx_perf_allocator_t* ucx_perf_mem_type_allocators[];

/* Device test dispatchers for each memory type */
typedef struct ucx_perf_device_dispatcher ucx_perf_device_dispatcher_t;
extern const ucx_perf_device_dispatcher_t* ucx_perf_mem_type_device_dispatchers[];
Expand Down
Loading