Skip to content

UCT/GDA: Use gpunetio submodule - #11122

Merged
yosefe merged 19 commits into
openucx:masterfrom
Artemy-Mellanox:topic/gpunetio-use
Mar 7, 2026
Merged

yosefe merged 19 commits into
openucx:masterfrom
Artemy-Mellanox:topic/gpunetio-use

Conversation

@Artemy-Mellanox

Copy link
Copy Markdown
Contributor

No description provided.

Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread debian/rules.in Resolved
Comment thread src/tools/perf/cuda/Makefile.am Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
Comment thread src/uct/ib/mlx5/gdaki/Makefile.am Resolved
Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
Comment thread config/m4/cuda.m4 Resolved
Comment thread test/gtest/ucp/cuda/test_kernels.h Outdated
#include <ucp/api/device/ucp_host.h>
#include <ucs/sys/device_code.h>

#if defined(__NVCC__) && !defined(__builtin_ia32_prefetch)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add into a common header for this workaround to avoid multiple definitions?

Comment thread config/m4/cuda.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread ucx.spec.in Resolved
Comment thread test/gtest/uct/cuda/test_kernels.h Resolved
Comment thread src/tools/perf/cuda/cuda_common.h Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/Makefile.am Resolved
Comment thread src/uct/ib/mlx5/gdaki/Makefile.am Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread config/m4/cuda.m4 Resolved
Comment thread src/ucs/sys/device_code.h Resolved
Comment thread src/uct/ib/mlx5/gdaki/common Resolved
Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Outdated
#include <doca_gpunetio_dev_verbs_qp.cuh>
#include <infiniband/mlx5dv.h> /* TODO add to gpunetio */
#include <cuda.h> /* TODO add to gpunetio */
#include <uct/ib/mlx5/gdaki/device/doca_gpunetio_dev_verbs_qp.cuh>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we include "device/doca_gpunetio_dev_verbs_qp.cuh" ?

Comment thread config/m4/cuda.m4 Resolved
Comment thread config/m4/cuda.m4 Resolved
Comment thread config/m4/cuda.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
tvegas1
tvegas1 previously approved these changes Feb 25, 2026
Comment thread src/tools/perf/cuda/Makefile.am Resolved
Comment thread src/uct/ib/mlx5/gdaki/configure.m4 Resolved
Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Outdated
#include <doca_gpunetio_dev_verbs_qp.cuh>
#include <infiniband/mlx5dv.h> /* TODO add to gpunetio */
#include <cuda.h> /* TODO add to gpunetio */
#include "device/doca_gpunetio_dev_verbs_qp.cuh"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move line 14 after line 10

Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
Comment thread src/uct/ib/mlx5/gdaki/Makefile.am Resolved
Comment thread src/uct/ib/mlx5/gdaki/Makefile.am Resolved
Comment thread debian/ucx-ib-mlx5-gda.install Resolved
Comment thread src/tools/perf/cuda/Makefile.am Resolved
Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
@Artemy-Mellanox
Artemy-Mellanox force-pushed the topic/gpunetio-use branch 2 times, most recently from 76f2c3b to 903c544 Compare March 2, 2026 09:54
Comment thread src/tools/perf/cuda/ucp_cuda_kernel.cu Resolved
Comment on lines +204 to 205
#if HAVE_DECL_MLX5DV_UMEM_MASK_DMABUF
struct mlx5dv_devx_umem_in umem_in = {};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we unite some common code with uct_gdaki_check_umem_dmabuf?

Comment thread src/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
Comment thread stubs/uct/ib/mlx5/gdaki/gdaki.cuh Resolved
Comment thread test/gtest/ucp/cuda/test_kernels.cu Resolved
uct_elem, comp);
result.producer_index = 0;
result.ready_index = 0;
#if HAVE_MLX5_DV

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should remove this code altogether (why need GDA specific code in UCP level test kernel?)
but can be in next PR

#include <ucp/api/device/ucp_device_impl.h>
#include <ucs/debug/log.h>
#include <common/cuda.h>
#include <cuda.h>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this include to common/cuda.h

#include <uct/ib/mlx5/gdaki/gdaki.cuh>
#if __has_include(<uct/ib/mlx5/gdaki/gdaki.cuh>) && __has_include(<infiniband/mlx5dv.h>)
# include <uct/ib/mlx5/gdaki/gdaki.cuh>
# define UCT_RC_MLX5_GDA_SUPPORTED 1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd propose unified naming:
UCT_TL_RC_MLX5_GDA
UCT_TL_RC_MLX5_GDA_ENABLE
UCT_HAVE_TL_RC_MLX5_GDA

@yosefe
yosefe merged commit d9b582e into openucx:master Mar 7, 2026
152 checks passed
jeynmann pushed a commit to jeynmann/ucx that referenced this pull request Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants