Skip to content

CUDA: enable the CUB path on HIP via hipCUB #26592

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Geramy wants to merge 11 commits into
base: master
Choose a base branch
from
+42 −44
Open

CUDA: enable the CUB path on HIP via hipCUB #26592

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e1b2fba
CUDA: enable the CUB path on HIP via hipCUB
Geramy Aug 4, 2026
aa58789
added missing newline.
Geramy Aug 4, 2026
624eb53
ci : install hipCUB headers
sszymczy Aug 5, 2026
85d25f6
cuda : add missing error checks
sszymczy Aug 5, 2026
5e5cd73
Update ggml/src/ggml-cuda/vendors/hip.h
Geramy Aug 5, 2026
f58f024
Made adjustments as requested, removing cub-compat and using vendor s…
Geramy Aug 5, 2026
d072d32
disabled tests as requested by @fairydreaming but i want to leave beh…
Geramy Aug 7, 2026
5dcaf72
changed use_cub to GGML_CUDA_USE_CUB as directed by @CISC and also mo…
Geramy Aug 12, 2026
527fcad
Resolve rocPRIM crashing.
Geramy Aug 28, 2026
861abb2
Merge branch 'master' into Geramy/argsort_top_k_cub_on_hip
Geramy Aug 28, 2026
f969e23
Merge branch 'master' into Geramy/argsort_top_k_cub_on_hip
Geramy Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions ggml/src/ggml-cuda/argsort.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
#include "argsort.cuh"

#ifdef GGML_CUDA_USE_CUB
# include <cub/cub.cuh>
# include "cub-compat.cuh"
// CCCL_* are not defined for hipCUB, so these stay disabled on HIP
# if (CCCL_MAJOR_VERSION >= 3 && CCCL_MINOR_VERSION >= 1)
# define STRIDED_ITERATOR_AVAILABLE
# include <cuda/iterator>
# endif
# endif // CCCL_MAJOR_VERSION >= 3 && CCCL_MINOR_VERSION >= 1
using namespace cub;
#endif // GGML_CUDA_USE_CUB

Expand Down
4 changes: 2 additions & 2 deletions ggml/src/ggml-cuda/common.cuh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,9 +107,9 @@
#define GGML_CUDA_CC_IS_QY2(cc) (cc >= GGML_CUDA_CC_QY2 && cc < GGML_CUDA_CC_PH1)
#define GGML_CUDA_CC_IS_PH1(cc) (cc >= GGML_CUDA_CC_PH1)

#if !defined(GGML_USE_HIP) && !defined(GGML_USE_MUSA) && CUDART_VERSION >= 11070
#if !defined(GGML_USE_MUSA) && (defined(GGML_USE_HIP) || CUDART_VERSION >= 11070)
# define GGML_CUDA_USE_CUB
#endif // !defined(GGML_USE_HIP) && !defined(GGML_USE_MUSA) && CUDART_VERSION >= 11070
#endif// !defined(GGML_USE_MUSA) && (defined(GGML_USE_HIP) || CUDART_VERSION >= 11070)

// PDL host-side support (cudaLaunchKernelEx) requires CUDART >= 11.8.
// However, this has been bugged in CTK < 12.3 for MSVC builds, see
Expand Down
7 changes: 7 additions & 0 deletions ggml/src/ggml-cuda/cub-compat.cuh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#pragma once
#if defined(GGML_USE_HIP)
# include <hipcub/hipcub.hpp>
namespace cub = hipcub;
#else
# include <cub/cub.cuh>
#endif
Comment thread
Geramy marked this conversation as resolved.
Outdated
2 changes: 1 addition & 1 deletion ggml/src/ggml-cuda/cumsum.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
#include "ggml.h"

#ifdef GGML_CUDA_USE_CUB
# include <cub/cub.cuh>
# include "cub-compat.cuh"
#endif // GGML_CUDA_USE_CUB

template<typename T, int BLOCK_SIZE>
Expand Down
2 changes: 1 addition & 1 deletion ggml/src/ggml-cuda/mean.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#include "reduce_rows.cuh"

#ifdef GGML_CUDA_USE_CUB
#include <cub/cub.cuh>
#include "cub-compat.cuh"
using namespace cub;
#endif // GGML_CUDA_USE_CUB

Expand Down
2 changes: 1 addition & 1 deletion ggml/src/ggml-cuda/ssm-scan.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#endif // !defined(GGML_USE_HIP) && !defined(GGML_USE_MUSA) && CUDART_VERSION >= 11070

#ifdef USE_CUB
#include <cub/cub.cuh>
#include "cub-compat.cuh"
using namespace cub;
#endif // USE_CUB
Comment thread
Geramy marked this conversation as resolved.
Outdated

Expand Down
2 changes: 1 addition & 1 deletion ggml/src/ggml-cuda/sum.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#include "sumrows.cuh"

#ifdef GGML_CUDA_USE_CUB
#include <cub/cub.cuh>
#include "cub-compat.cuh"
using namespace cub;
#endif // GGML_CUDA_USE_CUB

Expand Down
3 changes: 2 additions & 1 deletion ggml/src/ggml-cuda/top-k.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
#include "top-k.cuh"

#ifdef GGML_CUDA_USE_CUB
# include <cub/cub.cuh>
# include "cub-compat.cuh"
// hipCUB has no DeviceTopK and does not define CCCL_*, so this stays disabled on HIP
# if (CCCL_MAJOR_VERSION >= 3 && CCCL_MINOR_VERSION >= 2)
# define CUB_TOP_K_AVAILABLE
# include <cuda/iterator>
Expand Down
3 changes: 3 additions & 0 deletions ggml/src/ggml-cuda/vendors/hip.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@
#define cudaStreamPerThread hipStreamPerThread
#define cudaStreamSynchronize hipStreamSynchronize
#define cudaStreamWaitEvent hipStreamWaitEvent
#define cudaStreamIsCapturing hipStreamIsCapturing
#define cudaStreamCaptureStatus hipStreamCaptureStatus
#define cudaStreamCaptureStatusNone hipStreamCaptureStatusNone
Comment thread
Geramy marked this conversation as resolved.
Outdated
#define cudaGraphExec_t hipGraphExec_t
#define cudaGraphNode_t hipGraphNode_t
#define cudaKernelNodeParams hipKernelNodeParams
Expand Down
Loading