[rocThrust] Enable host-only rocThrust builds with g++/clang++

[umfranzw]

Motivation

Currently, all rocThrust build configurations depend on rocPRIM. Since rocPRIM requires a GPU, this makes it tricky to use rocThrust for host-only computation.

In addition, since building rocPRIM requires a hip-aware clang (eg. hipcc), so does building rocThrust. This means you can't use gcc or vanilla-clang to build rocThrust, even if you're only doing host-side compute.

Upstream thrust does allow users to build for host-only compute, and this is typically done with gcc. It turns out that there are some differences between the way that clang/hipcc and gcc interpret the C++ standard. This can lead to situations where applications that work when built with Thrust do not work when built with rocThrust. Addressing these situations creates additional load for the compiler team.

Technical Details

This change makes modifications so that rocThrust can be used in a host-only CPP mode that does not require rocPRIM. This allows you to use g++ or a non-HIP-enabled clang++ as a compiler. Changes include:

• Added a new cmake option, ROCTHRUST_DEVICE_SYSTEM, which is similar to Thrust's THRUST_DEVICE_SYSTEM macro. It can be set to one of: HIP, CUDA, TBB, OpenMP, or CPP. If it's set to HIP or CUDA, then the code links against device dependencies (eg. for HIP, it looks for rocPRIM). If it's set to CPP, then it does not link against device dependencies.

• Because the benchmarks, tests, and examples all currently depend on rocPRIM, I've added a cmake fatal error that triggers when rocThrust is built with device system CPP and benchmarks, tests, or examples are enabled. We can probably enable many of the tests going forward, but it'll require a bit of work, so it may be best to follow-up on that in a separate change.

• The cmake dependencies file has been tweaked so we don't try to fetch rocPRIM when ROCTHRUEST_DEVICE_SYSTEM == CPP.

• I've modified the cmake compiler verification and rocThrust device_system.h header to allow gcc to be used as a host compiler again.

• I updated a number of cases where we were assuming that we could call into rocPRIM as a fallback. I've added a macro _THRUST_USE_ROCPRIM that's set to 1 in cases where have access to it.

Test Plan

Since we can't currently run the normal tests, you can verify that this works by building the simple example below using g++ and vanilla-clang++.

• cd rocm-libraries/projects/rocthrust
• ROCM_PATH=/opt/rocm CXX=g++ cmake -B build -DBUILD_BENCHMARK=OFF -DBUILD_TEST=OFF -DROCTHRUST_DEVICE_SYSTEM=CPP
• cd build
• make install
• cd ~
• save the following example into a file called example.cpp:

#include "thrust/execution_policy.h"
#include "thrust/device_vector.h"
#include "thrust/host_vector.h"

#include <iostream>
#include <cassert>

int main(int argc, char* argv[])
{
	using T = int;
	thrust::host_vector<T> h_data = {1, 2, 3, 4, 5, 6};
	thrust::device_vector<T> d_data = h_data;
	thrust::inclusive_scan(thrust::device, d_data.begin(), d_data.end(), d_data.begin(), 0, thrust::plus<T>());
	h_data = d_data;
	assert(h_data == std::vector<T>({1, 3, 6, 10, 15, 21}));
	return 0;
}

Now build and run with:

• g++ -I /opt/rocm/include example.cpp
• ./a.out

Ensure that the example builds and runs successfully.
You can repeat this process using the vanilla clang++ located at /opt/rocm/llvm/bin/clang++. Note that you'll need to delete the build directory and start over, changing the CXX value in the second command to /opt/rocm/llvm/bin/clang++.

Test Result

The example compiles and runs correctly with both g++ and clang++.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #2363       +/-   ##
============================================
+ Coverage    54.54%   84.07%   +29.53%     
============================================
  Files           14      501      +487     
  Lines         3768    42472    +38704     
  Branches       578     5459     +4881     
============================================
+ Hits          2055    35707    +33652     
- Misses        1468     2499     +1031     
- Partials       245     4266     +4021     

Flag	Coverage Δ
hipFFT	?
rocThrust	84.07% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
projects/rocthrust/thrust/detail/alignment.h	100.00% <ø> (ø)
projects/rocthrust/thrust/detail/config/libcxx.h	100.00% <ø> (ø)

... and 513 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.