Revamp how libomp.so is found#853
Conversation
An upstream llvm change enables LLVM_ENABLE_PER_TARGET_RUNTIME_DIR by default for the openmp build. This installs the openmp libraries into /opt/rocm-ver/llvm/lib/x86_64-unknown-linux-gnu instead of /opt/rocm-ver/llvm/lib. Currenty, hipBLAS only uses /lib. Since hipBLAS uses gcc by default, find_package(OpenMP) will locate libgomp from the gnu installation. We would prefer to use libomp from the ROCm install. Query amdclang to find LLVM_TARGET_TRIPLE and use as a suffix for find_library(omp). On the other hand, the user can choose to use hipcc, which allows find_package(OpenMP) to properly locate libomp inside ROCm llvm. To include legacy support, I left the hardcoded -L with use of HIP_CLANG_ROOT as a fallback. This would be much simpler if hipBLAS defaulted to hipcc as the cmake compiler.
estewart08
requested review from
NaveenElumalaiAMD,
TorreZuk,
amcamd,
bragadeesh,
daineAMD,
mahmoodw and
rkamd
as code owners
| if (TARGET OpenMP::OpenMP_CXX) | ||
| set( COMMON_LINK_LIBS "OpenMP::OpenMP_CXX") |
There was a problem hiding this comment.
So aren't we missing the OpenMP::OpenMP_CXX link request on CUDA backend?
There was a problem hiding this comment.
True, I can move set( COMMON_LINK_LIBS ${LIBOMP_FOUND}) outside of the amd guard.
| # if there is no omp.h to find the client compilation will fail and this should be obvious, used to be REQUIRED | ||
| find_package(OpenMP) | ||
| if( CMAKE_CXX_COMPILER MATCHES ".*/hipcc$" ) | ||
| find_package(OpenMP) |
There was a problem hiding this comment.
Similar question we shouldn't require NON hipcc to find_package(OpenMP) so can this just be moved two lines above? Sorry I meant other than hipcc we still want find_package to set cmake target etc.
There was a problem hiding this comment.
When the cmake compiler is hipcc find_package finds the correct ROCm libomp.so. Otherwise, gcc will get libgomp.so from gcc installation. I would rather get the libomp from ROCm to be consistent. With your current configuration cmake adds libgomp.so with -L/rpaths -lomp to the ROCm library. So I am fairly certain libgomp is ignored.
Remove amd check for rpath addtion. If we use the libomp.so from ROCm we need the rpath.
|
Hi @estewart08 are you still active on this? |
|
I am going to pick this effort back up, so I would prefer it to stay open for now. |
|
@estewart08 if picking this up again for next release time is ticking.... |
|
@estewart08 you mentioned you were going to continue this work. Any ETA? |
|
I am currently looking into providing an openmp-config.cmake file that will provide the proper location of the library. No definitive ETA at the moment. |
@estewart08 @stellaraccident This would be a big help as we are now seeing issues with tighter redhat 10 rpath restrictions so avoiding the workaround by getting full cmake config could solve newer problems. |
I don't have full state on the set of constraints that need to be solved, but I've definitely developed... Let's call it experience... On this issue and agree we need to improve the situation. Agreed that some form of standalone cmake config for this seems like the right answer. If we want to look at details, maybe look at this in upcoming office hours. |
|
Closed in favor of: #1038 |
3 participants
An upstream llvm change enables
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR by default for the openmp build. This installs the openmp libraries into
/opt/rocm-ver/llvm/lib/x86_64-unknown-linux-gnuinstead of/opt/rocm-ver/llvm/lib. Currenty, hipBLAS only uses /lib.Since hipBLAS uses gcc by default,
find_package(OpenMP)will locate libgomp from the gnu installation. We would prefer to use libomp from the ROCm install. Query amdclang to find LLVM_TARGET_TRIPLE and use as a suffix forfind_library(omp).On the other hand, the user can choose to use hipcc, which allows
find_package(OpenMP)to properly locate libomp inside ROCm llvm.To include legacy support, I left the hardcoded
-L HIP_CLANG_ROOTas a fallback.This would be much simpler if hipBLAS defaulted to hipcc as the cmake compiler.
Summary of proposed changes:
find_library(omp).find_package(OpenMP).-L HIP_CLANG_ROOT/lib.find_package(LLVM)due torocm-llvm-dev/develnot being installed by default on various operating systems, which would not have the cmake config files.