Repository navigation
make hip-tests compileable with TheRock - #1624
Conversation
e6f7581 to
a37743c
Compare
|
With the amount of files changed, I would feel more confident with SMEs from the hip runtime team reviewing, plus links to successful logs from both the legacy CI systems once the corresponding changes are made there plus logs from TheRock build system, when paired with ROCm/TheRock#2001 |
| if(NOT THEROCK_BUILD_MODE) | ||
| # Setup catch2 manually | ||
| include(FetchContent) | ||
| FetchContent_Declare( | ||
| Catch2 | ||
| URL "https://rocm-third-party-deps.s3.us-east-2.amazonaws.com/Catch2-3.8.1.tar.gz" | ||
| DOWNLOAD_EXTRACT_TIMESTAMP OFF | ||
| URL_HASH SHA256=18b3f70ac80fccc340d8c6ff0f339b2ae64944782f8d2fca2bd705cf47cadb79 | ||
| ) | ||
| FetchContent_MakeAvailable(Catch2) | ||
| else() | ||
| # TheRock has already setup Catch2 for us | ||
| find_package(Catch2 REQUIRED) | ||
| endif() |
There was a problem hiding this comment.
Copying a comment from ROCm/TheRock#2001 (comment)
I don't think a custom flag (especially not one referencing TheRock) should really be needed here. Other subprojects use find_package without such requirements. See https://github.com/ROCm/TheRock/blob/main/docs/development/dependencies.md for best practices. I think this general pattern should work in this case too: https://github.com/ROCm/rocm-libraries/blob/68b549eb4d2ad4a936526a60f79e0bdd91e7ad34/projects/hipblas/cmake/get-rocm-cmake.cmake#L6-L13
find_package(PackageName ...)
if(NOT PackageName_FOUND)
FetchContent_Declare(
...
)
FetchContent_MakeAvailable(...)
find_pacakge(PackageName ... PATHS ...)
)There was a problem hiding this comment.
made some changes, have a look
There was a problem hiding this comment.
Thanks, the changes in projects/hip-tests/catch/CMakeLists.txt look better to me now.
By the way, reviews are easier if you avoid rewriting git history and force pushing. Prefer to use merges instead of history rewriting rebases to sync your branch and push new commits in response to comments. I can't easily see what specifically changed between my prior comment and now:
- This force push compare shows unrelated changes:
5bff44a to
b3d4de3
Compare
ef80c5c to
6199930
Compare
80af2c7 to
33548c0
Compare
87bf9b7 to
d7103b9
Compare
| TEST_SRC ${TEST_SRC} | ||
| TEST_TARGET_NAME build_tests | ||
| COMPILE_OPTIONS -std=c++17) No newline at end of file | ||
| TEST_TARGET_NAME build_tests) No newline at end of file |
There was a problem hiding this comment.
missing new-line at the end here
| TEST_SRC ${TEST_SRC} | ||
| TEST_TARGET_NAME build_tests | ||
| COMPILE_OPTIONS -std=c++17) No newline at end of file | ||
| TEST_TARGET_NAME build_tests) No newline at end of file |
There was a problem hiding this comment.
missing new-line at the end
| fp16_ops.cc | ||
| fp8_host.cc | ||
| fp8_e8m0.cc | ||
| #fp8_e8m0.cc |
There was a problem hiding this comment.
Can a comment be added as to why this is now commented out?
| constexpr size_t num_of_ops = 18; | ||
| constexpr size_t iters = 100; | ||
| Catch::Generators::RandomFloatingGenerator<float> input1_gen(2.2f, 10.f); | ||
| Catch::Generators::RandomFloatingGenerator<float> input1_gen(2.2f, 10.f, /*seed*/ 0x1234); |
There was a problem hiding this comment.
Is the intention to always use the same seed?
There was a problem hiding this comment.
At the moment yes, the old catch2 API did not accept any seed. The new one does.
There was a problem hiding this comment.
Are there any concerns that there is a gap in testing if it's always the same seed?
There was a problem hiding this comment.
I do not think so, but let me check the API of old catch2's interface and see how they handled seed.
| TEST_TARGET_NAME build_tests | ||
| COMPILE_OPTIONS -std=c++17) | ||
| TEST_TARGET_NAME build_tests) | ||
| target_link_libraries(GLInteropTest OpenGL::GL OpenGL::EGL GLUT::GLUT) No newline at end of file |
There was a problem hiding this comment.
missing new-line here
| message(STATUS "Doxygen was not found. Building test plan documentation will not be available") | ||
| endif() | ||
|
|
||
|
|
There was a problem hiding this comment.
One too many lines here
| inccmake_files = glob.glob(build_folder + "/**/*"+inc_cmake_pattern, recursive=True) | ||
| make_test_files_portable(inccmake_files) | ||
|
|
||
|
|
There was a problem hiding this comment.
extra lines at the end here
| #print(f"In relative_paths.py [1]: build folder: {build_folder}") | ||
| #print(f"In relative_paths.py [2]: install folder: {install_folder}") | ||
| else: | ||
| print(f"Args not provided. Please provide arg1-build folder-catch_tests, arg2-install folder") |
There was a problem hiding this comment.
Perhaps providing an example of the expected args would be helpful here.
There was a problem hiding this comment.
this script is a workaround script that handles some ctest/catch downsides and is only called within the hip-tests packaging build files. . I have an open item to completely get rid of this file and have a structural fix.
Also, the example is in the comment.
1st arg - build folder catch_tests
2nd arg - install folder the user wishes to install the tests.
| "/opt/rocm/bin/rocm_agent_enumerator | awk '$0 != \"gfx000\"' | xargs | sed -e 's/ /;/g' | " | ||
| "tr -d '\n'"; | ||
| constexpr std::string_view rocm_agent_enum = "rocm_agent_enumerator"; | ||
| std::filesystem::path default_rocm_bin_path = "/opt/rocm"; |
There was a problem hiding this comment.
Can we remove this default path value and still have it work?
There was a problem hiding this comment.
No, we need to find rocm_agent_enumerator. We allow user to point to it with ROCM_PATH, if we do not find it, we need a fallback path.
There was a problem hiding this comment.
OK. Note that this fallback probably should be removed for ROCm 8.0 onward.
There was a problem hiding this comment.
Not sure what you mean here. hip-tests need to find rocm_agent_enum, users can point to it via setting environment variable ROCM_PATH which is what we do in TheRock enablement cmake change. But if user has not defined that variable fallback path is checking /opt/rocm. Are we changing the installation directory of /opt/rocm in ROCm 8.0?
rahulc-gh
left a comment
There was a problem hiding this comment.
skipping windows CI gates based on - https://mlsejenkinsvm.amd.com/job/sandbox/job/hip-rt-psdb/9828/
## Motivation Resolved: SWDEV-566226 The current implementation of agents inside of rocprof-systems keeps just the minimal necessary set of information required for populating the `info_agent` table inside of rocpd database. There is a sufficient amount of data that is being left out from database, so this change should fix that and store the additional agent information as an `extdata` row inside of `info_agent` table. ## Technical Details This PR introduces additional filed inside of `agent` structure inside which is representing the JSON formatted string of all the additional information we can acquire about particular agent. This data is processed and added during the initial fetching of agents, and afterwards pushed inside of the database. [rocm-systems] ROCm/rocm-systems#1624 (commit 8e1aee6)
## Motivation This change enables hip-tests to be built on TheRock. This is the first step in getting hip-tests building/running with TheRock. ## Technical Details Enable hip-tests to be built with TheRock, when user enables testing via flag `THEROCK_BUILD_TESTING`. [hip-tests](https://github.com/ROCm/rocm-systems/tree/develop/projects/hip-tests/catch) are collection of tests that are used to test the hip-runtime. A corresponding PR is also required in hip-tests to make it compatible with TheRock, which are being done [here](ROCm/rocm-systems#1624). Subsequent PRs are required to run the built hip-tests which will follow after this PR. ROCm/rocm-systems#2187 - PR will fix the paths to be consistent across linux and windows. Below issues will be fixed with it https://github.com/ROCm/TheRock/pull/2373/files#r2582804642 #2373 (comment) #2001 (comment) #2001 (comment) ## Test Plan The build should complete end to end without failures, next set of changes will enable the running of tests. ## Test Result NA ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
## Motivation This change enables hip-tests to be built on TheRock. This is the first step in getting hip-tests building/running with TheRock. ## Technical Details Enable hip-tests to be built with TheRock, when user enables testing via flag `THEROCK_BUILD_TESTING`. [hip-tests](https://github.com/ROCm/rocm-systems/tree/develop/projects/hip-tests/catch) are collection of tests that are used to test the hip-runtime. A corresponding PR is also required in hip-tests to make it compatible with TheRock, which are being done [here](ROCm/rocm-systems#1624). Subsequent PRs are required to run the built hip-tests which will follow after this PR. ROCm/rocm-systems#2187 - PR will fix the paths to be consistent across linux and windows. Below issues will be fixed with it https://github.com/ROCm/TheRock/pull/2373/files#r2582804642 #2373 (comment) #2001 (comment) #2001 (comment) ## Test Plan The build should complete end to end without failures, next set of changes will enable the running of tests. ## Test Result NA ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
## Motivation This change enables hip-tests to be built on TheRock. This is the first step in getting hip-tests building/running with TheRock. ## Technical Details Enable hip-tests to be built with TheRock, when user enables testing via flag `THEROCK_BUILD_TESTING`. [hip-tests](https://github.com/ROCm/rocm-systems/tree/develop/projects/hip-tests/catch) are collection of tests that are used to test the hip-runtime. A corresponding PR is also required in hip-tests to make it compatible with TheRock, which are being done [here](ROCm/rocm-systems#1624). Subsequent PRs are required to run the built hip-tests which will follow after this PR. ROCm/rocm-systems#2187 - PR will fix the paths to be consistent across linux and windows. Below issues will be fixed with it https://github.com/ROCm/TheRock/pull/2373/files#r2582804642 #2373 (comment) #2001 (comment) #2001 (comment) ## Test Plan The build should complete end to end without failures, next set of changes will enable the running of tests. ## Test Result NA ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
## Motivation This change enables hip-tests to be built on TheRock. This is the first step in getting hip-tests building/running with TheRock. ## Technical Details Enable hip-tests to be built with TheRock, when user enables testing via flag `THEROCK_BUILD_TESTING`. [hip-tests](https://github.com/ROCm/rocm-systems/tree/develop/projects/hip-tests/catch) are collection of tests that are used to test the hip-runtime. A corresponding PR is also required in hip-tests to make it compatible with TheRock, which are being done [here](ROCm/rocm-systems#1624). Subsequent PRs are required to run the built hip-tests which will follow after this PR. ROCm/rocm-systems#2187 - PR will fix the paths to be consistent across linux and windows. Below issues will be fixed with it https://github.com/ROCm/TheRock/pull/2373/files#r2582804642 ROCm#2373 (comment) ROCm#2001 (comment) ROCm#2001 (comment) ## Test Plan The build should complete end to end without failures, next set of changes will enable the running of tests. ## Test Result NA ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
## Motivation Enable hip-tests execution on TheRock linux and windows. Aligns installation path of hip-tests on linux and Windows enables sharding of hip-tests ## Details Enable hip-tests execution on TheRock linux and windows. **This PR fixes -** #1354 **Hip-tests source that gets built/executed as part of this PR** https://github.com/ROCm/rocm-systems/tree/develop/projects/hip-tests/catch **Related PRs to enable build/execution of tests -** #2373 - enables building of tests ROCm/rocm-systems#1624 - prepares rocm-systems to be built with catchlib from upstream , compiler/hip detection, packaging change ROCm/rocm-systems#2187 - aligns installation path for linux and windows **few other script changes to have compatible builds on older build frameworks.** https://github.com/AMD-ROCm-Internal/build-infra/pull/316 https://github.com/AMD-Radeon-Driver/compute-utils/pull/169 https://github.com/AMD-Radeon-Driver/compute-utils/pull/171 hip-tests executed and verified successfully ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests. --------- Co-authored-by: Geo Min <geomin12@amd.com>
## Motivation Enable hip-tests execution on TheRock linux and windows. Aligns installation path of hip-tests on linux and Windows enables sharding of hip-tests ## Details Enable hip-tests execution on TheRock linux and windows. **This PR fixes -** #1354 **Hip-tests source that gets built/executed as part of this PR** https://github.com/ROCm/rocm-systems/tree/develop/projects/hip-tests/catch **Related PRs to enable build/execution of tests -** #2373 - enables building of tests ROCm/rocm-systems#1624 - prepares rocm-systems to be built with catchlib from upstream , compiler/hip detection, packaging change ROCm/rocm-systems#2187 - aligns installation path for linux and windows **few other script changes to have compatible builds on older build frameworks.** https://github.com/AMD-ROCm-Internal/build-infra/pull/316 https://github.com/AMD-Radeon-Driver/compute-utils/pull/169 https://github.com/AMD-Radeon-Driver/compute-utils/pull/171 hip-tests executed and verified successfully ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests. --------- Co-authored-by: Geo Min <geomin12@amd.com>


Motivation
Make changes through out hip-tests to make sure it compileable with TheRock
Technical Details
Test Plan
Existing tests should pass without any issues
Test Result
running tests
Submission Checklist