Skip to content

[windows] Patch TENSILE_LIBRARY_DIR in hipBLASLt. #783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ScottTodd merged 14 commits into from
Jul 9, 2025
Merged

[windows] Patch TENSILE_LIBRARY_DIR in hipBLASLt. #783

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
faf5a86
[windows] Patch TENSILE_LIBRARY_DIR in rocBLAS and hipBLASLt.
ScottTodd Jun 6, 2025
b099c87
[windows][ci] Enable hipBLASLt tests on Windows.
ScottTodd Jun 6, 2025
598fd07
Delete extra patch.
ScottTodd Jun 6, 2025
92c99ce
Merge remote-tracking branch 'upstream/main' into users/scotttodd/win…
ScottTodd Jun 6, 2025
59ce73a
Revert "[windows][ci] Enable hipBLASLt tests on Windows."
ScottTodd Jun 30, 2025
7709438
Merge remote-tracking branch 'upstream/main' into users/scotttodd/win…
ScottTodd Jun 30, 2025
be122d6
Patch rocblas_gentest.py invocation for better Windows compatibility.
ScottTodd Jul 1, 2025
6bea94d
Apply patch to rocBLAS next-cmake too.
ScottTodd Jul 2, 2025
374f26b
Merge remote-tracking branch 'upstream/main' into users/scotttodd/win…
ScottTodd Jul 7, 2025
fdc7610
Patch sibling search location in rocBLAS.
ScottTodd Jul 7, 2025
5d6c552
Enable hipblaslt tests on Windows again.
ScottTodd Jul 7, 2025
16d18fa
Drop gentest.py patch for now.
ScottTodd Jul 8, 2025
6493549
Rework hipBLASLt patch, drop rocBLAS patches.
ScottTodd Jul 8, 2025
031e3f8
Revert changes to patches.
ScottTodd Jul 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/test_hipblaslt.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,6 @@ jobs:
test_hipblaslt:
name: "hipBLASLt math-lib test"
runs-on: ${{ inputs.test_runs_on }}
# Currently, this test is not working with Windows. This test will be enabled once this library has been enabled
if: ${{ inputs.platform == 'linux' }}
Comment thread
ScottTodd marked this conversation as resolved.
Resolved

@ScottTodd ScottTodd Jun 6, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests pass on Windows gfx1151: https://github.com/ROCm/TheRock/actions/runs/15495327375/job/43649536258?pr=783

[----------] Global test environment tear-down
[==========] 5766 tests from 2 test suites ran. (470772 ms total)
[  PASSED  ] 5766 tests.
hipBLASLt version: 100000
hipBLASLt git version: c630f202
command line: B:\actions-runner\_work\TheRock\TheRock\build\bin\hipblaslt-test.exe --gtest_filter=*pre_checkin*

Odd checkout failures on rocBLAS and rocPRIM though:
https://github.com/ROCm/TheRock/actions/runs/15495327375/job/43649536566?pr=783

 Deleting the contents of 'B:\actions-runner\_work\TheRock\TheRock'
Error: File was unable to be removed Error: EPERM: operation not permitted, unlink 'B:\actions-runner\_work\TheRock\TheRock\build\bin\amd_comgr0605.dll'

@ScottTodd ScottTodd Jun 9, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failed again on a retry: https://github.com/ROCm/TheRock/actions/runs/15495327375/job/43740951661?pr=783#step:2:28

@amd-justchen can you debug on that runner (windows-strix-halo-gpu-rocm-1)?

@amd-justchen amd-justchen Jun 9, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! I'll take a look! I just about am done monitoring the CI CPU builders. That error looks like the same one before. I'll be able to remote in to the machine to troubleshoot.

@ScottTodd ScottTodd Jun 9, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect the build is only just now producing those files, so we should see if any permission/owner bits are set differently. Might need to change how those projects are built or make our CI workflows try harder to clean up the files during startup/shutdown.

A common post-job hook for a self-hosted runner deletes the source directory. We used these on IREE: https://github.com/iree-org/iree/tree/41dcee93c7157955d94973addf6770cecf926849/build_tools/github_actions/runner/config/hooks. See pre_job.sh, cleanup_workdir.sh, and https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/running-scripts-before-or-after-a-job.

@ScottTodd ScottTodd Jul 8, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh: https://github.com/ROCm/TheRock/actions/runs/16148483933/job/45579902751

Still failing with "File was unable to be removed" for rocblas, hipblaslt, rocprim, and hipcub tests. Going to retry those to see if the runners are flaky somehow.

strategy:
fail-fast: false
defaults:
Expand Down
5 changes: 3 additions & 2 deletions ...es/amd-mainline/hipBLASLt/0001-Guard-against-missing-directories-in-Tensile-Validat.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
From a785e82c8100d36a3d1eeafa74a9fbc356f15954 Mon Sep 17 00:00:00 2001
From 5b70d10580b63883bb0f971afd36f473307815c4 Mon Sep 17 00:00:00 2001
From: Scott Todd <scott.todd0@gmail.com>
Date: Wed, 21 May 2025 15:27:09 -0700
Subject: [PATCH] Guard against missing directories in Tensile Validators.py.
Subject: [PATCH 1/3] Guard against missing directories in Tensile
Validators.py.

---
tensilelite/Tensile/Toolchain/Validators.py | 10 ++++++++--
Expand Down
6 changes: 3 additions & 3 deletions ...002-next-cmake-Link-against-roctx64.patch → ...hipBLASLt/0002-Link-against-roctx64.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From e2cc40475092e7af4bb92b294b28046a33abf76b Mon Sep 17 00:00:00 2001
From 5aacae77fa532606e31e98a8ee27ee497084d9ae Mon Sep 17 00:00:00 2001
From: Stella Laurenzo <stellaraccident@gmail.com>
Date: Fri, 30 May 2025 18:19:06 -0700
Subject: [PATCH 2/2] [next-cmake] Link against roctx64.
Subject: [PATCH 2/3] Link against roctx64.

Prior, we were finding it but left it out of the link.
---
Expand Down Expand Up @@ -30,5 +30,5 @@ index d2ba9d75..3ba8eabc 100644

if(HIPBLASLT_ENABLE_ASAN)
--
2.43.0
2.45.1.windows.1

49 changes: 49 additions & 0 deletions ...es/amd-mainline/hipBLASLt/0003-Unify-HIPBLASLT_TENSILE_LIBRARY_DIR-across-Windows-a.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
From 996367c9bb3d8dae69654e88adeecb163ab0b282 Mon Sep 17 00:00:00 2001
From: Scott Todd <scott.todd0@gmail.com>
Date: Thu, 5 Jun 2025 17:48:02 -0700
Subject: [PATCH 3/3] Unify HIPBLASLT_TENSILE_LIBRARY_DIR across Windows and
Linux.

Splitting the structure leads to further complexity in all downstream projects and tooling.
---
library/CMakeLists.txt | 6 +-----
next-cmake/CMakeLists.txt | 6 +-----
2 files changed, 2 insertions(+), 10 deletions(-)

diff --git a/library/CMakeLists.txt b/library/CMakeLists.txt
index ccda26c5..5195f7d3 100644
--- a/library/CMakeLists.txt
+++ b/library/CMakeLists.txt
@@ -238,11 +238,7 @@ install(
)

if ( NOT BUILD_CUDA )
- if (WIN32)
- set( HIPBLASLT_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}hipblaslt/bin" CACHE PATH "path to tensile library" )
- else()
- set( HIPBLASLT_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}${CMAKE_INSTALL_LIBDIR}/hipblaslt" CACHE PATH "path to tensile library" )
- endif()
+ set( HIPBLASLT_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}${CMAKE_INSTALL_LIBDIR}/hipblaslt" CACHE PATH "path to tensile library" )
# For ASAN package, Tensile library files are not required
if( NOT ENABLE_ASAN_PACKAGING )
if( BUILD_SHARED_LIBS )
diff --git a/next-cmake/CMakeLists.txt b/next-cmake/CMakeLists.txt
index 3ba8eabc..0417749b 100644
--- a/next-cmake/CMakeLists.txt
+++ b/next-cmake/CMakeLists.txt
@@ -328,11 +328,7 @@ if(HIPBLASLT_ENABLE_HOST)
endif()

if(HIPBLASLT_ENABLE_DEVICE)
- if(WIN32)
- set(HIPBLASLT_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}hipblaslt/bin" CACHE PATH "path to tensile library")
- else()
- set(HIPBLASLT_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}${CMAKE_INSTALL_LIBDIR}/hipblaslt" CACHE PATH "path to tensile library")
- endif()
+ set(HIPBLASLT_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}${CMAKE_INSTALL_LIBDIR}/hipblaslt" CACHE PATH "path to tensile library")
rocm_install(
DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/Tensile/library
DESTINATION ${HIPBLASLT_TENSILE_LIBRARY_DIR}
--
2.45.1.windows.1

6 changes: 3 additions & 3 deletions patches/amd-mainline/rocBLAS/0001-Update-packaging-in-pip-venv-to-stabilize-setuptools.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 1eac4ebdccefefec6d69f38da8b4fdc21ad5ae68 Mon Sep 17 00:00:00 2001
From c65e49d61ed401b9e685cc89ce6b9fed85cc1086 Mon Sep 17 00:00:00 2001
From: Scott Todd <scott.todd0@gmail.com>
Date: Thu, 6 Mar 2025 15:16:19 -0800
Subject: [PATCH 1/5] Update 'packaging' in pip venv to stabilize setuptools.
Subject: [PATCH 1/3] Update 'packaging' in pip venv to stabilize setuptools.

When using setuptools>=71.0.0 and packaging<22.0, setuptools may fail with `TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'`.
---
Expand Down Expand Up @@ -29,5 +29,5 @@ index 951755ae..7c9c6cd1 100644
execute_process(
RESULT_VARIABLE rc
--
2.43.0
2.45.1.windows.1

8 changes: 4 additions & 4 deletions patches/amd-mainline/rocBLAS/0002-Find-rocm_smi-via-config-files.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 7413f1b2b01f9ddc03ccff4b4e607af6261225af Mon Sep 17 00:00:00 2001
From a50492466bdd13bcfe43767637673286cd1b7252 Mon Sep 17 00:00:00 2001
From: Marius Brehler <marius.brehler@amd.com>
Date: Wed, 12 Mar 2025 14:26:45 +0000
Subject: [PATCH 2/2] Find `rocm_smi` via config files
Subject: [PATCH 2/3] Find `rocm_smi` via config files

Use the config files provided by upstream instead of a custom finder.
---
Expand All @@ -11,7 +11,7 @@ Use the config files provided by upstream instead of a custom finder.
delete mode 100644 clients/cmake/FindROCmSMI.cmake

diff --git a/clients/CMakeLists.txt b/clients/CMakeLists.txt
index 12bd44d7..dc61a102 100644
index 153800a3..7945778e 100644
--- a/clients/CMakeLists.txt
+++ b/clients/CMakeLists.txt
@@ -193,8 +193,8 @@ if( BUILD_CLIENTS_BENCHMARKS OR BUILD_CLIENTS_TESTS)
Expand Down Expand Up @@ -83,5 +83,5 @@ index 698f6884..00000000
- IMPORTED_LOCATION "${ROCM_SMI_LIBRARY}"
- INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "${ROCM_SMI_ROOT}/include")
--
2.43.0
2.45.1.windows.1

31 changes: 31 additions & 0 deletions patches/amd-mainline/rocBLAS/0003-Unify-ROCBLAS_TENSILE_LIBRARY_DIR-across-Windows-and.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
From cd43be77cf5dc4c2cf46d2449c4dddc186da3880 Mon Sep 17 00:00:00 2001
From: Scott Todd <scott.todd0@gmail.com>
Date: Thu, 5 Jun 2025 17:51:14 -0700
Subject: [PATCH 3/3] Unify ROCBLAS_TENSILE_LIBRARY_DIR across Windows and
Linux.

Splitting the structure leads to further complexity in all downstream projects and tooling.
---
library/src/CMakeLists.txt | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/library/src/CMakeLists.txt b/library/src/CMakeLists.txt
index f4bdfb5f..e39c74cf 100644
--- a/library/src/CMakeLists.txt
+++ b/library/src/CMakeLists.txt
@@ -820,11 +820,7 @@ rocm_install_targets(
)

if( BUILD_WITH_TENSILE )
- if (WIN32)
- set( ROCBLAS_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}/bin/rocblas" CACHE PATH "path to tensile library" )
- else()
- set( ROCBLAS_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}${CMAKE_INSTALL_LIBDIR}/rocblas" CACHE PATH "path to tensile library" )
- endif()
+ set( ROCBLAS_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}${CMAKE_INSTALL_LIBDIR}/rocblas" CACHE PATH "path to tensile library" )
# For ASAN package, Tensile library files(which are not shared libraries) are not required
if( NOT ENABLE_ASAN_PACKAGING )
if( BUILD_SHARED_LIBS )
--
2.45.1.windows.1