[ScottTodd]

This gets hipBLASLt tests passing on Windows in TheRock (see ROCm/TheRock#783), for ROCm/TheRock#681. It also enables PyTorch builds on Windows to use hipBLASLt without needing to manually copy folders around in the install directory.

This directory structure is now consistent with rocBLAS:

rocm-libraries/projects/rocblas/library/src/CMakeLists.txt

Lines 825 to 830 in c7fe831

	if( BUILD_WITH_TENSILE )
	if (WIN32)
	set( ROCBLAS_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}/bin/rocblas" CACHE PATH "path to tensile library" )
	else()
	set( ROCBLAS_TENSILE_LIBRARY_DIR "\${CPACK_PACKAGING_INSTALL_PREFIX}${CMAKE_INSTALL_LIBDIR}/rocblas" CACHE PATH "path to tensile library" )
	endif()

[davidd-amd]

We will need to reboot tests

[ScottTodd]

We will need to reboot tests

Yep. @geomin12 the "TheRock CI" workflows were skipped, so required checks there won't be satisfied until that part of #479 is resolved. We should also get hipblaslt and other subprojects added to here:

rocm-libraries/.github/scripts/therock_matrix.py

Lines 1 to 16 in dc76f81

	"""
	This dictionary is used to map specific file directory changes to the corresponding build flag and tests
	"""
	subtree_to_project_map = {
	"projects/rocprim": "prim",
	"projects/rocthrust": "prim",
	"projects/hipcub": "prim"
	}
	project_map = {
	"prim": {
	"cmake_options": "-DTHEROCK_ENABLE_PRIM=ON -DTHEROCK_ENABLE_ALL=OFF",
	"project_to_test": "rocprim, rocthrust, hipcub",
	"subtree_checkout": "projects/rocprim\nprojects/hipcub\nprojects/rocthrust",
	},
	}

Not sure about the "mci/rocm-libraries/" workflow, at least from a public developer perspective... I can look at the details and ask around internally with my AMD account though.

[ScottTodd]

Not sure about the "mci/rocm-libraries/" workflow, at least from a public developer perspective... I can look at the details and ask around internally with my AMD account though.

I followed the link and clicked "rebuild".

[geomin12]

We will need to reboot tests

Yep. @geomin12 the "TheRock CI" workflows were skipped, so required checks there won't be satisfied until that part of #479 is resolved. We should also get hipblaslt and other subprojects added to here:

rocm-libraries/.github/scripts/therock_matrix.py

Lines 1 to 16 in dc76f81

	"""
	This dictionary is used to map specific file directory changes to the corresponding build flag and tests
	"""
	subtree_to_project_map = {
	"projects/rocprim": "prim",
	"projects/rocthrust": "prim",
	"projects/hipcub": "prim"
	}
	project_map = {
	"prim": {
	"cmake_options": "-DTHEROCK_ENABLE_PRIM=ON -DTHEROCK_ENABLE_ALL=OFF",
	"project_to_test": "rocprim, rocthrust, hipcub",
	"subtree_checkout": "projects/rocprim\nprojects/hipcub\nprojects/rocthrust",
	},
	}

Not sure about the "mci/rocm-libraries/" workflow, at least from a public developer perspective... I can look at the details and ask around internally with my AMD account though.

yes, I plan on adding those cmake config fairly soon and getting those workflows updated

[TorreZuk]

Looks good to me if your windows tests pass.

You could fix the string "path to tensile library" to be "path to tensileLite library" (or whatever name should be the standard naming for tensile lite) but that is fine to clarify in separate PR as widespread documentation needs to be changed to avoid confusion. We shouldn't have two different libraries both referred to as tensile.

Would have been nice to have the math-CI windows pipelines running as well as a reference point, as PRs were made by multiple groups but never merged.

[ScottTodd]

@ellosel (or other hipblaslt developers) any recommendations for names to use or other text to include in these comments?

[ScottTodd]

Also yes, we'll get all the relevant CI pipelines hooked up so these sorts of changes can be tested by automation.

I'm juggling a bunch of patches but once I land them all I think we'll have CI-produced PyTorch wheels on Windows that can pass sanity checks. I built with all my patches except this one today and got this far:

>>> import torch
>>> batch_matrix1 = torch.ones(10, 2, 3, device="cuda")
>>> batch_matrix2 = torch.ones(10, 3, 4, device="cuda")
>>> result = torch.bmm(batch_matrix1, batch_matrix2)

rocBLAS error: Cannot read D:\projects\TheRock\external-builds\pytorch\.venv\Lib\site-packages\_rocm_sdk_libraries_gfx110X_dgpu\bin\/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1100
 List of available TensileLibrary Files :

rocBLAS error: Could not initialize Tensile host:
directory_iterator::directory_iterator: The system cannot find the path specified.: "D:\projects\TheRock\external-builds\pytorch\.venv\Lib\site-packages\_rocm_sdk_libraries_gfx110X_dgpu\bin\/rocblas/library"

Once the patches are landed and some remaining CI pipelines are wired up we'll have those sorts of PyTorch tests running on ~every commit, for both Linux and Windows.

[ScottTodd]

All workflow runs completed. Can someone from CODEOWNERS please review this?

[ScottTodd]

Ping? Still waiting on a code owner review.

[stellaraccident]

Fine for now. In the future, we want consistent directory layouts for kernel libraries across the operating systems (i.e. kernel libraries in bin is a bug). But fine to make this consistent for the moment.