[oneMKL][spblas] updated sparse::gemm routines spec, which supports d…

…ense matrix storage layout, and introduced layout enum class. (#363)

source/elements/oneMKL/source/architecture/api_design.inc.rst

@@ -232,6 +232,32 @@ Each enumeration value comes with two names: A single-character name (the tradit
               -  Index arrays for an input matrix are provided using one-based (Fortran style) index values.  That is, indices start at 1.
 
 
+      .. _onemkl_enum_layout:
+
+      .. rubric:: layout
+         :name: layout
+         :class: sectiontitle
+
+      The ``layout`` type specifies how a dense matrix ``A`` with leading dimension ``lda`` is stored as one dimensional array in memory. 
+      The layouts are traditionally provided in one of two forms: C/C++-style using ``row_major`` layout,
+      or Fortran-style using ``column_major`` layout. The ``layout`` type can take the following values:
+
+      .. container:: tablenoborder
+
+         .. list-table::
+            :header-rows: 1
+
+            * -  Short Name
+              -  Long Name
+              -  Description
+            * -  ``layout::R``
+              -  ``layout::row_major``
+              -  For row major layout, the elements of each row of a dense matrix ``A`` are contiguous in memory while the elements of each column are at distance ``lda`` from the element in the same column and the previous row.
+            * -  ``layout::C``
+              -  ``layout::col_major``
+              -  For column major layout, the elements of each column a dense matrix ``A`` are contiguous in memory while the elements of each row are at distance ``lda`` from the element in the same row and the previous column.
+
+
 .. note::
         :ref:`onemkl_appendix` may contain other API design decisions or recommendations that may be of use to the general developer of oneMKL, but which may not necessarily be part of the oneMKL specification.
 

source/elements/oneMKL/source/domains/spblas/gemm.rst

@@ -16,9 +16,18 @@ matrix product defined as
 
 .. math::
 
-   C \leftarrow \alpha \text{op}(A) B + \beta C
+   C \leftarrow \alpha \cdot \text{op}(A) \cdot \text{op}(B) + \beta \cdot C
 
-where :math:`\alpha` and :math:`\beta` are scalars, :math:`B` and :math:`C` are dense matrices and :math:`A` is a sparse matrix. Dense matrix storage is in row-major format. Sparse matrix formats are compressed sparse row (CSR) formats. 
+where :math:`\alpha` and :math:`\beta` are scalars, :math:`A` is a sparse matrix, :math:`B` and :math:`C` are dense matrices, :math:`\text{op}()` is a matrix modifier for :math:`A` and :math:`B` using the following description:
+
+.. math::
+
+    \text{op}(A) = \begin{cases} A,& \text{ oneapi::mkl::transpose::nontrans}\\ A^{T},& \text{ oneapi::mkl::transpose::trans}\\A^{H},& \text{ oneapi::mkl::transpose::conjtrans} \end{cases}
+
+
+and :math:`\text{op}(A)` is an ``m``-by-``k`` matrix , :math:`\text{op}(B)` is an ``k``-by-``columns`` matrix, and :math:`C` is an ``m``-by-``columns`` matrix.
+
+Dense matrix storage is in either row-major or column-major format. Sparse matrix formats are compressed sparse row (CSR) formats. 
 
 
 .. _onemkl_sparse_gemm_buffer:
@@ -33,7 +42,9 @@ gemm (Buffer version)
    namespace oneapi::mkl::sparse {
 
       void gemm (sycl::queue                          &queue, 
-                 oneapi::mkl::transpose               transpose_val, 
+                 oneapi::mkl::layout                  dense_matrix_layout, 
+                 oneapi::mkl::transpose               transpose_A, 
+                 oneapi::mkl::transpose               transpose_B, 
                  const fp                             alpha, 
                  oneapi::mkl::sparse::matrix_handle_t A_handle, 
                  sycl::buffer<fp, 1>                  &B, 
@@ -56,8 +67,18 @@ gemm (Buffer version)
          kernels execution.
 
 
-    transpose_val
-         Specifies operation ``op()`` on input matrix. The possible options
+    dense_matrix_layout
+         Specifies the storage scheme in memory for the dense matrices. Note that this layout applies to both :math:`B` and :math:`C` dense matrices.
+         The possible options are described in :ref:`onemkl_enum_layout` enum class.
+
+
+    transpose_A
+         Specifies operation ``op()`` on input matrix :math:`A`. The possible options
+         are described in :ref:`onemkl_enum_transpose` enum class.
+
+
+    transpose_B
+         Specifies operation ``op()`` on input matrix :math:`B`. The possible options
          are described in :ref:`onemkl_enum_transpose` enum class.
 
 
@@ -70,26 +91,57 @@ gemm (Buffer version)
 
 
     B
-         The dense matrix in the sparse-dense matrix product. A one dimensional SYCL memory object containing an array of size at least ``cols*ldb``, where ``cols`` = the number of columns of matrix :math:`op(A)`.
+         The input dense matrix :math:`B` in the sparse matrix-dense matrix product. :math:`B` is a one dimensional SYCL memory object containing an array of size:
+
+      .. list-table::
+         :header-rows: 1
+
+         * -
+           - ``B`` not transposed
+           - ``B`` transposed
+         * - Row major
+           - ``B`` is an ``k``-by-``columns`` matrix so must have size at least ``k``\ \*\ ``ldb``.
+           - ``B`` is an ``columns``-by-``k`` matrix so must have size at least ``columns``\ \*\ ``ldb``
+         * - Column major
+           - ``B`` is an ``k``-by-``columns`` matrix so must have size at least ``ldb``\ \*\ ``columns``.
+           - ``B`` is an ``columns``-by-``k`` matrix so must have size at least ``ldb``\ \*\ ``k``
+
+      See :ref:`matrix-storage` for more details.
+
 
     columns
-         Number of columns of matrix, :math:`C`.
+         Number of columns of matrix :math:`C`.
 
 
     ldb
-         Specifies the leading dimension of matrix, :math:`B`. Should be greater than or equal to the number of columns of :math:`B` which is ``columns``.
+         Specifies the leading dimension of matrix :math:`B`. It must be positive.
+
+      .. list-table::
+         :header-rows: 1
+
+         * -
+           - ``B`` not transposed
+           - ``B`` transposed
+         * - Row major
+           - ``ldb`` must be at least ``columns``.
+           - ``ldb`` must be at least ``k``.
+         * - Column major
+           - ``ldb`` must be at least ``k``.
+           - ``ldb`` must be at least ``columns``.
 
 
     beta
          Specifies the scalar ``beta``.
 
 
     C
-         The dense matrix input/output array.  A one-dimensional SYCL memory object containing an array of size at least ``rows*ldc``, where ``rows`` = the number of rows
-         of matrix :math:`op(A)`.
+         The dense matrix input/output array.  A one-dimensional SYCL memory object containing an array of size at least ``m``\ \*\ ``ldc`` if row_major layout is used to store dense matrices
+         or at least ``ldc``\ \*\ ``columns`` if column_major layout is used to store dense matrices.
+
 
     ldc
-         Specifies the leading dimension of matrix :math:`C` . Must be greater than or equal to the number of columns of :math:`C` which is ``columns``.
+         Specifies the leading dimension of matrix :math:`C`.
+         Must be positive and at least ``columns`` if row major layout is used to store dense matrices or at least ``m`` if column major layout is used to store dense matrices.
 
 
 .. container:: section
@@ -130,7 +182,9 @@ gemm (USM version)
    namespace oneapi::mkl::sparse {
 
       sycl::event gemm (sycl::queue                           &queue, 
-                        oneapi::mkl::transpose                transpose_val, 
+                        oneapi::mkl::layout                   dense_matrix_layout, 
+                        oneapi::mkl::transpose                transpose_A, 
+                        oneapi::mkl::transpose                transpose_B, 
                         const fp                              alpha, 
                         oneapi::mkl::sparse::matrix_handle_t  A_handle, 
                         const fp                              *B, 
@@ -153,34 +207,80 @@ gemm (USM version)
          kernels execution.
 
 
-    transpose_val
-         Specifies operation ``op()`` on input matrix. The possible options
+    dense_matrix_layout
+         Specifies the storage scheme in memory for the dense matrices. Note that this layout applies to both :math:`B` and :math:`C` dense matrices.
+         The possible options are described in :ref:`onemkl_enum_layout` enum class.
+
+
+    transpose_A
+         Specifies operation ``op()`` on input matrix :math:`A`. The possible options
          are described in :ref:`onemkl_enum_transpose` enum class.
 
+
+    transpose_B
+         Specifies operation ``op()`` on input matrix :math:`B`. The possible options
+         are described in :ref:`onemkl_enum_transpose` enum class.
+
+
     alpha
          Specifies the scalar :math:`\alpha`.
 
+
     A_handle
          Handle to object containing sparse matrix, :math:`A`. Created using the oneapi::mkl::sparse::set_csr_data routine.
 
+
     B 
-         The dense matrix in the sparse-dense matrix product. A device accessible USM object containing an array of size at least ``cols*ldb``, where ``cols`` = the number of columns of matrix :math:`op(A)`.
+         The dense matrix in the sparse-dense matrix product. A device accessible USM object containing an array of size:
+
+      .. list-table::
+         :header-rows: 1
+
+         * -
+           - ``B`` not transposed
+           - ``B`` transposed
+         * - Row major
+           - ``B`` is an ``k``-by-``columns`` matrix so must have size at least ``k``\ \*\ ``ldb``.
+           - ``B`` is an ``columns``-by-``k`` matrix so must have size at least ``columns``\ \*\ ``ldb``
+         * - Column major
+           - ``B`` is an ``k``-by-``columns`` matrix so must have size at least ``ldb``\ \*\ ``columns``.
+           - ``B`` is an ``columns``-by-``k`` matrix so must have size at least ``ldb``\ \*\ ``k``
+
+      See :ref:`matrix-storage` for more details.
+
 
     columns
          Number of columns of matrix :math:`C`.
 
+
     ldb
-         Specifies the leading dimension of matrix :math:`B`. Should be greater than or equal to the number of columns of :math:`B`.
+         Specifies the leading dimension of matrix :math:`B`. It must be positive.
+
+      .. list-table::
+         :header-rows: 1
+
+         * -
+           - ``B`` not transposed
+           - ``B`` transposed
+         * - Row major
+           - ``ldb`` must be at least ``columns``.
+           - ``ldb`` must be at least ``k``.
+         * - Column major
+           - ``ldb`` must be at least ``k``.
+           - ``ldb`` must be at least ``columns``.
+
 
     beta
          Specifies the scalar ``beta``.
 
+
     C
-         The dense matrix input/output array.  A device accessible USM object containing an array of size at least ``rows*ldc``, where ``rows`` = the number of rows
-         of matrix :math:`op(A)`.
+         The dense matrix input/output array.  A device accessible USM object containing an array of size at least ``m``\ \*\ ``ldc`` if row_major layout is used to store dense matrices
+         or at least ``ldc``\ \*\ ``columns`` if column_major layout is used to store dense matrices.
 
     ldc
-         Specifies the leading dimension of matrix :math:`C` . Must be greater than or equal to ``columns``.
+         Specifies the leading dimension of matrix :math:`C`.
+         Must be positive and at least ``columns`` if row major layout is used to store dense matrices or at least ``m`` if column major layout is used to store dense matrices.
 
     dependencies
          List of events that oneapi::mkl::sparse::gemm routine depends on.