Skip to content

Add support for BLAS and LAPACK dependencies (continued) #14773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mgorny wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add support for BLAS and LAPACK dependencies (continued) #14773

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion ci/ciimage/arch/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ pkgs=(
doxygen vulkan-headers vulkan-icd-loader vulkan-validation-layers openssh mercurial gtk-sharp-3 qt5-tools
libwmf cmake netcdf-fortran openmpi nasm gnustep-base gettext
python-lxml hotdoc rust-bindgen qt6-base qt6-tools qt6-declarative wayland wayland-protocols
intel-oneapi-mkl
blas blas64 cblas cblas64 lapack lapack64 lapacke lapacke64
intel-oneapi-mkl intel-oneapi-openmp openblas tbb
# cuda
)

Expand Down
3 changes: 2 additions & 1 deletion ci/ciimage/fedora/image.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"env": {
"CI": "1",
"SKIP_STATIC_BOOST": "1",
"MESON_CI_JOBNAME": "linux-fedora-gcc"
"MESON_CI_JOBNAME": "linux-fedora-gcc",
"MKL_THREADING_LAYER": "GNU"
}
}
16 changes: 15 additions & 1 deletion ci/ciimage/fedora/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,29 @@ pkgs=(
doxygen vulkan-devel vulkan-validation-layers-devel openssh lksctp-tools-devel objfw mercurial gtk-sharp3-devel libpcap-devel gpgme-devel
qt5-qtbase-devel qt5-qttools-devel qt5-linguist qt5-qtbase-private-devel
qt6-qtdeclarative-devel qt6-qtbase-devel qt6-qttools-devel qt6-linguist qt6-qtbase-private-devel
libwmf-devel valgrind cmake openmpi-devel nasm gnustep-base-devel gettext-devel ncurses-devel
libwmf-devel valgrind cmake openmpi-devel nasm gnustep-base-devel gettext-devel ncurses-devel hwloc-devel
libxml2-devel libxslt-devel libyaml-devel glib2-devel json-glib-devel libgcrypt-devel wayland-devel wayland-protocols-devel
openblas-devel blas-devel lapack-devel intel-oneapi-mkl-devel
# HACK: remove npm once we switch back to hotdoc sdist
nodejs-npm
)

# Sys update
dnf -y upgrade

# Add Intel oneAPI repository for mkl
cat > /etc/yum.repos.d/oneAPI.repo <<EOF
[oneAPI]
name=Intel® oneAPI repository
baseurl=https://yum.repos.intel.com/oneapi
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
# use low priority or it will replace openmpi-devel
priority=10000
EOF

# Install deps
dnf -y install "${pkgs[@]}"
# HACK: build hotdoc from git repo since current sdist is broken on modern compilers
Expand Down
3 changes: 3 additions & 0 deletions ci/ciimage/gentoo/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ pkgs_stable=(
net-print/cups
dev-util/vulkan-headers
media-libs/vulkan-loader
sci-libs/openblas

# misc
app-admin/sudo
Expand Down Expand Up @@ -86,6 +87,7 @@ pkgs_latest=(

# ~arch only
sci-libs/scalapack
sci-libs/mkl
)
pkgs=( "${pkgs_stable[@]}" "${pkgs_latest[@]}" )

Expand Down Expand Up @@ -117,6 +119,7 @@ cat <<-EOF > /etc/portage/package.use/ci
dev-lang/rust-bin clippy rustfmt
dev-libs/boost python
sci-libs/hdf5 cxx
sci-libs/lapack lapacke

# slimmed binpkg, nomesa
media-libs/libsdl2 -opengl -wayland -alsa -dbus -gles2 -udev -vulkan
Expand Down
16 changes: 15 additions & 1 deletion ci/ciimage/opensuse/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ pkgs=(
boost-devel libboost_date_time-devel libboost_filesystem-devel libboost_locale-devel
libboost_headers-devel libboost_test-devel libboost_log-devel libboost_regex-devel
libboost_python3-devel libboost_regex-devel
blas-devel cblas-devel lapack-devel lapacke-devel openblas-devel intel-oneapi-mkl-devel
# HACK: remove npm once we switch back to hotdoc sdist
npm
)
Expand All @@ -27,8 +28,21 @@ pkgs=(
zypper --non-interactive patch --with-update --with-optional
zypper --non-interactive update

# Add Intel oneAPI repository for mkl
cat > /etc/zypp/repos.d/oneAPI.repo <<-EOF
[oneAPI]
name=Intel® oneAPI repository
baseurl=https://yum.repos.intel.com/oneapi
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
# use low priority or it will replace openmpi-devel
priority=10000
EOF

# Install deps
zypper install -y "${pkgs[@]}"
zypper --gpg-auto-import-keys install -y "${pkgs[@]}"
# HACK: build hotdoc from git repo since current sdist is broken on modern compilers
# change back to 'hotdoc' once it's fixed
install_python_packages git+https://github.com/hotdoc/hotdoc
Expand Down
2 changes: 2 additions & 0 deletions ci/ciimage/ubuntu-rolling/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ pkgs=(
itstool
openjdk-11-jre
jq
libblas-dev liblapack-dev liblapacke-dev libblas64-dev liblapack64-dev liblapacke64-dev
libopenblas-dev libopenblas64-dev libmkl-dev
)

sed -i '/^Types: deb/s/deb/deb deb-src/' /etc/apt/sources.list.d/ubuntu.sources
Expand Down
146 changes: 146 additions & 0 deletions docs/markdown/Dependencies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,152 @@ to what is provided by the C runtime libraries.

`method` may be `auto`, `builtin` or `system`.

## BLAS and LAPACK

*(added 1.11.0)*

Enables compiling and linking against BLAS and LAPACK libraries. BLAS and
LAPACK are generic APIs, which can be provided by a number of different
implementations. It is possible to request either any implementation that
provides the API, or a specific implementation like OpenBLAS or MKL.
Furthermore, a preferred order may be specified, as well as the default
integer size (LP64/32-bit or ILP64/64-bit) and whether to detect the Fortran
and/or C APIs.

The common API between all BLAS and LAPACK dependencies uses the `modules`
keyword, with possible values:

- `'interface: lp64'` (default) or `'interface: ilp64'`: to select the default
integer size
Copy link
Copy Markdown
Member

@jpakkane jpakkane Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand why you did this, but it's conceptually not really correct. An "interface" is not really a module. Could we have some other way of expressing this information that is more explicit about what it does?

Copy link
Copy Markdown
Contributor Author

@mgorny mgorny Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a specific suggestion? From the discussion on #10921 I got the impression that it's a good enough compromise. Eli was rather opposed to adding new keyword arguments to dependency() over this, and the only clean alternative I can think of is creating a whole new module for this — and I'm not really convinced it's worth the effort. There's also the option of duplicating dependencies and having a ton of openblas-lp64, openblas-ilp64, mkl-lp64-iomp and so on — but I'm not convinced this will actually be cleaner.

Copy link
Copy Markdown
Member

@dcbaker dcbaker Jul 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking this too. CPS has the concept of "configurations" but I'm not really sure that this is a configuration either. We have a meeting tomorrow, I will bring this up with them and see what they think.

Copy link
Copy Markdown
Member

@jpakkane jpakkane Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory you could do something like:

blas_dep = dependency('blas',
  modules: [...],
  configuration: {'interface': 'ilp64'})

But we'd need to properly review that to make sure it works in all (or at least most) use cases.

Copy link
Copy Markdown
Member

@jpakkane jpakkane Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another big question is whether they provide different pkg-config names. That is, if there is both an openblas and openblas-ilp64 already, then using those names makes sense.

Copy link
Copy Markdown
Member

@dnicolodi dnicolodi Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason to use fake module arguments to pass additional properties to use for dependency selection is to avoid adding more keyword arguments to the dependency() function. IIUC the main reason for not adding more keyword arguments is that it sucks to have a generic function to allow arguments that apply only to a limited subset of cases.

The possible way out is to allow a Python's kwargs-style signature where the dependency() function forwards additional arguments to the concrete implementation of the dependency finding classes. IIUC kwargs-style arguments are not desired in the Meson language. However, using fake modules strings to pass arbitrary arguments effectively implements kwarg-style arguments circumventing the language limitations. It has all the drawbacks of allowing kwarg-style arguments plus all the drawbacks of having to implement parsing and error reporting in the implementation of the concrete dependency implementation.

If adding kwarg-style arguments to the Meson language is really not going to happen, maybe we should add to the dependency() function a properties arguments that takes a dictionary:

blas_dep = dependency('blas', properties: {'threading': true, 'interface': 'ilp64'})
gtest_dep = dependency('gtest', properties: {'bazel': false, 'main': true})
boost_dep = dependency('boost', modules: ['foo', 'bar'], properties: {'threading': true})

Copy link
Copy Markdown
Contributor Author

@mgorny mgorny Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, something accepting a dictionary would definitely be preferable for me over a list with custom key-value syntax.

Copy link
Copy Markdown
Member

@dcbaker dcbaker Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the example above is mixing "configurations" and "modules"/"components". I would say the gtest "main" example is a module/component, because it's an extra piece added to the core of gtest. boost with threading is a configuration, since you can have single- or multi- threaded implementations of a given module/component. configurations likely need to be a mapping or an array, because you'll likely want to be able to give a tri-state to the configuration like a UserFeature option does, "must have", "can have", "must not have".

dependencies(
  'boost',
  modules : ['foo', 'bar'],
  configurations : {'debug': get_option('debug'), 'threading': true, 'asan': false}
)

I wish I'd gotten around to dealing with CPS stuff sooner, because I think it would help with this particular issue to have all of that work done :/

Copy link
Copy Markdown
Member

@dnicolodi dnicolodi Aug 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say the gtest "main" example is a module/component, because it's an extra piece added to the core of gtest.

Agreed.

boost with threading is a configuration, since you can have single- or multi- threaded implementations of a given module/component.

I used the name properties instead of configuration (and I like configuration much more) but I didn't use a module for threading, thus I'm not sure I understand this objection.

configurations likely need to be a mapping or an array, because you'll likely want to be able to give a tri-state to the configuration like a UserFeature option does, "must have", "can have", "must not have".

Sure. I didn't specify which values can be associated to configuration keys. I guess for some of them a tri-state makes sense.

Copy link
Copy Markdown
Contributor

@h-vetinari h-vetinari Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish I'd gotten around to dealing with CPS stuff sooner, because I think it would help with this particular issue to have all of that work done :/

Would you mind opening an issue that tracks all the the CPS-related stuff? I've looked around in the repo but couldn't find anything. I could open an issue myself, but it's not likely to be of high quality (though of course you could then edit it as you see fit), because I don't know this effort well. Let me know if for whatever reason you'd still like me to do that.

- `'cblas'`: require the dependency to provide CBLAS (in addition to BLAS,
which is always present)
- `'lapack'`: require the dependency to provide LAPACK

The `get_variable` method on the returned dependency object may be used to
query the symbol suffix for the found library. Different libraries use
different symbol suffixes, and those are needed to link against the library
from C/Fortran code. Those suffixes may be fixed, like is the case for MKL and
Accelerate, or depend on the build configuration of the library, like is the
case for OpenBLAS.

Usage may look like:

```meson
dep_mkl = dependency('mkl', modules: ['interface: ilp64', 'cblas', 'lapack'])

# Or, to select one of multiple BLAS/LAPACK implementations:

blas = dependency(
['accelerate', 'flexiblas', 'mkl', 'openblas', 'blas'],
modules: ['interface: ilp64', 'cblas', 'lapack'],
)
blas_symbol_suffix = blas.get_variable('symbol_suffix')
if blas_symbol_suffix != ''
# The define here will be specific to the users code
_args_blas = ['-DBLAS_SYMBOL_SUFFIX=' + blas_symbol_suffix]
endif
blas_dep = declare_dependency(dependencies: [blas], compile_args: _args_blas)

# Adding the generic BLAS or LAPACK as a fallback, this may improve portability
blas_dep = dependency(['accelerate', 'mkl', 'openblas', 'blas')
lapack_dep = dependency(['accelerate', 'mkl', 'openblas', 'lapack')
```

Note that it is not necessary to specify both `'lapack'` and `'blas'` for the
same build target, because LAPACK itself depends on BLAS.


### Specific BLAS and LAPACK implementations

#### OpenBLAS

The `version` and `interface` keywords may be passed to request the use of a
specific version and interface, correspondingly:

```meson
openblas_dep = dependency('openblas',
version : '>=0.3.21',
modules: [
'interface: ilp64', # can be lp64 or ilp64
'cblas',
'lapack',
]
)
```

If OpenBLAS is installed in a nonstandard location *with* pkg-config files,
you can set `PKG_CONFIG_PATH`. Alternatively, you can specify
`openblas_includedir` and `openblas_librarydir` in your native or cross machine
file, this works also if OpenBLAS is installed *without* pkg-config files:

```ini
[properties]
openblas_includedir = '/path/to/include_dir' # should contain openblas_config.h
openblas_librarydir = '/path/to/library_dir'
```

OpenBLAS build configurations may include different symbol suffixes for ILP64 builds,
differences in library and pkg-config file naming conventions, and with or without
CBLAS or LAPACK. Meson will autodetect common symbol suffix conventions and validate
that, if `'cblas'` or `'lapack'` modules are requested, the found library actually
was built with support for those APIs.

OpenBLAS can be built with either pthreads or OpenMP threading. Information on
this is not available through Meson.

#### Intel MKL

MKL is built in a fixed build configuration per release by Intel, and always
includes both LP64 and ILP64, and CBLAS and LAPACK. MKL provides a number of
options for the threading layer - these can be selected through an MKL-specific
`'threading'` module:

```meson
mkl_dep = dependency('mkl',
version: '>=2023.2.0',
modules: [
interface: 'lp64', # options are 'lp64' or 'ilp64'
threading: 'seq', # options are 'seq', 'iomp', 'gomp', 'tbb'
]
)
```

In addition to all combinations of LP64/ILP64, threading, and dynamic/static
libraries, MLK provides a "Single Dynamic Library" (SDL). To select it, use the
`'sdl'` module. The SDL defaults to the LP64 interface and Intel OpenMP
(`'iomp'`) threading, with switching other interface and threading options
being accessible at runtime. When using the `'sdl'` module, either leave out
the interface and threading modules or use values that match the SDL defaults -
anything else will raise an error.

### Accelerate

Apple's Accelerate framework provides BLAS and LAPACK. It received a major update
in macOS 13.3, with LAPACK updated from 3.2 to 3.9, ILP64 support added, and
many longstanding bugs fixed. The `accelerate` dependency therefore only supports
this new version, and won't return a result on macOS <13.3 (note: the
corresponding 13.3 XCode SDK must also be installed) or if
`MACOS_DEPLOYMENT_TARGET` is set to a version lower than 13.3.

```meson
accelerate_dep = dependency('accelerate', modules: [interface: 'lp64'])
```

Note that the default, older Accelerate version can still be detected through
Meson's `appleframeworks` detection method:

```meson
dependency('Accelerate')
# Or:
dependency('appleframeworks', modules : 'Accelerate')
```
Those methods do not support any of the BLAS/LAPACK-specific modules.

### Other BLAS and LAPACK libraries

Specific support for other BLAS/LAPACK libraries is not (yet) included. They
can still be detected via the regular pkg-config or CMake support.

## Blocks

Enable support for Clang's blocks extension.
Expand Down
8 changes: 8 additions & 0 deletions mesonbuild/dependencies/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
BuiltinDependency, SystemDependency, get_leaf_external_dependencies)
from .detect import find_external_dependency, get_dep_identifier, packages, _packages_accept_language


__all__ = [
'Dependency',
'InternalDependency',
Expand Down Expand Up @@ -227,6 +228,13 @@ def __init__(self, name: str, environment: 'Environment', kwargs: T.Dict[str, T.
'libssl': 'misc',
'objfw': 'misc',

# From blas_lapack:
'accelerate': 'blas_lapack',
'blas': 'blas_lapack',
'lapack': 'blas_lapack',
'mkl': 'blas_lapack',
'openblas': 'blas_lapack',

# From platform:
'appleframeworks': 'platform',

Expand Down
Loading
Loading