diff --git a/projects/amdsmi/CHANGELOG.md b/projects/amdsmi/CHANGELOG.md index 61c50626e233..7040c2129db3 100644 --- a/projects/amdsmi/CHANGELOG.md +++ b/projects/amdsmi/CHANGELOG.md @@ -203,6 +203,11 @@ Full documentation for amd_smi_lib is available at [https://rocm.docs.amd.com/pr - **Fixed GPU metrics returning no data on integrated GPUs**. - `amdsmi_get_gpu_metrics_info()` and `amdsmi_get_gpu_activity()` failed on APUs such as gfx1103, leaving PLX temperature, GPU utilization and `amd-smi metric` empty. Those devices now report the values they provide with the rest marked N/A, and any GPU without a PLX sensor now reports N/A there instead of a placeholder reading. +- **Fixed the `amdsmi` Python module from TheRock's `rocm-sdk-core` wheel failing outside `share/amd_smi`**. + - Copied into site-packages, the module could not load `libamd_smi.so.27` unless `torch` was imported first. It now loads the library from the `_rocm_sdk_core` package on `sys.path` when no usable system library is found, and reuses the copy PyTorch already loaded instead of mapping a second one. + - The copy TheRock ships in `rocm-sdk-core` loads only that wheel's library, never a system `libamd_smi.so` from another ROCm release. + - The module refuses a library newer than itself, whose structures can be larger than the module allocates, and warns about an older one. `AMDSMI_LIB_OVERRIDE` still loads any library. + - **Fixed `amd-smi` printing a Python traceback when an unknown NIC or switch is selected**. - `amd-smi static --nic 999` and `--switch 999` failed while building the "device not found" error, so the command exited `1` with a traceback and no readable message. `--json` and `--csv` produced no parseable output. - Both now report `Can not find a device: NIC '999'` (or `SWITCH`) and exit `196`, matching `--gpu`, `--cpu`, and `--core`. diff --git a/projects/amdsmi/cmake_modules/utils.cmake b/projects/amdsmi/cmake_modules/utils.cmake index fdc8c3b493cf..2a26c1b88d22 100644 --- a/projects/amdsmi/cmake_modules/utils.cmake +++ b/projects/amdsmi/cmake_modules/utils.cmake @@ -41,6 +41,9 @@ function(get_version_from_file REL_FILE_PATH ITEM) set(OUTPUT_ITEM "0") if(EXISTS "${FILE_PATH}") + # Reconfigure when the version changes, so the version configured into + # files such as the Python module's _version.py matches the library. + set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS "${FILE_PATH}") file(READ ${FILE_PATH} file_contents) string(REGEX MATCHALL "AMDSMI_LIB_VERSION_${ITEM} *[0-9]+" OUTPUT_STR "${file_contents}") list(LENGTH OUTPUT_STR OUTPUT_STR_LENGTH) diff --git a/projects/amdsmi/docs/install/install.md b/projects/amdsmi/docs/install/install.md index db549d8393d2..9eb1e721e9ab 100644 --- a/projects/amdsmi/docs/install/install.md +++ b/projects/amdsmi/docs/install/install.md @@ -218,6 +218,12 @@ Nightly builds of the ROCm Core SDK (including AMD SMI) are published by Set this after activating the target virtual environment; a `PYTHONPATH` pointing at another ROCm installation would override its Python package. + This wrapper never loads a system `libamd_smi.so`. It finds + `lib/libamd_smi.so.` in the `_rocm_sdk_core` package through + `sys.path`, so a copy of the `amdsmi` directory placed elsewhere in the + environment also loads it. It refuses a library newer than itself; see + [Packaging and install paths](../packaging.md). + ## Optional and advanced installation Use these optional procedures for CLI autocompletion and advanced setups, such diff --git a/projects/amdsmi/docs/packaging.md b/projects/amdsmi/docs/packaging.md index fb758cd3771b..ff9e25d5352e 100644 --- a/projects/amdsmi/docs/packaging.md +++ b/projects/amdsmi/docs/packaging.md @@ -12,7 +12,7 @@ the loader contract that `py-interface/amdsmi_wrapper.py` implements. | System package (deb/rpm) | `/opt/rocm/lib/libamd_smi.so.` (+ `ld.so.conf.d` entry) | Installed into the system interpreter's `site-packages`/`dist-packages` **and** `share/amd_smi` | SONAME via the dynamic linker | | TheRock native package (`amdrocm-amdsmi`) | `/opt/rocm/core-X.Y/lib/libamd_smi.so.` | `/opt/rocm/core-X.Y/share/amd_smi/amdsmi` only; importable after adding `share/amd_smi` to `PYTHONPATH` | Resolved relative to the wrapper (`../../../lib`) | | TheRock tarball | `/lib/libamd_smi.so.` in the extracted tree | `/share/amd_smi/amdsmi`; add `share/amd_smi` to `PYTHONPATH` for direct imports | Resolved relative to the wrapper (`../../../lib`) | -| ROCm via pip (TheRock `rocm-sdk-core`) | `/lib/libamd_smi.so.` | `/share/amd_smi/amdsmi`; add `share/amd_smi` to `PYTHONPATH` for direct imports | Resolved relative to the wrapper (`../../../lib`) | +| ROCm via pip (TheRock `rocm-sdk-core`) | `/lib/libamd_smi.so.` | `/share/amd_smi/amdsmi`; add `share/amd_smi` to `PYTHONPATH` for direct imports | Resolved relative to the wrapper (`../../../lib`); a copy elsewhere, such as in site-packages, uses `_rocm_sdk_core/lib` | | ROCm via pip in a venv | Same as above, inside the venv | Same as above, inside the venv | Same as above | | Community-built PyPI `amdsmi` releases through 7.0.2 | Not bundled; requires a matching ROCm `libamd_smi.so` | Interpreter `site-packages` | Legacy loader checks `ROCM_HOME` before `ROCM_PATH`, then the dynamic linker and `/opt/rocm/lib` | | Bundled-library wheel (`BUILD_PYTHON_WHEEL=ON`) | Bundled `libamd_smi_python.so` next to the wrapper | Interpreter `site-packages` | The bundled `.so`; system fallback is disabled | @@ -41,9 +41,28 @@ The current `py-interface/amdsmi_wrapper.py` selects the library in this order 2. A bundled `libamd_smi_python.so` next to the wrapper — the bundled-library wheel. 3. The SONAME resolved relative to the wrapper (`parents[3]/lib`) — the TheRock `share/amd_smi` layout, where a venv has no `ld.so.conf.d` entry. -4. The bare SONAME via the dynamic linker — the system deb/rpm. - -Steps 3 and 4 are skipped when `_AMDSMI_ALLOW_SYSTEM_FALLBACK` is `False`. The +4. In the copy TheRock ships in `rocm-sdk-core` (it sets + `_AMDSMI_ROCM_SDK_CORE = True`): `lib/` in the `_rocm_sdk_core` + package, then stop. That copy never loads a system `libamd_smi.so`, which + could come from another ROCm release. +5. The bare SONAME via the dynamic linker — the system deb/rpm. +6. `lib/` in the `_rocm_sdk_core` package, for a wrapper without that + flag. + +Steps 4 and 6 locate `_rocm_sdk_core` through `sys.modules` or `sys.path` +without importing it, so they also work when the module is copied out of +`share/amd_smi`, for example into site-packages. Each file in steps 3, 4 and 6 +is first opened with `RTLD_NOLOAD`, so a copy already loaded in the process is +reused; PyTorch's `_amdsmi_cdll_hook` would otherwise redirect the load to +`$ROCM_PATH/lib`. + +Steps 3 to 6 check the library version against the module's `_version.py`. A +library with a newer minor version, or another major version, is closed and +skipped, because SONAME `27` structures have grown between minor releases and +an older module allocates too little for them. An older library loads with a +warning. `AMDSMI_LIB_OVERRIDE` and the bundled library skip this check. + +Steps 3 to 6 are skipped when `_AMDSMI_ALLOW_SYSTEM_FALLBACK` is `False`. The committed wrapper and the system package keep it `True`; the wheel build flips it to `False`, so the bundled-library wheel does not automatically load a system `libamd_smi.so` (which could be a different version and would risk diff --git a/projects/amdsmi/py-interface/README.md b/projects/amdsmi/py-interface/README.md index 0cc3c6d8281e..9342598c03e1 100644 --- a/projects/amdsmi/py-interface/README.md +++ b/projects/amdsmi/py-interface/README.md @@ -24,7 +24,7 @@ The following install modes expose the same `import amdsmi` entry point: | Mode | What ships | Loader resolves to | |------|-----------|--------------------| | System package (`amd-smi-lib` rpm/deb) | The wrapper installed directly into the system Python's `site-packages` so plain `import amdsmi` works. The shared library lives at `/opt/rocm/lib/libamd_smi.so` and is registered with the dynamic linker via `ldconfig`. | `libamd_smi.so` resolved by the dynamic linker (SONAME). | -| TheRock native package, tarball, or ROCm pip install | The wrapper under `/share/amd_smi/amdsmi`; add `/share/amd_smi` to `PYTHONPATH` for direct imports. | `/lib/libamd_smi.so.`, resolved relative to the wrapper. | +| TheRock native package, tarball, or ROCm pip install | The wrapper under `/share/amd_smi/amdsmi`; add `/share/amd_smi` to `PYTHONPATH` for direct imports. | `/lib/libamd_smi.so.`, resolved relative to the wrapper. A `rocm-sdk-core` wrapper copied elsewhere, such as into site-packages, loads `lib/libamd_smi.so.` from the `_rocm_sdk_core` package. | | Community-built PyPI releases through 7.0.2 | Only the Python wrapper in `/amdsmi/`. Its version must match the ROCm release providing the library. | Matching ROCm `libamd_smi.so`; the legacy loader checks `ROCM_HOME` before `ROCM_PATH`. | | Bundled-library wheel (`BUILD_PYTHON_WHEEL=ON`) | The wrapper plus a SONAME-renamed `libamd_smi_python.so` directly inside `/amdsmi/`. | `libamd_smi_python.so` next to the wrapper. | @@ -56,7 +56,7 @@ wrapper you intend to use. | Variable | Purpose | |----------|---------| -| `AMDSMI_LIB_OVERRIDE` | Absolute path to a `libamd_smi*.so` to load **instead of** the auto-detected one. Intended for local development against an in-tree build (e.g. `AMDSMI_LIB_OVERRIDE=$PWD/build/libamd_smi.so python3 -c "import amdsmi"`) and for ABI-compatibility tests that need to point the wrapper at a curated alternate library. When set, it takes precedence over both the pip-bundled and system libraries. | +| `AMDSMI_LIB_OVERRIDE` | Absolute path to a `libamd_smi*.so` to load **instead of** the auto-detected one. Intended for local development against an in-tree build (e.g. `AMDSMI_LIB_OVERRIDE=$PWD/build/libamd_smi.so python3 -c "import amdsmi"`) and for ABI-compatibility tests that need to point the wrapper at a curated alternate library. When set, it takes precedence over both the pip-bundled and system libraries, and the library is loaded even if it is newer than the wrapper, which the wrapper otherwise refuses. | The legacy PyPI wrappers do not implement `AMDSMI_LIB_OVERRIDE` or the current missing-library behavior described below. diff --git a/projects/amdsmi/py-interface/amdsmi_wrapper.py b/projects/amdsmi/py-interface/amdsmi_wrapper.py index 70935ae10583..c971fdc8a026 100644 --- a/projects/amdsmi/py-interface/amdsmi_wrapper.py +++ b/projects/amdsmi/py-interface/amdsmi_wrapper.py @@ -153,7 +153,7 @@ def char_pointer_cast(string, encoding='utf-8'): # --------------------------------------------------------------------------- # Dynamic library loading # --------------------------------------------------------------------------- -# This wrapper supports three self-contained install paths: +# This wrapper supports four self-contained install paths: # # 1. pip wheel # The wheel ships ``libamd_smi_python.so`` next to this file. @@ -168,6 +168,10 @@ def char_pointer_cast(string, encoding='utf-8'): # Wrapper at ``/share/amd_smi/amdsmi/``, library at # ``/lib/`` -- resolved by fixed relative path. # +# 4. the copy TheRock ships in its rocm-sdk-core wheel, also once copied +# into site-packages: library at ``<_rocm_sdk_core>/lib/``. +# TheRock marks that copy by setting ``_AMDSMI_ROCM_SDK_CORE``. +# # A user installs ONE of these packages. We never combine paths # from them -- no ROCM_HOME / ROCM_PATH ladders, no walking up to a # ROCm root, no LD_LIBRARY_PATH probing. ``AMDSMI_LIB_OVERRIDE`` stays @@ -190,6 +194,117 @@ def char_pointer_cast(string, encoding='utf-8'): # system library (which would risk symbol conflicts inside PyTorch / JAX). _AMDSMI_ALLOW_SYSTEM_FALLBACK = True +# TheRock flips this to True in the copy it ships in the rocm-sdk-core wheel +# (build_tools/build_python_packages.py). That copy then loads only +# rocm-sdk-core's library, never a system libamd_smi.so found by SONAME, so a +# module from one ROCm release is not paired with another release's library. +_AMDSMI_ROCM_SDK_CORE = False + + +def _module_version(): + """(major, minor) of the build this module came from, read from the + _version.py that is configured beside it, or None when it is absent.""" + import re + + try: + text = (Path(__file__).resolve().parent / "_version.py").read_text() + except OSError: + return None + m = re.search(r'__version__\s*=\s*["\'](\d+)\.(\d+)', text) + return (int(m.group(1)), int(m.group(2))) if m else None + + +def _check_library_version(lib, path): + """Reject a library newer than this module: SONAME 27 has grown structs + between minor releases, and an older module's buffers are too small for + them. An older library only lacks newer functions, so it is a warning.""" + expected = _module_version() + if expected is None: + return + try: + get_lib_version = lib.amdsmi_get_lib_version + except AttributeError: + return + # amdsmi_version_t starts with uint32 major, minor; the buffer is larger + # than the struct so no future layout can overflow it. + version = (ctypes.c_uint32 * 64)() + if get_lib_version(version) != 0: + return + found = (version[0], version[1]) + if found[0] != expected[0] or found[1] > expected[1]: + raise OSError( + "loading %s returned AMD SMI library %d.%d, which this %d.%d Python module cannot " + "use; set AMDSMI_LIB_OVERRIDE to a library path to load it anyway" + % ((path,) + found + expected) + ) + if found[1] < expected[1]: + import warnings + + warnings.warn( + "loading %s returned AMD SMI library %d.%d, older than this %d.%d Python module; " + "newer functions are unavailable" % ((path,) + found + expected), + stacklevel=3, + ) + + +def _unload(lib): + """Drop the reference ctypes.CDLL took on a library the version check + rejected, so a later load by SONAME, or one PyTorch's hook redirects, + cannot return it.""" + import _ctypes + + handle = getattr(lib, "_handle", None) + if handle is not None and hasattr(_ctypes, "dlclose"): + _ctypes.dlclose(handle) + + +def _rocm_sdk_core_dir(): + """The _rocm_sdk_core package directory, found the way an import would + find it but without importing it, so import guards that reject third-party + packages cannot break the load.""" + import sys + + module = sys.modules.get("_rocm_sdk_core") + if getattr(module, "__file__", None): + return Path(module.__file__).parent + for entry in sys.path: + if not isinstance(entry, str): + continue + candidate = Path(os.path.abspath(entry)) / "_rocm_sdk_core" + try: + if (candidate / "__init__.py").is_file(): + return candidate + except OSError: + continue + return None + + +def _open_library(path, mode, errors, pinned=True): + """Open path, reusing it when it is already loaded. PyTorch's import hook + can redirect a plain load to another libamd_smi; with RTLD_NOLOAD the + redirect can only return a copy the process has already loaded. A pinned + path is then opened through ctypes.cdll, which the hook does not wrap, so + the file asked for is the file loaded.""" + noload = getattr(os, "RTLD_NOLOAD", 0) + for attempt in ([mode | noload] if noload else []) + [mode]: + try: + if attempt == mode and pinned: + lib = ctypes.cdll.LoadLibrary(str(path)) + else: + lib = ctypes.CDLL(str(path), mode=attempt) + except OSError as err: + if attempt == mode: + errors.append(str(err)) + continue + try: + _check_library_version(lib, path) + except OSError as err: + errors.append(str(err)) + _unload(lib) + return None + return lib + return None + def _load_library(): """Load the AMD SMI shared library. @@ -198,8 +313,10 @@ def _load_library(): 1. ``AMDSMI_LIB_OVERRIDE`` env var (ABI-test escape hatch). 2. ``libamd_smi_python.so`` next to this file (pip wheel). 3. ``/lib/`` relative to this file (relocatable ROCm tree). - 4. SONAME via the dynamic linker (system rpm / deb); skipped when - _AMDSMI_ALLOW_SYSTEM_FALLBACK is False (pip wheel). + 4. In rocm-sdk-core's copy: ``_rocm_sdk_core/lib/``, then stop. + 5. SONAME via the dynamic linker (system rpm / deb). + 6. ``_rocm_sdk_core/lib/``, for a copy without TheRock's flag. + 3-6 skip a library newer than this module. """ mode = getattr(ctypes, "RTLD_LOCAL", 0) @@ -222,16 +339,43 @@ def _load_library(): # amdsmi_interface.py resolves librocm-core.so the same way. A # present-but-unloadable file (missing deps) must not shadow the system # linker lookup, so fall through on OSError. + candidates = [] here = Path(__file__).resolve() if len(here.parents) > 3: - relocatable = here.parents[3] / "lib" / _AMDSMI_LIB_SONAME - if relocatable.exists(): - try: - return ctypes.CDLL(str(relocatable), mode=mode), str(relocatable) - except OSError: - pass - - return ctypes.CDLL(_AMDSMI_LIB_SONAME, mode=mode), _AMDSMI_LIB_SONAME + candidates.append(here.parents[3] / "lib" / _AMDSMI_LIB_SONAME) + if _AMDSMI_ROCM_SDK_CORE: + sdk_core = _rocm_sdk_core_dir() + if sdk_core is not None: + candidates.append(sdk_core / "lib" / _AMDSMI_LIB_SONAME) + + errors = [] + for path in candidates: + if path.exists(): + lib = _open_library(path, mode, errors) + if lib is not None: + return lib, str(path) + + if _AMDSMI_ROCM_SDK_CORE: + if sdk_core is None: + errors.append("_rocm_sdk_core not found on sys.path") + raise OSError( + "no usable %s from rocm-sdk-core: %s" + % (_AMDSMI_LIB_SONAME, "; ".join(errors) or "no file at " + ", ".join(map(str, candidates))) + ) + lib = _open_library(_AMDSMI_LIB_SONAME, mode, errors, pinned=False) + if lib is not None: + return lib, _AMDSMI_LIB_SONAME + + # No system library: a module copied beside TheRock's rocm-sdk-core wheel + # by hand, without TheRock's flag, still finds that wheel's library. + sdk_core = _rocm_sdk_core_dir() + if sdk_core is not None: + path = sdk_core / "lib" / _AMDSMI_LIB_SONAME + if path.exists(): + lib = _open_library(path, mode, errors) + if lib is not None: + return lib, str(path) + raise OSError("; ".join(errors)) class _MissingLibrary: diff --git a/projects/amdsmi/tests/python/test_abi_compat.py b/projects/amdsmi/tests/python/test_abi_compat.py index f1ac6bc25fe8..d429bbe1cb9c 100644 --- a/projects/amdsmi/tests/python/test_abi_compat.py +++ b/projects/amdsmi/tests/python/test_abi_compat.py @@ -18,6 +18,7 @@ # and ctypes.CDLL is monkey-patched to a FakeCDLL that simulates the # library surface. +import _ctypes import ast import builtins import ctypes @@ -31,6 +32,7 @@ import tempfile import unittest from pathlib import Path +from unittest import mock from types import ModuleType REPO_ROOT = Path(__file__).resolve().parents[2] @@ -75,6 +77,8 @@ class FakeCDLL: """ def __init__(self, path, mode=0, available_symbols=None): + if mode & getattr(os, "RTLD_NOLOAD", 0): + raise OSError("%s: not loaded" % path) self._path = path self._mode = mode self._available = set(available_symbols or ()) @@ -165,12 +169,15 @@ def __init__(self, available): def __enter__(self): self._orig = ctypes.CDLL + self._orig_dlltype = ctypes.cdll._dlltype available = self._available ctypes.CDLL = lambda path, mode=0: FakeCDLL(path, mode, available) + ctypes.cdll._dlltype = ctypes.CDLL return self def __exit__(self, *exc): ctypes.CDLL = self._orig + ctypes.cdll._dlltype = self._orig_dlltype class _LibOverrideEnvMixin: @@ -398,6 +405,271 @@ def _cdll(path, mode=0): self.assertIn(str(reloc), attempted) self.assertEqual(mod._loaded_lib_path, mod._AMDSMI_LIB_SONAME) + def _sdk_core_site(self, rocm_sdk_core=True): + import re as _re + + soname = _re.search(r'_AMDSMI_LIB_SONAME = "([^"]+)"', WRAPPER_SRC.read_text()).group(1) + site = self._tmp / "site-packages" + pkg = site / "amdsmi" + pkg.mkdir(parents=True) + text = WRAPPER_SRC.read_text() + if rocm_sdk_core: + flag = "_AMDSMI_ROCM_SDK_CORE = False" + self.assertEqual(text.count(flag), 1) + text = text.replace(flag, "_AMDSMI_ROCM_SDK_CORE = True") + (pkg / "amdsmi_wrapper.py").write_text(text) + sdk_core = site / "_rocm_sdk_core" + (sdk_core / "lib").mkdir(parents=True) + (sdk_core / "__init__.py").write_text("") + sdk_lib = sdk_core / "lib" / soname + sdk_lib.write_bytes(b"") + return site, pkg, sdk_lib + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_rocm_sdk_core_library_found_from_site_packages(self): + # The module installed or copied into site-packages beside TheRock's + # rocm-sdk-core wheel must load _rocm_sdk_core/lib/, without + # importing _rocm_sdk_core, so a restrictive import guard cannot stop it. + site, pkg, sdk_lib = self._sdk_core_site() + sys.path.insert(0, str(site)) + try: + with _ProfileModeLikeGuard(), _Patch(STABLE_SYMBOLS): + mod = _import_wrapper_from(str(pkg), self._modname) + finally: + sys.path.remove(str(site)) + self.assertEqual(mod._loaded_lib_path, str(sdk_lib)) + self.assertNotIn("_rocm_sdk_core", sys.modules) + + def _import_with(self, site, pkg, cdll, hook=None): + sys.path.insert(0, str(site)) + orig, orig_dlltype = ctypes.CDLL, ctypes.cdll._dlltype + ctypes.cdll._dlltype = cdll + ctypes.CDLL = hook or cdll + try: + return _import_wrapper_from(str(pkg), self._modname) + finally: + ctypes.CDLL, ctypes.cdll._dlltype = orig, orig_dlltype + sys.path.remove(str(site)) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_rocm_sdk_core_copy_never_loads_system_library(self): + # rocm-sdk-core's copy must not pair with a system libamd_smi.so from + # another release when its own library cannot be loaded. + site, pkg, sdk_lib = self._sdk_core_site() + attempted = [] + + def _cdll(path, mode=0): + attempted.append(str(path)) + if str(path) == str(sdk_lib): + raise OSError("simulated missing dependency") + return FakeCDLL(path, mode, STABLE_SYMBOLS) + + mod = self._import_with(site, pkg, _cdll) + self.assertIn(str(sdk_lib), attempted) + self.assertNotIn(mod._AMDSMI_LIB_SONAME, attempted) + self.assertIsNone(mod._loaded_lib_path) + self.assertIn("simulated missing dependency", str(mod._libraries["libamd_smi.so"]._err)) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_system_copy_ignores_rocm_sdk_core(self): + # Without TheRock's flag (the rpm/deb copy) the loader keeps today's + # order and never picks up a rocm-sdk-core library from sys.path. + site, pkg, sdk_lib = self._sdk_core_site(rocm_sdk_core=False) + attempted = [] + + def _cdll(path, mode=0): + attempted.append(str(path)) + return FakeCDLL(path, mode, STABLE_SYMBOLS) + + mod = self._import_with(site, pkg, _cdll) + self.assertNotIn(str(sdk_lib), attempted) + self.assertEqual(mod._loaded_lib_path, mod._AMDSMI_LIB_SONAME) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_copy_without_flag_falls_back_to_rocm_sdk_core(self): + # A module copied beside rocm-sdk-core without TheRock's flag tries the + # system library first, then rocm-sdk-core's. + site, pkg, sdk_lib = self._sdk_core_site(rocm_sdk_core=False) + attempted = [] + + def _cdll(path, mode=0): + attempted.append(str(path)) + if str(path) != str(sdk_lib): + raise OSError("%s: cannot open shared object file" % path) + return FakeCDLL(path, mode, STABLE_SYMBOLS) + + mod = self._import_with(site, pkg, _cdll) + self.assertLess(attempted.index(mod._AMDSMI_LIB_SONAME), attempted.index(str(sdk_lib))) + self.assertEqual(mod._loaded_lib_path, str(sdk_lib)) + + @unittest.skipUnless( + WRAPPER_SRC.is_file() and hasattr(os, "RTLD_NOLOAD"), "needs the wrapper and RTLD_NOLOAD" + ) + def test_loaded_candidate_is_reused_before_a_plain_load(self): + # Under PyTorch's import hook a plain load can be redirected to another + # libamd_smi; asking for the candidate with RTLD_NOLOAD first returns + # the copy PyTorch already loaded. + site, pkg, sdk_lib = self._sdk_core_site() + attempted = [] + + def _cdll(path, mode=0): + attempted.append((str(path), bool(mode & os.RTLD_NOLOAD))) + if mode & os.RTLD_NOLOAD and str(path) == str(sdk_lib): + return FakeCDLL(path, 0, STABLE_SYMBOLS) + raise OSError("not loaded") + + mod = self._import_with(site, pkg, _cdll) + self.assertEqual(attempted, [(str(sdk_lib), True)]) + self.assertEqual(mod._loaded_lib_path, str(sdk_lib)) + + @unittest.skipUnless( + WRAPPER_SRC.is_file() and hasattr(os, "RTLD_NOLOAD"), "needs the wrapper and RTLD_NOLOAD" + ) + def test_plain_load_is_not_redirected_by_ctypes_hook(self): + # With nothing loaded, PyTorch 2.14's hook sends any libamd_smi.so* + # open to $ROCM_HOME's copy; the rocm-sdk-core copy must still load + # the file it asked for, not a same-version system library. + site, pkg, sdk_lib = self._sdk_core_site() + system_lib = str(self._tmp / "rocm" / "lib" / "libamd_smi.so") + plain = lambda path, mode=0: FakeCDLL(path, mode, STABLE_SYMBOLS) + + def _hook(path, mode=0): + if Path(path).name.startswith("libamd_smi.so"): + path = system_lib + return plain(path, mode) + + mod = self._import_with(site, pkg, plain, hook=_hook) + self.assertEqual(mod._loaded_lib_path, str(sdk_lib)) + self.assertEqual(mod._libraries["libamd_smi.so"]._path, str(sdk_lib)) + + @unittest.skipUnless( + WRAPPER_SRC.is_file() and hasattr(os, "RTLD_NOLOAD"), "needs the wrapper and RTLD_NOLOAD" + ) + def test_loaded_system_library_is_reused_before_a_plain_soname_load(self): + # Without TheRock's flag the SONAME lookup also asks for a loaded copy + # first, so PyTorch's hook cannot load a second one beside it. + site, pkg, sdk_lib = self._sdk_core_site(rocm_sdk_core=False) + attempted = [] + + def _cdll(path, mode=0): + attempted.append((str(path), bool(mode & os.RTLD_NOLOAD))) + if mode & os.RTLD_NOLOAD and str(path) == "libamd_smi.so.27": + return FakeCDLL(path, 0, STABLE_SYMBOLS) + raise OSError("not loaded") + + mod = self._import_with(site, pkg, _cdll) + self.assertEqual(attempted, [("libamd_smi.so.27", True)]) + self.assertEqual(mod._loaded_lib_path, mod._AMDSMI_LIB_SONAME) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_empty_sys_path_entry_is_the_working_directory(self): + # "" is the working directory (python -c, python -m), so a + # _rocm_sdk_core found only there must still be used. + site, pkg, sdk_lib = self._sdk_core_site() + cwd = os.getcwd() + os.chdir(site) + sys.path.insert(0, "") + try: + with _Patch(STABLE_SYMBOLS): + mod = _import_wrapper_from(str(pkg), self._modname) + finally: + sys.path.remove("") + os.chdir(cwd) + self.assertEqual(Path(mod._loaded_lib_path).resolve(), sdk_lib.resolve()) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_rocm_sdk_core_copy_names_the_missing_library(self): + site, pkg, sdk_lib = self._sdk_core_site() + sdk_lib.unlink() + sys.path.insert(0, str(site)) + try: + with _Patch(STABLE_SYMBOLS): + mod = _import_wrapper_from(str(pkg), self._modname) + finally: + sys.path.remove(str(site)) + self.assertIsNone(mod._loaded_lib_path) + self.assertIn(str(sdk_lib), str(mod._libraries["libamd_smi.so"]._err)) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + @unittest.skipIf(hasattr(os, "geteuid") and os.geteuid() == 0, "root ignores permissions") + def test_unusable_sys_path_entries_are_skipped(self): + # The import system skips non-str and unreadable sys.path entries, so + # the _rocm_sdk_core lookup must not fail on them either. + site, pkg, sdk_lib = self._sdk_core_site() + locked = self._tmp / "locked" + (locked / "inner").mkdir(parents=True) + locked.chmod(0) + entries = [b"/nonexistent", str(locked / "inner"), str(site)] + sys.path[:0] = entries + try: + with _Patch(STABLE_SYMBOLS): + mod = _import_wrapper_from(str(pkg), self._modname) + finally: + del sys.path[: len(entries)] + locked.chmod(0o700) + self.assertEqual(mod._loaded_lib_path, str(sdk_lib)) + + def _versioned_cdll(self, versions, attempted): + class _Versioned(FakeCDLL): + @property + def _handle(self): + return self._path + + def __getattr__(self, name): + if name != "amdsmi_get_lib_version": + return FakeCDLL.__getattr__(self, name) + major, minor = versions[self._path] + + def _get(buf): + buf[0], buf[1] = major, minor + return 0 + + return _get + + def _cdll(path, mode=0): + attempted.append(str(path)) + return _Versioned(str(path), mode, STABLE_SYMBOLS) + + return _cdll + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_newer_library_is_rejected(self): + # SONAME 27 grew structs between minor releases, so a module must not + # bind a library newer than the build it came from. + site, pkg, sdk_lib = self._sdk_core_site() + (pkg / "_version.py").write_text('__version__ = "27.1.0+abc"\n') + attempted = [] + cdll = self._versioned_cdll({str(sdk_lib): (27, 2)}, attempted) + with mock.patch.object(_ctypes, "dlclose", create=True) as dlclose: + mod = self._import_with(site, pkg, cdll) + self.assertIsNone(mod._loaded_lib_path) + self.assertIn("27.2", str(mod._libraries["libamd_smi.so"]._err)) + dlclose.assert_called_once_with(str(sdk_lib)) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_older_library_warns(self): + site, pkg, sdk_lib = self._sdk_core_site() + (pkg / "_version.py").write_text('__version__ = "27.2.0"\n') + attempted = [] + with self.assertWarns(UserWarning): + mod = self._import_with( + site, pkg, self._versioned_cdll({str(sdk_lib): (27, 1)}, attempted) + ) + self.assertEqual(mod._loaded_lib_path, str(sdk_lib)) + + @unittest.skipUnless(WRAPPER_SRC.is_file(), "amdsmi_wrapper.py not found") + def test_newer_system_library_is_closed_before_the_fallback(self): + # Without TheRock's flag a newer library found by SONAME is refused and + # closed, and rocm-sdk-core's library is loaded instead. + site, pkg, sdk_lib = self._sdk_core_site(rocm_sdk_core=False) + (pkg / "_version.py").write_text('__version__ = "27.1.0"\n') + attempted = [] + cdll = self._versioned_cdll({sdk_lib.name: (27, 2), str(sdk_lib): (27, 1)}, attempted) + with mock.patch.object(_ctypes, "dlclose", create=True) as dlclose: + mod = self._import_with(site, pkg, cdll) + self.assertEqual(mod._loaded_lib_path, str(sdk_lib)) + dlclose.assert_called_once_with(sdk_lib.name) + class DisableSystemFallbackToolTest(unittest.TestCase): """tools/disable_system_fallback.py flips the loader flag exactly once.""" diff --git a/projects/amdsmi/tools/generator.py b/projects/amdsmi/tools/generator.py index 3a963b588979..3f3d0c74862c 100644 --- a/projects/amdsmi/tools/generator.py +++ b/projects/amdsmi/tools/generator.py @@ -209,7 +209,7 @@ def main(): # --------------------------------------------------------------------------- # Dynamic library loading # --------------------------------------------------------------------------- -# This wrapper supports three self-contained install paths: +# This wrapper supports four self-contained install paths: # # 1. pip wheel # The wheel ships ``libamd_smi_python.so`` next to this file. @@ -224,6 +224,10 @@ def main(): # Wrapper at ``/share/amd_smi/amdsmi/``, library at # ``/lib/`` -- resolved by fixed relative path. # +# 4. the copy TheRock ships in its rocm-sdk-core wheel, also once copied +# into site-packages: library at ``<_rocm_sdk_core>/lib/``. +# TheRock marks that copy by setting ``_AMDSMI_ROCM_SDK_CORE``. +# # A user installs ONE of these packages. We never combine paths # from them -- no ROCM_HOME / ROCM_PATH ladders, no walking up to a # ROCm root, no LD_LIBRARY_PATH probing. ``AMDSMI_LIB_OVERRIDE`` stays @@ -246,6 +250,117 @@ def main(): # system library (which would risk symbol conflicts inside PyTorch / JAX). _AMDSMI_ALLOW_SYSTEM_FALLBACK = True +# TheRock flips this to True in the copy it ships in the rocm-sdk-core wheel +# (build_tools/build_python_packages.py). That copy then loads only +# rocm-sdk-core's library, never a system libamd_smi.so found by SONAME, so a +# module from one ROCm release is not paired with another release's library. +_AMDSMI_ROCM_SDK_CORE = False + + +def _module_version(): + \"\"\"(major, minor) of the build this module came from, read from the + _version.py that is configured beside it, or None when it is absent.\"\"\" + import re + + try: + text = (Path(__file__).resolve().parent / "_version.py").read_text() + except OSError: + return None + m = re.search(r'__version__\\s*=\\s*["\\'](\\d+)\\.(\\d+)', text) + return (int(m.group(1)), int(m.group(2))) if m else None + + +def _check_library_version(lib, path): + \"\"\"Reject a library newer than this module: SONAME 27 has grown structs + between minor releases, and an older module's buffers are too small for + them. An older library only lacks newer functions, so it is a warning.\"\"\" + expected = _module_version() + if expected is None: + return + try: + get_lib_version = lib.amdsmi_get_lib_version + except AttributeError: + return + # amdsmi_version_t starts with uint32 major, minor; the buffer is larger + # than the struct so no future layout can overflow it. + version = (ctypes.c_uint32 * 64)() + if get_lib_version(version) != 0: + return + found = (version[0], version[1]) + if found[0] != expected[0] or found[1] > expected[1]: + raise OSError( + "loading %s returned AMD SMI library %d.%d, which this %d.%d Python module cannot " + "use; set AMDSMI_LIB_OVERRIDE to a library path to load it anyway" + % ((path,) + found + expected) + ) + if found[1] < expected[1]: + import warnings + + warnings.warn( + "loading %s returned AMD SMI library %d.%d, older than this %d.%d Python module; " + "newer functions are unavailable" % ((path,) + found + expected), + stacklevel=3, + ) + + +def _unload(lib): + \"\"\"Drop the reference ctypes.CDLL took on a library the version check + rejected, so a later load by SONAME, or one PyTorch's hook redirects, + cannot return it.\"\"\" + import _ctypes + + handle = getattr(lib, "_handle", None) + if handle is not None and hasattr(_ctypes, "dlclose"): + _ctypes.dlclose(handle) + + +def _rocm_sdk_core_dir(): + \"\"\"The _rocm_sdk_core package directory, found the way an import would + find it but without importing it, so import guards that reject third-party + packages cannot break the load.\"\"\" + import sys + + module = sys.modules.get("_rocm_sdk_core") + if getattr(module, "__file__", None): + return Path(module.__file__).parent + for entry in sys.path: + if not isinstance(entry, str): + continue + candidate = Path(os.path.abspath(entry)) / "_rocm_sdk_core" + try: + if (candidate / "__init__.py").is_file(): + return candidate + except OSError: + continue + return None + + +def _open_library(path, mode, errors, pinned=True): + \"\"\"Open path, reusing it when it is already loaded. PyTorch's import hook + can redirect a plain load to another libamd_smi; with RTLD_NOLOAD the + redirect can only return a copy the process has already loaded. A pinned + path is then opened through ctypes.cdll, which the hook does not wrap, so + the file asked for is the file loaded.\"\"\" + noload = getattr(os, "RTLD_NOLOAD", 0) + for attempt in ([mode | noload] if noload else []) + [mode]: + try: + if attempt == mode and pinned: + lib = ctypes.cdll.LoadLibrary(str(path)) + else: + lib = ctypes.CDLL(str(path), mode=attempt) + except OSError as err: + if attempt == mode: + errors.append(str(err)) + continue + try: + _check_library_version(lib, path) + except OSError as err: + errors.append(str(err)) + _unload(lib) + return None + return lib + return None + def _load_library(): \"\"\"Load the AMD SMI shared library. @@ -254,8 +369,10 @@ def _load_library(): 1. ``AMDSMI_LIB_OVERRIDE`` env var (ABI-test escape hatch). 2. ``libamd_smi_python.so`` next to this file (pip wheel). 3. ``/lib/`` relative to this file (relocatable ROCm tree). - 4. SONAME via the dynamic linker (system rpm / deb); skipped when - _AMDSMI_ALLOW_SYSTEM_FALLBACK is False (pip wheel). + 4. In rocm-sdk-core's copy: ``_rocm_sdk_core/lib/``, then stop. + 5. SONAME via the dynamic linker (system rpm / deb). + 6. ``_rocm_sdk_core/lib/``, for a copy without TheRock's flag. + 3-6 skip a library newer than this module. \"\"\" mode = getattr(ctypes, "RTLD_LOCAL", 0) @@ -278,16 +395,43 @@ def _load_library(): # amdsmi_interface.py resolves librocm-core.so the same way. A # present-but-unloadable file (missing deps) must not shadow the system # linker lookup, so fall through on OSError. + candidates = [] here = Path(__file__).resolve() if len(here.parents) > 3: - relocatable = here.parents[3] / "lib" / _AMDSMI_LIB_SONAME - if relocatable.exists(): - try: - return ctypes.CDLL(str(relocatable), mode=mode), str(relocatable) - except OSError: - pass - - return ctypes.CDLL(_AMDSMI_LIB_SONAME, mode=mode), _AMDSMI_LIB_SONAME + candidates.append(here.parents[3] / "lib" / _AMDSMI_LIB_SONAME) + if _AMDSMI_ROCM_SDK_CORE: + sdk_core = _rocm_sdk_core_dir() + if sdk_core is not None: + candidates.append(sdk_core / "lib" / _AMDSMI_LIB_SONAME) + + errors = [] + for path in candidates: + if path.exists(): + lib = _open_library(path, mode, errors) + if lib is not None: + return lib, str(path) + + if _AMDSMI_ROCM_SDK_CORE: + if sdk_core is None: + errors.append("_rocm_sdk_core not found on sys.path") + raise OSError( + "no usable %s from rocm-sdk-core: %s" + % (_AMDSMI_LIB_SONAME, "; ".join(errors) or "no file at " + ", ".join(map(str, candidates))) + ) + lib = _open_library(_AMDSMI_LIB_SONAME, mode, errors, pinned=False) + if lib is not None: + return lib, _AMDSMI_LIB_SONAME + + # No system library: a module copied beside TheRock's rocm-sdk-core wheel + # by hand, without TheRock's flag, still finds that wheel's library. + sdk_core = _rocm_sdk_core_dir() + if sdk_core is not None: + path = sdk_core / "lib" / _AMDSMI_LIB_SONAME + if path.exists(): + lib = _open_library(path, mode, errors) + if lib is not None: + return lib, str(path) + raise OSError("; ".join(errors)) class _MissingLibrary: