Repository navigation
rocmPackages: 7.2.3 -> 10.1.0 #573885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rocmPackages: 7.2.3 -> 10.1.0 #573885
Changes from all commits
3f92427
6a0b03d
72f4d36
3df5955
9045bd0
b649366
c6c3a29
c5ea90e
c451456
dd8b0cd
a0866fb
cda386e
f47e942
69f78ea
62570cb
24b423c
7ed4a84
7e948fc
1d3f1d0
66076f0
e15ea47
1b422cc
179db62
f907ab4
5680940
386f847
34eede4
5b5e802
f088acc
c68c639
dad7aff
468b86a
2d2884a
f9b059f
3544b89
925eadb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -188,6 +188,25 @@ | |
| authentication layer before exposing it to other users. | ||
| - `services.autobrr.secretFile` has been removed, as autobrr no longer uses a session secret since version 1.82.0. Remove the option from your configuration. | ||
|
|
||
| - `rocmPackages` has been updated from ROCm 7.2.4 to ROCm 10.1.0. AMD skipped | ||
| the 8.x and 9.x series. ROCm 10 breaks ABI and API: | ||
| `amdsmi` moves to the 27.0.0 ABI (`libamd_smi.so.27`, several clock/throttle | ||
| APIs removed), `rocsparse` drops `rocsparse_indextype_u16`, `hipcub` removes | ||
| its deprecated iterators and operators, and `miopen` drops its OpenCL backend. | ||
| Update out-of-tree packages that compile against `rocmPackages` for these | ||
| changes. The ROCm LLVM toolchain is now LLVM 24. | ||
|
|
||
| `rocmPackages.rocmlir` now builds from `ROCm/rocmlirTriton`, the rocMLIR | ||
| fork that AMDMIGraphX 10.1 pins, instead of `ROCm/rocMLIR`. | ||
|
|
||
| `rocmPackages.rocm-bandwidth-test` has been removed; upstream retired it. | ||
|
|
||
| `composable_kernel`, the `miopen` CK solvers, and `vllm` no longer support | ||
| `gfx900`/`gfx906` (Vega / MI25 / MI50 / Radeon VII) on ROCm 10. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This and the rocmlir are good reasons to go via rocmPackage_10 rather than a breaking change on master. |
||
| `rocblas`/`hipblas`-only consumers such as `llama-cpp` and `ollama` still | ||
| build for those targets. To stay on ROCm 7.2.x, use the NixOS 26.05 | ||
| stable channel. | ||
|
|
||
| - Artalk has been updated to 2.10.0. Its default configuration and data | ||
| directory discovery changed; see the [upstream migration | ||
| guide](https://artalk.js.org/en/guide/releases/v2.10.0.html) when invoking | ||
|
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Extracted this change to #574457. As a note: the comment added here is redundant with the commit message, and is too much about the change rather than the content. The purpose is also fairly apparently without the comment. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -261,11 +261,27 @@ goBuild (finalAttrs: { | |
| # (cuda_v12 / cuda_v13 / rocm_v7_1 / rocm_v7_2 / vulkan / cuda_jetpack*). | ||
| rocmMajorVersion = lib.versions.major rocmPackages.clr.version; | ||
| rocmMinorVersion = lib.versions.minor rocmPackages.clr.version; | ||
| # cmake/local.cmake only defines ROCm backend labels rocm_v7_1 (Windows) | ||
| # and rocm_v7_2 (Linux); the two "currently share build settings" and the | ||
| # label is an upstream *interface tag*, deliberately decoupled from the | ||
| # actual ROCm toolchain version (which comes from rocmPackages via | ||
| # buildInputs + AMDGPU_TARGETS). Our rocmPackages is now ROCm 10, so the | ||
| # derived rocm_v10_0 would hit `Unknown ROCm backend` (no such preset). | ||
| # Map any ROCm >= 8 to the Linux rocm_v7_2 build path — the GGML_HIP | ||
| # runner it produces is version-agnostic and links our ROCm 10 libs. | ||
| # TODO: file an upstream ollama PR adding an explicit rocm_v10_0 | ||
| # preset (llama/server/CMakePresets.json) + local.cmake branch, then drop | ||
| # this remap. | ||
| rocmLinuxBackend = | ||
| if lib.versionAtLeast rocmMajorVersion "8" then | ||
| "rocm_v7_2" | ||
| else | ||
| "rocm_v${rocmMajorVersion}_${rocmMinorVersion}"; | ||
| llamaBackend = | ||
|
Comment on lines
+275
to
280
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. conditionalizing like this makes little sense when only one rocmPackages variant is in tree
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ollama/ollama#18435. It may build but it's not clear if that means supported. |
||
| if enableCuda then | ||
| "cuda_v${cudaMajorVersion}" | ||
| else if enableRocm then | ||
| "rocm_v${rocmMajorVersion}_${rocmMinorVersion}" | ||
| rocmLinuxBackend | ||
| else if enableVulkan then | ||
| "vulkan" | ||
| else | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,6 +3,8 @@ | |
| stdenv, | ||
| cmake, | ||
| ninja, | ||
| pkg-config, | ||
| libdrm, | ||
| fetchFromGitHub, | ||
| python3, | ||
| opencv, | ||
|
|
@@ -81,6 +83,10 @@ stdenv'.mkDerivation (finalAttrs: { | |
| ] | ||
| ++ lib.optionals rocmSupport [ | ||
| rocmPackages.clr | ||
| # ROCm 10's rocm_smi-config.cmake does find_dependency(PkgConfig) (rocm-smi-lib | ||
| # now requires libdrm_amdgpu.pc), pulled in transitively via find_package(Torch) | ||
| # -> Caffe2 LoadHIP -> rocm_smi, so pkg-config must be on the host path. | ||
|
Comment on lines
+86
to
+88
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. slop comment that isn't needed |
||
| pkg-config | ||
| ] | ||
| ++ lib.optionals cudaSupport [ | ||
| cudaPackages.cuda_nvcc | ||
|
|
@@ -106,8 +112,10 @@ stdenv'.mkDerivation (finalAttrs: { | |
| rocmPackages.hipblaslt | ||
| rocmPackages.rocblas | ||
| rocmPackages.rocsolver | ||
| rocmPackages.rocsolver | ||
| rocmPackages.miopen | ||
| # rocm_smi-config.cmake (pulled via find_package(Torch)->Caffe2 LoadHIP) runs | ||
| # pkg_check_modules(libdrm) in ROCm 10, so libdrm.pc must be on PKG_CONFIG_PATH. | ||
|
Comment on lines
+116
to
+117
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. slop comment that isn't needed |
||
| libdrm | ||
| ] | ||
| ++ lib.optionals cudaSupport [ | ||
| cudaPackages.cuda_cudart | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,98 +24,140 @@ | |
|
|
||
| assert lib.assertMsg (ompdGdbSupport -> ompdSupport) "OMPD GDB support requires OMPD support!"; | ||
|
|
||
| stdenv.mkDerivation (finalAttrs: { | ||
| pname = "openmp"; | ||
| inherit version; | ||
|
|
||
| src = | ||
| if monorepoSrc != null then | ||
| runCommand "openmp-src-${version}" | ||
| { | ||
| inherit (monorepoSrc) passthru; | ||
| strictDeps = true; | ||
| __structuredAttrs = true; | ||
| } | ||
| '' | ||
| mkdir -p "$out" | ||
| cp -r ${monorepoSrc}/cmake "$out" | ||
| cp -r ${monorepoSrc}/openmp "$out" | ||
| '' | ||
| else | ||
| src; | ||
|
|
||
| sourceRoot = "${finalAttrs.src.name}/openmp"; | ||
|
|
||
| outputs = [ | ||
| "out" | ||
| "dev" | ||
| ]; | ||
|
|
||
| patches = | ||
| lib.optional (lib.versionOlder release_version "19") (getVersionFile "openmp/fix-find-tool.patch") | ||
| ++ [ | ||
| (getVersionFile "openmp/run-lit-directly.patch") | ||
| let | ||
| # LLVM 23 removed the legacy standalone OpenMP build (cmake <llvm>/openmp now | ||
| # aborts with a FATAL_ERROR). OpenMP must be built through the runtimes entry | ||
| # point (cmake <llvm>/runtimes -DLLVM_ENABLE_RUNTIMES=openmp), which supplies | ||
| # the project() call and the LLVM cmake modules. Older LLVMs keep the | ||
| # standalone layout. | ||
| useRuntimesBuild = lib.versionAtLeast release_version "23"; | ||
| in | ||
| stdenv.mkDerivation ( | ||
| finalAttrs: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Undo unnecessary indenting. |
||
| { | ||
| pname = "openmp"; | ||
| inherit version; | ||
|
|
||
| src = | ||
| if monorepoSrc != null then | ||
| runCommand "openmp-src-${version}" | ||
| { | ||
| inherit (monorepoSrc) passthru; | ||
| strictDeps = true; | ||
| __structuredAttrs = true; | ||
| } | ||
| ( | ||
| if useRuntimesBuild then | ||
| '' | ||
| mkdir -p "$out/llvm" | ||
| cp -r ${monorepoSrc}/cmake "$out" | ||
| cp -r ${monorepoSrc}/runtimes "$out" | ||
| cp -r ${monorepoSrc}/openmp "$out" | ||
| cp -r ${monorepoSrc}/llvm/cmake "$out/llvm/" | ||
| # openmp/runtime/unittests unconditionally pulls in gtest from | ||
| # ../third-party/unittest, so it must be present even with tests off. | ||
| cp -r ${monorepoSrc}/third-party "$out" | ||
| '' | ||
| else | ||
| '' | ||
| mkdir -p "$out" | ||
| cp -r ${monorepoSrc}/cmake "$out" | ||
| cp -r ${monorepoSrc}/openmp "$out" | ||
| '' | ||
| ) | ||
| else | ||
| src; | ||
|
|
||
| sourceRoot = "${finalAttrs.src.name}/${if useRuntimesBuild then "runtimes" else "openmp"}"; | ||
|
|
||
| outputs = [ | ||
| "out" | ||
| "dev" | ||
| ]; | ||
|
|
||
| nativeBuildInputs = [ | ||
| cmake | ||
| python3 | ||
| perl | ||
| ninja | ||
| pkg-config | ||
| lit | ||
| ]; | ||
|
|
||
| propagatedNativeBuildInputs = [ | ||
| checkPhaseThreadLimitHook | ||
| ]; | ||
|
|
||
| buildInputs = [ | ||
| llvm | ||
| ] | ||
| ++ lib.optionals (ompdSupport && ompdGdbSupport) [ | ||
| python3 | ||
| ]; | ||
|
|
||
| strictDeps = true; | ||
|
|
||
| cmakeFlags = [ | ||
| (lib.cmakeBool "LIBOMP_ENABLE_SHARED" ( | ||
| !stdenv.hostPlatform.isStatic && stdenv.hostPlatform.hasSharedLibraries | ||
| )) | ||
| (lib.cmakeBool "LIBOMP_OMPD_SUPPORT" ompdSupport) | ||
| (lib.cmakeBool "LIBOMP_OMPD_GDB_SUPPORT" ompdGdbSupport) | ||
| (lib.cmakeFeature "CLANG_TOOL" "${clang-unwrapped}/bin/clang") | ||
| (lib.cmakeFeature "OPT_TOOL" "${llvm}/bin/opt") | ||
| (lib.cmakeFeature "LINK_TOOL" "${llvm}/bin/llvm-link") | ||
| ] | ||
| ++ devExtraCmakeFlags; | ||
|
|
||
| doCheck = false; | ||
|
|
||
| checkTarget = "check-openmp"; | ||
|
|
||
| preCheck = '' | ||
| patchShebangs ../tools/archer/tests/deflake.bash | ||
| ''; | ||
|
|
||
| __structuredAttrs = true; | ||
|
|
||
| meta = llvm_meta // { | ||
| homepage = "https://openmp.llvm.org/"; | ||
| description = "Support for the OpenMP language"; | ||
| longDescription = '' | ||
| The OpenMP subproject of LLVM contains the components required to build an | ||
| executable OpenMP program that are outside the compiler itself. | ||
| Contains the code for the runtime library against which code compiled by | ||
| "clang -fopenmp" must be linked before it can run and the library that | ||
| supports offload to target devices. | ||
| ''; | ||
| # "All of the code is dual licensed under the MIT license and the UIUC | ||
| # License (a BSD-like license)": | ||
| license = with lib.licenses; [ | ||
| mit | ||
| ncsa | ||
| patches = | ||
| lib.optional (lib.versionOlder release_version "19") (getVersionFile "openmp/fix-find-tool.patch") | ||
| ++ [ | ||
| (getVersionFile "openmp/run-lit-directly.patch") | ||
| ]; | ||
|
|
||
| nativeBuildInputs = [ | ||
| cmake | ||
| python3 | ||
| perl | ||
| ninja | ||
| pkg-config | ||
| lit | ||
| ]; | ||
|
|
||
| propagatedNativeBuildInputs = [ | ||
| checkPhaseThreadLimitHook | ||
| ]; | ||
|
|
||
| buildInputs = [ | ||
| llvm | ||
| ] | ||
| ++ lib.optionals (ompdSupport && ompdGdbSupport) [ | ||
| python3 | ||
| ]; | ||
| }; | ||
| }) | ||
|
|
||
| strictDeps = true; | ||
|
|
||
| cmakeFlags = [ | ||
| (lib.cmakeBool "LIBOMP_ENABLE_SHARED" ( | ||
| !stdenv.hostPlatform.isStatic && stdenv.hostPlatform.hasSharedLibraries | ||
| )) | ||
| (lib.cmakeBool "LIBOMP_OMPD_SUPPORT" ompdSupport) | ||
| (lib.cmakeBool "LIBOMP_OMPD_GDB_SUPPORT" ompdGdbSupport) | ||
| (lib.cmakeFeature "CLANG_TOOL" "${clang-unwrapped}/bin/clang") | ||
| (lib.cmakeFeature "OPT_TOOL" "${llvm}/bin/opt") | ||
| (lib.cmakeFeature "LINK_TOOL" "${llvm}/bin/llvm-link") | ||
| ] | ||
| ++ lib.optionals useRuntimesBuild [ | ||
| # Build through the runtimes entry point (see useRuntimesBuild above). | ||
| (lib.cmakeFeature "LLVM_ENABLE_RUNTIMES" "openmp") | ||
| # We disable tests (doCheck = false); avoid pulling in third-party/unittest, | ||
| # which the trimmed source tree does not contain. | ||
| (lib.cmakeBool "LLVM_INCLUDE_TESTS" false) | ||
| ] | ||
| ++ devExtraCmakeFlags; | ||
|
|
||
| doCheck = false; | ||
|
|
||
| checkTarget = "check-openmp"; | ||
|
|
||
| preCheck = '' | ||
| patchShebangs ../tools/archer/tests/deflake.bash | ||
| ''; | ||
|
|
||
| __structuredAttrs = true; | ||
|
|
||
| meta = llvm_meta // { | ||
| homepage = "https://openmp.llvm.org/"; | ||
| description = "Support for the OpenMP language"; | ||
| longDescription = '' | ||
| The OpenMP subproject of LLVM contains the components required to build an | ||
| executable OpenMP program that are outside the compiler itself. | ||
| Contains the code for the runtime library against which code compiled by | ||
| "clang -fopenmp" must be linked before it can run and the library that | ||
| supports offload to target devices. | ||
| ''; | ||
| # "All of the code is dual licensed under the MIT license and the UIUC | ||
| # License (a BSD-like license)": | ||
| license = with lib.licenses; [ | ||
| mit | ||
| ncsa | ||
| ]; | ||
| }; | ||
| } | ||
| # Kept conditional so that derivations for older LLVMs stay unchanged. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. rebuild avoidance should typically be a TODO and get fixed on staging as a followup in our usual dev process
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. https://github.com/llvm/llvm-project/blob/llvmorg-20.1.8/openmp/runtime/cmake/LibompExports.cmake#L49
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like the
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ROCm/llvm-project@ceab53e#diff-3a672d4745f00e038440e0bc905155bff4847b16ad54051ea3e8127a04b62e5b Initial commit turned it on for
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. #575951 Has the non rocm specific changes.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| // lib.optionalAttrs useRuntimesBuild { | ||
| # The runtimes build writes generated headers (omp.h, ...) back into the | ||
| # openmp/ source tree, which is a sibling of sourceRoot and therefore left | ||
| # read-only by unpackPhase (only sourceRoot gets `chmod u+w`). Make the whole | ||
| # checkout writable. | ||
| preConfigure = '' | ||
| chmod -R u+w .. | ||
| ''; | ||
| } | ||
| ) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -55,6 +55,7 @@ | |
| openssl, | ||
| numactl, | ||
| llvmPackages, | ||
| libdrm, | ||
|
|
||
| # dependencies | ||
| filelock, | ||
|
|
@@ -399,6 +400,16 @@ buildPythonPackage.override { inherit stdenv; } (finalAttrs: { | |
| --replace-fail "list(APPEND ATen_HIP_INCLUDE \''${CMAKE_CURRENT_SOURCE_DIR}/../../../third_party/composable_kernel/include)" "" \ | ||
| --replace-fail "list(APPEND ATen_HIP_INCLUDE \''${CMAKE_CURRENT_SOURCE_DIR}/../../../third_party/composable_kernel/library/include)" "" | ||
| '' | ||
| + lib.optionalString rocmSupport '' | ||
| # ROCm 10's rocprofiler-sdk ships an OMPT plugin whose one-time init dlopens a | ||
| # library while libomp holds its (non-recursive) init ticket lock: OpenBLAS's | ||
| # constructor queries the CPU count -> libomp __kmp_middle_initialize -> | ||
| # ompt_pre_init -> rocprofiler-sdk -> dlopen -> OpenBLAS ctor -> libomp again, | ||
| # busy-waiting on the held lock forever, so `import torch` hangs (surfaced as a | ||
| # pythonImportsCheck hang; also hits runtime, e.g. vllm). Disable the OMPT tool | ||
| # interface by default (setdefault, so a user wanting rocprofv3 can override). | ||
| sed -i '/^ if USE_GLOBAL_DEPS:$/i\ os.environ.setdefault("OMP_TOOL", "disabled")' torch/__init__.py | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should be a patch or a use of substituteInPlace and shouldn't be modifying the runtime environment via source code changes.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| '' | ||
| # Remove PyTorch's FindCUDAToolkit.cmake and use CMake's default. | ||
| # NOTE: Parts of pytorch rely on unmaintained FindCUDA.cmake with custom patches to support e.g. | ||
| # newer architectures (sm_90a). We do want to delete vendored patches, but have to keep them | ||
|
|
@@ -625,6 +636,13 @@ buildPythonPackage.override { inherit stdenv; } (finalAttrs: { | |
| ++ lib.optionals rocmSupport [ | ||
| rocmtoolkit_joined | ||
| rocmPackages.clr # Added separately so setup hook applies | ||
| # ROCm 10's rocm-smi exports a rocm_smi-config.cmake that does | ||
| # pkg_check_modules(_ROCM_SMI_DRM REQUIRED libdrm) (its public headers pull in | ||
| # libdrm/drm.h via kfd_ioctl.h). rocm-smi propagates libdrm, but torch consumes | ||
| # it through the rocmtoolkit_joined symlinkJoin, which drops propagatedBuildInputs | ||
| # -> libdrm.pc is off PKG_CONFIG_PATH and torch's LoadHIP find_package(rocm_smi) | ||
| # fails at configure. Add libdrm directly so the pkg-config hook finds libdrm.pc. | ||
|
Comment on lines
+639
to
+644
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. unneeded slop comment |
||
| libdrm | ||
| ]; | ||
|
|
||
| dependencies = [ | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I should have been more specific on
romlir. A drop in replacement does not mean replace the nixpkgs copy with something different.rocmlirupstream is still actively being developed.This change should be a few different things:
migraphxupdated to depend on thatrcomlirpackage why it doesn't have a 10.1 version update.