-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
cudaPackages.cudnn: use libcublas if available #168755
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,11 @@ | ||
| { stdenv | ||
| , lib | ||
| , cudatoolkit | ||
| , cudatoolkit ? null | ||
| , libcublas ? null | ||
| , zlib ? null | ||
| , fetchurl | ||
| , addOpenGLRunpath | ||
| , autoPatchelfHook | ||
| , autoAddOpenGLRunpathHook | ||
| , # The distributed version of CUDNN includes both dynamically liked .so files, | ||
| # as well as statically linked .a files. However, CUDNN is quite large | ||
| # (multiple gigabytes), so you can save some space in your nix store by | ||
|
|
@@ -25,6 +28,8 @@ assert (hash != null) || (sha256 != null); | |
| let | ||
| majorMinorPatch = version: lib.concatStringsSep "." (lib.take 3 (lib.splitVersion version)); | ||
| version = majorMinorPatch fullVersion; | ||
| # Use libcublas if available | ||
| withoutCudaToolkit = libcublas != null; | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What's the reason to have a version with libcublas and another without? Is it purely for backwards compatibility with derivations still based on cudatoolkit?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah ok, so we only have redist packages for CUDA 11.4-11.6 currently. But according to https://developer.download.nvidia.com/compute/cuda/redist/ there are redist packages for all CUDA 11.x versions. @FRidh is there a reason that we don't have redist packages for CUDA 11.0-11.3 as well as 11.4-11.6?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we move this boolean flag to the top-level? We shouldn't introduce more
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
there are unfortunately only manifest files for 11.4 to 11.6
@SuperSandro2000 could you stop this nonsense. There is nothing wrong with adding
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh I see. There are manifest files for all of 11.x but they are mostly empty up until 11.4.2
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
It is playing with fire. It is very easy to get things wrong and don't even notice it. |
||
| in stdenv.mkDerivation { | ||
| name = "cudatoolkit-${cudatoolkit.majorVersion}-cudnn-${version}"; | ||
|
|
||
|
|
@@ -35,7 +40,19 @@ in stdenv.mkDerivation { | |
| inherit url hash sha256; | ||
| }; | ||
|
|
||
| nativeBuildInputs = [ addOpenGLRunpath ]; | ||
| nativeBuildInputs = [ | ||
| autoAddOpenGLRunpathHook | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn't know there way such a hook. Is this new?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes I created it in the redist packages PR. |
||
| autoPatchelfHook | ||
| ]; | ||
|
|
||
| buildInputs = [ | ||
| stdenv.cc.cc.lib | ||
| ] ++ lib.optionals withoutCudaToolkit [ | ||
| libcublas | ||
| zlib | ||
| ] ++ lib.optionals (!withoutCudaToolkit) [ | ||
| cudatoolkit | ||
| ]; | ||
|
|
||
| # Some cuDNN libraries depend on things in cudatoolkit, eg. | ||
| # libcudnn_ops_infer.so.8 tries to load libcublas.so.11. So we need to patch | ||
|
|
@@ -46,15 +63,6 @@ in stdenv.mkDerivation { | |
| installPhase = '' | ||
| runHook preInstall | ||
|
|
||
| function fixRunPath { | ||
| p=$(patchelf --print-rpath $1) | ||
| patchelf --set-rpath "''${p:+$p:}${lib.makeLibraryPath [ stdenv.cc.cc cudatoolkit.lib ]}:${cudatoolkit}/lib:\$ORIGIN/" $1 | ||
| } | ||
|
|
||
| for sofile in {lib,lib64}/lib*.so; do | ||
| fixRunPath $sofile | ||
| done | ||
|
|
||
| mkdir -p $out | ||
| cp -a include $out/include | ||
| [ -d "lib/" ] && cp -a lib $out/lib | ||
|
|
@@ -66,18 +74,6 @@ in stdenv.mkDerivation { | |
| runHook postInstall | ||
| ''; | ||
|
|
||
| # Set RUNPATH so that libcuda in /run/opengl-driver(-32)/lib can be found. | ||
| # See the explanation in addOpenGLRunpath. | ||
| postFixup = '' | ||
| for lib in $out/lib/lib*.so; do | ||
| addOpenGLRunpath $lib | ||
| done | ||
| ''; | ||
|
|
||
| propagatedBuildInputs = [ | ||
| cudatoolkit | ||
| ]; | ||
|
|
||
| passthru = { | ||
| inherit cudatoolkit; | ||
| majorVersion = lib.versions.major version; | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Required regardless of which cudatoolkit distribution: cudnn on master is broken
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting, I thought ldd showed all was good before.