python3Packages.torch: allow lazy loading libnvrtc by SomeoneSerge · Pull Request #297590 · NixOS/nixpkgs

SomeoneSerge · 2024-03-21T00:44:50Z

Description of changes

Fixes #296179, refactors autoAddDriverRunapth and autoAddCudaCompatHook

CC @yannham

Things done

Add a 👍 reaction to pull requests you find important.

SomeoneSerge · 2024-03-21T01:03:54Z

pkgs/development/cuda-modules/setup-hooks/auto-add-driver-runpath-hook.sh

CC @jonringer

ryanswrt · 2024-03-21T08:26:08Z

I tried this against my setup using torch-bin and the issue persists; do the fixes in torch/default.nix need to be applied to bin.nix?

SomeoneSerge · 2024-03-21T12:07:25Z

pkgs/development/cuda-modules/setup-hooks/auto-fix-elf-files.sh

I tried this against my setup using torch-bin and the issue persists; do the fixes in torch/default.nix need to be applied to bin.nix?

I didn't look at torch-bin although it's only natural it should be broken the same way.
I also noticed a typo in the setup-hook because of which the libnvrtc path got lost in torch too.

Suggested change

newPath="${newPath}${elfAddRunpathSuffix:+:}${elfAddRunpathSuffix}"

newPath="${newPath}${elfAddRunpathsSuffix:+:}${elfAddRunpathsSuffix}"

pkgs/development/cuda-modules/setup-hooks/auto-fix-elf-files.sh

yannham

If I understood correctly, instead of patching elf files for each path, we accumulate all the paths in an array and patch everything at once in the end, letting us change the order.

I really like this approach. We can only specify "insert before" currently, which isn't a complete way to control the path ordering, but I guess it's amply sufficient in practice.

yannham · 2024-03-26T15:15:31Z

pkgs/development/cuda-modules/setup-hooks/auto-fix-elf-files.sh

if autoFixElfFiles is only used there, I wonder if it's not simpler to inline it. As much as I liked my higher-order solution 😛 (or, another solution if we want to keep the code nicely modular in those different bash functions, would be to hardcode elfAddRunpathsAction instead of fixAction). Unless you envision autoFixElfFiles to be useful for other things?

pkgs/development/cuda-modules/setup-hooks/auto-fix-elf-files.sh

python3Packages.torch-bin: fix lazy nvrtc

Fixes NixOS#296179

SomeoneSerge · 2024-03-26T21:51:18Z

pkgs/development/cuda-modules/saxpy/default.nix

omg, I recovered the string-splitting in the last force-push, but I broke cuda_compat again because I do the string-splitting too late 🙈

breakds · 2024-05-28T18:18:41Z

Wondering is this PR mainly pending resolving the conflicts? I see the comments from #296179 about jetson being affected - is that still a concern?

Thanks!

SomeoneSerge · 2024-05-28T20:18:20Z

Wondering is this PR mainly pending resolving the conflicts?
@breakds

Pretty much, the PR stalled after the hooks were moved from cudaPackages to the top-level. We'd have to start over. But also, the fix to the pytorch problem specifically amounts to adding "${lib.getLib cudaPackages.cuda_nvrtc}/lib" to the RUNPATHs, that can be temporarily hacked in without touching autoFixElfFiles or anything...

EDIT: I'm also getting more skeptical about the bash approach, maybe we should've gone with e.g. python3Minimal (or perl, whatever is cheaper to bootstrap)

breakds · 2024-05-28T21:23:16Z

@SomeoneSerge Thanks for the clarification!

Pretty much, the PR stalled after the hooks were moved from cudaPackages to the top-level.

Is this related to the following warnings that I saw recently?

trace: warning: cudaPackages.autoAddDriverRunpath is deprecated, use pkgs.autoAddDriverRunpath instead
trace: warning: cudaPackages.autoAddDriverRunpath is deprecated, use pkgs.autoAddDriverRunpath instead
trace: warning: cudaPackages.autoFixElfFiles is deprecated, use pkgs.autoFixElfFiles instead
trace: warning: cudaPackages.autoAddOpenGLRunpathHook is deprecated, use pkgs.autoAddDriverRunpath instead

the fix to the pytorch problem specifically amounts to adding "${lib.getLib cudaPackages.cuda_nvrtc}/lib" to the RUNPATHs

I saw your fix mentioned in another thread which seems like a good temporary solution. Is it not acceptable to patch it like that at this moment?

I'm also getting more skeptical about the bash approach

I also find that bash scripts are not the easiest to read and to maintain ... but it is probably my problem :(

SomeoneSerge · 2024-05-28T21:48:15Z

I saw your fix mentioned in another thread which seems like a good temporary solution. Is it not acceptable to patch it like that at this moment?

Yes and you're welcome to go ahead and implement it!

CMCDragonkai · 2024-06-04T14:44:49Z

Would these be relevant to tensorflow?

SomeoneSerge · 2024-06-05T16:18:29Z

Would these be relevant to tensorflow?

Does it dlopen libnvrtc?

SomeoneSerge · 2025-11-13T14:55:16Z

pkgs/development/cuda-modules/setup-hooks/auto-add-cuda-compat-runpath.sh

+arrayInsertBefore() {
+    local -n arrayRef="$1" # Namerefs, bash >= 4.3:
+    local pattern="$2"
+    local item="$3"
+    shift 3

-  if [[ $# -eq 0 ]]; then
-    echo "addCudaCompatRunpath: no library path provided" >&2
-    exit 1
-  elif [[ $# -gt 1 ]]; then
-    echo "addCudaCompatRunpath: too many arguments" >&2
-    exit 1
-  elif [[ "$1" == "" ]]; then
-    echo "addCudaCompatRunpath: empty library path" >&2
-    exit 1
-  else
-    libPath="$1"
-  fi
+    local i
+    local foundMatch=

-  origRpath="$(patchelf --print-rpath "$libPath")"
-  patchelf --set-rpath "@libcudaPath@:$origRpath" "$libPath"
+    local -a newArray
+    for i in "${arrayRef[@]}" ; do
+        if [[ "$i" == "$pattern" ]] ; then
+            newArray+=( "$item" )
+            foundMatch=1
+        fi
+        newArray+=( "$i" )
+    done
+    if [[ -z "$foundMatch" ]] ; then
+        newArray+=( "$item" )
+    fi
+    arrayRef=( "${newArray[@]}" )
 }

-postFixupHooks+=("autoFixElfFiles addCudaCompatRunpath")
+
+if [[ -n "@libcudaPath@" ]] ; then
+    arrayInsertBefore elfPrependRunpaths "@driverLink@/lib" "@libcudaPath@"
+fi


related: #385960

SomeoneSerge · 2025-11-13T14:55:57Z

Stale for a long time, need to revisit the motivation and the situation, etc, etc.

EDIT: Maybe closed prematurely? #461334

SomeoneSerge added the 6.topic: cuda Parallel computing platform and API label Mar 21, 2024

github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Mar 21, 2024

SomeoneSerge commented Mar 21, 2024

View reviewed changes

SomeoneSerge requested a review from ConnorBaker March 21, 2024 01:10

ofborg bot requested review from samuela, teh, thoughtpolice and tscholak March 21, 2024 01:18

SomeoneSerge commented Mar 21, 2024

View reviewed changes

pkgs/development/cuda-modules/setup-hooks/auto-fix-elf-files.sh Outdated Show resolved Hide resolved

yannham approved these changes Mar 26, 2024

View reviewed changes

wegank added the 12.approvals: 1 This PR was reviewed and approved by one person. label Mar 26, 2024

SomeoneSerge added 4 commits March 26, 2024 17:53

cudaPackages.autoFixElfFiles: elfAddRunpaths shorthand: init

a725317

python3Packages.torch-bin: fix lazy nvrtc

cudaPackages.autoFixElfFiles: decrease default verbosity

a724abd

cudaPackages.saxpy: re-enable eval for Jetsons

2906f88

python3Packages.torch: cuda: support dlopen(libnvrtc)

e5dc228

Fixes NixOS#296179

SomeoneSerge force-pushed the fix/python3Packages.torch/libnvrtc branch from 9294ff8 to 5ca0617 Compare March 26, 2024 17:54

python3Packages.torch-bin: fix lazy nvrtc

026df8f

SomeoneSerge force-pushed the fix/python3Packages.torch/libnvrtc branch from 5ca0617 to 026df8f Compare March 26, 2024 21:24

SomeoneSerge commented Mar 26, 2024

View reviewed changes

SomeoneSerge marked this pull request as draft March 26, 2024 21:51

SomeoneSerge mentioned this pull request Apr 3, 2024

cudaPackages.autoAddDriverRunpath: drop the redundant copy of the hook #301176

Merged

13 tasks

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Apr 5, 2024

SomeoneSerge mentioned this pull request May 18, 2024

python311Packages.tensorflow-bin: 2.15.0 -> 2.16.1 #311945

Merged

13 tasks

breakds mentioned this pull request May 29, 2024

python3Packages.torch: add libnvrtc to torch's runpath #315694

Merged

13 tasks

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 2, 2025

SomeoneSerge commented Nov 13, 2025

View reviewed changes

SomeoneSerge closed this Nov 13, 2025

github-project-automation bot moved this from New to ✅ Done in CUDA Team Nov 13, 2025

	newPath="${newPath}${elfAddRunpathSuffix:+:}${elfAddRunpathSuffix}"
	newPath="${newPath}${elfAddRunpathsSuffix:+:}${elfAddRunpathsSuffix}"

Uh oh!

Conversation

SomeoneSerge commented Mar 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

SomeoneSerge Mar 21, 2024

Choose a reason for hiding this comment

Uh oh!

ryanswrt commented Mar 21, 2024

Uh oh!

SomeoneSerge Mar 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yannham left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yannham Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SomeoneSerge Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

breakds commented May 28, 2024

Uh oh!

SomeoneSerge commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

breakds commented May 28, 2024

Uh oh!

SomeoneSerge commented May 28, 2024

Uh oh!

CMCDragonkai commented Jun 4, 2024

Uh oh!

SomeoneSerge commented Jun 5, 2024

Uh oh!

SomeoneSerge Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

SomeoneSerge commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

SomeoneSerge commented Mar 21, 2024 •

edited

Loading

SomeoneSerge Mar 21, 2024 •

edited

Loading

yannham left a comment •

edited

Loading

SomeoneSerge commented May 28, 2024 •

edited

Loading

SomeoneSerge commented Nov 13, 2025 •

edited

Loading