{_cuda.extensions,pkgsCuda,pkgsForCudaArch}: init

[ConnorBaker]

This PR:

• Includes CUDA language section documentation fixups and new sections (including a CUDA User Guide!)
• Introduces and documents _cuda.extensions, pkgsCuda, and pkgsForCudaArch
• Fixes package set leakage in versioned, non-default instances of the CUDA package set (e.g., cudaPackages_11_8, cudaPackages_12_2, etc.) and documents cudaPackages.pkgs

Note

There is a increase to evaluation time for attributes which draw from the non-default CUDA package set due to the implementation of the fix for package set leakage.

As an example, consider catboost, which requires cudaPackages_11, and saw a slight slowdown from about 0.49 seconds to about 0.55 seconds:

Before

{
  "cpuTime": 0.4951170086860657,
  "envs": {
    "bytes": 14589216,
    "elements": 1034109,
    "number": 789543
  },
  "gc": {
    "cycles": 1,
    "heapSize": 402915328,
    "totalBytes": 182882816
  },
  "list": {
    "bytes": 1729208,
    "concats": 22189,
    "elements": 216151
  },
  "nrAvoided": 846465,
  "nrExprs": 217900,
  "nrFunctionCalls": 711585,
  "nrLookups": 381381,
  "nrOpUpdateValuesCopied": 4538315,
  "nrOpUpdates": 45631,
  "nrPrimOpCalls": 417981,
  "nrThunks": 908748,
  "sets": {
    "bytes": 89235808,
    "elements": 5467638,
    "number": 109600
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 8,
    "Value": 24
  },
  "symbols": {
    "bytes": 464088,
    "number": 44826
  },
  "time": {
    "cpu": 0.4951170086860657,
    "gc": 0.002,
    "gcFraction": 0.004039449190621771
  },
  "values": {
    "bytes": 52030200,
    "number": 2167925
  }
}

After

{
  "cpuTime": 0.5533679723739624,
  "envs": {
    "bytes": 22146784,
    "elements": 1569057,
    "number": 1199291
  },
  "gc": {
    "cycles": 1,
    "heapSize": 402915328,
    "totalBytes": 275970768
  },
  "list": {
    "bytes": 2658768,
    "concats": 31839,
    "elements": 332346
  },
  "nrAvoided": 1285255,
  "nrExprs": 217978,
  "nrFunctionCalls": 1073751,
  "nrLookups": 590551,
  "nrOpUpdateValuesCopied": 6945741,
  "nrOpUpdates": 65022,
  "nrPrimOpCalls": 644075,
  "nrThunks": 1375075,
  "sets": {
    "bytes": 135716224,
    "elements": 8314615,
    "number": 167649
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 8,
    "Value": 24
  },
  "symbols": {
    "bytes": 464266,
    "number": 44837
  },
  "time": {
    "cpu": 0.5533679723739624,
    "gc": 0.002,
    "gcFraction": 0.003614231577985893
  },
  "values": {
    "bytes": 77944752,
    "number": 3247698
  }
}

The largest measured increase in evaluation time comes from pkgs/top-level/release-cuda.nix, which instantiates every CUDA package set, jumping from about 7.7 seconds to about 25.6 seconds:

Before

{
  "cpuTime": 7.651072025299072,
  "envs": {
    "bytes": 438816984,
    "elements": 31661704,
    "number": 23190419
  },
  "gc": {
    "cycles": 9,
    "heapSize": 1829953536,
    "totalBytes": 3203691584
  },
  "list": {
    "bytes": 60658712,
    "concats": 1043070,
    "elements": 7582339
  },
  "nrAvoided": 24049709,
  "nrExprs": 1224124,
  "nrFunctionCalls": 20635918,
  "nrLookups": 10945781,
  "nrOpUpdateValuesCopied": 47363986,
  "nrOpUpdates": 1770862,
  "nrPrimOpCalls": 9565514,
  "nrThunks": 27756355,
  "sets": {
    "bytes": 1151189808,
    "elements": 67027422,
    "number": 4921941
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 8,
    "Value": 24
  },
  "symbols": {
    "bytes": 937835,
    "number": 81382
  },
  "time": {
    "cpu": 7.651072025299072,
    "gc": 2.613,
    "gcFraction": 0.34152076877068227
  },
  "values": {
    "bytes": 977363952,
    "number": 40723498
  }
}

After

{
  "cpuTime": 25.631572723388672,
  "envs": {
    "bytes": 1542351136,
    "elements": 110687785,
    "number": 82106107
  },
  "gc": {
    "cycles": 13,
    "heapSize": 7483875328,
    "totalBytes": 12488948416
  },
  "list": {
    "bytes": 194446808,
    "concats": 3671610,
    "elements": 24305851
  },
  "nrAvoided": 87539852,
  "nrExprs": 1224245,
  "nrFunctionCalls": 75285426,
  "nrLookups": 35311482,
  "nrOpUpdateValuesCopied": 236831655,
  "nrOpUpdates": 6330797,
  "nrPrimOpCalls": 35780617,
  "nrThunks": 94455780,
  "sets": {
    "bytes": 4987623664,
    "elements": 298972000,
    "number": 12754479
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 8,
    "Value": 24
  },
  "symbols": {
    "bytes": 937946,
    "number": 81387
  },
  "time": {
    "cpu": 25.631572723388672,
    "gc": 7.534,
    "gcFraction": 0.2939343629556241
  },
  "values": {
    "bytes": 3475471944,
    "number": 144811331
  }
}

Given the small number of packages which use a non-default version of the CUDA package set:

• catboost:

  nixpkgs/pkgs/top-level/all-packages.nix

  Line 7802 in 846783c

  cudaPackages = cudaPackages_11;

• dcgm:

  nixpkgs/pkgs/by-name/dc/dcgm/package.nix

  Lines 9 to 10 in 846783c

  	cudaPackages_11,
  	cudaPackages_12,

• Old releases of magma:

  nixpkgs/pkgs/development/libraries/science/math/magma/generic.nix

  Lines 45 to 46 in 846783c

  	effectiveCudaPackages =
  	if strings.versionOlder version "2.7.1" then cudaPackages_11 else cudaPackages;

• python3Packages.paddlepaddle:

  nixpkgs/pkgs/development/python-modules/paddlepaddle/default.nix

  Line 13 in 846783c

  cudaPackages_11 ? { },

• python3Packages.tensorflow-build:

  nixpkgs/pkgs/top-level/python-packages.nix

  Line 17604 in 846783c

  cudaPackagesTF = pkgs.cudaPackages_11;

• python3Packages.tensorrt:

  nixpkgs/pkgs/top-level/python-packages.nix

  Line 17671 in 846783c

  cudaPackages = pkgs.cudaPackages_11;

I don't believe this increase to be a cause for concern for most users.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[ConnorBaker]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 406568 --extra-nixpkgs-config '{ allowAliases = false; allowBroken = false; allowUnfree = true; checkMeta = true; contentAddressedByDefault = false; cudaCapabilities = [ "8.9" ]; cudaSupport = true; }'
Commit: 0500b6b25b93692efa4bb2b1ff00088a69fb78f7

---

x86_64-linux

✅ 5 packages built:

• cudaPackages.nsight_systems
• cudaPackages.nsight_systems.bin
• cudaPackages_11.nsight_systems
• cudaPackages_11.nsight_systems.bin
• nixpkgs-manual

[ConnorBaker]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 406568
Commit: 0500b6b25b93692efa4bb2b1ff00088a69fb78f7

---

x86_64-darwin

✅ 1 package built:

• nixpkgs-manual

---

aarch64-darwin

✅ 1 package built:

• nixpkgs-manual

---

x86_64-linux

✅ 5 packages built:

• cudaPackages.nsight_systems
• cudaPackages.nsight_systems.bin
• cudaPackages_11.nsight_systems
• cudaPackages_11.nsight_systems.bin
• nixpkgs-manual

[ConnorBaker]

Thank you for the review! Excellent suggestions -- I made them all, squashed, and rebased.

[djacu]

[SomeoneSerge]

FTR we'd discussed this with Connor, and it is a known concern that we here are bending the Nixpkgs' "public API" to compensate for Nix's CLI UX issues. This is somewhat evidenced by there being no in-tree consumers for selecting specific CUDA architectures. Still the intuitive way to achieve the desired UX "now", and I can't really judge what's worth what...

As such, even today this UX kinda could be implemented in a separate layer. Like we could literally write a builder-pattern wrapper for nixpkgs going like (foo.buildPlatform.x86_64-linux.hostPlatform.aarch64-linux.cuda."86".withOverlay (fi: pre: ...)).result, and a cli tool around such wrapper.

[SomeoneSerge]

Another remembrance: prior to introduction of pkgsRocm the idea was to make nixpkgsFun public, and that too might be helpful with the separation into layers, in particular with moving pkgsRocm and pkgsCuda* from stages to the level of all-packages.nix

[SomeoneSerge]

Not obvious from the title, but the biggest and most important change of the PR is probably this: the re-instantiation of nixpkgs from inside various cudaPackages_XX_Y package sets. An obvious layering violation and a very unfortunate situation, but there doesn't seem to be any (affordable) way around this. This is mostly needed because of a handful of fairly exceptional packages like mpi and ucc (circular dependencies in the "coarsened" graph where cudaPackages are a single super-node), but... really is needed?

[SomeoneSerge]

Hmmm on the (hundred and) second thought... the bug here was that when we (cudaPackages) consumed ucc/ucx which we knew depended, in turn, on "us" we hadn't configured them with the appropriate overrides. Now we are doing this, but we pass the overrides using a "sledgehammer", i.e. via overlays. We could, in fact, just do the explicit local overrides. I'd argue it would be more in the current spirit of Nixpkgs to do that.

The proposed overalys-based approach is more systemic (ensures we don't forget local overrides again, ensures transitive dependencies are handled correctly) at the cost of introducing (another) layering violation

[LunNova]

Late addition: on the ROCm side we seem to be getting by with just overriding things like ucx as we feel it's needed rather than an overlayed pkgs but I can see the stack of things to override may get increasingly difficult over time to apply; maybe we'll end up copying this approach later.

[ConnorBaker]

This was done explicitly because there's no way to know all transitive dependencies have been overridden (or even over overridden correctly!). Rather than playing a never-ending game of whack-a-mole (and I don't know how one would find out about misconfigurations without someone reporting an issue) this was the best solution I could come up with. The stack has enough problems; I didn't want missing overrides in massive transitive closures to be one of them.

[LunNova]

(and I don't know how one would find out about misconfigurations without someone reporting an issue)

Some horrifying test that asserts on equality of pkgsCuda.cudaPackages.x matching cudaPackages.x for everything in it?

but yeah very whack-a-mole

[SomeoneSerge]

getting by with just overriding things like ucx as we feel

Yeah gets problematic with the diamond deps like tf->cudA,tf->mpi->ucx->cudB,cuda->ucx

Some horrifying test that asserts on equality

I think we should be able to just break cyclic dependencies, in particular the cuda->ucx part. I'd be in for switching from cudaPackages.pkgs to asserts too, as long as we manage to keep the obvious targets (torch, tf, triton?) functional...

[SomeoneSerge]

To sum up: I truly wish I could say "no, and we should do X instead" to any of this, but the fact is the PR reflects the complexity of the problem... Thank you for shoveling this, Connor. And thank you for the reviews Dan

[SomeoneSerge]

Also a record concerning processes because I've been increasingly often late with reviews. So far we've seen arguments but no explicit rules about "how long one should wait for reviews prior to merging, what kind of changes are ok without reviews,,&c", and my personal opinions currently are that:

• having explicit rules would be net harmful,
• it's expected that people are sometimes unavailable,
• it's also expected, that other people may go ahead and change stuff even if some "vested parties" are unavailable; that's the risk that comes with "being unavailable"

In case of this PR specifically, the nixpkgs-side commx that happened was more or less the following:

• We'd talked through these changes with Connor during some video calls,
• The PR had received reviews by another maintainer, Dan.
• Connor still messaged me with another reminder, effectively ensuring "consent".

I think these interactions involved managing conflicts of interests (e.g. contributor v. employee), and managing perceived senses of "ownership" ([imo] nobody rightfully "owns" anything in Nixpkgs, but feelings get hurt). None of the interactions were as such obligatory, and time surely could have been saved. There's likely a bunch of PRs in the backlog for which it should've been saved. Time and, I imagine, frustration. There's got to be, somewhere, the balance where things do move forward, but no angry discourse "willy-nilly merge" threads are opened. I'm hoping we're not too far from the balance

[nixpkgs-ci]

Successfully created backport PR for release-25.05:

• [Backport release-25.05] {_cuda.extensions,pkgsCuda,pkgsForCudaArch}: init #427462

[trofi]

Bisect says fc2c350 cudaPackages: fix package set leakage introduced a minor eval failure as missing attribute dereference:

$ nix eval --impure --expr 'with import ./. {}; cudaPackages_11_0.tests.test-opencv-then-torch.buildCommand'
evaluation warning: CUDA versions older than 12.0 will be removed in Nixpkgs 25.05; see the 24.11 release notes for more information
evaluation warning: CUDA versions older than 12.0 will be removed in Nixpkgs 25.05; see the 24.11 release notes for more information
error:
       … while evaluating the attribute 'tests.test-opencv-then-torch.buildCommand'
         at /tmp/nb/nixpkgs/pkgs/build-support/trivial-builders/default.nix:80:17:
           79|         enableParallelBuilding = true;
           80|         inherit buildCommand name;
             |                 ^
           81|         passAsFile = [ "buildCommand" ] ++ (derivationArgs.passAsFile or [ ]);

       … while calling the 'getAttr' builtin
         at <nix/derivation-internal.nix>:50:17:
           49|     value = commonAttrs // {
           50|       outPath = builtins.getAttr outputName strict;
             |                 ^
           51|       drvPath = strict.drvPath;

       … while calculating requiredPythonModules for python3.13-torch-2.7.1:

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: attribute 'cuda_nvcc' missing
       at /tmp/nb/nixpkgs/pkgs/development/python-modules/triton/default.nix:144:44:
          143|       TRITON_NVDISASM_PATH = lib.getExe' cudaPackages.cuda_nvdisasm "nvdisasm";
          144|       TRITON_CUDACRT_PATH = lib.getInclude cudaPackages.cuda_nvcc;
             |                                            ^
          145|       TRITON_CUDART_PATH = lib.getInclude cudaPackages.cuda_cudart;

[ConnorBaker]

Can you tell we’re not running the tests in any CI ;)

That failure makes sense — that attribute is only available from 11.4 and on. Truly everything prior should be removed at this point.

Although, Jetson devices (special aarch64-linux) don’t get that until 11.8, unless you use JetPack NixOS, which repackages Debian installers for 11.4 to get those attributes.

Hmmm.

It’s probably fine to remove everything before 11.8 from an in-tree perspective, I’ll try to do that when I get some time.