Skip to content

cudaPackages: fix multiplex builder isSupported check#281806

Merged
SomeoneSerge merged 1 commit intoNixOS:masterfrom
ConnorBaker:fix/cudaPackages-multiplex-builder-isSupported-check
Jan 19, 2024
Merged

cudaPackages: fix multiplex builder isSupported check#281806
SomeoneSerge merged 1 commit intoNixOS:masterfrom
ConnorBaker:fix/cudaPackages-multiplex-builder-isSupported-check

Conversation

@ConnorBaker
Copy link
Contributor

@ConnorBaker ConnorBaker commented Jan 18, 2024

Description of changes

The predicate was incorrect because package.platform is the name of an NVIDIA redistArch and so will always be correct (the releases.nix expression is based on the manifests NVIDIA provides). Instead we should check that package.platform is the same as the one we're trying to build for.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Co-Authored-By: Yann Hamdaoui <yann.hamdaoui@tweag.io>
@ConnorBaker ConnorBaker self-assigned this Jan 18, 2024
@ConnorBaker ConnorBaker added the 6.topic: cuda Parallel computing platform and API label Jan 18, 2024
@ConnorBaker ConnorBaker marked this pull request as ready for review January 18, 2024 15:45
Copy link
Contributor

@SomeoneSerge SomeoneSerge Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❯ nix eval -I nixpkgs=flake:github:ConnorBaker/nixpkgs/fix/cudaPackages-multiplex-builder-isSupported-check -f '<nixpkgs>' --arg config '{ allowUnfree = true; cudaSupport = true; }' \
  cudaPackagesGoogle.cudnn_8_6.brokenConditions
{ "CUDA version is too new" = true; "CUDA version is too old" = false; }

...expected: all false, tensorflow.outPath evaluat-able without allowBroken

Cf. #281568

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the platform regression is independent of the cuda release regression

@GaetanLepage
Copy link
Contributor

Result of nixpkgs-review pr 281806 run on x86_64-linux 1

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jan 18, 2024
Copy link
Contributor

@SomeoneSerge SomeoneSerge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I see the regression.

❯ nix eval -f https://github.com/NixOS/nixpkgs/archive/a33557d1b95a8ec416c9c047723b5bb179b57d99.zip --argstr system aarch64-linux --arg config '{ cudaCapabilities = [ "6.2" ]; cudaEnableForwardCompat = false; cudaSupport = true; allowUnfree = true; }' -L cudaPackages.cudnn.src.url
"https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-x86_64/cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar.xz"
❯ nix eval -f . --argstr system aarch64-linux --arg config '{ cudaCapabilities = [ "6.2" ]; cudaEnableForwardCompat = false; cudaSupport = true; allowUnfree = true; }' -L cudaPackages.cudnn.src.url
"https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-aarch64/cudnn-linux-aarch64-8.9.5.30_cuda12-archive.tar.xz"

Probably introduced by the same commit where I added builtins.sort preferable .... This is horrifying, we still need something much much much simpler in place of our selection logic, even after the recent refactors...

@SomeoneSerge SomeoneSerge merged commit 8da872f into NixOS:master Jan 19, 2024
@ConnorBaker ConnorBaker deleted the fix/cudaPackages-multiplex-builder-isSupported-check branch March 15, 2024 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: cuda Parallel computing platform and API 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants