Skip to content

magma: fix #220343 and separate CUDA build/run-time dependencies#220366

Merged
samuela merged 3 commits intoNixOS:masterfrom
ConnorBaker:feat/magma-split-out-cuda-nativeBuildInputs
Mar 22, 2023
Merged

magma: fix #220343 and separate CUDA build/run-time dependencies#220366
samuela merged 3 commits intoNixOS:masterfrom
ConnorBaker:feat/magma-split-out-cuda-nativeBuildInputs

Conversation

@ConnorBaker
Copy link
Contributor

@ConnorBaker ConnorBaker commented Mar 9, 2023

Description of changes

Goal is to shrink the size of the closure, or modularize the components which are downloaded.

Also fixes #220343.

My hope is that, for packages which do not fully support CUDA redistributables, by creating separate symlinked copies of what cudatoolkit would provide we can improve the user experience.

Closure before:

/nix/store/qmnr18aqd08zdkhka695ici96k6nzirv-libunistring-1.0         	   1.7M	   1.7M
/nix/store/vv6rlzln7vhxk519rdsrzmhhlpyb5q2m-libidn2-2.3.2            	 254.1K	   2.0M
/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224           	  28.9M	  30.8M
/nix/store/205vsmbfhq1q2vhgskpqyymqvba4mscp-gcc-11.3.0-lib           	   7.5M	  38.3M
/nix/store/2w4k8nvdyiggz717ygbbxchpnxrqc6y9-gcc-12.2.0-lib           	   7.8M	  38.6M
/nix/store/x6nnam5hk44mljbk782rcbd92jlnz8r6-pcre-8.45                	 514.4K	  31.3M
/nix/store/4vkv3rzky44hp2b8r13d8hr4ykvqhvwh-gnugrep-3.7              	 773.2K	  32.1M
/nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15             	   1.6M	  32.4M
/nix/store/mg9l7phyhvi16p9g8g3g8fbyj4mr79gq-zlib-1.2.13              	 125.6K	  31.0M
/nix/store/6g6d4la7xsizvr8qg91f56jiqx149iqq-binutils-2.40-lib        	   2.7M	  33.7M
/nix/store/izdfacm4wfmfjbgl1k3qlfjslkkrb2kg-gfortran-12.2.0-lib      	  10.7M	  41.6M
/nix/store/yzbam1rxs24z2apzpdpgqwi5fcwbid6z-openblas-0.3.21          	  26.1M	  67.7M
/nix/store/azvwwb4994qm7b1fcz8gc3gnlf8zwzi4-blas-3                   	  53.3M	 121.0M
/nix/store/ia34rbsa6d2dalzk5f7hy5jp9zazpv24-gmp-with-cxx-6.2.1       	 729.2K	  39.3M
/nix/store/b8wwwcng8c0snvmcawvhiny4b2gr6yhh-mpfr-4.2.0               	 774.0K	  40.1M
/nix/store/jn9kg98dsaajx4mh95rb9r5rf2idglqh-attr-2.5.1               	  78.8K	  30.9M
/nix/store/bw9s084fzmb5h40x98mfry25blj4cr9r-acl-2.3.1                	 108.9K	  31.0M
/nix/store/jvl8dr21nrwhqywwxcl8di4j55765gvy-gmp-with-cxx-stage4-6.2.1	 730.4K	  39.3M
/nix/store/bg8f47vihykgqcgblxkfk9sbvc4dnksa-coreutils-9.1            	   1.4M	  40.9M
/nix/store/lp8qrhb6hs42jwbapzq20l05jf4kyicq-glibc-2.35-224-bin       	   3.0M	  33.8M
/nix/store/wyzaa007anaxxhmrcbffm597v14n2mxs-linux-headers-6.1        	   6.0M	   6.0M
/nix/store/pqnd39aq2sksad2zvswjcpkqdc7ig3f9-glibc-2.35-224-dev       	   2.2M	  42.0M
/nix/store/fhzz4yrdy17czwc9i4swhlpcp445inzb-binutils-2.40            	  28.2M	  69.6M
/nix/store/xnrwkaidhxjsb70c2bnrl6j0mmr0y3qg-expand-response-params   	  16.4K	  30.9M
/nix/store/qkshx57xqsr4g4v8ga0jn8jrgnyaam3f-binutils-wrapper-2.40    	  48.5K	  84.8M
/nix/store/f6172g5agk13134pdx5hl1qjmw8a4sdw-libmpc-1.3.1             	 273.4K	  40.4M
/nix/store/fdfcva0q4zgnm1gpc7wmz4cgq7c3hxx1-isl-0.20                 	   2.5M	  41.8M
/nix/store/vns64fmwpqz8himgfdy8h7i29h9gbryc-gcc-11.3.0               	 180.6M	 242.2M
/nix/store/ds6ivg31k3l0pjhhf3s769bkpmafa54g-gcc-wrapper-11.3.0       	  55.2K	 278.4M
/nix/store/l0d6c6wa5sk4wx70x77k3d5clmny7sw6-cuda_cudart-11.7.60      	   6.2M	   6.2M
/nix/store/mxcbimi0gs021wwyfik63km6v26pzhp8-libcublas-11.10.1.25     	   1.1G	   1.2G
/nix/store/ss8jw175bhs3cmlzmmhsbpqihy457ids-libcusparse-11.7.3.50    	 488.1M	 488.1M
/nix/store/rbm214ji7jqk58g8shhkmhvl6xflblz0-cuda_cupti-11.7.50       	  88.4M	  88.4M
/nix/store/virnlrgvysrn1nl6bra798fy6x731nj0-cuda_nvprof-11.7.50      	   9.9M	 129.1M
/nix/store/zrpivbb39kiyij6dns4pz4wh8nik6m3y-cuda_nvcc-11.7.64        	 114.8M	 393.2M
/nix/store/mpj8ij172g466nncmnfqpz694pa746dr-cuda-redist-11.7         	  63.0K	   2.1G
/nix/store/ib9ckb796i7nqj0fy6iankvhyfc6az70-magma-2.7.1              	 429.4M	   2.6G

Closure after:

/nix/store/qmnr18aqd08zdkhka695ici96k6nzirv-libunistring-1.0         	   1.7M	   1.7M
/nix/store/vv6rlzln7vhxk519rdsrzmhhlpyb5q2m-libidn2-2.3.2            	 254.1K	   2.0M
/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224           	  28.9M	  30.8M
/nix/store/205vsmbfhq1q2vhgskpqyymqvba4mscp-gcc-11.3.0-lib           	   7.5M	  38.3M
/nix/store/mxcbimi0gs021wwyfik63km6v26pzhp8-libcublas-11.10.1.25     	   1.1G	   1.2G
/nix/store/ss8jw175bhs3cmlzmmhsbpqihy457ids-libcusparse-11.7.3.50    	 488.1M	 488.1M
/nix/store/23ygxbxapna1d80v447mnpfa2n78ay0x-cuda-redist-11.7         	  10.5K	   1.6G
/nix/store/2w4k8nvdyiggz717ygbbxchpnxrqc6y9-gcc-12.2.0-lib           	   7.8M	  38.6M
/nix/store/x6nnam5hk44mljbk782rcbd92jlnz8r6-pcre-8.45                	 514.4K	  31.3M
/nix/store/4vkv3rzky44hp2b8r13d8hr4ykvqhvwh-gnugrep-3.7              	 773.2K	  32.1M
/nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15             	   1.6M	  32.4M
/nix/store/mg9l7phyhvi16p9g8g3g8fbyj4mr79gq-zlib-1.2.13              	 125.6K	  31.0M
/nix/store/6g6d4la7xsizvr8qg91f56jiqx149iqq-binutils-2.40-lib        	   2.7M	  33.7M
/nix/store/izdfacm4wfmfjbgl1k3qlfjslkkrb2kg-gfortran-12.2.0-lib      	  10.7M	  41.6M
/nix/store/yzbam1rxs24z2apzpdpgqwi5fcwbid6z-openblas-0.3.21          	  26.1M	  67.7M
/nix/store/azvwwb4994qm7b1fcz8gc3gnlf8zwzi4-blas-3                   	  53.3M	 121.0M
/nix/store/ia34rbsa6d2dalzk5f7hy5jp9zazpv24-gmp-with-cxx-6.2.1       	 729.2K	  39.3M
/nix/store/b8wwwcng8c0snvmcawvhiny4b2gr6yhh-mpfr-4.2.0               	 774.0K	  40.1M
/nix/store/jn9kg98dsaajx4mh95rb9r5rf2idglqh-attr-2.5.1               	  78.8K	  30.9M
/nix/store/bw9s084fzmb5h40x98mfry25blj4cr9r-acl-2.3.1                	 108.9K	  31.0M
/nix/store/jvl8dr21nrwhqywwxcl8di4j55765gvy-gmp-with-cxx-stage4-6.2.1	 730.4K	  39.3M
/nix/store/bg8f47vihykgqcgblxkfk9sbvc4dnksa-coreutils-9.1            	   1.4M	  40.9M
/nix/store/lp8qrhb6hs42jwbapzq20l05jf4kyicq-glibc-2.35-224-bin       	   3.0M	  33.8M
/nix/store/wyzaa007anaxxhmrcbffm597v14n2mxs-linux-headers-6.1        	   6.0M	   6.0M
/nix/store/pqnd39aq2sksad2zvswjcpkqdc7ig3f9-glibc-2.35-224-dev       	   2.2M	  42.0M
/nix/store/fhzz4yrdy17czwc9i4swhlpcp445inzb-binutils-2.40            	  28.2M	  69.6M
/nix/store/xnrwkaidhxjsb70c2bnrl6j0mmr0y3qg-expand-response-params   	  16.4K	  30.9M
/nix/store/qkshx57xqsr4g4v8ga0jn8jrgnyaam3f-binutils-wrapper-2.40    	  48.5K	  84.8M
/nix/store/f6172g5agk13134pdx5hl1qjmw8a4sdw-libmpc-1.3.1             	 273.4K	  40.4M
/nix/store/fdfcva0q4zgnm1gpc7wmz4cgq7c3hxx1-isl-0.20                 	   2.5M	  41.8M
/nix/store/vns64fmwpqz8himgfdy8h7i29h9gbryc-gcc-11.3.0               	 180.6M	 242.2M
/nix/store/ds6ivg31k3l0pjhhf3s769bkpmafa54g-gcc-wrapper-11.3.0       	  55.2K	 278.4M
/nix/store/l0d6c6wa5sk4wx70x77k3d5clmny7sw6-cuda_cudart-11.7.60      	   6.2M	   6.2M
/nix/store/rbm214ji7jqk58g8shhkmhvl6xflblz0-cuda_cupti-11.7.50       	  88.4M	  88.4M
/nix/store/virnlrgvysrn1nl6bra798fy6x731nj0-cuda_nvprof-11.7.50      	   9.9M	 129.1M
/nix/store/zrpivbb39kiyij6dns4pz4wh8nik6m3y-cuda_nvcc-11.7.64        	 114.8M	 393.2M
/nix/store/fmv2m75gvby3jg106ibs4k956yl59b66-cuda-native-redist-11.7  	  53.3K	 497.8M
/nix/store/zwjfybn9vah5vkxmxl9qlawxmx7zmagp-magma-2.7.1              	 429.4M	   2.6G
Details

Common derivations:

/nix/store/qmnr18aqd08zdkhka695ici96k6nzirv-libunistring-1.0         	   1.7M	   1.7M
/nix/store/vv6rlzln7vhxk519rdsrzmhhlpyb5q2m-libidn2-2.3.2            	 254.1K	   2.0M
/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224           	  28.9M	  30.8M
/nix/store/205vsmbfhq1q2vhgskpqyymqvba4mscp-gcc-11.3.0-lib           	   7.5M	  38.3M
/nix/store/2w4k8nvdyiggz717ygbbxchpnxrqc6y9-gcc-12.2.0-lib           	   7.8M	  38.6M
/nix/store/x6nnam5hk44mljbk782rcbd92jlnz8r6-pcre-8.45                	 514.4K	  31.3M
/nix/store/4vkv3rzky44hp2b8r13d8hr4ykvqhvwh-gnugrep-3.7              	 773.2K	  32.1M
/nix/store/5ynbf6wszmggr0abwifdagrixgnya5vy-bash-5.2-p15             	   1.6M	  32.4M
/nix/store/mg9l7phyhvi16p9g8g3g8fbyj4mr79gq-zlib-1.2.13              	 125.6K	  31.0M
/nix/store/6g6d4la7xsizvr8qg91f56jiqx149iqq-binutils-2.40-lib        	   2.7M	  33.7M
/nix/store/izdfacm4wfmfjbgl1k3qlfjslkkrb2kg-gfortran-12.2.0-lib      	  10.7M	  41.6M
/nix/store/yzbam1rxs24z2apzpdpgqwi5fcwbid6z-openblas-0.3.21          	  26.1M	  67.7M
/nix/store/azvwwb4994qm7b1fcz8gc3gnlf8zwzi4-blas-3                   	  53.3M	 121.0M
/nix/store/ia34rbsa6d2dalzk5f7hy5jp9zazpv24-gmp-with-cxx-6.2.1       	 729.2K	  39.3M
/nix/store/b8wwwcng8c0snvmcawvhiny4b2gr6yhh-mpfr-4.2.0               	 774.0K	  40.1M
/nix/store/jn9kg98dsaajx4mh95rb9r5rf2idglqh-attr-2.5.1               	  78.8K	  30.9M
/nix/store/bw9s084fzmb5h40x98mfry25blj4cr9r-acl-2.3.1                	 108.9K	  31.0M
/nix/store/jvl8dr21nrwhqywwxcl8di4j55765gvy-gmp-with-cxx-stage4-6.2.1	 730.4K	  39.3M
/nix/store/bg8f47vihykgqcgblxkfk9sbvc4dnksa-coreutils-9.1            	   1.4M	  40.9M
/nix/store/lp8qrhb6hs42jwbapzq20l05jf4kyicq-glibc-2.35-224-bin       	   3.0M	  33.8M
/nix/store/wyzaa007anaxxhmrcbffm597v14n2mxs-linux-headers-6.1        	   6.0M	   6.0M
/nix/store/pqnd39aq2sksad2zvswjcpkqdc7ig3f9-glibc-2.35-224-dev       	   2.2M	  42.0M
/nix/store/fhzz4yrdy17czwc9i4swhlpcp445inzb-binutils-2.40            	  28.2M	  69.6M
/nix/store/xnrwkaidhxjsb70c2bnrl6j0mmr0y3qg-expand-response-params   	  16.4K	  30.9M
/nix/store/qkshx57xqsr4g4v8ga0jn8jrgnyaam3f-binutils-wrapper-2.40    	  48.5K	  84.8M
/nix/store/f6172g5agk13134pdx5hl1qjmw8a4sdw-libmpc-1.3.1             	 273.4K	  40.4M
/nix/store/fdfcva0q4zgnm1gpc7wmz4cgq7c3hxx1-isl-0.20                 	   2.5M	  41.8M
/nix/store/vns64fmwpqz8himgfdy8h7i29h9gbryc-gcc-11.3.0               	 180.6M	 242.2M
/nix/store/ds6ivg31k3l0pjhhf3s769bkpmafa54g-gcc-wrapper-11.3.0       	  55.2K	 278.4M
/nix/store/l0d6c6wa5sk4wx70x77k3d5clmny7sw6-cuda_cudart-11.7.60      	   6.2M	   6.2M
/nix/store/mxcbimi0gs021wwyfik63km6v26pzhp8-libcublas-11.10.1.25     	   1.1G	   1.2G
/nix/store/ss8jw175bhs3cmlzmmhsbpqihy457ids-libcusparse-11.7.3.50    	 488.1M	 488.1M
/nix/store/rbm214ji7jqk58g8shhkmhvl6xflblz0-cuda_cupti-11.7.50       	  88.4M	  88.4M
/nix/store/virnlrgvysrn1nl6bra798fy6x731nj0-cuda_nvprof-11.7.50      	   9.9M	 129.1M
/nix/store/zrpivbb39kiyij6dns4pz4wh8nik6m3y-cuda_nvcc-11.7.64        	 114.8M	 393.2M

Derivations unique to before:

/nix/store/mpj8ij172g466nncmnfqpz694pa746dr-cuda-redist-11.7         	  63.0K	   2.1G
/nix/store/ib9ckb796i7nqj0fy6iankvhyfc6az70-magma-2.7.1              	 429.4M	   2.6G

Derivations unique to after:

/nix/store/23ygxbxapna1d80v447mnpfa2n78ay0x-cuda-redist-11.7         	  10.5K	   1.6G
/nix/store/fmv2m75gvby3jg106ibs4k956yl59b66-cuda-native-redist-11.7  	  53.3K	 497.8M
/nix/store/zwjfybn9vah5vkxmxl9qlawxmx7zmagp-magma-2.7.1              	 429.4M	   2.6G
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ConnorBaker ConnorBaker self-assigned this Mar 9, 2023
@ConnorBaker ConnorBaker added the 6.topic: cuda Parallel computing platform and API label Mar 9, 2023
@ofborg ofborg bot requested a review from tbenst March 9, 2023 18:07
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 9, 2023
@ConnorBaker ConnorBaker changed the title magma: make a distinction between CUDA build/run-time dependencies magma: separate CUDA build/run-time dependencies; fix #220343 Mar 9, 2023
@ConnorBaker ConnorBaker changed the title magma: separate CUDA build/run-time dependencies; fix #220343 magma: fix #220343 and separate CUDA build/run-time dependencies Mar 9, 2023
@ConnorBaker ConnorBaker force-pushed the feat/magma-split-out-cuda-nativeBuildInputs branch from d8b393e to 37c8d5f Compare March 9, 2023 19:34
@ConnorBaker ConnorBaker marked this pull request as ready for review March 9, 2023 19:36
@ConnorBaker
Copy link
Contributor Author

@NixOS/cuda-maintainers

@ConnorBaker ConnorBaker force-pushed the feat/magma-split-out-cuda-nativeBuildInputs branch from 37c8d5f to c5c6b76 Compare March 9, 2023 21:11
@ConnorBaker ConnorBaker requested a review from samuela March 9, 2023 21:14
@ConnorBaker ConnorBaker force-pushed the feat/magma-split-out-cuda-nativeBuildInputs branch 2 times, most recently from b8041ce to ebd45bb Compare March 9, 2023 22:23
@ConnorBaker
Copy link
Contributor Author

ConnorBaker commented Mar 9, 2023

@SomeoneSerge I just understood what you meant now when you asked whether setting CUDAHOSTCXX was still needed. You're right, it's not necessary since you're setting it here:

Build fines without it, too: no more errors when trying to detect the CUDA version during CMake's configurePhase.

@ConnorBaker ConnorBaker force-pushed the feat/magma-split-out-cuda-nativeBuildInputs branch from ebd45bb to c4905c0 Compare March 9, 2023 22:32
@ConnorBaker ConnorBaker force-pushed the feat/magma-split-out-cuda-nativeBuildInputs branch from c4905c0 to 2b47a7a Compare March 9, 2023 23:02
@ConnorBaker
Copy link
Contributor Author

@tbenst this should be fully ready for you to review now, sorry about the delay!

@ConnorBaker ConnorBaker force-pushed the feat/magma-split-out-cuda-nativeBuildInputs branch from 2b47a7a to 0cf4ddd Compare March 13, 2023 22:40
@SomeoneSerge
Copy link
Contributor

Result of nixpkgs-review pr 220366 --extra-nixpkgs-config '{ cudaCapabilities = [ "8.6" ]; }' run on x86_64-linux 1

11 packages built:
  • magma (magma-cuda ,magma_2_7_1)
  • magma-hip
  • magma_2_6_2
  • python310Packages.torchWithCuda
  • python310Packages.torchWithCuda.dev
  • python310Packages.torchWithCuda.dist
  • python310Packages.torchWithCuda.lib
  • python310Packages.torchWithRocm
  • python310Packages.torchWithRocm.dev
  • python310Packages.torchWithRocm.dist
  • python310Packages.torchWithRocm.lib

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-45/26397/1

@samuela
Copy link
Member

samuela commented Mar 16, 2023

cc @tbenst friendly ping as maintainer

@tbenst
Copy link
Contributor

tbenst commented Mar 21, 2023

cc @tbenst friendly ping as maintainer

Nice work on this! I’m hoping to finish my PhD soon(TM) and had to step away from Nix in favor of productivity so I’m not in a position to review / maintain right now. Sorry about that! Please feel free to remove me as a maintainer.

@ConnorBaker ConnorBaker force-pushed the feat/magma-split-out-cuda-nativeBuildInputs branch from c312bd2 to dcbb5ed Compare March 21, 2023 19:37
@ConnorBaker
Copy link
Contributor Author

I removed @tbenst and added myself as a maintainer.

@samuela are we okay to merge?

Copy link
Member

@samuela samuela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! @SomeoneSerge does this look ok to you?

@samuela
Copy link
Member

samuela commented Mar 21, 2023

Nice work on this! I’m hoping to finish my PhD soon(TM) and had to step away from Nix in favor of productivity so I’m not in a position to review / maintain right now. Sorry about that! Please feel free to remove me as a maintainer.

No problem, @tbenst . Best of luck with your defense!

@ofborg ofborg bot added the 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. label Mar 21, 2023
@SomeoneSerge
Copy link
Contributor

Result of nixpkgs-review pr 220366 --extra-nixpkgs-config '{ cudaCapabilities = [ "8.6" ]; }' run on x86_64-linux 1

11 packages built:
  • magma (magma-cuda ,magma_2_7_1)
  • magma-hip
  • magma_2_6_2
  • python310Packages.torchWithCuda
  • python310Packages.torchWithCuda.dev
  • python310Packages.torchWithCuda.dist
  • python310Packages.torchWithCuda.lib
  • python310Packages.torchWithRocm
  • python310Packages.torchWithRocm.dev
  • python310Packages.torchWithRocm.dist
  • python310Packages.torchWithRocm.lib

@samuela samuela merged commit d8e1593 into NixOS:master Mar 22, 2023
@samuela
Copy link
Member

samuela commented Mar 22, 2023

Great, thanks so much @ConnorBaker and @SomeoneSerge !

@ConnorBaker ConnorBaker deleted the feat/magma-split-out-cuda-nativeBuildInputs branch March 22, 2023 04:29
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-46/26872/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: cuda Parallel computing platform and API 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes.

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

magma: lexicographic sorting of CUDA capabilities is not future-proof

5 participants