fix `release-cuda` eval errors by ruro · Pull Request #379768 · NixOS/nixpkgs

ruro · 2025-02-06T06:37:28Z

Currently, the nix-community CUDA builder has a bunch of Eval Errors. This PR fixes some of the "lower hanging fruit" among those eval errors.

You can see the list of current eval errors by opening the "Evaluation Errors" tab for the nixpkgs:cuda jobset. Note that the "Evaluation Errors" tab for individual runs is currently broken, so only the jobset-level "Evaluation Errors" tab is populated (which contains evaluation errors for the latest job).

You can reproduce the evaluation errors locally by running

hydra-eval-jobs pkgs/top-level/release-cuda.nix -I . --arg 'supportedSystems' '[ "x86_64-linux" ]'

(hydra-eval-jobs is provided by the hydra package). When reviewing this PR, I recommend running the above command on each commit starting with the merge base and then diffing the json data printed on stdout to verify that each commit does what I claim it does.

Things done

a8e9088 rename the jobs so that they don't repeat the cudaPackages* prefix twice

This is mostly a cosmetic change so that the job name matches the nixpkgs attribute path.

The next bunch of commits remove the accidental use of non-standard aliases for package names and then sets allowAliases = false to ensure that aliases aren't used in the future.

e0d45da removes Theano from the list of evaluated packages (Theano was removed from nixpkgs)
108123b removes pytorch which is a deprecated alias for torch which is already being evaluated
5c0092e changes Keras (deprecated alias) to keras
0c12821 changes scikitimage (deprecated alias) to scikit-image
8027d0b sets allowAliases = false (this ensures that aliases aren't used accidentally in the future)

Notably, the last commit also stops trying to evaluate the deprecated cudaPackages_10_* package sets and should also automatically stop evaluating CUDA versions prior to 12.0 after 25.05 gets released.

7d9676c some minor refactoring and helpful comments in release-cuda.nix

The next few commits disable the evaluation of "known" broken derivations by moving some brokenConditions to badPlatformConditions:

0f7fe9c cudnn
ebd16a4 nsight_systems
992b385 nvidia_driver

Additionally, the cudaPackages.tensorrt package is marked with hydraPlatforms = none, because this package is fundamentally impossible to build on Hydra due to requireFiled sources and its non-redistributable LICENSE.

98e9f51 tensorrt

Applying the above changes should fix 238 out of 259 eval errors that are currently happening in the nixpkgs:cuda jobset on the nix-community builders. The remaining 21 errors are slightly more tricky, so I am leaving them for a later PR.

Add a 👍 reaction to pull requests you find important.

mweinelt · 2025-02-07T03:47:41Z

Requires proper scoping of commit messages, if you don't intend them to get squashed.

ruro · 2025-02-08T11:59:51Z

@mweinelt my bad, I totally forgot about the commit messages. I am fine with either squashing or not. Although, I guess, the changes are big enough to keep the separate commits.

The "commit conventions for pkgs" want me to start the commits with a (pkg-name): prefix, but all the commits in this PR modify pkgs/top-level/release-cuda.nix, so I am not sure what would be the appropriate prefix in this case. I guess, I could prefix the 8 commits that mention specific packages to those packages, but that still leaves the 4 "generic" refactoring commits.

P.S.: looks like this requires a manual rebase anyway

ruro · 2025-02-08T12:50:51Z

I rebased and updated the commit names to be more in line with the commit conventions. Let me know if I missed anything else.

zowoq · 2025-02-11T07:58:55Z

Triggered an eval with these changes: https://hydra.nix-community.org/eval/265502

ruro · 2025-02-11T10:35:42Z

@zowoq thanks! Would I be correct in assuming that you've rebased this PR on top of 58edd1e (264446) before running it?

Unfortunately, it's a bit hard to verify whether the eval errors decreased as expected since the Evaluation Errors tab for individual runs is still broken on Hydra (and I've unfortunately missed the short window of time when this was the latest run, and you could see the Evaluation Errors on the main jobset page).

Also, the cudaPackages.cudaPackages.x -> cudaPackages.x rename makes it slightly inconvenient to verify that the build results are unaffected, but you can kind of infer that they are unaffected based on the package count decreasing by one (due to the removal of the duplicated python3Packages.pytorch.x86_64-linux in favor of the still succeeding python3Packages.torch.x86_64-linux).

zowoq · 2025-02-11T12:15:31Z

Would I be correct in assuming that you've rebased this PR on top of 58edd1e (264446) before running it?

Yes.

I ran it again to get the errors:

in job ‘cudaPackages_12_3.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_11_5.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_12_2.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘truecrack-cuda.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate truecrack-3.6: «broken»: is marked as broken

in job ‘cudaPackages_11_8.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘python3Packages.bpycv.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: fn-0.4.3 not supported for interpreter python3.12

in job ‘cudaPackages_11_0.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_12_0.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_11_1.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘colmap.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘tts.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate python3.12-torch-2.5.1: «unfree»: has an unfree license (‘bsd3 issl unfreeRedistributable’)

in job ‘cudaPackages_11_6.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_11_2.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_11.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_12_1.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘python3Packages.mxnet.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate mxnet-1.9.1: «broken»: is marked as broken

in job ‘cudaPackages_11_3.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘cudaPackages_11_4.cuda-samples.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure

in job ‘pixinsight.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate pixinsight-1.8.9-3-20240625: «unfree»: has an unfree license (‘unfree’)

in job ‘python3Packages.pymc.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Failed to evaluate python3.12-pytensor-2.26.4: «broken»: is marked as broken

in job ‘python3Packages.boxx.x86_64-linux’:
error:
       … while calling the 'deepSeq' builtin
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:479:35:
          478|     in
          479|     if drv == null then null else deepSeq drv' drv';
             |                                   ^
          480|

       … while evaluating the attribute 'outPath'
         at /nix/store/xqkfwfp7m4z00ir6464mj9x1nyrsm5zs-source/lib/customisation.nix:468:13:
          467|           value = commonAttrs // {
          468|             outPath = output.outPath;
             |             ^
          469|             drvPath = output.drvPath;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: fn-0.4.3 not supported for interpreter python3.12

ruro · 2025-02-11T12:32:00Z

👍 Looks correct to me.

Here is the same list of errors, but without the stack traces, one line per eval error.

in job ‘cudaPackages_12_3.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_11_5.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_12_2.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘truecrack-cuda.x86_64-linux’:                   error: Failed to evaluate truecrack-3.6: «broken»: is marked as broken
in job ‘cudaPackages_11_8.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘python3Packages.bpycv.x86_64-linux’:            error: fn-0.4.3 not supported for interpreter python3.12
in job ‘cudaPackages_11_0.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_12_0.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_11_1.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘colmap.x86_64-linux’:                           error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘tts.x86_64-linux’:                              error: Failed to evaluate python3.12-torch-2.5.1: «unfree»: has an unfree license (‘bsd3 issl unfreeRedistributable’)
in job ‘cudaPackages_11_6.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_11_2.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_11.cuda-samples.x86_64-linux’:     error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_12_1.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘python3Packages.mxnet.x86_64-linux’:            error: Failed to evaluate mxnet-1.9.1: «broken»: is marked as broken
in job ‘cudaPackages_11_3.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘cudaPackages_11_4.cuda-samples.x86_64-linux’:   error: Failed to evaluate freeimage-unstable-2021-11-01: «insecure»: is marked as insecure
in job ‘pixinsight.x86_64-linux’:                       error: Failed to evaluate pixinsight-1.8.9-3-20240625: «unfree»: has an unfree license (‘unfree’)
in job ‘python3Packages.pymc.x86_64-linux’:             error: Failed to evaluate python3.12-pytensor-2.26.4: «broken»: is marked as broken
in job ‘python3Packages.boxx.x86_64-linux’:             error: fn-0.4.3 not supported for interpreter python3.12

As mentioned previously, I'll probably work on the remaining errors in separate PRs.

pkgs/top-level/release-cuda.nix

prusnak · 2025-02-14T12:40:15Z

As mentioned previously, I'll probably work on the remaining errors in separate PRs.

truecrack-cuda fixed in truecrack: fix cuda #382028
python3Packages.boxx (and python3Packages.bpycv which depends on that) fixed in python3Packages.boxx: fix build #382056
python3Packages.pytensor is marked as broken because it is not compatible with numpy 2.x but this is being fixed in upstream as we speak: Make PyTensor compatible with numpy 2.0 pymc-devs/pytensor#1194

Rename the jobs from cudaPackages*.cudaPackages*.blah to just cudaPackages*.blah so that they match the nixpkgs attribute paths.

The theano package itself was originally removed in NixOS#313148.

Both `pytorch` and `torch` refer to the same package, but only the second one is the canonical name. The canonical `torch` name is already mentioned once a bit lower in the file.

Both `Keras` and `keras` refer to the same package, but only the second one is the canonical name.

Both `scikitimage` and `scikit-image` refer to the same package, but only the second one is the canonical name.

Using primary/canonical names for packages makes it easier to keep track of stuff. Additionally, this stops the recursion into cudaPackages* from including fake "aliases" for old (unsupported) cuda versions.

ruro · 2025-02-17T16:54:47Z

Rebased to fix merge conflict.

pkgs/top-level/release-cuda.nix

ruro · 2025-03-11T22:08:55Z

As discussed in one of the threads and on matrix. Commits in the latest force-push remove the overengineered filterPackagePredicates logic in favor of just setting badPlatformsConditions and hydraPlatforms for problematic packages as appropriate.

This should be ready to merge unless someone has any further concerns.

SomeoneSerge · 2025-03-11T22:42:06Z

pkgs/top-level/release-cuda.nix

+  # of their meta.{hydraPlatforms,platforms,badPlatforms} attributes
  packageSets = builtins.filter (lib.strings.hasPrefix "cudaPackages") (builtins.attrNames pkgs);
-  evalPackageSet = pset: mapTestOn { ${pset} = packagePlatforms pkgs.${pset}; };
+  evalPackageSetPlatforms = lib.genAttrs packageSets (pset: packagePlatforms pkgs.${pset});


IIRC eval in evalPackageSet was meant as a verb (it was a function); the new variable is an attrset? packageSets? platformPackageSets?

Does the new code generate identical tree structure? AFAIU python3Packages.tensorflow.x86_64-linux in https://hydra.nix-community.org/job/nixpkgs/cuda/python3Packages.tensorflow.x86_64-linux corresponds to a path in this attrset

This refactoring is technically no longer needed. We can drop commit 7d9676c, if you want (or we can keep the refactoring, but bikeshed the variable name a bit). This commit does not change any behavior (it just inlines the function call that used to be done on line 162).

I just left this refactor in for now, because I found the original structure a bit unintuitive.

evalPackageSetPlatforms is a nested attrSet where each leaf contains a list of Platforms on which each Package should be evaluated. I admit that the name could use some improvements.

So for example evalPackageSetPlatforms.cudaPackages.cudnn == [ "x86_64-linux" ], but evalPackageSetPlatforms.cudaPackages_12_3.cudnn_8_0 == [ ] (because of the badPlatformConditions).

mapTestOn then converts this attrset into an attrset of actual jobs that Hydra is going to evaluate/build by looking up those attribute paths in pkgs. The only change to the job names were done in a8e9088 - 0c12821 (and those changes should have made the job names match the attr paths better). So yes, the job name should match the attribute path in nixpkgs.

I find the current setup a bit easier to follow, because evalPackageSetPlatforms has the exact same structure as the explicit/literal attrset defined later in the file (with all the somepackage = linux; lines).

Perhaps it should be called something along the lines of packageSetPlatformsToEval or packagePlatformsToEval or packageSetPlatforms.

The only change to the job names was done in a8e9088.

Ah, gotcha. H'm I really can't remember why I did ${pset} = ... in the first place

We can drop commit 7d9676c, if you want (or we can keep the refactoring, but bikeshed the variable name a bit)

Either of the two options sounds fine

PS I played with

❯ nix repl -I nixpkgsRuro=https://github.com/ruro/nixpkgs/archive/fix-release-cuda-eval-errors.tar.gz nix-repl> orig = import <nixpkgs/pkgs/top-level/release-cuda.nix> { } nix-repl> new = import <nixpkgsRuro/pkgs/top-level/release-cuda.nix> { }

...and everything seems good

ruro · 2025-03-12T08:56:32Z

@SomeoneSerge Force pushed the refactoring changes as discussed on matrix.

SomeoneSerge · 2025-03-12T12:49:27Z

@ruro thank you so much for your contributions and for the persistence!

NixOSInfra added the 12.first-time contribution This PR is the author's first one; please be gentle! label Feb 6, 2025

github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Feb 6, 2025

ruro marked this pull request as ready for review February 6, 2025 16:56

ruro force-pushed the fix-release-cuda-eval-errors branch from d707bff to c65ed11 Compare February 8, 2025 12:47

zowoq requested a review from SomeoneSerge February 11, 2025 07:58

SomeoneSerge reviewed Feb 13, 2025

View reviewed changes

pkgs/top-level/release-cuda.nix Outdated Show resolved Hide resolved

SomeoneSerge reviewed Feb 13, 2025

View reviewed changes

pkgs/top-level/release-cuda.nix Outdated Show resolved Hide resolved

SomeoneSerge reviewed Feb 13, 2025

View reviewed changes

pkgs/top-level/release-cuda.nix Outdated Show resolved Hide resolved

ruro added 6 commits February 17, 2025 19:38

release-cuda: fix job names

a8e9088

Rename the jobs from cudaPackages*.cudaPackages*.blah to just cudaPackages*.blah so that they match the nixpkgs attribute paths.

python3Packages.theano: remove from release-cuda

e0d45da

The theano package itself was originally removed in NixOS#313148.

python3Packages.torch remove duplicate from release-cuda

108123b

Both `pytorch` and `torch` refer to the same package, but only the second one is the canonical name. The canonical `torch` name is already mentioned once a bit lower in the file.

python3Packages.keras: switch to primary alias in release-cuda

5c0092e

Both `Keras` and `keras` refer to the same package, but only the second one is the canonical name.

python3Packages.scikit-image: switch to primary alias in release-cuda

0c12821

Both `scikitimage` and `scikit-image` refer to the same package, but only the second one is the canonical name.

release-cuda: disable deprecated aliases

8027d0b

Using primary/canonical names for packages makes it easier to keep track of stuff. Additionally, this stops the recursion into cudaPackages* from including fake "aliases" for old (unsupported) cuda versions.

ruro force-pushed the fix-release-cuda-eval-errors branch from c65ed11 to cf624a8 Compare February 17, 2025 16:43

ruro force-pushed the fix-release-cuda-eval-errors branch from cf624a8 to 04edda0 Compare March 3, 2025 12:35

SomeoneSerge reviewed Mar 11, 2025

View reviewed changes

pkgs/top-level/release-cuda.nix Outdated Show resolved Hide resolved

ruro force-pushed the fix-release-cuda-eval-errors branch from 04edda0 to 98e9f51 Compare March 11, 2025 22:04

github-actions bot added the 6.topic: cuda Parallel computing platform and API label Mar 11, 2025

github-project-automation bot added this to CUDA Team Mar 11, 2025

github-project-automation bot moved this to New in CUDA Team Mar 11, 2025

github-actions bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Mar 11, 2025

nix-owners bot requested review from ConnorBaker, mdaiter and samuela March 11, 2025 22:12

SomeoneSerge reviewed Mar 11, 2025

View reviewed changes

ruro added 5 commits March 12, 2025 04:42

cudaPackages.cudnn: brokenConditions -> badPlatformsConditions

85f3648

cudaPackages.nsight_systems: brokenConditions -> badPlatformsConditions

bdc9f1d

cudaPackages.nvidia_driver: brokenConditions -> badPlatformsConditions

64db2f4

cudaPackages.tensorrt: set hydraPlatforms to none

4e27f20

release-cuda: minor refactoring and comments

eeaac37

ruro force-pushed the fix-release-cuda-eval-errors branch from 98e9f51 to eeaac37 Compare March 12, 2025 08:55

SomeoneSerge merged commit 1fc5ce1 into NixOS:master Mar 12, 2025
27 checks passed

github-project-automation bot moved this from New to ✅ Done in CUDA Team Mar 12, 2025

Uh oh!

Conversation

ruro commented Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

mweinelt commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruro commented Feb 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruro commented Feb 8, 2025

Uh oh!

zowoq commented Feb 11, 2025

Uh oh!

ruro commented Feb 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zowoq commented Feb 11, 2025

Uh oh!

ruro commented Feb 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

prusnak commented Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruro commented Feb 17, 2025

Uh oh!

Uh oh!

ruro commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SomeoneSerge Mar 11, 2025

Choose a reason for hiding this comment

Uh oh!

ruro Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SomeoneSerge Mar 12, 2025

Choose a reason for hiding this comment

Uh oh!

SomeoneSerge Mar 12, 2025

Choose a reason for hiding this comment

Uh oh!

ruro commented Mar 12, 2025

Uh oh!

Uh oh!

SomeoneSerge commented Mar 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ruro commented Feb 6, 2025 •

edited

Loading

mweinelt commented Feb 7, 2025 •

edited

Loading

ruro commented Feb 8, 2025 •

edited

Loading

ruro commented Feb 11, 2025 •

edited

Loading

ruro commented Feb 11, 2025 •

edited

Loading

prusnak commented Feb 14, 2025 •

edited

Loading

ruro commented Mar 11, 2025 •

edited

Loading

ruro Mar 11, 2025 •

edited

Loading