llama-cpp: fix cuda support

[happysalada]

Description of changes

this fixes cuda support.
an attempt at addressing #272569

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[happysalada]

@SomeoneSerge thank you for the detailed feedback, I think I integrated everything you pointed out.
let me know if anything else!

[happysalada]

Result of nixpkgs-review pr 277709 run on x86_64-linux 1

[happysalada]

I've switched a system with cudaSupport to this PR and tested ollama works

[happysalada]

@SomeoneSerge gave it another round of changes. Let me know.

[teto]

if all the settings are exclusive, one solution could be to replace the booleans with a string target and check the string with checkListOfEnum "GPU target" [ "cuda" "rocm "openblas" ] ;

[SomeoneSerge]

if all the settings are exclusive, one solution could be to replace the booleans with a string target and check the string with checkListOfEnum "GPU target" [ "cuda" "rocm "openblas" ] ;

I'm not sure what exactly is the situation upstream wrt the interaction of these options, so in a way the bool flags might be justified. They let the end-user try the cursed things out, and we do communicate the supported variants using broken. Otherwise, I like the suggestion

[SomeoneSerge]

We should wait for Ofborg, but otherwise I think the PR is ready (within the scope suggested by the title)

Thanks @happysalada!

[happysalada]

Result of nixpkgs-review pr 277709 run on x86_64-linux 1

[happysalada]

I've tested ollama on a system with cudaSupport with this PR.