python3Packages.pytorch: repair for darwin

[Et7f3]

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[wegank]

@ofborg build python3Packages.pytorch

[SomeoneSerge]

This issue seems to be specific to nixpkgs

I'm not sure I fully understand the comment from the linked code, but is there, from nixpkgs point of view, a "correct way" to detect aligned_alloc availability that we could try and merge upstream? Maybe @Calvin-L, the author of the comment, knows

I think it's OK to merge this patch for now though, as soon as we see that it fixes the build

[Calvin-L]

is there, from nixpkgs point of view, a "correct way" to detect aligned_alloc availability that we could try and merge upstream?

Please don't take this answer as definitive, but...

There is no proper feature test macro for aligned_alloc, so PyTorch (and every project that uses aligned_alloc) would probably need a full-blown feature test: compile a program that uses aligned_alloc and verify that the compiled program functions as expected. What a pain! That's an incredibly onerous requirement. I bet they won't like it, especially since PyTorch probably compiles correctly on recent versions of MacOS and only fails on exotic environments like Nixpkgs.

More background/brain dump:

AFAICT, PyTorch is checking for C++17 language support in the compiler and assuming that implies the presence of aligned_alloc. That's not quite a correct deduction, even though aligned_alloc is part of the C++17 standard. The relationship between C++17_{language constructs} and C++17_{headers and libraries} is very fuzzy. PyTorch's check is enough to guarantee that the compiler supports C++17 language constructs, but not enough to guarantee that the standard library actually supports aligned_alloc.

Nixpkgs uses a very old set of standard library headers on Darwin, so C++17_{headers and libraries} support on that platform is a bit sketchy, hence the failure. I suspect you would see a similar failure if you installed modern Clang on a very old version of MacOS and tried to install PyTorch---but there are probably only 2 or 3 people on the planet with setups like that. Well, those 2 or 3 plus everyone who uses Nixpkgs on Darwin.

[Et7f3]

Ah timeout 😮‍💨 2h30 on my machine

[SomeoneSerge]

Seems like it worked https://hydra.nixos.org/build/215847815