pkgsLinux: set crossSystem instead of localSystem

[tie]

Description of changes

As the doc implies, we should be using crossSystem to build for the Linux platform. Using localSystem just happens to occasionally hit cached builds. See also #316659 (comment) and https://github.com/NixOS/nixpkgs/runs/25855763929

See also #282401, #294725

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[tie]

The last tpm2-tss commit would likely cause a lot of rebuilds since it’s systemd dependency. I’ll rebase to the staging branch later.

[tie]

OK, apparently some Linux-specific packages are still getting pulled from hostPkgs (mostly for full systemd, for some reason, e.g. bpftool). Marking the PR as ready for review so we’d at least get an evaluation failure instead of confusing build failure for now.

[tie]

Ah, I see #293573. That still seems like a really weird and niche use case to me — I mean, it already requires linux builder, so why not run tests there? Instead, I think we should be able to cross-compile NixOS from macOS with some effort.

[baloo]

This will need to retarget to staging (because of tpm2-tss), but for the tpm2 changes I don't see issues.

[roberth]

See comment.
To summarize, building in a VM is preferable over cross compilation for various reasons.

[roberth]

This is meant to be a natively built Linux package set, ie a non-cross build that's potentially done on a different machine. Do you think we non-cross would be a clearer description than natively built in the doc above?
Note that the "machine" is easy to set up with for example linux-builder.

By changing to cross compilation, we make the user experience worse and increase cost.

• Cross compilation, despite how well it works in Nixpkgs, does not permit tests to be run in package builds
• We don't have a cache for darwin->linux cross builds, afaik. These builds would be unnecessary extra builds, costing compute and cache storage (ofborg, hydra, cache.nixos.org, etc)

Also note that if you're in a team using a mix of Linux and Darwin and you're doing deployment with cross builds, your performing unnecessary redeployments depending on who's initiating it, causing unnecessary disruptions as system services are "updated" to an equivalent package on a different store path. You could solve this by deploying from dedicated infrastructure, but then you might as well use that for remote builds as well.
More importantly though, non-cross builds are more likely to work.
All in all, non-cross deployments are simpler and more robust, so I would not default to cross compilation anywhere, including here.

Perhaps your goal could be achieved by adding pkgsLinuxCross, but then I still wouldn't make the test framework use that for the reasons above.

[tie]

That sounds reasonable given the current state of cross-compilation of Nixpkgs (though I wouldn’t say it works well, rather it works for well-tested {local,cross}System pairs, but that doesn’t cover glibc → glibc static cross-build, NixOS built from macOS, and a lot of other cases).

I think we’ve had a similar discussion in NixOS/nix#10291 about this use case. W.r.t. tests, these do not change the resulting package output. So it doesn’t really matter where the tests are run as long the closures are byte-for-byte identical and reproducible. That is, as long as we can test that a certain set of packages is identical independent of the localSystem, it doesn’t really matter which derivation runs the tests (assuming that at least one does). Sure, that requires content-addressed derivations to get this right, but then there are a lot of other places that should be fixed before that.

I understand your point, but I do want to improve the current state of affairs. I’ll open separate PR for package-specific commits I’ve pushed to this branch, without this particular change.

[roberth]

I'd previously asked the infra team to provide a Linux builder, but they were concerned about a deadlock issue. I've now opened an issue with alternatives that avoid the problem (and it's also actually properly visible, unlike chat in Matrix).

• Fix Darwin passthru VM tests with remote builders infra#439