lib.systems: introduce toolchain, cc, and bintools attributes

[RossComputerGuy]

Things done

Replaces useLLVM, useArocc, and useZig with toolchain, cc, linker, and bintools attributes. This might not produce any rebuilds but we'll see. This has the advantage of preventing using${compiler} flags from colliding and not working correctly if we were to stack multiple pkgs* together.

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[tomberek]

The main idea and motivation is in: pkgs/stdenv/cross/default.nix

[K900]

ack, would be nice to make this rebuild 0

[RossComputerGuy]

@K900 there was a typo with xwayland, that should be fixed now. It'd be great if we can get this PR through before it becomes more work to rebase.

[RossComputerGuy]

Rebasing, we also have an stdenv board now. I hope this doesn't stall out again because there's a bunch of conflicts and I don't want to rebase this regularly.

[philiptaron]

Thanks for your work here Tristan. I'm looking to @emilazy to give the ✅ , but I think we are agreed:

1. Using well-known strings with symbolic meaning is the right direction, rather than is* and use* functions attached to the stdenv.
2. Splitting apart the concept of the "toolchain" into "cc", "bintools", "cxxlib", "unwinderlib", "rtlib", and "linker" allows derivations to tune their behavior wisely in the face of the combinatorial explosion of possibilities.
3. We should expect to continue to evolve this system over the course of the next few releases as the codebase adopts and reacts to these well-known strings -- and introduces more, because the innovation train isn't stopping soon.

I'd like to do a build on staging once the merge conflicts are resolved and at least one of @reckenrode or @emilazy accepts, then merge.

What say you all?

[philiptaron]

Not blocking:

If (lib.listToAttrs (lib.map (key: lib.nameValuePair key final.${key}) (lib.attrNames toolchain))) were extracted to a let binding, we could write a good assertion message here.

[RossComputerGuy]

This might be fine as is. I'm not sure, maybe just a comment would be okay? Let's see what others think.

[emilazy]

I am mostly worried about whether elaborate is in the hot path of anything. I think that since these values didn’t exist before it’s okay if the error message is bad for now. assert toolchain == lib.mapAttrs (key: _: final.${key}) toolchain seems better and surely faster, regardless.

[RossComputerGuy]

I am mostly worried about whether elaborate is in the hot path of anything. I think that since these values didn’t exist before it’s okay if the error message is bad for now.

I think the people who tend to change these sorts of things are in the minority. They likely would be in a Matrix chat one of us are in anyway so clarification on the failure is possible there. We shouldn't rely on that but I don't see many people utilizing this feature until it is flushed out more.

assert toolchain == lib.mapAttrs (key: _: final.${key}) toolchain seems better and surely faster, regardless.

Oh yeah, that is a better way of checking.

[philiptaron]

me getting confused: why would this be the hostPlatform and not the buildPlatform's linker?

[RossComputerGuy]

According to https://nixos.org/manual/nixpkgs/stable/#ssec-cross-platform-parameters, host is the platform where things run. The things we run are being built with a different linker. crossPlatform in nixpkgs goes to hostPlatform and targetPlatform.

[philiptaron]

this file in particular looks a lot better

[philiptaron]

I like these changes a lot.

[RossComputerGuy]

Considering the size of this PR, complexity, available time, and other factors I think this should be fine as is. We always can work on improving this with additional work. This is something I very much would like to get into 25.05 so I can work on other work like reworking the CPU model support and going towards LLVM bootstrapping inside the stdenv.

I know @philiptaron mentioned he'll start a build up this weekend so we'll see what happens from that.

[emilazy]

I think that a PR being huge and complex and having limited available qualified reviewer time are the opposite of reasons to rush a PR…

I still intend to try and find the time to review this over the weekend. However, I continue to think it would be best for everyone to split it up into stages as I previously recommended, since this is still only part of the picture as it is. Landing in 25.05 or not shouldn’t matter for follow‐up work, since these are purely read‐only properties that wouldn’t be exposed interfaces for platform specification or lead to the deprecation of the existing useLLVM etc. until they can be properly wired up to the bootstrap as discussed.

[RossComputerGuy]

Ok, I've split it into just the first commit as a MVP for this feature. This likely will be split into 4 or 6 subsequent PR's.

[emilazy]

Thanks, I appreciate you splitting off the migration changes since it’ll make them easier to review without blocking this improvement! It looks good to me now modulo a few nits.

[emilazy]

Will this not override the input when these are set? (Might just be final being named confusingly…)

[RossComputerGuy]

No because args is merged on top, that's how libc, isStatic, and a bunch of other things work.

[emilazy]

I think this needs to stay outside of toolchain, since overriding it was already technically “supported” (regardless of whether it actually did anything useful) so it’s a breaking change to include it in the checks, sadly.

[RossComputerGuy]

Alright, understandable.

[emilazy]

I think this needs a cxxrtlib to go with it, per #365057 (comment).

[RossComputerGuy]

Alright, should we imply the cxxrtlib based on the cxxlib or should it be based on the "toolchain" in use?

[emilazy]

I think this and maybe other things might be wrong for MinGW + threads + etc., but I’m not sure what would be right and it can probably wait until this work actually starts touching bootstrap in a way that will flag it up.

[RossComputerGuy]

Yeah, I think @Ericson2314 would know? However, he can be busy and difficult to get ahold of sometimes. If he can clarify like before the end of the week then I think we can go from there but if it's longer than a week then follow up PR's can fix it.

[emilazy]

I am mostly worried about whether elaborate is in the hot path of anything. I think that since these values didn’t exist before it’s okay if the error message is bad for now. assert toolchain == lib.mapAttrs (key: _: final.${key}) toolchain seems better and surely faster, regardless.

[RossComputerGuy]

Thank you @emilazy, I've addressed the feedback now.

[nix-owners]

The PR's base branch is set to master, but 932 commits from the python-updates branch are included. Make sure you know the right base branch for your changes, then:

• If the changes should go to the python-updates branch, change the base branch to python-updates
• If the changes should go to the master branch, rebase your PR onto the merge base with the master branch:

  # git rebase --onto $(git merge-base upstream/master HEAD) $(git merge-base upstream/python-updates HEAD)
  git rebase --onto effac2a6624f7486f081f59e74578203fc1596d3 f9cfc40c66860d770edc93fc630b54bedf103c03
  git push --force-with-lease

[RossComputerGuy]

Accidentally did maintainers/scripts/auto-rebase/run.sh upstream/staging.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/5394

[RossComputerGuy]

I've realized that this might be necessary to make crossStdenv since it would make overriding the toolchain inside the stdenv easier. I have #409851 which implements some of the new logic this PR allows for.

[emilazy]

Thanks, this looks like a good basis for migrating checks and then the bootstrap. A few details left but I think we can land this soon.

There is one design point I am still unsure about, which is the stringly‐typed nature of the API. Using the cxx vs. c++ ambiguity as an example, a check like stdenv.hostPlatform.cxxlib == "libc++" would be silently wrong right now. That might just be part of the Nix experience, but it is possible we could do better and make this less error‐prone – e.g. expose something stdenv.hostPlatform.cxxlib == lib.toolchain.cxxlib.libcxx to compare against, or lib.toolchain.isLibcxx stdenv.hostPlatform.

Maybe the existing lib.systems.inspect functionality offers everything we need here and we can just define patterns? But I think we should figure out what we want these checks to look like before migrating them across the tree. @alyssais Do you have any opinions on this?

[emilazy]

@reckenrode What do we want for Darwin here? I’m not sure what the balance of LLVM vs. cctools is currently.

Maybe it would be best to split this up into separate variables, but I’m not sure what the correct split would be.

[reckenrode]

Darwin’s bintools is almost all LLVM except for the following: ranlib, otool, install_name_tool, and ld. I’m currently testing a change to use ranlib from LLVM, so it might only be the last three for 25.11.

I’d really like to change the linker to ld64 to match what Darwin is actually using. Technically, cctools has another linker, but it’s old and not used anymore.

[reckenrode]

(In short, I think llvm is right for Darwin for bintools here.)

[emilazy]

We don’t use libcxxrt on any platform right now (but in the future it will probably make sense to use libc++ and libcxxrt on FreeBSD rather than the current libstdc++ plus libsupc++ stack). This should always be libsupc++ when using libstdc++.

Also, we should be consistent around cxx vs. c++ if we diverge from project names. I think I would favour using the upstream project names (so libstdc++, libc++abi, libcxxrt).

[RossComputerGuy]

I've switched to replace ++ with xx. The idea behind using xx is then it's a bit nicer to interact with directly since you don't have to wrap in a string if it is an attribute name.

[emilazy]

Suggested change

	"libunwind-darwin-host"
	"libunwind-system"

“Host” is confusing wrt. hostPlatform. I think we don’t need to encode the Darwin detail because that can be conditioned on separately.

[reckenrode]

If we’re going to distinguish the system libunwind on Darwin, should we also distinguish using the system libc++ (once that lands)?

[RossComputerGuy]

Renamed

[emilazy]

I think this should just be libgcc. _s is for shared library.

[RossComputerGuy]

Done

[emilazy]

I don’t think we should set these while they’re redundant to useLLVM (since they couldn’t be any other possible value, and adjusting them won’t work, and it means that e.g. lib.systems.examples.aarch64-freebsd // { useLLVM = false; } becomes broken for no good reason).

[RossComputerGuy]

It's necessary for eval to work.

[RossComputerGuy]

Since we have lib.systems.toolchain, we can use that to easily help clean out stdenv.hostPlatform of the toolchain attributes. Now that logic is handled outside of this file for setting the new toolchain attributes.

[RossComputerGuy]

but it is possible we could do better and make this less error‐prone – e.g. expose something stdenv.hostPlatform.cxxlib == lib.toolchain.cxxlib.libcxx to compare against, or lib.toolchain.isLibcxx stdenv.hostPlatform.

100% agree on this

[alyssais]

Maybe the existing lib.systems.inspect functionality offers everything we need here and we can just define patterns? But I think we should figure out what we want these checks to look like before migrating them across the tree. @alyssais Do you have any opinions on this?

Not especially, although reusing an existing mechanism sounds nice if it's workable.

[reckenrode]

Maybe the existing lib.systems.inspect functionality offers everything we need here and we can just define patterns? But I think we should figure out what we want these checks to look like before migrating them across the tree. @alyssais Do you have any opinions on this?

If we’re going to distinguish the system libunwind and/or libc++ on Darwin, I would like to have some way to do these checks that treats them as libunwind and libc++ when you don’t care about that detail. Otherwise, everywhere that checks will have to condition on both variants, check a prefix, or be wrong for Darwin. (Obviously, there needs to be a way to distinguish when you do care, but I doubt that will be needed much outside of the LLVM derivation.)

[RossComputerGuy]

If we’re going to distinguish the system libunwind and/or libc++ on Darwin, I would like to have some way to do these checks that treats them as libunwind and libc++ when you don’t care about that detail. Otherwise, everywhere that checks will have to condition on both variants, check a prefix, or be wrong for Darwin. (Obviously, there needs to be a way to distinguish when you do care, but I doubt that will be needed much outside of the LLVM derivation.)

Agreed, I think having a way to distinguish between a specific unwindlib (Apple's vs LLVM's) and a generic unwindlib (GCC vs LLVM) is definitely needed. I was thinking this could be done inside the specific derivations. We could have isAppleUnwind, isLLVM, and isGNU in the passthru.

[RossComputerGuy]

Would be nice to continue on this.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/6078