rust: make pkgsCross.wasi32.rustPlatform.buildRustPackage work

[jcaesar]

Description of changes

This PR makes it possible to e.g. build and run rust packages like nix shell .#pkgsCross.wasi32.charasay -c charasay.wasm say moo (assuming you have wasm32-wasi in your emulated systems).

Without this PR, building the cross rustc fails with

thread 'main' panicked at src/core/build_steps/compile.rs:370:17:
Target "wasm32-unknown-wasi" does not have a "wasi-root" ```

I think this is neat, but it admittedly isn't super useful, as a lot of packages fail to compile due to failing C dependencies or some common dependencies assuming that all wasm is for the web. I've tested the 469 rust packages in by-name, and only 60¹ compile. Even those that do compile might fail at runtime, e.g. due to std::thread not being supported and panicking when attempting to create a thread (like alejandra).

I'm only aware of two mildly useful things that work, namely rq and qrtool. But I think it's nice to have this working.

¹ aarch64-esr-decoder, aba, action-validator, adrs, bunbun, cargo-autoinherit, cargo-bloat, cargo-bump, cargo-shear, cargo-swift, catppuccin-whiskers, catppuccinifier-cli, cdwe, circom, clippy-sarif, cmd-wrapped, commitmsgfmt, diesel-cli-ext, dipc, elf-info, emacs-lsp-booster, esbuild-config, french-numbers, galerio, git-upstream, hadolint-sarif, hvm, hyperlink, icnsify, ifrextractor-rs, keedump, kickstart, ldproxy, little_boxes, ludtwig, lutgen, majima, mcumgr-client, mdsh, ndstrim, obsidian-export, okolors, parallel-disk-usage, pdfrip, polylux2pdfpc, preserves-tools, prettypst, protoc-gen-rust-grpc, qrtool, rq, rs-tftpd, rusti-cal, rusty-diceware, sarif-fmt, shellcheck-sarif, tera-cli, thud, toml-cli, typstfmt, unison-fsmonitor, wit-bindgen

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[jcaesar]

I've re-verified that this is the best solution I can come up with. I've added comments on the parts that aren't entirely starightforward. @Kranzes, if you only removed your reviewer status because this was still a draft, can I ask you to have a look at some point?

[jcaesar]

Thank you for the review.

(The reason I wasn't particularly keen on touching wasilibc is that the following takes ages to build. The floorp failure also happens on master, so that's unrelated.)

---

Result of nixpkgs-review pr 323161 run on x86_64-linux 1

1 package blacklisted:

• nixos-install-tools

3 packages failed to build:

• floorp
• floorp-unwrapped
• floorp-unwrapped.debug

32 packages built:

• adoptopenjdk-icedtea-web
• betterbird
• betterbird-unwrapped
• betterbird-unwrapped.debug
• eyewitness
• firefox
• firefox-beta
• firefox-beta-unwrapped
• firefox-beta-unwrapped.debug
• firefox-beta-unwrapped.symbols
• firefox-devedition
• firefox-devedition-unwrapped
• firefox-devedition-unwrapped.debug
• firefox-devedition-unwrapped.symbols
• firefox-esr
• firefox-esr-115-unwrapped
• firefox-esr-115-unwrapped.debug
• firefox-esr-115-unwrapped.symbols
• firefox-mobile
• firefox-unwrapped
• firefox-unwrapped.debug
• firefox-unwrapped.symbols
• firefoxpwa
• librewolf
• librewolf-unwrapped
• librewolf-unwrapped.debug
• sitespeed-io
• slimerjs
• thunderbird
• thunderbird-unwrapped
• thunderbird-unwrapped.debug
• thunderbird-unwrapped.symbols

[alyssais]

I tried building pkgsCross.wasi32.qrtool, but it failed to build wasm32-unknown-wasi-rustc-1.78.0.

thread 'main' panicked at src/core/build_steps/compile.rs:400:17:
Target "wasm32-unknown-wasi" does not have a "wasi-root" key in Config.toml
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:13

[jcaesar]

@alyssais Could you tell me exactly what you're trying to build? The following work for me (should be the same thing):

nix build  -vL github:NixOS/nixpkgs/56df69358a7c06f415c33c3cb9543e41b872c220#pkgsCross.wasi32.rq
nix build  -vL github:NixOS/nixpkgs/pull/323161/head#pkgsCross.wasi32.rq

(qrtool will fail in installPhase because of #289517 / #308283. Try rq instead if the above does get past the failure you saw in rustc.)

[Edit:] Did you try that on master / some other PR? Because that's nearly the same error as in the opening post for this PR.

[alyssais]

Looks like I might have mixed up which branch I was building — sorry!

[jcaesar]

Here I'd thought I could finally make progress because found a decent workaround for the mess in the rust build environment setup (namely: do actually use linkers that are either named *-gcc or *-clang (just like actually passed to rustc). There's a longer rationale in the commit message.)

And already, three new messes have appeared.

• There's now three wasi targets (-p1, -p1-threads, -p2), and that isn't being dealt with anywhere yet, at all
• rustc now uses llvm fired clang, which doesn't build for wasi. (I don't fully understand what's going on)
• rustc started to expect wasi targets to use a built-in linker (i.e. placed at …/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/). Fixed by a single substituteInPlace, but :/. (Placing a linker in that path is possible, but annoying since x86_64-unknown-linux-gnu is a symlink to a different derivation thanks to fastCross)

I think I'll let this rest.

[jcaesar]

This is a time sink I can't see myself working on in the near future. It's not quite in line with nixpkgs goals, and if I'd be looking to make something of practical value, I'd look for wali these days.