rust: cargo: Use rustc and cargo built on Build for cross-compiling

[mehmooda]

When cross-compiling a rust package, all we need is the std library compiled
for the target. This uses the final stage compiler which was built for Build
and then uses that as a stage0 compiler for target std library.

It also copies the rust binary from pkgsBuildBuild so that it find the new
lib/rustlib directory.

We also need to create a cargo wrapper which will use the "new" rust compiler

Closes #207435.

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.11 Release Notes (or backporting 22.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[Ericson2314]

I think this sort of thing is good to have.

[mehmooda]

I think this is ready now.
I'll queue up a clean up to staging for the if else null into a lib.optionalString after this and master gets merged into staging.

[mehmooda]

TIL: if you ln -s a directory straight onto the output, it works when nix is installed single user but fails when nix is running as a daemon due to it trying to chown the existing directory. Using lndir instead works.

[06kellyjac]

anything else holding this up other than conflicts?

[yu-re-ka]

This does not work with the current version of Rust. The compiler even after being copied to the new location is still looking for core crate exclusively in its original path, and this seems to be the compiled in sysroot (rustc --print sysroot).
Once more a wakeup call for someone to come up with a rustc wrapper ;)

We can still work around this however, by rewriting this reference in the binary to the new $out. But for that, the derivation name length has to match.

[winterqt]

Once more a wakeup call for someone to come up with a rustc wrapper ;)

If it's compiled in, how would a wrapper help?

[yu-re-ka]

It can be altered with the --sysroot parameter

[yu-re-ka]

cross-compiling fd fails with error: failed to get bitcode from object file for LTO (Bitcode section not found in object file), presumably because of other assumptions that are compiled into the rustc binary

[yu-re-ka]

The std lib produced with fastCross is a stage 0 std lib, and thus does not include LLVM bitcode.

https://github.com/rust-lang/rust/blob/master/src/bootstrap/compile.rs#L409

[yu-re-ka]

This should now work (again). Please review.

[kloenk]

LGTM

resolved

[yu-re-ka]

@ofborg build pkgsCross.aarch64-multiplatform.fd

[yu-re-ka]

@ofborg build pkgsCross.aarch64-multiplatform.fd

[yu-re-ka]

@ofborg build pkgsCross.musl64.fd

[yu-re-ka]

@ofborg build pkgsStatic.fd

[yu-re-ka]

Note: pkgsStatic.fd on aarch64-linux breakage is unrelated; was already broken on master

[infinisil]

This doesn't appear to have fixed the linked issue, see #207435 (comment). Ping @winterqt

[yu-re-ka]

That a rustc derivation appears in the list is to be expected. Within that derivation is should only build the std lib though. 15 Sept 2023 02:03:38 Silvan Mosberger ***@***.***>:

…

This doesn't appear to have fixed the linked issue, see #207435 (comment)[#207435 (comment)]. Ping @winterqt[https://github.com/winterqt] — Reply to this email directly, view it on GitHub[#176694 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AURNSZN3Y426FJVGFY7BN3TX2OLNJANCNFSM5YCOZKQA]. You are receiving this because you commented. [Tracking image][https://github.com/notifications/beacon/AURNSZOKZ7GSJ7AXGI4ANELX2OLNJA5CNFSM5YCOZKQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOM2E4KIQ.gif]