Skip to content

buildRustPackage: factor out cargoCheckHook and cargoInstallHook#113193

Merged
Mic92 merged 6 commits intoNixOS:stagingfrom
danieldk:cargo-install-hook
Feb 18, 2021
Merged

buildRustPackage: factor out cargoCheckHook and cargoInstallHook#113193
Mic92 merged 6 commits intoNixOS:stagingfrom
danieldk:cargo-install-hook

Conversation

@danieldk
Copy link
Contributor

@danieldk danieldk commented Feb 15, 2021

Motivation for this change

Converts the last parts of buildRustPackage to hooks. There is two APIs changes:

  1. cargoParallelTestThreads suggests that this attribute sets the number of threads used during tests, while it is actually a boolean option (use 1 thread or NIX_BUILD_CORES threads). In the hook, this is replaced by a more canonical name dontUseCargoParallelTests.
  2. preBuild/postBuild are run outside pushd/popd in buildPhase, to make it possible to hookify the binary/library collection.

Wrt. (2) I went over all derivations that use buildRustPackage and preBuild/postBuild. I could only find a problematic preBuild in wasmer, which is fixed in this PR.

I also checked cargoBuildFlags in all derivations that use them and fixed feature quotations in diesel-cli and wasmer.

Related issues:

Follow-up to:

CC @FRidh @Mic92

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. 8.has: documentation This PR adds or changes documentation labels Feb 15, 2021
@ofborg ofborg bot requested review from Br1ght0ne, fgaz, numkem and symphorien February 15, 2021 11:49
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Feb 15, 2021
@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Feb 16, 2021

I could only find a problematic preBuild in wasmer, which is fixed in this PR.

wasmer will always be a special children eh package.

Thank you for your effort!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add that to the changelog?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. There was another API change (removal of the target argument of buildRustPackage). I can do a PR for the release notes after this one that summarizes all the changes.

API change:

`cargoParallelTestThreads` suggests that this attribute sets the
number of threads used during tests, while it is actually a boolean
option (use 1 thread or NIX_BUILD_CORES threads). In the hook, this
is replaced by a more canonical name `dontUseCargoParallelTests`.
Copy link
Member

@Br1ght0ne Br1ght0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

preBuild is now run before changing to buildAndTestSubdir, so use full
path to tests/client.rs in preBuild.
@ofborg ofborg bot requested a review from Br1ght0ne February 17, 2021 17:14
@Mic92
Copy link
Member

Mic92 commented Feb 17, 2021

I am doing one last test building firefox and some rust stuff.

@Mic92 Mic92 merged commit 236b56c into NixOS:staging Feb 18, 2021
threads=1
fi

argstr="--${cargoBuildType} --target @rustTargetPlatformSpec@ --frozen";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like you changed:

  • ${lib.optionalString (checkType == "release") "--release"} -> --${cargoBuildType}

But there are crates out there that on checkType, like cargo-deb. There may be others too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an oversight. We should pass checkType as cargoCheckType and then use that in the hook.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how many packages are out there using that, but I added this to work around issues with packages where cargo test only works with debug, so this shouldn't be dropped entirely IMHO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, it's an oversight. I'll do a PR that fixes this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bobrik added a commit to bobrik/nixpkgs that referenced this pull request Feb 25, 2021
The `checkType` attribute does not apply after NixOS#113193,
and since we're changing tests anyway, let's run them
in the release mode to avoid the need for `checkType`.

This fixes the otherwise broken build.
@nh2 nh2 mentioned this pull request Apr 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants