rust: find nested packages in git repositories

[steveej]

rust: find nested packages in git repositories

this mimics the heuristic cargo uses for finding crates in their git
repositories (cargo-issue-1462).

Motivation for this change

buildRustPackage fails when used with the cargoLock argument and the package in the git dependency is not found in the top-level Cargo.toml. An example would this dependency specification:

cargo-test-support = { git = "https://github.com/rust-lang/cargo", branch = "rust-1.53.0" }

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• 21.11 Release Notes (or backporting 21.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[andir]

Can we have a test case for this?

[harlantwood]

+1 this would be really useful for us!

I don’t have enough context to properly review this, but would appreciate this feature.

[steveej]

Can we have a test case for this?

got one now 🙂

here's the error message the test throws when b2aa19e is reverted.

patching sources
Executing cargoSetupPostPatchHook
Validating consistency between /build/git-dependency-rev-non-workspace-nested-crate//Cargo.lock and /build/cargo-vendor-dir/Cargo.lock
Finished cargoSetupPostPatchHook
configuring
building
Executing cargoBuildHook
++ env CC_x86_64-unknown-linux-gnu=/nix/store/s5hkav7whndbfz0szshpb46h4idqdq9a-gcc-wrapper-10.3.0/bin/cc CXX_x86_64-unknown-linux-gnu=/nix/store/s5hkav7whndbfz0szshpb46h4idqdq9a-gcc-wrapper-10.3.0/bin/c++ CC_x86_64-unknown-linux-gnu=/nix/store/s5hkav7whndbfz0szshpb46h4idqdq9a-gcc-wrapper-10.3.0/bin/cc CXX_x86_64-unknown-linux-gnu=/nix/store/s5hkav7whndbfz0szshpb46h4idqdq9a-gcc-wrapper-10.3.0/bin/c++ cargo build -j 16 --target x86_64-unknown-linux-gnu --frozen --release
building '/nix/store/7zds35bsj1mbijcbprhzc5si3bv9n9rn-cargo-vendor-dir.drv'...
building '/nix/store/hgk9q95jlwcwcqmc0m7zn6jx7zr3krsi-cargo-vendor-dir.drv'...
building '/nix/store/s3rwmyv748p2p8dhf7wbzjlpx0ybg1ji-cargo-vendor-dir.drv'...
building '/nix/store/sw5f7rg4adsw3gkxqz89az13mxx6dimy-cargo-vendor-dir.drv'...
building '/nix/store/cl37yqbx67h7w5300g8nxz3alf8lyrzk-git-dependency-rev-0.1.0.drv'...
building '/nix/store/cdmbk2sq7f6fpfkyca3d4p2q81myyj59-git-dependency-0.1.0.drv'...
building '/nix/store/hhx5v9rgwqsqj90plhmjkd56ppsxpkal-git-dependency-branch-0.1.0.drv'...
error: no matching package named `cargo-test-support` found
location searched: https://github.com/rust-lang/cargo?branch=rust-1.53.0#4369396c
required by package `git-dependency-rev-non-workspace-nested-crate v0.1.0 (/build/git-dependency-rev-non-workspace-nested-crate)`
building '/nix/store/f7drs4qbv2nwvzy6kakqhxg01fl3ak9h-git-dependency-tag-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/x04kms2zjviz9dxrx05i5iaclilvvacg-git-dependency-rev
source root is git-dependency-rev
Executing cargoSetupPostUnpackHook
unpacking source archive /nix/store/bxfsvbzif6vixsykm2ih4ls7d0m5p5vf-cargo-vendor-dir
builder for '/nix/store/nz2klkl3ffj41d94j0yi6xd3v0amf3ll-git-dependency-rev-non-workspace-nested-crate-0.1.0.drv' failed with exit code 101
error: build of '/nix/store/cdmbk2sq7f6fpfkyca3d4p2q81myyj59-git-dependency-0.1.0.drv', '/nix/store/cl37yqbx67h7w5300g8nxz3alf8lyrzk-git-dependency-rev-0.1.0.drv', '/nix/store/f7drs4qbv2nwvzy6kakqhxg01fl3ak9h-git-dependency-tag-0.1.0.drv', '/nix/store/hhx5v9rgwqsqj90plhmjkd56ppsxpkal-git-dependency-branch-0.1.0.drv', '/nix/store/nz2klkl3ffj41d94j0yi6xd3v0amf3ll-git-dependency-rev-non-workspace-nested-crate-0.1.0.drv' failed

[risicle]

Test fails on macos 10.15:

running: "/nix/store/s5yid9p7xgdn2bnh6yys2817y1a71jpr-clang-wrapper-7.1.0/bin/cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-arch" "x86_64" "-I" "/private/tmp/nix-build-git-dependency-rev-non-workspace-nested-crate-0.1.0.drv-0/git-dependency-rev-non-workspace-nested-crate/target/x86_64-apple-darwin/release/build/libgit2-sys-0daeab34138c53c0/out/include" "-I" "libgit2/src" "-I" "libgit2/deps/http-parser" "-I" "libgit2/deps/pcre" "-I" "/private/tmp/nix-build-git-dependency-rev-non-workspace-nested-crate-0.1.0.drv-0/git-dependency-rev-non-workspace-nested-crate/target/x86_64-apple-darwin/release/build/libssh2-sys-23ff351112efb397/out/include" "-I" "/private/tmp/nix-build-git-dependency-rev-non-workspace-nested-crate-0.1.0.drv-0/git-dependency-rev-non-workspace-nested-crate/target/x86_64-apple-darwin/release/build/libz-sys-b4adeaa667061859/out/include" "-fvisibility=hidden" "-DGIT_REGEX_BUILTIN=1" "-DHAVE_STDINT_H=1" "-DHAVE_MEMMOVE=1" "-DNO_RECURSE=1" "-DNEWLINE=10" "-DPOSIX_MALLOC_THRESHOLD=10" "-DLINK_SIZE=2" "-DPARENS_NEST_LIMIT=250" "-DMATCH_LIMIT=10000000" "-DMATCH_LIMIT_RECURSION=MATCH_LIMIT" "-DMAX_NAME_SIZE=32" "-DMAX_NAME_COUNT=10000" "-DSHA1DC_NO_STANDARD_INCLUDES=1" "-DSHA1DC_CUSTOM_INCLUDE_SHA1_C=\"common.h\"" "-DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C=\"common.h\"" "-o" "/private/tmp/nix-build-git-dependency-rev-non-workspace-nested-crate-0.1.0.drv-0/git-dependency-rev-non-workspace-nested-crate/target/x86_64-apple-darwin/release/build/libgit2-sys-0daeab34138c53c0/out/build/libgit2/src/streams/stransport.o" "-c" "libgit2/src/streams/stransport.c"
  cargo:warning=libgit2/src/streams/stransport.c:13:10: fatal error: 'Security/SecureTransport.h' file not found
  cargo:warning=#include <Security/SecureTransport.h>
  cargo:warning=         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
  cargo:warning=1 error generated.
  exit status: 1

[risicle]

👍 FWIW it works with just Security

[steveej]

@risicle

+1 FWIW it works with just Security

cool, i took the liberty to amend and force-push to the macos fix.

[steveej]

@andir the test case is in place.