Skip to content

Comments

FreeBSD: Use separate files for packages#299347

Merged
Ericson2314 merged 1 commit intoNixOS:masterfrom
obsidiansystems:freebsd-by-name
Apr 1, 2024
Merged

FreeBSD: Use separate files for packages#299347
Ericson2314 merged 1 commit intoNixOS:masterfrom
obsidiansystems:freebsd-by-name

Conversation

@Ericson2314
Copy link
Member

Description of changes

There are a number of packages that we ought to be able to autocall, but cannot because we need to add manual arguments just to avoid splicing. This sucks but is the right call for now --- the conclusion should be not that by-name is bad, but that splicing is bad.

This tries to do nothing but move things around; hashes are almost unchanged. @rhelmot then has more changes to do on top of this, which will be easier to review since code will be modified in place rather than moved around and modified at the same time.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@Ericson2314 Ericson2314 requested a review from alyssais March 27, 2024 00:49
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Mar 27, 2024
@alyssais
Copy link
Member

@infinisil just want to check — are we ready for having non-top-level package sets using by-name?

@Ericson2314 Ericson2314 force-pushed the freebsd-by-name branch 2 times, most recently from 3de3fa6 to 6bf738a Compare March 27, 2024 14:09
@Ericson2314
Copy link
Member Author

Converted the rest of the packages now too (not just the ones @rhelmot did).

Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we ready for having non-top-level package sets using by-name?

Not really tbh, and I'd love to have some helping hands to make that a reality. I just opened NixOS/nixpkgs-vet#31 to track this and give a rough idea how to make it reality. Would appreciate some helping hands for that :)

Comment on lines 22 to 25
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function differs somewhat substantially from how pkgs/by-name works:

  • No sharding (wouldn't want that here though)
  • Individual files are allowed too, it doesn't have to be directories like with pkgs/by-name. Only allowing directories was an intentional decision with pkgs/by-name, because it keeps it more consistent and easier to change.
  • Files are allowed to refer to files outside them, unlike pkgs/by-name which forbids this. This doesn't have the benefit of pkgs/by-name self-containedness of packages
  • Recursive attribute sets can be created from directories, unlike pkgs/by-name, which requires all attributes to be derivations. This means that it's not easily possible to act on the result, requiring hacks like recurseIntoAttrs and co.
  • Similarly, it doesn't restrict package.nix to just derivations.
  • And more..

While I don't think this is great, it's not necessarily a big problem here. The main point I want to make is that we shouldn't confuse pkgs/by-name with this function.

Because of that it might be best to at least rename this directory and add some explanation about how it's not like pkgs/by-name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK yeah I didn't even realize the function was unrelated in origin. I'll rename the dir, and then I think we can do the rest as follow-up.

There are a number of packages that we ought to be able to autocall, but
cannot because we need to add manual arguments just to avoid splicing.
This sucks but is the right call for now --- the conclusion should be
not that auto-calling is bad, but that splicing is bad.

This tries to do nothing but move things around; hashes are almost
unchanged. @rhelmot then has more changes to do on top of this, which
will be easier to review since code will be modified in place rather
than moved around and modified at the same time.
@Ericson2314
Copy link
Member Author

OK renamed so it is no longer confused with the "real" by-name.

@Ericson2314 Ericson2314 changed the title FreeBSD: Use "pkgs by name" pattern FreeBSD: Use separate files for packages Mar 28, 2024
@alyssais
Copy link
Member

Shouldn't this result in the same drvs as before?

@Ericson2314
Copy link
Member Author

@alyssais there is a slight difference in the cross-ness of a setup hook itself (based on nix-diff). I decided it didn't matter to me.

@Ericson2314
Copy link
Member Author

 $ nix-diff /nix/store/zcig1vwadj1dkr7a4fz9v8cdlnl0zf6c-zlib-x86_64-unknown-freebsd13-1.3.1.drv /nix/store/8g7az950jckfza2ab21460hgwfyc5zhg-zlib-x86_64-unknown-freebsd13-1.3.1.drv
- /nix/store/zcig1vwadj1dkr7a4fz9v8cdlnl0zf6c-zlib-x86_64-unknown-freebsd13-1.3.1.drv:{out}
+ /nix/store/8g7az950jckfza2ab21460hgwfyc5zhg-zlib-x86_64-unknown-freebsd13-1.3.1.drv:{out}
• The input derivation named `stdenv-linux` differs
  - /nix/store/46f158ry1v2mp6mbnamhi2sr242q730k-stdenv-linux.drv:{out}
  + /nix/store/4p69y4r1hbbmrda2psfnl4wzcyc6dfmc-stdenv-linux.drv:{out}
  • The input derivation named `x86_64-unknown-freebsd13-clang-wrapper-17.0.6` differs
    - /nix/store/q4vij7ypsyvj5vlv751ndcp0kfwadka3-x86_64-unknown-freebsd13-clang-wrapper-17.0.6.drv:{out}
    + /nix/store/4p38vsvjb4jl87js3aqrk24cjd3nypbz-x86_64-unknown-freebsd13-clang-wrapper-17.0.6.drv:{out}
    • The input derivation named `libc-x86_64-unknown-freebsd13-13.1.0` differs
      - /nix/store/30r2k79r24ys37hqfxnc3ms2pzkgc1vg-libc-x86_64-unknown-freebsd13-13.1.0.drv:{out}
      + /nix/store/0zrv27bsfnp1gsm2zx51x84wlz0da3s9-libc-x86_64-unknown-freebsd13-13.1.0.drv:{out}
      • The input derivation named `csu-freebsd-x86_64-unknown-freebsd13-13.1.0` differs
        - /nix/store/9l99rqm97k9z8a2gsyhr0m856hv561fh-csu-freebsd-x86_64-unknown-freebsd13-13.1.0.drv:{out}
        + /nix/store/35q243q8ak7cp44mi9r6vghmpra98fvm-csu-freebsd-x86_64-unknown-freebsd13-13.1.0.drv:{out}
        • The input derivation named `freebsd-setup-hook` differs
          - /nix/store/12qiff9g43f7cqz2sh6i98qfa0si25qw-freebsd-setup-hook.drv:{out}
          + /nix/store/59879crmp465cqxsairmvcn5ng2q0qr8-freebsd-setup-hook.drv:{out}
          • The set of input source names do not match:
              - cross-file.conf
          • The input derivation named `stdenv-linux` differs
            - /nix/store/2ppq6frac5y7nkvc081i52lgrrc75f85-stdenv-linux.drv:{out}
            + /nix/store/4vx0hvfmnwvsc7fif345xs9zdjaxkkpr-stdenv-linux.drv:{out}
            • The set of input derivation names do not match:
                + acl-2.3.2
                + attr-2.5.2
                + binutils-2.41
                + binutils-wrapper-2.41
                + ed-1.20.1
                + expand-response-params
                + gcc-13.2.0
                + gcc-wrapper-13.2.0
                + glibc-2.38-44
                + gmp-6.3.0
                + gmp-with-cxx-6.3.0
                + gnu-config-2023-09-19
                + isl-0.20
                + libidn2-2.3.7
                + libmpc-1.3.1
                + libunistring-1.1
                + linux-headers-6.7
                + mpfr-4.2.1
                + pcre2-10.43
                + xgcc-13.2.0
                + zlib-1.3.1
            • The set of input derivations named `bzip2-1.0.8` do not match
            • The set of input derivations named `xz-5.6.1` do not match
            • The environments do not match:
                + allowedRequisites=/nix/store/dwxz2p17xwmivwl55qkfji1dp2qd2846-gzip-1.13 /nix/store/dwxz2p17xwmivwl55qkfji1dp2qd2846-gzip-1.13 /nix/store/q5ggwzjigzwqy4vjd6gvbs9jh4z33mrb-bzip2-1.0.8-bin /nix/store/vwrm7xc2fha9imwzyz4xm5cvdm45l8m6-bzip2-1.0.8 /nix/store/2zvi5q6fvrmznavnqgzc947wssilv9vy-xz-5.6.1-bin /nix/store/yyqzw7xvsrn3h2zrvincbs1b291yzx8c-xz-5.6.1 /nix/store/5lr5n3qa4day8l1ivbwlcby2nknczqkq-bash-5.2p26 /nix/store/5lr5n3qa4day8l1ivbwlcby2nknczqkq-bash-5.2p26 /nix/store/j2y057vz3i19yh4zjsan1s3q256q15rd-binutils-2.41 /nix/store/k9479b5zabc5ncvb3dmvk2jpqq4avjs0-binutils-2.41-lib /nix/store/43dnc4h7x0p8ybaq38aaxkhy722kzrx2-coreutils-9.4 /nix/store/43dnc4h7x0p8ybaq38aaxkhy722kzrx2-coreutils-9.4 /nix/store/y6lacwx4602ah1nafpm7jfm5bwlafhg4-diffutils-3.10 /nix/store/y6lacwx4602ah1nafpm7jfm5bwlafhg4-diffutils-3.10 /nix/store/6xyv6b1l5zvzd23scbim221rxkcp8g9w-findutils-4.9.0 /nix/store/6xyv6b1l5zvzd23scbim221rxkcp8g9w-findutils-4.9.0 /nix/store/n2lgzbb7gcy48br83cyzbm3f1k3jkip9-gawk-5.2.2 /nix/store/n2lgzbb7gcy48br83cyzbm3f1k3jkip9-gawk-5.2.2 /nix/store/5pyacja52fxmx0k1b27c44nl40v82gbk-gmp-with-cxx-6.3.0 /nix/store/5pyacja52fxmx0k1b27c44nl40v82gbk-gmp-with-cxx-6.3.0 /nix/store/19zx24xlq03ljfill0173a8fn046pvk8-gnumake-4.4.1 /nix/store/19zx24xlq03ljfill0173a8fn046pvk8-gnumake-4.4.1 /nix/store/kqb7yncgd6a17nlgbixhcvcwync0nqg0-gnused-4.9 /nix/store/kqb7yncgd6a17nlgbixhcvcwync0nqg0-gnused-4.9 /nix/store/n40cmp7fqdx66pfxpwqxmgq9card3dma-gnutar-1.35 /nix/store/n40cmp7fqdx66pfxpwqxmgq9card3dma-gnutar-1.35 /nix/store/nk45h8dy9zlx4d0b9ghf5xnj9pypyqgh-gnugrep-3.11 /nix/store/nk45h8dy9zlx4d0b9ghf5xnj9pypyqgh-gnugrep-3.11 /nix/store/fd127i4i2bcswx22gqhvwdbapwj4jq5s-patch-2.7.6 /nix/store/fd127i4i2bcswx22gqhvwdbapwj4jq5s-patch-2.7.6 /nix/store/spiwhs9xa145l8qa5i51f1rcpkfd5jvq-patchelf-0.15.0 /nix/store/spiwhs9xa145l8qa5i51f1rcpkfd5jvq-patchelf-0.15.0 /nix/store/900m2vcva703m9vwnip1fbl0fpwwq722-ed-1.20.1 /nix/store/900m2vcva703m9vwnip1fbl0fpwwq722-ed-1.20.1 /nix/store/a0abh546l9zbx6wf12m8ncwgngvw55y8-file-5.45 /nix/store/a0abh546l9zbx6wf12m8ncwgngvw55y8-file-5.45 /nix/store/9v67dgmg1r9sb92h4c0drh35j2z5cnv4-attr-2.5.2 /nix/store/kq60cqfd2kb7mr6mlbbqbb8li7bkznrb-acl-2.3.2 /nix/store/zbaajn0dk65kwvrcglhli1j4xx4vhvvj-zlib-1.3.1 /nix/store/q3wiyzfhghb8spcsb1hcqcnsvgn57226-pcre2-10.43 /nix/store/n9sq1bvghs9z0qg6cmwg27y4jmszwgqi-libidn2-2.3.7 /nix/store/77yhmwrwism02371kzyda4d127kdwdnf-libunistring-1.1 /nix/store/1rm6sr6ixxzipv5358x0cmaw8rs84g2j-glibc-2.38-44 /nix/store/6jk1d1m5j9d8gjyq79zqlgqqs9j3gcwn-glibc-2.38-44-dev /nix/store/2ksh88m9fnnmj8xn5a2a0z2q9vakbjpj-glibc-2.38-44-bin /nix/store/zb3gzfab3v2q5bib8chihr2k885zj1n4-linux-headers-6.7 /nix/store/5ac6jr3asky426rc5qaxrf3wkhzpw1cm-binutils-wrapper-2.41 /nix/store/kvlhk0gpm2iz1asbw1xjac2ch0r8kyw9-gcc-wrapper-13.2.0 /nix/store/rqga421d43q40blrrgmiw820p01a4nba-gcc-13.2.0 /nix/store/agp6lqznayysqvqkx4k1ggr8n1rsyi8c-gcc-13.2.0-lib /nix/store/n0k7d16gi7mj7bwwm1vlrvhcd1bs3qcd-expand-response-params /nix/store/6k373ff6rgylb9w6l2qlagb5xdv0fcsf-gcc-13.2.0-libgcc /nix/store/3sxwxqzkkrgpgaibkm27ggb9kjbzdy31-xgcc-13.2.0-libgcc /nix/store/4xwfmpljvazd1zbp9f179i33n1acq0cy-update-autotools-gnu-config-scripts-hook /nix/store/8dqbwvzppprxjy98ryh30zrbjhh7g25k-gnu-config-2023-09-19 /nix/store/04qi91bvjwn9s0p23wqy2gjnmshk6h11-gmp-6.3.0 /nix/store/dw1k0g9b40x79258qc7gy5sx6m2kd8w2-libmpc-1.3.1 /nix/store/w5x48fqsx3lvgx7s2i5mx5s901w5d7p0-mpfr-4.2.1 /nix/store/vrj0g16ci387zm1sapqh764qqjf85hk0-isl-0.20 /nix/store/spiwhs9xa145l8qa5i51f1rcpkfd5jvq-patchelf-0.15.0 /nix/store/4xwfmpljvazd1zbp9f179i33n1acq0cy-update-autotools-gnu-config-scripts-hook /nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh /nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh /nix/store/wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh /nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh /nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh /nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh /nix/store/pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh /nix/store/jivxp510zxakaaic7qkrb7v1dd2rdbw9-multiple-outputs.sh /nix/store/ilaf1w22bxi6jsi45alhmvvdgy4ly3zs-patch-shebangs.sh /nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh /nix/store/xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh /nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh /nix/store/wmknncrif06fqxa16hpdldhixk95nds0-strip.sh
          • Skipping environment comparison
        • The input derivation named `include-freebsd-x86_64-unknown-freebsd13-13.1.0` differs
          - /nix/store/5dmi8zwm2sd81jc6wxrc89m6p2slvala-include-freebsd-x86_64-unknown-freebsd13-13.1.0.drv:{out}
          + /nix/store/x5lbqaswwjyfzax9svigdjzxwija8104-include-freebsd-x86_64-unknown-freebsd13-13.1.0.drv:{out}
          • The input derivation named `freebsd-setup-hook` differs
            • These two derivations have already been compared
          • Skipping environment comparison
        • Skipping environment comparison
      • The input derivation named `freebsd-setup-hook` differs
        • These two derivations have already been compared
      • The input derivation named `include-freebsd-x86_64-unknown-freebsd13-13.1.0` differs
        • These two derivations have already been compared
      • Skipping environment comparison
    • The input derivation named `libcxx-x86_64-unknown-freebsd13-17.0.6` differs
      - /nix/store/4vnd887wwkc33klgn2bqi1wp38hi6r5l-libcxx-x86_64-unknown-freebsd13-17.0.6.drv:{dev}
      + /nix/store/dcwzmzsri52p2149183arzp9szhmdw1h-libcxx-x86_64-unknown-freebsd13-17.0.6.drv:{dev}
      • The input derivation named `libcxxrt-x86_64-unknown-freebsd13-unstable-2024-02-05` differs
        - /nix/store/16q5vdq7b82lwpdqhd3amx2xa3jydadh-libcxxrt-x86_64-unknown-freebsd13-unstable-2024-02-05.drv:{out}
        + /nix/store/hs8l52pqmivvandalhawkhz1f30a0n48-libcxxrt-x86_64-unknown-freebsd13-unstable-2024-02-05.drv:{out}
        • The input derivation named `stdenv-linux` differs
          - /nix/store/jfs75caqc4z72ms4dsjym8i52hn2gn18-stdenv-linux.drv:{out}
          + /nix/store/3n3zzbzi1qkxkqirq1q7p97z7jxi38wa-stdenv-linux.drv:{out}
          • The input derivation named `x86_64-unknown-freebsd13-clang-wrapper-17.0.6` differs
            - /nix/store/v84vydna8hxmszvhbvbgamvdfjcpmq2y-x86_64-unknown-freebsd13-clang-wrapper-17.0.6.drv:{out}
            + /nix/store/hf7w958nw0nkx8fxisfj0b5gd0ghzsyn-x86_64-unknown-freebsd13-clang-wrapper-17.0.6.drv:{out}
            • The input derivation named `libc-x86_64-unknown-freebsd13-13.1.0` differs
              • These two derivations have already been compared
            • The input derivation named `x86_64-unknown-freebsd13-llvm-binutils-wrapper-17.0.6` differs
              - /nix/store/1krhwcdn0pvpldfyfj1fv7wxvimd2ah1-x86_64-unknown-freebsd13-llvm-binutils-wrapper-17.0.6.drv:{out}
              + /nix/store/584l6cgzsrh2l5l12xj2an6sxmzdz16m-x86_64-unknown-freebsd13-llvm-binutils-wrapper-17.0.6.drv:{out}
              • The input derivation named `libc-x86_64-unknown-freebsd13-13.1.0` differs
                • These two derivations have already been compared
              • Skipping environment comparison
            • Skipping environment comparison
          • Skipping environment comparison
        • Skipping environment comparison
      • The input derivation named `libunwind-x86_64-unknown-freebsd13-17.0.6` differs
        - /nix/store/za7s6dqglriylkh3xhwccv1fy5gzzxaa-libunwind-x86_64-unknown-freebsd13-17.0.6.drv:{dev}
        + /nix/store/k8w9mmg5ryynamng9j6996cgxj3bi0mg-libunwind-x86_64-unknown-freebsd13-17.0.6.drv:{dev}
        • The input derivation named `stdenv-linux` differs
          • These two derivations have already been compared
        • Skipping environment comparison
      • The input derivation named `stdenv-linux` differs
        • These two derivations have already been compared
      • Skipping environment comparison
    • The input derivation named `libunwind-x86_64-unknown-freebsd13-17.0.6` differs
      • These two derivations have already been compared
    • The input derivation named `x86_64-unknown-freebsd13-llvm-binutils-wrapper-17.0.6` differs
      • These two derivations have already been compared
    • Skipping environment comparison
  • Skipping environment comparison
• Skipping environment comparison

The new way is actually more correct I think ---- the setup hook itself is not cross-compiled.

@Ericson2314 Ericson2314 merged commit 48020a2 into NixOS:master Apr 1, 2024
@Ericson2314 Ericson2314 deleted the freebsd-by-name branch April 1, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants