[staging] gnugrep: 3.7 -> 3.11

[dasJ]

Also replace myself as a maintainer

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[dasJ]

Draft for now because this breaks PCRE

[dasJ]

Should work now, @trofi can you verify that I didn't break the bootstrap tools?

[trofi]

At least on linux (and I think on darwin as well) we will need to add pcre2 to allowed references (or substitute from pcre?). Otherwise stdenv fails to build as:

$ nix build --no-link -f. stdenv
error: output '/nix/store/jy1wr873b4ib5gplrg3118z8gjs2bpa1-stdenv-linux' is not allowed to refer to the following paths:
         /nix/store/grd3jawsc8xchi0zg92k91m1n5piasbj-pcre2-10.42

[dasJ]

Do I need them in persistent as well?

Edit: I just added it as well because it made sense. Also switched from pcre to gnugrep.pcre2 to be more consistent

[trofi]

This script runs against current bootstrapTools binaries that nixpkgs uses. It will have to have two worlds simultaneously:

• grep with pcre
• grep with pcre2

As it it fails to unpack current bootstrapTools:

$ nix build --no-link github:NixOS/nixpkgs/pull/228714/merge#stdenv -L
...
bootstrap-tools> patching /nix/store/l4xssxrlz7mdpqr4blfh774q6v56849k-bootstrap-tools/lib/libpcre2*
bootstrap-tools> stat: No such file or directory

[trofi]

Similar to glibc case it has to handle our current bootstrapTools as well.

[dasJ]

I have successfully built stdenv from this branch now for x86_64-linux

[trofi]

LGTM. We can fix the

./pkgs/stdenv/linux/bootstrap-tools-musl/scripts/unpack-bootstrap-tools.sh
./pkgs/stdenv/linux/bootstrap-tools/scripts/unpack-bootstrap-tools.sh

a while after (hopefully before anyone tries to use newer bootstrapTools.

[Artturin]

Suggested change

	checkInputs = [ perl ];
	nativeCheckInputs = [ perl ];

[vcunat]

Today's cutoff for "breaking changes" in 23.05, so I wonder if this is worth squeezing in. (I don't feel confident enough to merge now.)

[dasJ]

I personally wouldn't risk it, it's not like people have been anxiously waiting for this gnugrep update ;)

[vcunat]

musl is in trouble, but fortunately only on aarch64:
https://hydra.nixos.org/build/221679820

[risicle]

This appears to break pkgsStatic.gnugrep for me on x86_64 linux, with 4 tests failing: test-nl_langinfo-mt, test-setlocale_null-mt-one, test-setlocale_null-mt-all, test-thread_create.

[wegank]

So, add !stdenv.hostPlatform.isMusl to doCheck?

[vcunat]

Yes, I'd use that as the simplest workaround, in case nothing better is submitted. It's surely better than to have no musl-based stdenv. For example, we could conditionally drop just the tests failing now (a tiny minority).

[dasJ]

I'm currently investigating

[vcunat]

Hmm, on x86_64-darwin it always gets stuck during tests (four times at least now). That's on Hydra: https://hydra.nixos.org/build/222612843#tabs-buildsteps
Locally it did pass for me on the first try – so maybe something about rosetta. I don't know, but I expect we'd conditionally drop tests again?

[dasJ]

so doCheck = !stdenv.hostPlatform.isDarwin?

[wegank]

IIRC, @reckenrode also mentioned the x86_64-darwin issue, albeit for sandbox builds.

[reckenrode]

I encountered it trying to work on the Darwin stdenv bump on x86_64-darwin. It always hangs on my Apple Silicon MBP, but it builds on our Intel iMac. The MBP is running 13.4 while the iMac is still on 13.3.1(a). I don’t know if OS version makes a difference, or if it’s a Rosetta 2 issue the test is triggering.

[vcunat]

Apparently it succeeded in the end

nix build /nix/store/cprxh7g44qgvygmc2hv3l6xg3qkyl6sx-gnugrep-3.11 --dry-run

so I guess I'll let it be for now and reconsider doCheck = !stdenv.hostPlatform.isDarwin once it gets stuck again? (which I expect to happen in some future staging-next)

[reckenrode]

How long did it take to run? I left mine going for hours, and it never finished.

[vcunat]

I don't know how to find that. The log is easy to get, as the URL doesn't change on retries, but I can't see how to find which job built it (and see the time and machine).

[Artturin]

I don't know how to find that. The log is easy to get, as the URL doesn't change on retries, but I can't see how to find which job built it (and see the time and machine).

checkPhase completed in 2 minutes 32 seconds

[reckenrode]

Are any of the builders actually x86_64-darwin hardware? It’s still stuck for me (Rosetta 2 on an M1 Max).

[1/158/203 built, 0.0 MiB DL] building gnugrep-3.11 (checkPhase): FAIL: test-c-stack.sh

It’s been stuck at that for hours. I even tried to rework my stdenv stuff to build gnugrep later (with clang 16), but it didn’t make a difference.

[vcunat]

I believe all are aarch64-darwin currently. @cole-h should have a better grasp on this.

[cole-h]

ofborg has 3 x86 and 2 aarch64 macs, with the aarch64 macs enabling Rosetta and taking x86 builds whenever The Scheduler desires.

[dasJ]

Can anyone with Rosetta test this patch?

diff --git a/pkgs/tools/text/gnugrep/default.nix b/pkgs/tools/text/gnugrep/default.nix
index 75c6f695073..2494ed1e49b 100644
--- a/pkgs/tools/text/gnugrep/default.nix
+++ b/pkgs/tools/text/gnugrep/default.nix
@@ -17,7 +17,8 @@ stdenv.mkDerivation {
   };

   # Some gnulib tests fail on Musl: https://github.com/NixOS/nixpkgs/pull/228714
-  postPatch = if stdenv.hostPlatform.isMusl then ''
+  # They also seem to fail on Rosetta: https://github.com/NixOS/nixpkgs/pull/228714#issuecomment-1574300624
+  postPatch = if stdenv.hostPlatform.isMusl || stdenv.hostPlatform.system == "x86_64-darwin" then ''
     sed -i 's:gnulib-tests::g' Makefile.in
   '' else null;

[vcunat]

@cole-h: this was too big for OfBorg, I meant Hydra where it did succeed after several attempts (but I don't know on which machine).

[cole-h]

According to https://hydra.nixos.org/machines, we only have 1 "real" x86_64-darwin machine (destined-gannet). (There's stirred-oryx too, but it's been dead basically since forever.)

[cole-h]

Can anyone with Rosetta test this patch?

diff --git a/pkgs/tools/text/gnugrep/default.nix b/pkgs/tools/text/gnugrep/default.nix
index 75c6f695073..2494ed1e49b 100644
--- a/pkgs/tools/text/gnugrep/default.nix
+++ b/pkgs/tools/text/gnugrep/default.nix
@@ -17,7 +17,8 @@ stdenv.mkDerivation {
   };

   # Some gnulib tests fail on Musl: https://github.com/NixOS/nixpkgs/pull/228714
-  postPatch = if stdenv.hostPlatform.isMusl then ''
+  # They also seem to fail on Rosetta: https://github.com/NixOS/nixpkgs/pull/228714#issuecomment-1574300624
+  postPatch = if stdenv.hostPlatform.isMusl || stdenv.hostPlatform.system == "x86_64-darwin" then ''
     sed -i 's:gnulib-tests::g' Makefile.in
   '' else null;

Worked for me:

vin@catacendre nixpkgs % file result/bin/grep
result/bin/grep: Mach-O 64-bit executable x86_64
vin@catacendre nixpkgs % realpath ./result
/nix/store/vpcabk9qbxssqj3qcy9c3nag465mi3zw-gnugrep-3.11