The nar.sh checks fail for some Linux machines

[yannham]

Platform

• Linux:
• macOS
• WSL

Additional information

When building nix as a dependency of another project through the github:nixos/nix flake on a Linux runner, latest Nix (I've tried several commits, last was 322d2c7) fails to build (more specifically, the tests fail). The host is a Zen 4 EPYC with x86_64 Linux (NixOS 24.05). Filesystem is ZFS.

On another local Linux machine (Linux 6.10.11-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.10.11-1 (2024-09-22) x86_64 GNU/Linux), the project builds correctly.

The failing test is nar.sh, more specifically this part:

https://github.com/NixOS/nix/blob/master/tests/functional/nars.sh#L91-L100

It seems that Darwin and Linux have a different expected behaviour here, but for some reason the Linux runner specifically ends up having Darwin's behaviour.

Output

Output

nix>     +(nars.sh:94) [[ Linux = Darwin ]]
nix>     +(nars.sh:97) nix-store --restore /build/nix-test/nars/out
nix>     error: path '/build/nix-test/nars/out/â' already exists
nix>     ++(nars.sh:97) onError
nix>     ++(/build/source/tests/functional/common/functions.sh:237) set +x
nix>     nars.sh: test failed at:
nix>       main in nars.sh:97

Full output available here: https://github.com/tweag/nickel/actions/runs/11016505458/job/30592178626?pr=2037#step:5:3894

Priorities

Add 👍 to issues you find important.

[yannham]

After some quick investigation, it seems that utf8only is set to true in our setup and normalization to formD. Looking at the NAR test, it seems that it expects Linux to never perform unicode normalization of file names, but this assumption appears to be broken in that precise setup (Linux+ZFS with utf8only=true+normalize != none).

[yannham]

This test has been added 3 weeks ago by @edolstra in 7a765a6
I can propose some solution to special-case ZFS + normalize, but it's going to be a good paragraph of bash at least, and I'm not sure why nix is testing that at all, so waiting for more input first.

[puckipedia]

oh, someone actually uses zfs' normalize in the wild, impressive

i'd probably change the test to check that either it created both store paths (probably with ls rather than test -e, because iirc -e will also normalise?), or that it failed with an EEXIST

[yannham]

Thanks @puckipedia. It makes sense that we just test the two existing possibilities but without assuming anything based on the current system, as long as one we get one of the two accepted outcomes. I have a patch in my fork, currently testing it on the runner. I'll open a PR once that works.

[YorikSar]

@puckipedia normalization=formD actually came from https://openzfs.github.io/openzfs-docs/Getting%20Started/NixOS/Root%20on%20ZFS.html#system-installation where it's recommended. Do you think it's a mistake? What are the downsides of using it?