Skip errors

[AriFordsham]

Some NARINFO downloads fail for me. Sometimes mirror-nix can't parse them for some reason, sometimes my content filter doesn't like the look of them.

When a Narinfo fails, mirror-nix stops. I think it should continue downloading the rest.

Is this project maintained? has it been superseded by something else?

[AleXoundOS]

Thank you for the report. The project is not actively maintained (even the readme is not actual). Though, since nobody else could succeed that far, at least I can answer some questions. So, it's really worked for me for nixpkgs commit in example directory (though with a little patch) resulting in ~400 GiB of compressed nix cache.

sometimes my content filter doesn't like the look of them

What "content filter" are you talking about?

I didn't experience broken NARINFOs. But in this case you can at least dump their urls into a file (--dump-narinfo-urls), then try to filter valid/invalid (using curl/wget plus some scripting maybe), then write some script that retrieves NAR urls from NARINFOs and downloads NARs. Or patch the code.
Failure to parse NARINFO is abnormal. Maybe NARINFO format has changed. Dunno.

[AriFordsham]

Thank you.

I actually am having more success right now with the simple scripts from https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369.

They seem at least as fast. I'm not sure if the memory leak bugs (NixOS/nix#1681, NixOS/nix#2774, NixOS/nix#3386) that apparently provoked this project are stil an issue, the scripts do seem to get stuck sometimes. I've crudely hacked around this by disabling the swap file, it seems better but not sorted. Next I'll try reducing the physical memory (I'm running in a VM) to get misbehaving paths to give up sooner.

I'm using quite a strict content filter (https://netfree.link) that apparently objects to certain character sequences in certain files. That's fine - I'll have to do without those packages. With @ejpcmac 's script from the Discourse, those paths simply error out and continue. mirror-nix Just throws it's hands up and stops.

I understand this is an unmaintained project that I'll have to hack around to get to work, and I might have to do without some packages. If I could change one thing, though, it would be to keep going in the face of unsuccessful downloads, parses etc. rather than failing.

[AleXoundOS]

I actually am having more success right now with the simple scripts from https://discourse.nixos.org/t/using-nixos-in-an-isolated-environment/3369.

It's difficult to judge about success until you "battle"-test the downloaded cache in isolated environment trying to build/nixos-rebuild a bunch of stuff, including overrides which require rebuild from sources, etc. I think, indirectly, you can be more or less confident if you have ~400 GiB of cache for a single nixpkgs commit (I think it has gotten only bigger over the past year). The algorithm mirror-nix implements is something like a "spider" which traces all relationships between derivations in both directions (not only a dependency tree, but referrers too, producing a much bigger network).
One package that is definitely missing using mirror-nix is nvidia xorg driver. You will have to download it manually.

If I could change one thing, though, it would be to keep going in the face of unsuccessful downloads, parses etc. rather than failing.

Sounds reasonable.