Skip to content

copy-tarballs: drop perl bindings#361700

Merged
mweinelt merged 1 commit intoNixOS:masterfrom
Mic92:copy-tarballs
Oct 2, 2025
Merged

copy-tarballs: drop perl bindings#361700
mweinelt merged 1 commit intoNixOS:masterfrom
Mic92:copy-tarballs

Conversation

@Mic92
Copy link
Member

@Mic92 Mic92 commented Dec 4, 2024

This hopefully makes it easier to re-write this script in a language that people understand. Because it's shelling out, it's likely slower but hopefully still fast enough for our purposes.

Things done

I only tested local dry run without any credentials...

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@Mic92
Copy link
Member Author

Mic92 commented Dec 4, 2024

@vcunat can you give me some code how to test it?

@Mic92 Mic92 mentioned this pull request Dec 4, 2024
13 tasks
@Mic92
Copy link
Member Author

Mic92 commented Dec 4, 2024

Ok. Looks like I also should replace hashFile

@github-actions github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Dec 4, 2024
@Mic92 Mic92 force-pushed the copy-tarballs branch 3 times, most recently from 9a65523 to 32beb24 Compare December 4, 2024 09:24
@vcunat
Copy link
Member

vcunat commented Dec 4, 2024

Nit: is it worth using --argstr when you can directly anti-quote the values into the nix expression? (at least I do assume that it's at least as easy in Perl as in Bash)

@Mic92 Mic92 force-pushed the copy-tarballs branch 3 times, most recently from f9fcb5e to 89093ba Compare December 4, 2024 10:08
@Mic92
Copy link
Member Author

Mic92 commented Dec 4, 2024

Nit: is it worth using --argstr when you can directly anti-quote the values into the nix expression? (at least I do assume that it's at least as easy in Perl as in Bash)

It's less mental overhead for me to have to think about potential escaping issues, so I find it easier to read this way.

@mweinelt
Copy link
Member

mweinelt commented Jun 7, 2025

Tested and runs into the following error on 4792576

error:
       … while calling the 'map' builtin
         at /home/tarball-mirror/nixpkgs/maintainers/scripts/find-tarballs.nix:26:17:
           25|
           26|   uniqueFiles = map (x: x.file) (genericClosure {
             |                 ^
           27|     startSet = map (file: {
       … while calling the 'genericClosure' builtin
         at /home/tarball-mirror/nixpkgs/maintainers/scripts/find-tarballs.nix:26:34:
           25|
           26|   uniqueFiles = map (x: x.file) (genericClosure {
             |                                  ^
           27|     startSet = map (file: {
       (stack trace truncated; use '--show-trace' to show the full, detailed trace)
       error: attribute 'outputHashAlgo' missing
       at /home/tarball-mirror/nixpkgs/maintainers/scripts/find-tarballs.nix:38:12:
           37|     isPatch = (drv ? postFetch && drv.postFetch != "");
           38|     type = drv.outputHashAlgo;
             |            ^
           39|     name = drv.name;
./maintainers/scripts/copy-tarballs.pl: evaluation failed

@mweinelt
Copy link
Member

Bisected and fixed in #414869.

Now I applied this patch for nixos/infra to pluto.nixos.org and mirror-tarballs.service works again.

@mweinelt mweinelt moved this to In Progress in Infra operational issues Jun 11, 2025
mweinelt added a commit to NixOS/infra that referenced this pull request Jun 12, 2025
With NixOS 25.05 there is no nix_2_18 anymore and now we're on an newer
version and the perl bindings changed.
github-merge-queue bot pushed a commit to NixOS/infra that referenced this pull request Jun 12, 2025
tarball-mirror: apply github.com/NixOS/nixpkgs/pull/361700
@mweinelt mweinelt marked this pull request as ready for review June 12, 2025 16:21
@wegank wegank added the 12.approvals: 1 This PR was reviewed and approved by one person. label Jun 13, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 3, 2025
@mweinelt
Copy link
Member

Can you rebase?

This hopefully makes it easier to re-write this script in a language
that people understand. Because it's shelling out, it's likely slower
but hopefully still fast enough for our purposes.
@Mic92
Copy link
Member Author

Mic92 commented Sep 30, 2025

@mweinelt Done

@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 30, 2025
@mweinelt mweinelt added this pull request to the merge queue Oct 2, 2025
Merged via the queue into NixOS:master with commit 007e768 Oct 2, 2025
29 of 31 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Infra operational issues Oct 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

Development

Successfully merging this pull request may close these issues.

5 participants