Fix Savannah snapshot downloads by switching from cgit to gitweb#523
Conversation
The cgit snapshot service on Savannah GNU.org has been disabled/broken, returning HTTP 400 errors for all snapshot requests. This affects the emacs update CI job which downloads source tarballs for building. The error manifested as: error: unable to download 'https://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-<sha>.tar.gz': HTTP error 400 This is a known infrastructure issue affecting multiple projects: - https://lists.gnu.org/r/guix-devel/2025-01/msg00170.html - NixOS/nixpkgs#346264 The recommended workaround from the Guix community discussion is to use gitweb URLs instead of cgit URLs for downloading snapshots: Old: https://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-<sha>.tar.gz New: https://git.savannah.gnu.org/gitweb/?p=emacs.git;a=snapshot;h=<sha>;sf=tgz The --name flag is required because gitweb URLs contain semicolons which nix-prefetch-url incorrectly interprets as part of the store path name.
|
I kept trying to fix my nix code to install emacs with some customization, but finally gave up. I think the fetcher in https://github.com/nix-community/emacs-overlay/blob/master/overlays/emacs.nix probably needs some adjustments as well. The above is only part of the fix, I guess. |
|
|
||
| output_branch=$(echo $branch | sed s/"\/"/"_"/) | ||
| digest=$(nix-prefetch-url --unpack "https://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-${commit_sha}.tar.gz") | ||
| digest=$(nix-prefetch-url --unpack --name "emacs-${commit_sha}.tar.gz" "https://git.savannah.gnu.org/gitweb/?p=emacs.git;a=snapshot;h=${commit_sha};sf=tgz") |
There was a problem hiding this comment.
Essentially, the gitweb snapshot service is no more reliable than the cgit snapshot service. They may decide to also disable gitweb snapshot in the future.
What is more reliable here is the git protocol. We should use that instead. Our use cases should be covered well by fetchgit.
I think the fetcher in https://github.com/nix-community/emacs-overlay/blob/master/overlays/emacs.nix probably needs some adjustments as well. The above is only part of the fix, I guess.
Yes. Here we update info about source and we use this source info in overlays/emacs.nix.
NOTE: There's a GitHub mirror maintained by tarsius. It doesn't currently include the feature/igc branch, but it will once the GitHub actions next run.
I am not a fan of adding more indirection.
I checked a few repos and their cgit snapshot services are all disabled. CC @normalcea (author of NixOS/nixpkgs#435964 and NixOS/nixpkgs#427024)
|
@jian-lin Sorry, I don't have plans to keep working on this. |
ghost
closed this
|
No worries. I’ll fix this, probably this weekend. |
See previous discussion: nix-community#523 This is fairly slow because the Savannah git repository doesn't seem to support blob filtration. Given that this is normally run via a CI job that seems minor.
Upstream Emacs disabled their cgit snapshot services again. It is more future-proofing to fetch Emacs source via git. More context: nix-community#523 (comment) Note that this patch does not affect Emacs forks whose source code is hosted on and fetched via GitHub. Fixes: nix-community#519 Ref: nix-community#523 Co-authored-by: Lin Jian <me@linj.tech>
Upstream Emacs disabled their cgit snapshot services again. It is more future-proofing to fetch Emacs source via git. More context: nix-community#523 (comment) Note that this patch does not affect Emacs forks whose source code is hosted on and fetched via GitHub. Ref: nix-community#523 Co-authored-by: Lin Jian <me@linj.tech>
Upstream Emacs disabled their cgit snapshot services again. It is more future-proofing to fetch Emacs source via git. More context: #523 (comment) Note that this patch does not affect Emacs forks whose source code is hosted on and fetched via GitHub. Ref: #523 Co-authored-by: Lin Jian <me@linj.tech>
Upstream Emacs disabled their cgit snapshot services again. It is more future-proofing to fetch Emacs source via git. More context: nix-community#523 (comment) Note that this patch does not affect Emacs forks whose source code is hosted on and fetched via GitHub. Ref: nix-community#523 Co-authored-by: Lin Jian <me@linj.tech>
3 participants
The cgit snapshot service on Savannah GNU.org has been disabled/broken, returning HTTP 400 errors for all snapshot requests. This affects the emacs update CI job which downloads source tarballs for building.
The error manifested as:
error: unable to download 'https://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-.tar.gz': HTTP error 400
This is a known infrastructure issue affecting multiple projects:
The recommended workaround from the Guix community discussion is to use gitweb URLs instead of cgit URLs for downloading snapshots:
Old: https://git.savannah.gnu.org/cgit/emacs.git/snapshot/emacs-.tar.gz
New: https://git.savannah.gnu.org/gitweb/?p=emacs.git;a=snapshot;h=;sf=tgz