Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changing checksums #17

Open
ajs124 opened this issue Feb 18, 2021 · 2 comments
Open

changing checksums #17

ajs124 opened this issue Feb 18, 2021 · 2 comments

Comments

@ajs124
Copy link
Contributor

ajs124 commented Feb 18, 2021

When re-running composer2nix, I observed some downloads of the same commits not resulting in the same checksums, e.g.

           src = composerEnv.buildZipPackage {
             name = "webmozart-assert-ab2cb0b3b559010b75981b1bdce728da3ee90ad6";
             src = fetchurl {
-              url = https://api.github.com/repos/webmozart/assert/zipball/ab2cb0b3b559010b75981b1bdce728da3ee90ad6;
-              sha256 = "16dp6yybqq2ayw67fq00mdyq0wblmsqi8qnbr8mszs9h4mj0ggpx";
+              url = "https://api.github.com/repos/webmozart/assert/zipball/ab2cb0b3b559010b75981b1bdce728da3ee90ad6";
+              sha256 = "0mzkqxzng8kkzwyn06dq3qh3r0kx8kki7y95rbac1s4ays184yz7";
             };
           };

or

           src = composerEnv.buildZipPackage {
             name = "webmozart-assert-ab2cb0b3b559010b75981b1bdce728da3ee90ad6";
             src = fetchurl {
-              url = https://api.github.com/repos/webmozart/assert/zipball/ab2cb0b3b559010b75981b1bdce728da3ee90ad6;
-              sha256 = "16dp6yybqq2ayw67fq00mdyq0wblmsqi8qnbr8mszs9h4mj0ggpx";
+              url = "https://api.github.com/repos/webmozart/assert/zipball/ab2cb0b3b559010b75981b1bdce728da3ee90ad6";
+              sha256 = "0mzkqxzng8kkzwyn06dq3qh3r0kx8kki7y95rbac1s4ays184yz7";
             };
           };

(the url only changes because it's quoted now)

I'm not sure what the best fix here is, but maybe using fetchzip instead of fetchurl would help.

@svanderburg
Copy link
Owner

That's a good one. fetchzip is not a solution that we can generally apply, because it will only work when downloading from GitHub.

Probably a good structural solution is to change fetchgit in Nixpkgs to cope with the side effect that causes the hash to change, most likely a timestamp.

@lheckemann
Copy link

fetchgit isn't even being used here, and fetchgit should be free of this. The problem is that the _zip_s generated by GitHub aren't deterministic, even though their content is (and the zips generated by github are what's referenced by composer.lock, directly by URL). This is why fetchzip exists and why it might be an appropriate solution (folding the fetchurl and the buildZipPackage into a single deterministic derivation).

The issue I encountered when trying to implement this is that the URLs don't end with .zip, so stdenv tries to unpack it as a tarball. I've opened NixOS/nixpkgs#118662 to remedy this, but would be glad if anyone has a prettier approach.

jtojnar added a commit to fossar/composition-c4 that referenced this issue May 31, 2021
Had I remembered this, I probably would not spend my time writing c4. Well, 🤷‍♀

At least lockfile-based approach is more suitable for my use case.
Also, unlike the two other tools, we do not suffer from GitHub zipballs being non-deterministic:

svanderburg/composer2nix#17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants