Was 1.6.0 retagged?

[rwhogg]

On May 17, the SHA-256 sum of https://github.com/kubernetes/kops/archive/1.6.0.tar.gz was 39ef8382d7557c4eacf9678feae42f473f2a9c436f4a518dfcf6a630eea6c2ce . As of today, it seems to be 483da291fc5a7a72c151e15ab586e6106f807564894669070705d2e1762e5595 .

Was 1.6.0 retagged? I'm not sure why the checksums would have changed.

Homebrew/Linuxbrew users currently cannot install kops from source as a result (see Homebrew/homebrew-core#13900).

Thanks!

[ilovezfs]

@justinsb gentle ping on this.

[justinsb]

Sorry for delay.

It was not deliberately retagged. Checking my logs etc.

[ilovezfs]

The difference between the tarballs appears to be

Josephs-MacBook-Pro:tmp joe$ diff AAA/kops-1.6.0/vendor/k8s.io/kubernetes/pkg/version/base.go BBB/kops-1.6.0/vendor/k8s.io/kubernetes/pkg/version/base.go
54c54
< 	gitVersion   string = "v0.0.0-master+f69a313"
---
> 	gitVersion   string = "v0.0.0-master+f69a313ce"
Josephs-MacBook-Pro:tmp joe$ diff AAA/kops-1.6.0/vendor/k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/version/base.go BBB/kops-1.6.0/vendor/k8s.io/kubernetes/staging/src/k8s.io/client-go/pkg/version/base.go
54c54
< 	gitVersion   string = "v0.0.0-master+f69a313"
---
> 	gitVersion   string = "v0.0.0-master+f69a313ce"

[ilovezfs]

I'm wondering if this could somehow be related to Homebrew/homebrew-core#13792 (comment).

Though I'm not sure how that could be unless there was a retagging.

[justinsb]

So I have the logs from when I published:

> make release-github
shipbot -tag 1.6.0 -config .shipbot.yaml
I0517 00:07:06.607205   24170 main.go:121] listing github releases for kubernetes/kops
I0517 00:07:06.890650   24170 main.go:141] SHA is "f69a313ce72de29339b0a60c1e30aef35e12ad4e"
I0517 00:07:06.890716   24170 main.go:150] creating github release for kubernetes/kops/1.6.0
I0517 00:07:07.042173   24170 main.go:157] listing github release assets for kubernetes/kops/1.6.0
I0517 00:07:07.089122   24170 main.go:225] creating github release assets for kubernetes/kops/1.6.0 "kops-darwin-amd64"
I0517 00:07:07.089163   24170 main.go:231] Uploading "/release/src/k8s.io/kops/.build/dist/darwin/amd64/kops"
I0517 00:07:11.702167   24170 main.go:237] uploaded asset: github.ReleaseAsset{ID:3891712, URL:"https://api.github.com/repos/kubernetes/kops/releases/assets/3891712", Name:"kops-darwin-amd64", Label:"", State:"uploaded", ContentType:"application/octet-stream", Size:120450944, DownloadCount:0, CreatedAt:github.Timestamp{2017-05-17 04:07:07 +0000 UTC}, UpdatedAt:github.Timestamp{2017-05-17 04:07:11 +0000 UTC}, BrowserDownloadURL:"https://github.com/kubernetes/kops/releases/download/untagged-a1d8fced88eca6fe02b1/kops-darwin-amd64", Uploader:github.User{Login:"justinsb", ID:100893, AvatarURL:"https://avatars1.githubusercontent.com/u/100893?v=3", HTMLURL:"https://github.com/justinsb", GravatarID:"", Type:"User", SiteAdmin:false, URL:"https://api.github.com/users/justinsb", EventsURL:"https://api.github.com/users/justinsb/events{/privacy}", FollowingURL:"https://api.github.com/users/justinsb/following{/other_user}", FollowersURL:"https://api.github.com/users/justinsb/followers", GistsURL:"https://api.github.com/users/justinsb/gists{/gist_id}", OrganizationsURL:"https://api.github.com/users/justinsb/orgs", ReceivedEventsURL:"https://api.github.com/users/justinsb/received_events", ReposURL:"https://api.github.com/users/justinsb/repos", StarredURL:"https://api.github.com/users/justinsb/starred{/owner}{/repo}", SubscriptionsURL:"https://api.github.com/users/justinsb/subscriptions"}}
I0517 00:07:11.702460   24170 main.go:225] creating github release assets for kubernetes/kops/1.6.0 "kops-linux-amd64"
I0517 00:07:11.702512   24170 main.go:231] Uploading "/release/src/k8s.io/kops/.build/dist/linux/amd64/kops"
I0517 00:07:15.058386   24170 main.go:237] uploaded asset: github.ReleaseAsset{ID:3891713, URL:"https://api.github.com/repos/kubernetes/kops/releases/assets/3891713", Name:"kops-linux-amd64", Label:"", State:"uploaded", ContentType:"application/octet-stream", Size:72709344, DownloadCount:0, CreatedAt:github.Timestamp{2017-05-17 04:07:11 +0000 UTC}, UpdatedAt:github.Timestamp{2017-05-17 04:07:15 +0000 UTC}, BrowserDownloadURL:"https://github.com/kubernetes/kops/releases/download/untagged-a1d8fced88eca6fe02b1/kops-linux-amd64", Uploader:github.User{Login:"justinsb", ID:100893, AvatarURL:"https://avatars1.githubusercontent.com/u/100893?v=3", HTMLURL:"https://github.com/justinsb", GravatarID:"", Type:"User", SiteAdmin:false, URL:"https://api.github.com/users/justinsb", EventsURL:"https://api.github.com/users/justinsb/events{/privacy}", FollowingURL:"https://api.github.com/users/justinsb/following{/other_user}", FollowersURL:"https://api.github.com/users/justinsb/followers", GistsURL:"https://api.github.com/users/justinsb/gists{/gist_id}", OrganizationsURL:"https://api.github.com/users/justinsb/orgs", ReceivedEventsURL:"https://api.github.com/users/justinsb/received_events", ReposURL:"https://api.github.com/users/justinsb/repos", StarredURL:"https://api.github.com/users/justinsb/starred{/owner}{/repo}", SubscriptionsURL:"https://api.github.com/users/justinsb/subscriptions"}}

So the git tag for 1.6.0 was f69a313ce72de29339b0a60c1e30aef35e12ad4e , which is still the git tag, so we haven't moved the git tag: https://github.com/kubernetes/kops/tree/1.6.0

Sadly shipbot doesn't print the SHAs of the file it uploads (it should, and I'll fix that separately). But I did double-check the SHAs of the two kops executables and they match.

[justinsb]

@ilovezfs I'm a little confused by the linked issue, but here's more head-scratching material:

https://github.com/kubernetes/kops/blob/1.6.0/vendor/k8s.io/kubernetes/pkg/version/base.go#L54

gitVersion string = "v0.0.0-master+$Format:%h$"

Here's the problem .. the github release tarball has expanded $Format (the zipfile has the same thing):

gitVersion string = "v0.0.0-master+f69a313ce"

I have no idea if that is correct/expected. It is certainly surprising to me. But this would explain how we might not have retagged, but github expansion would change the SHA of the expanded source tarball.

BTW Thank you for finding & looking at this!

[justinsb]

(That thanks goes to @ilovezfs and @rwhogg of course, along with anyone else looking at this!)

[ilovezfs]

@justinsb This seems to be Git magic from the way git archive and keyword expansion work:

http://git-memo.readthedocs.io/en/latest/git_archive.html#git-keyword-expansion
https://git-scm.com/book/en/v2/Customizing-Git-Git-Attributes

[justinsb]

@ilovezfs yes, you're right. I think this means we should probably use %H - i.e. the full commit hash - or else the SHA of the exported version might change. It is surprising to me that github would regenerate the tar/zip files - they are created automatically.

(If anyone knows if we can / should do something to "lock them" then let me know!)

As this is actually in kubernetes, I'm going to open an issue suggesting this. The project in question should certainly be made aware, and it does help that it is kubernetes/kubernetes :-)

[ilovezfs]

Since it appears to be the git archive command on the backend, it's subject to change at the whim of upstream Git whenever GitHub changes Git versions. So I think the only one way to 100% guarantee the tarball checksum won't morph at some point would be to upload your own release tarball in addition to the auto-generated one.

[ilovezfs]

I've opened Homebrew/homebrew-core#13935 to update the checksum.

[chrislovecnm]

What is the resolution for this?

[chrislovecnm]

Can we close?

[ilovezfs]

I don't think the underlying issue has been resolved, so I don't see why it would be closed.

[ilovezfs]

@justinsb FYI the checksum for 1.7.0 has now changed as well, and I have yet to be able to reconstruct the original, even using the method described here: Homebrew/homebrew-core#18044 (comment). I was able to reconstruct all of the other checksums that had changed as described in that issue, so something is going on here that's different from the others.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close