Checksum failing for faraday_middleware 0.11.0 and bundler 1.14.0.pre.x

[chrismo]

I've tried this with 1.13.7, 1.14.0.pre.1 and 1.14.0.pre.2. Checksum is fine in 1.13, but shows a mismatch with 1.14

Bundler cannot continue installing faraday_middleware (0.11.0).
The checksum for the downloaded `faraday_middleware-0.11.0.gem` does not match the checksum given by the server. This means the contents of the downloaded gem is different from what
was uploaded to the server, and could be a potential security issue.

To resolve this issue:
1. delete the downloaded gem located at: `/Users/chrismo/.bundle/ruby/2.3.0/gems/faraday_middleware-0.11.0/faraday_middleware-0.11.0.gem`
2. run `bundle install`

If you wish to continue installing the downloaded gem, and are certain it does not pose a security issue despite the mismatching checksum, do the following:
1. run `bundle config disable.checksum_validation true` to turn off checksum verification
2. run `bundle install`

(More info: The expected SHA256 checksum was "c56bb3661af1d5057e578d72d2f9e93ab52630b74ebe37ce1118c93a6ddff56b", but the checksum for the downloaded gem was
"ff2bdbb6bdd5b2293f8ce7cb93572318dd068506e12f987f04a65209a2bc69cb".)

I've searched around for other checksum problems in bundler issues, but didn't hit anything that seemed a dupe. I need to get to a stopping point later today and I can continue digging into this.

[segiddins]

I don't think the is an issue with bundler -- it's just reporting that the servers checksum for the gem doesn't match the checksum of the downloaded gem file.

[indirect]

Looks like the sum in the database is wrong?

$ gem fetch faraday_middleware -v 0.11.0
$ shasum -a256 faraday_middleware-0.11.0.gem
ff2bdbb6bdd5b2293f8ce7cb93572318dd068506e12f987f04a65209a2bc69cb  faraday_middleware-0.11.0.gem

[chrismo]

Just an intermittent coincidence then? ... Hmmm, I just ran it about 7 times with 1.13.7 and 7 times with 1.14.0.pre.2 ... all 7 successful with 1.13 and all 7 failed with 1.14. I didn't know about any fetching differences in 1.14?

[segiddins]

In the CHANGELOG for 1.14.0.pre.1

gem checksums returned by rubygems.org are validated when installing gems (#4464, @segiddins)

[chrismo]

Ah, turns out I was unaware of #4851 ... so ... this is a non-issue.

It does make me wonder how many other gems might be out there that have this problem. I may take a swipe at installing a whole bunch of popular ones just in case.

[segiddins]

since that version was just pushed in the past few days, my guess might be a recent bug in rubygems.org

[chrismo]

I decided to write-up a script that would work from the rubygems data dump and use bundler/inline to install gems individually from the most downloaded onward. It's slow, but can handle a decent sample size to see if there are any other occurrences of this or not. Obvs this isn't widespread, enough downloads of the pre-releases of 1.14 have happened, but ... not difficult to test many other gems just in case.

[indirect]

We can also run this in a rubygems console, fetching gems from S3 and comparing their checksums to what we have in the database. It worries me that there are any bad checksums, since we should have taken a sum and saved it when the gem was first created.

[chrismo]

Ah yes, server-side, just re-calc the checksum and compare to db value would run much faster.

[chrismo]

My script made it through about 700 of the most popular gems, then I re-ran for most recent ones, and it made it through 1,000 of those - only faraday_middleware being a problem. Fairly small sample sizes as percentages go, but it's something. Guess they should try re-releasing faraday_middleware or someone in RubyGems can try to shore up the checksum server-side?

[indirect]

@chrismo I'd be more comfortable with them re-releasing, since that guarantees we aren't accidentally verifying the checksum of a gem that was somehow maliciously replaced! But if that's not an option for some reason let me know and we can try the prod console option.

[iMacTia]

Hello guys,

I released faraday_middleware 0.11.0 and didn't notice this issue immediately.
Assuming this was a once-in-a-time issue, I can try re-releasing the gem and see if that fix the issue.
Will let you know once it's done so you can double-check

[iMacTia]

I've just released v0.11.0.1
This time, I've also tried to install it locally through a bundle update and it worked fine.
Can you please double-check if everything is good on your side as well?

Thanks

[tjgrathwell]

I guess this information may belong on some sort of RubyGems bug, but we're seeing a similar checksum mismatch on the gem azure-core (I'm not the author, just a user):

$ gem fetch azure-core -v 0.1.6
$ shasum -a256 azure-core-0.1.6.gem
68ebac6994412a775584478961747a215d0844737a8b9a91405a0ac77fe6c514  azure-core-0.1.6.gem

require 'net/http'
require 'yaml'
response = Net::HTTP.get(URI('https://rubygems.org/api/v1/versions/azure-core.yaml'))
YAML.load(response).find { |version| version['number'] == '0.1.6' }['sha']
# => "b04f7f994e3abc31003139a7ff8a682980c1d064a906a3290d5c369685bc478b"

The SHAs for the previous version, 0.1.5, do match.

0.1.5 was pushed on Sep 22, 2016
0.1.6 was pushed on Nov 26, 2016

[segiddins]

I think we might want to bring this over to https://github.com/rubygems/rubygems.org

[tjgrathwell]

okay I made a thing ^^

[colby-swandale]

is this still a problem?

[ryanwjackson]

I'm running into it.

[segiddins]

Closing as a rubygems.org issue, since there's nothing for us to do about this in bundler itself.