Checksum of /versions mismatch when server returns weak etags

[derektamsen]

With bundler 1.12.5 running bundle update --verbose I am seeing the following error message:

bundle update --verbose
HTTP GET https://internal.gemstash.server/versions
HTTP 302 Found
HTTP GET https://rubygems.org/versions
HTTP 200 OK
HTTP GET https://internal.gemstash.server/versions
HTTP 302 Found
HTTP GET https://rubygems.org/versions
HTTP 200 OK
The checksum of /versions does not match the checksum provided by the server! Something is wrong (local checksum is "\"f3e85d51952e0136d9a61006a82ab686\"", was expecting "W/\"f3e85d51952e0136d9a61006a82ab686\"").
HTTP GET https://internal.gemstash.server/api/v1/dependencies
HTTP 200 OK

This is against an internal gemstash server which is returning a 302 to https://rubygems.org. I believe #4472 may have introduced the issue but only when the web server returns a weak ETag.

The /versions file on rubygems.org returns a weak ETag ETag: W/"26dd9dcb85113838e25ce1952ad06fa5" which is denoted by the W/. However, /versions served from https://index.rubygems.org serves a strong ETag Etag: "0e6cafa9793dec153155a64e9c2c5a9d"

This may have been introduced due to rails switching the default ETag type from strong to weak in the recent 5.0.0 release. rails/rails#17573

$ ruby -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin15]
$ gem -v
2.5.1

[indirect]

@derektamsen we probably need to special-case gemstash to request index.rubygems.org for /versions and /info/*. The canonical source for the index files is index.rubygems.org, not rubygems.org.

/cc @smellsblue

[derektamsen]

@indirect Thanks for the quick response. I was going to open a PR to gsub out the 'W/'. Do you think it's better to fix gemstash instead of the gsub in bundler?

Was going to change response_etag = response["ETag"] to response_etag = response["ETag"].sub(/^W\//, '') at https://github.com/bundler/bundler/blob/master/lib/bundler/vendor/compact_index_client/lib/compact_index_client/updater.rb#L53

[indirect]

Getting rid of the W/ won't help at all. The URL rubygems.org/versions is the wrong file. You need index.rubygems.org/versions instead.

[derektamsen]

I opened an issue against the gemstash project to track the issue.

[ToadJamb]

I just thought I would add that the fix for me was to change source 'https://www.rubygems.org' to source 'https://rubygems.org'.