Not compatible with urllib3 2.0

[dgilman]

No description provided.

[Mr0grog]

Ah, yeah, at the very least, this fix for gzip encoding problems in Wayback will need to be removed or done in a different way for v2:

wayback/wayback/_client.py

Lines 260 to 289 in 6d360e9

	#####################################################################
	# HACK: handle malformed Content-Encoding headers from Wayback.
	# When you send `Accept-Encoding: gzip` on a request for a memento, Wayback
	# will faithfully gzip the response body. However, if the original response
	# from the web server that was snapshotted was gzipped, Wayback screws up the
	# `Content-Encoding` header on the memento response, leading any HTTP client to
	# *not* decompress the gzipped body. Wayback folks have no clear timeline for
	# a fix, hence the workaround here. More info in this issue:
	# https://github.com/edgi-govdata-archiving/web-monitoring-processing/issues/309
	#
	# This subclass of urllib3's response class identifies the malformed headers
	# and repairs them before instantiating the actual response object, so when it
	# reads the body, it knows to decode it correctly.
	#
	# See what we're overriding from urllib3:
	# https://github.com/urllib3/urllib3/blob/a6ec68a5c5c5743c59fe5c62c635c929586c429b/src/urllib3/response.py#L499-L526
	class WaybackResponse(HTTPConnectionPool.ResponseCls):
	@classmethod
	def from_httplib(cls, httplib_response, **response_kwargs):
	headers = httplib_response.msg
	pairs = headers.items()
	if ('content-encoding', '') in pairs and ('Content-Encoding', 'gzip') in pairs:
	del headers['content-encoding']
	headers['Content-Encoding'] = 'gzip'
	return super().from_httplib(httplib_response, **response_kwargs)
	HTTPConnectionPool.ResponseCls = WaybackResponse
	# END HACK
	#####################################################################

I had thought we set constraints for urllib3 v1.x, but it looks like we didn’t, which is causing your test failures on #117. Thanks for calling this out.

I think the first thing that needs doing here is constraining the dependency to urllib3 >=1.20 <2 as a quick fix, and then we need to fix the above issue in order to be compatible with v2 (or switch to httpx, see #58).

[Mr0grog]

Released v0.4.2 with correct dependency constraints.