Conversation
| let client_builder = if self.automatic_decompression { | ||
| client_builder | ||
| } else { | ||
| client_builder.no_gzip().no_brotli().no_deflate().no_zstd() | ||
| }; |
There was a problem hiding this comment.
NB: I added this to get reqwest to stop proactively decompressing streams when it's not what we expect (specifically, when we intentionally want to fetch an archive without transparently decompressing it, even if the origin hints that a client could).
I think this is actually a better stub-in than what we have elsewhere, e.g. in RegistryClient where we use just Accept-Encoding: identity. The latter is purely a hint that (IIUC) the origin could still ignore with its response headers, whereas this forcefully prevents reqwest from attempting decompression.
(We don't want to use this everywhere obviously, e.g. we still want origins to serve us compressed JSON responses. But it seems like a more ideal solution during range requests.)
|
Noting: there's a decent chance we don't want to merge this as-is, since dropping the current EOL PyPy versions is somewhat aggressive. PyPy's maintainer notes in pypy/pypy#5353 (comment) that they plan to backfill more targz archives for older (EOL) versions, in which case we could defer the drop and keep this purely as a bz2-to-gzip transition. |
uv test inventory changesThis PR no longer changes the tests when compared with the latest |
Signed-off-by: William Woodruff <william@yossarian.net>
Signed-off-by: William Woodruff <william@yossarian.net>
Signed-off-by: William Woodruff <william@yossarian.net>
Signed-off-by: William Woodruff <william@yossarian.net>
|
Let's merge #20265 first, and then I'll deconflict here. (That has my decompression bugfix cherry-picked in.) |
|
Closing because the main relevant part of this was merged, and the decompression disablement tweak is no longer needed. |
Summary
This accomplishes two things:
.tar.bzto.tar.gzarchives for PyPy distributions, per Provide zstd downloads pypy/pypy#5353Changing from bz2 to gzip archives on
python.orgalso revealed a bug: our default HTTP client stack performs transparent decompression andpython.orgserves archives withContent-Type: application/x-tar, Content-Encoding: gzip, so reqwest would prematurely decompress the request (and thus break our expectation of a well-formed gzip for extraction). This didn't happen with bzip2 because reqwest doesn't know how to decompress bzip2 streams by default.Test Plan
Bump the PyPy snapshots where appropriate.
NFC, maybe. We'll see what the existing tests do.