Why zlib encoder encodes into different string from python's zlib

[midstreeeam]

Hey guys I just find a problem working with the lib.

I found for a same string, set with same compression level, the encode result from encoder.finish() is different from python's zlib.compress()

Although all the two string works, and the decoded string is same, I still want to know what is the defualtly different from flate2's encoder and python's zlib encoder.

[Byron]

GPT says:

Even though both flate2 and Python's zlib.compress() implement DEFLATE and produce valid compressed data, the differences in header information, compression strategy, and internal optimizations lead to different outputs. However, both are compatible with DEFLATE's decompression, ensuring the decoded string remains the same. If you're looking for identical compressed outputs, you'd need to ensure that both libraries are configured in exactly the same way, including header generation and compression settings, which can sometimes be difficult due to differences in their API design.

Here is a link to all the details.

[oyvindln]

The default rust backend is originally a port of miniz which uses some slightly different parameters and algorithms for match searching and block splitting compared to the standard zlib library which results in slightly different results. (Why it was done that way in the original miniz librar I don't know, maybe for ease of implementation or something.) I believe the python function uses the standard c zlib library under the hood.

Worth noting that zlib-ng also produces slightly different results compared to normal zlib due to some speed tradeoffs (I believe they use a minimum match size of 4 instead 3 at least as noted here), and some linux distros, notably fedora have moved to swap out standard zlib to zlib-ng in compat mode so even using zlib you aren't guaranteed a reproducible result unless you vendor it.