rapidgzip

[jianshu93]

Hello team,

Some recent parallel decompression available: https://github.com/mxmlnkn/rapidgzip/tree/master .

Not sure it is relevant or not since all those bindings themselves are not related to parallel implementation or not.

Thanks,

Jianshu

[Byron]

That's impressive work! Who would have thought that gzip can one day go at the speed of more than 2GB/s using multiple cores!

Something I found a bit magical is how an index can be produced in parallel as well even though a 'block' (for lack of proper term) depends on the previous one, which at least in theory prevents any parallel work unless one parallelises work within 32kb of data.

But that's just me rambling, as I don't have enough expertise to know if rapidzip could be integrated into flate2 or if any other benefits would be possible.

[oyvindln]

It depends a bit on whether it requires the whole data to operate or if it can work on a partial stream.