Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please release 1.8.5 #299

Closed
wdroste opened this issue Nov 28, 2021 · 10 comments
Closed

Please release 1.8.5 #299

wdroste opened this issue Nov 28, 2021 · 10 comments

Comments

@wdroste
Copy link

wdroste commented Nov 28, 2021

There's a critical bug in #292 / #293 that's breaking one of our applications. It looks like the last attempt at release failed can you please resolve this.

@danielfernandez
Copy link

This is critical for us too. Any attempts to use pure-java on a snappy-compressed file end up with a java.lang.IllegalArgumentException: newLimit > capacity being thrown due to the uncompressed buffer being resized beyond its capacity due to the wrong use of uncompressed.position() fixed in #293.

Please release 1.1.8.5 as soon as possible.

@wdroste
Copy link
Author

wdroste commented Dec 6, 2021

@xerial can you please update this. It's been since Oct 20th this fix has been in.

@devaraj-kavali
Copy link

we are also facing this issue, any plans to release this bug fix.

@ebuildy
Copy link

ebuildy commented Jul 2, 2022

Hello @xerial , thanks you for this library, any update on this one please?

I didnt find any workaround to make it work with Spark 3.2.1 on kubernetes.

Thanks you very much!

@ruleeeer
Copy link

ruleeeer commented Sep 8, 2022

@xerial Hi,thank you for this library.
This bug has been fixed for a year.
Would be grateful if 1.1.8.5 could be released.

@braiscouce
Copy link

Hi,

Do you know when the version with this fix will be released?

Regards.

@xerial
Copy link
Owner

xerial commented Oct 11, 2022

#293 introduces a binary change, and it will not work in some platforms like AIX/ppc,ppc64, s390x, etc., which are not covered by docker-based cross-compilers.

If @odaira can rebuild native libraries for these platforms as he did 4 years ago, it's safe to release. Otherwise, we need to drop support for these platforms.

@xerial
Copy link
Owner

xerial commented Oct 11, 2022

Pure-java support was an attempt to remove such a dependency on human labor, but it seems the current pure-java snappy code is not properly implemented and caused another issue, so it's blocking the project to ship a new release.

@danielfernandez
Copy link

danielfernandez commented Oct 18, 2022

@xerial The change in #293 is meant to affect the pure-java implementation only, and it simply refactors the moment in which the size of the compressed/uncompressed buffers is computed, so that the offset used as argument at the limit operation is not wrongly affected by the previous compression/uncompression operation.

Could you please elaborate on how this fix affects the AIX/ppc, ppc64, etc. environments? Do you mean that, by applying it, only the pure-java implementation would be fixed but not the native libraries? And if so... wouldn't it be possible to release 1.1.8.5 with this fix only, and keep a ticket open for the platforms that cannot benefit from this change?

This library is used by the Apache Hadoop libraries for snappy compression/decompression of large data files in data lakes, which means a really huge amount of corporate users. But in practice this issue makes it unusable in production, forcing anyone who needs to use this in a stable environment to keep their own fork 😞

@xerial
Copy link
Owner

xerial commented Jan 28, 2023

Released https://github.com/xerial/snappy-java/releases/tag/v1.1.9.0, which no longer supports pure-java mode. Some native libraries for special platforms like AIX are not included.

@xerial xerial closed this as completed Jan 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants