fix NoClassDefFoundError - io.airlift.compress.lz4.UnsafeUtil

[newur]

Motivation

Java client fails with Java 16 while receiving a message. Error:

[org.apa.pul.cli.imp.ClientCnx] (pulsar-client-io-2-1) [localhost/127.0.0.1:6650] Got exception java.lang.NoClassDefFoundError: Could not initialize class io.airlift.compress.lz4.UnsafeUtil
	at io.airlift.compress.lz4.Lz4RawDecompressor.decompress(Lz4RawDecompressor.java:60)
	at org.apache.pulsar.common.compression.CompressionCodecLZ4.decode(CompressionCodecLZ4.java:91)
	at org.apache.pulsar.client.impl.ConsumerImpl.uncompressPayloadIfNeeded(ConsumerImpl.java:1507)
	at org.apache.pulsar.client.impl.ConsumerImpl.messageReceived(ConsumerImpl.java:1037)
	at org.apache.pulsar.client.impl.ClientCnx.handleMessage(ClientCnx.java:447)
	at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:186)
...

Can be reproduced with the official example for Java Client from the docu while using JDK 16.

Workaround

Locally I use pulsar-client-original, and exclude the aircompressor 16. Example:

<dependency>
  <groupId>org.apache.pulsar</groupId>
  <artifactId>pulsar-client-original</artifactId>
  <version>${pulsar.version}</version>
  <exclusions>
    <exclusion>
      <groupId>io.airlift</groupId>
      <artifactId>aircompressor</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<!-- https://mvnrepository.com/artifact/io.airlift/aircompressor -->
<dependency>
  <groupId>io.airlift</groupId>
  <artifactId>aircompressor</artifactId>
  <version>0.19</version>
</dependency>

Modifications

Upgrade io.airlift.aircompressor to version 19.

Does this pull request potentially affect one of the following parts:

• Dependencies yes upgrade

[merlimat]

@newur We also need to update the version in the license files:

Run src/check-binary-license ./distribution/server/target/apache-pulsar-*-bin.tar.gz
io.airlift-aircompressor-0.19.jar unaccounted for in LICENSE
io.airlift-aircompressor-0.16.jar mentioned in LICENSE, but not bundled
aircompressor-0.19.jar unaccounted for in lib/presto/LICENSE
aircompressor-0.19.jar unaccounted for in lib/presto/LICENSE
aircompressor-0.16.jar mentioned in lib/presto/LICENSE, but not bundled
src/check-binary-license: line 120: echo: write error: Broken pipe

It looks like there are issues with the LICENSE/NOTICE.
Error: Process completed with exit code 2.

You can find them at:

• distribution/server/src/assemble/LICENSE.bin.txt
• pulsar-sql/presto-distribution/LICENSE

[newur]

For the record: deep down the following error is thrown when using JDK 16 and aircompressor <19:

Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module

Could not make it work with VM parameters like --add-opens java.base/java.nio=ALL-UNNAMED.

Also see: airlift/aircompressor#125

[eolivelli]

LGTM

[newur]

I cannot reproduce the failing unit tests locally. Checkout tag v2.8.0, changed the the aircompressor version and looks okay...

[eolivelli]

It is a flaky test.
I restarted the job. Don't worry