[fix] EXPERIMENT: Switch docker base image to avoid mixing musl & glibc libraries at runtime

[lhotari]

Motivation

While testing Pulsar 4.0.0-preview.1, some compatibility issues with the Alpine base image have shown up.
It appears that the solution in the base image is something that will cause further compatibility issues. Mixing musl and glibc at runtime is known to cause compatibility and stability issues.

https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/24647#note_176723 states it clearly:
"Combining glibc and musl runtimes is basically all but guaranteed to create an unstable environment, unless the system is appropriately configured (glibc side uses glibc binaries only, and vice versa)."

https://github.com/sgerrand/alpine-pkg-glibc/issues/194#issuecomment-1510140972 mentions:
"You really need everything pure musl, or everything musl (but works with gcompat or libc6-compat shim-like tools...) or everything glibc using this package)"

In our case, we are mixing pure musl libraries and glibc compiled libraries with our Alpine base image configuration which should be avoided.

Another reason to switch to a base image that is optimized for running Java JVM applications. The current Alpine image crashes the JVM in unexpected ways. #23348/#23361 is an example of this.

java.lang.OutOfMemoryError: unable to create a native thread: possibly out of memory or process/resource limits reached.

There's also one more release blocker which could be due to the mixed musl & glibc environment: #23306

Why mixing musl & glibc at runtime should be avoided

• blog post by Ariadne Conill, there is no such thing as a "glibc based alpine image"
• comments explaining reasons:
  • Add Alpine glibc images docker-library/official-images#10779 (comment)
  • Add Alpine glibc images docker-library/official-images#10779 (comment)
  • https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/24647#note_176723

Modifications

Use https://hub.docker.com/r/bellsoft/liberica-runtime-container as the base image. (Apache NiFi uses a bellsoft/liberica-openjdk-debian, so Liberica is already used in ASF.)
It's the Liberica OpenJDK's official base image provided by BellSoft. It uses Alpaquita Linux as the OS, which is similar to Alpine and uses apk as the package manager. Alpaquita Linux is optimized for running JVM applications.

The image size is very similar as before.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[lhotari]

It looks like test image building fails due to missing supervisor package. It is available also with pip install: http://supervisord.org/installing.html

[lhotari]

It looks like test image building fails due to missing supervisor package. It is available also with pip install: http://supervisord.org/installing.html

supervisor has been installed, but there is some remaining issue.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.57%. Comparing base (bbc6224) to head (f6498f5).
Report is 611 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23366      +/-   ##
============================================
+ Coverage     73.57%   74.57%   +0.99%     
- Complexity    32624    33962    +1338     
============================================
  Files          1877     1934      +57     
  Lines        139502   145132    +5630     
  Branches      15299    15870     +571     
============================================
+ Hits         102638   108231    +5593     
+ Misses        28908    28604     -304     
- Partials       7956     8297     +341     

Flag	Coverage Δ
inttests	27.69% <ø> (+3.10%)	⬆️
systests	24.39% <ø> (+0.07%)	⬆️
unittests	73.92% <ø> (+1.08%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 599 files with indirect coverage changes