Building Elasticsearch

Below versions of Elasticsearch are available in respective distributions at the time of creation of these build instructions.

The instructions provided below specify the steps to build Elasticsearch 8.15.2 on Linux on IBM Z for following distributions:

RHEL (8.8, 8.10, 9.2, 9.4, 9.5)
SLES 15 SP6
Ubuntu (20.04, 22.04)

Prerequisites:

Docker packages are provided for RHEL, SLES and Ubuntu in their respective repositories. More information about Docker CE can be found here
Ensure the current user belongs to group docker:

Use the below command to add group docker if it does not exist:
```
sudo groupadd docker
```
Use the below command to add current user to group docker if it has not been done:
```
sudo usermod -aG docker $USER && newgrp docker
```
Docker Compose is required for some elasticsearch integration tests. Docker Compose should be installed on all platforms where it is available if the integration tests are going to be run.

General Notes

When following the steps below please use a standard permission user unless otherwise specified.
A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Build and Install Elasticsearch

1. Build using script

If you want to build Elasticsearch using manual steps, go to STEP 2.

Use the following commands to build Elasticsearch using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Elasticsearch/8.15.2/build_elasticsearch.sh

# Build Elasticsearch
bash build_elasticsearch.sh  [Provide -t option for executing build with tests, Provide -j for java to use from [Temurin17, OpenJDK17, Temurin21, OpenJDK21], if not specified, the script will use default java as Temurin17]

If the build completes successfully, go to STEP 9. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

2. Install build dependencies

export SOURCE_ROOT=/<source_root>/
export GRADLE_USER_HOME=$SOURCE_ROOT/.gradle
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Elasticsearch/8.15.2/patch"

RHEL (8.8, 8.10, 9.2, 9.4, 9.5)

sudo yum install -y curl git gzip tar wget patch make gcc gcc-c++

SLES 15 SP6

sudo zypper install -y curl git gzip tar wget patch make gcc gcc-c++ fontconfig dejavu-fonts

Ubuntu (20.04, 22.04)

sudo apt-get update
sudo apt-get install -y curl git gzip tar wget patch locales make gcc g++
sudo locale-gen en_US.UTF-8

3. Install Java

With Eclipse Adoptium Temurin Runtime
- Download and install Eclipse Adoptium Temurin Runtime (Java 17) from here.
- Download and install Eclipse Adoptium Temurin Runtime (Java 21) from here.

With OpenJDK 17

RHEL (8.8, 8.10, 9.2, 9.4, 9.5)

sudo yum install -y java-17-openjdk-devel

SLES 15 SP6

sudo zypper install -y java-17-openjdk java-17-openjdk-devel

Ubuntu (20.04, 22.04)
```
sudo apt-get install -y openjdk-17-jdk
```

With OpenJDK 21

RHEL (8.8, 8.10, 9.2, 9.4, 9.5)

sudo yum install -y java-21-openjdk-devel

SLES 15 SP6

sudo zypper install -y java-21-openjdk java-21-openjdk-devel

Ubuntu (20.04, 22.04)
```
sudo apt-get install -y openjdk-21-jdk
```

If running tests with OpenJDK, create a symbolic link to the java home directory to work around a gradle problem where gradle does not recognize the distro provided OpenJDK

sudo mkdir -p /opt/java/
sudo ln -s <Path to JDK> /opt/java/jdk

Note: At the time of creation of these build instructions, Elasticsearch was verified with Eclipse Adoptium Temurin Runtime (Java 17 build 17.0.11+9, Java 21 build 21.0.3+9), OpenJDK 17 and 21.

4. Set the environment variables

export LANG="en_US.UTF-8"
export JAVA_HOME=<Path to JDK>
export ES_JAVA_HOME=/<Path to JDK>/
export PATH=$ES_JAVA_HOME/bin:$PATH

Note: Ensure system locale is set up correctly for Elasticsearch to build without encoding errors.

5. Build Prerequite Projects

5.1 JANSI v2.4.0

cd $SOURCE_ROOT
git clone -b jansi-2.4.0 https://github.com/fusesource/jansi.git
cd jansi
make clean-native native OS_NAME=Linux OS_ARCH=s390x

mkdir -p $SOURCE_ROOT/jansi-jar
cd $SOURCE_ROOT/jansi-jar
wget https://repo1.maven.org/maven2/org/fusesource/jansi/jansi/2.4.0/jansi-2.4.0.jar
jar xvf jansi-2.4.0.jar
cd org/fusesource/jansi/internal/native/Linux
mkdir s390x
cp $SOURCE_ROOT/jansi/target/native-Linux-s390x/libjansi.so s390x/
cd $SOURCE_ROOT/jansi-jar
jar cvf jansi-2.4.0.jar .

mkdir -p $SOURCE_ROOT/.gradle/caches/modules-2/files-2.1/org.fusesource.jansi/jansi/2.4.0/321c614f85f1dea6bb08c1817c60d53b7f3552fd/
cp jansi-2.4.0.jar $SOURCE_ROOT/.gradle/caches/modules-2/files-2.1/org.fusesource.jansi/jansi/2.4.0/321c614f85f1dea6bb08c1817c60d53b7f3552fd/
export sha256=$(sha256sum jansi-2.4.0.jar | awk '{print $1}')

5.2 Docker Images for osixia/openldap

cd $SOURCE_ROOT
git clone -b v1.2.0 https://github.com/osixia/docker-light-baseimage.git
cd docker-light-baseimage/
curl -sSL "${PATCH_URL}/docker-light-baseimage.patch" | git apply -
make build
cd $SOURCE_ROOT
git clone -b v1.4.0 https://github.com/osixia/docker-openldap.git
cd docker-openldap/
curl -sSL "${PATCH_URL}/docker-openldap.patch" | git apply -
make build

5.3 zstd build from source

cd $SOURCE_ROOT
export ZSTD_VERSION=1.5.5
wget https://github.com/facebook/zstd/archive/refs/tags/v$ZSTD_VERSION.tar.gz
tar -xzvf v$ZSTD_VERSION.tar.gz
cd zstd-$ZSTD_VERSION
make DESTDIR=$(pwd)/_build install

6. Download Elasticsearch and apply patches

cd $SOURCE_ROOT
git clone -b v8.15.2 https://github.com/elastic/elasticsearch
cd elasticsearch

Apply gradle patches to create s390x distribution

curl -sSL "${PATCH_URL}/elasticsearch.patch" | git apply -
sed -i 's|6cd91991323dd7b2fb28ca93d7ac12af5a86a2f53279e2b35827b30313fd0b9f|'"${sha256}"'|g' $SOURCE_ROOT/elasticsearch/gradle/verification-metadata.xml

git fetch --tags
latest_tag=$(git tag | sort -V | tail -n1)
latest_tag="${latest_tag:1}-SNAPSHOT"
sed -i 's|${project.version}|'"${latest_tag}"'|g' $SOURCE_ROOT/elasticsearch/x-pack/plugin/ml/build.gradle
mkdir -p $SOURCE_ROOT/elasticsearch/libs/
cp -r $SOURCE_ROOT/zstd-$ZSTD_VERSION/_build/usr/local/lib/ $SOURCE_ROOT/elasticsearch/libs/zstd/
export LD_LIBRARY_PATH=$SOURCE_ROOT/elasticsearch/libs/zstd/${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
sudo ldconfig

Since the latest Elasticsearch version has added docker-export files in their repo, we also need to do the same for s390x. So create the following directories -

mkdir -p $SOURCE_ROOT/elasticsearch/distribution/packages/s390x-rpm/
mkdir -p $SOURCE_ROOT/elasticsearch/distribution/packages/s390x-deb/
mkdir -p $SOURCE_ROOT/elasticsearch/distribution/archives/linux-s390x-tar/
mkdir -p $SOURCE_ROOT/elasticsearch/distribution/docker/ubi-docker-s390x-export/
mkdir -p $SOURCE_ROOT/elasticsearch/distribution/docker/cloud-docker-s390x-export/
mkdir -p $SOURCE_ROOT/elasticsearch/distribution/docker/cloud-ess-docker-s390x-export/
mkdir -p $SOURCE_ROOT/elasticsearch/distribution/docker/docker-s390x-export/
mkdir -p $SOURCE_ROOT/elasticsearch/distribution/docker/ironbank-docker-s390x-export/

Inside each directory create a `build.gradle` file with the following content -

// This file is intentionally blank. All configuration of the
// export is done in the parent project.

7. Build

cd $SOURCE_ROOT/elasticsearch
CPU_NUM="$(grep -c ^processor /proc/cpuinfo)"
export GRADLE_USER_HOME=$SOURCE_ROOT/.gradle
./gradlew :distribution:archives:linux-s390x-tar:assemble --max-workers="$CPU_NUM"  --parallel

8. Testing (Optional)

Build Docker image for OpenLDAP test

cd $SOURCE_ROOT/elasticsearch/x-pack/test/idp-fixture/src/main/resources/openldap/
docker build -f Dockerfile -t docker.elastic.co/elasticsearch-dev/openldap-fixture:1.0 .

Run test

cd $SOURCE_ROOT/elasticsearch
export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF8"
export JAVA_HOME=/<Path to JDK>/
export RUNTIME_JAVA_HOME=/<Path to JDK>/ # Only with Temurin 17, 21
export RUNTIME_JAVA_HOME=/opt/java/jdk   # Only with OpenJDK 17, 21
./gradlew --continue test internalClusterTest -Dtests.haltonfailure=false -Dtests.jvm.argline="-Xss2m"

Notes:

Environment Variables needed for rerunning tests have been added to $HOME/setenv.sh. To set the Environment Variables needed to rerun tests, please run: source $HOME/setenv.sh
On RHEL 8 and SLES with OpenJDK, you may need to change the system crypto policy with a command like: 'sudo update-crypto-policies --set LEGACY' in order for all security tests to pass.
On RHEL 9 with OpenJDK, some security tests will fail even with the LEGACY crypto policy.
If there is an stack overflow error, increase the -Xss arg value in the above command.
Some X-Pack test cases will fail as X-Pack plugins are not supported on s390x, such as Machine Learning features.
The node.processors setting is now bounded by the number of available processors. Some X-Pack Test case may fail for this reason. To fix this, ensure the value of node.processors setting does not exceed the number of available processors.
For more information regarding Elasticsearch testing, please refer to their testing documentation.
User can also create distributions as deb, rpm and docker using below commands.

./gradlew :distribution:packages:s390x-deb:assemble
./gradlew :distribution:packages:s390x-rpm:assemble
./gradlew :distribution:docker:docker-s390x-export:assemble

9. Install Elasticsearch

cd $SOURCE_ROOT/elasticsearch
sudo mkdir /usr/share/elasticsearch
sudo tar -xzf distribution/archives/linux-s390x-tar/build/distributions/elasticsearch-8.15.2-SNAPSHOT-linux-s390x.tar.gz -C /usr/share/elasticsearch --strip-components 1
sudo ln -sf /usr/share/elasticsearch/bin/* /usr/bin/

sudo /usr/sbin/groupadd elastic
sudo chown [username]:elastic -R /usr/share/elasticsearch/

Update configurations to disable unsupported xpack.ml

sudo echo 'xpack.ml.enabled: false' >> /usr/share/elasticsearch/config/elasticsearch.yml

10. Verify Elasticsearch Server

> elasticsearch --version
Version: 8.15.2-SNAPSHOT, Build: tar/98adf7bf6bb69b66ab95b761c9e5aadb0bb059a3/2024-12-10T09:31:12.266253939Z, JVM: 21.0.3

11. Start Elasticsearch Server

elasticsearch &
export ELASTIC_PASSWORD=[PASSWORD]

Use curl --cacert /usr/share/elasticsearch/config/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200 when running Elasticsearch. The output should be similar to this:

{
  "name" : "22105eb7d1a5",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "YudHgajfSZaYaop7zvUojQ",
  "version" : {
    "number" : "8.15.2-SNAPSHOT",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "98adf7bf6bb69b66ab95b761c9e5aadb0bb059a3",
    "build_date" : "2024-12-10T09:31:12.266253939Z",
    "build_snapshot" : true,
    "lucene_version" : "9.11.1",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

12. Cleanup

cd $SOURCE_ROOT
rm -rf $SOURCE_ROOT/temurin.tar.gz
rm -rf $SOURCE_ROOT/v1.5.5.tar.gz
rm -rf $SOURCE_ROOT/jansi
rm -rf $SOURCE_ROOT/jansi-jar

Notes:

The password for the elastic user can be found in the output of the elasticsearch & command above.
On RHEL 9 with OpenJDK 17, it might be necessary to add the command line options --tls-max 1.1 --ciphers DEFAULT@SECLEVEL=0 to the curl command above.

References:

https://www.elastic.co/products/elasticsearch

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building Elasticsearch

Building Elasticsearch

Prerequisites:

Build and Install Elasticsearch

1. Build using script

2. Install build dependencies

3. Install Java

4. Set the environment variables

5. Build Prerequite Projects

5.1 JANSI v2.4.0

5.2 Docker Images for osixia/openldap

5.3 zstd build from source

6. Download Elasticsearch and apply patches

Since the latest Elasticsearch version has added docker-export files in their repo, we also need to do the same for s390x. So create the following directories -

Inside each directory create a `build.gradle` file with the following content -

7. Build

8. Testing (Optional)

9. Install Elasticsearch

10. Verify Elasticsearch Server

11. Start Elasticsearch Server

12. Cleanup

References:

Clone this wiki locally

Building Elasticsearch

Building Elasticsearch

Prerequisites:

Build and Install Elasticsearch

1. Build using script

2. Install build dependencies

3. Install Java

4. Set the environment variables

5. Build Prerequite Projects

5.1 JANSI v2.4.0

5.2 Docker Images for osixia/openldap

5.3 zstd build from source

6. Download Elasticsearch and apply patches

Since the latest Elasticsearch version has added docker-export files in their repo, we also need to do the same for s390x. So create the following directories -

Inside each directory create a build.gradle file with the following content -

7. Build

8. Testing (Optional)

9. Install Elasticsearch

10. Verify Elasticsearch Server

11. Start Elasticsearch Server

12. Cleanup

References:

Clone this wiki locally

Inside each directory create a `build.gradle` file with the following content -