Building librdkafka

Building librdkafka

Below versions of librdkafka are available in respective distributions at the time of creation of these build instructions:

• RHEL (8.8, 8.10) have 1.6.1-1.el8
• RHEL (9.2, 9.4, 9.5) have 1.6.1-102.el9
• SLES 15 SP6 has 0.11.6-1.6.1

The instructions provided below specify the steps to build librdkafka version 2.8.0 on Linux on IBM Z for following distributions:

• RHEL (8.8, 8.10, 9.2, 9.4, 9.5)
• SLES 15 SP6
• Ubuntu (20.04, 22.04, 24.04, 24.10)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

1. Build and Install Librdkafka

1.1. Build using script

If you want to build librdkafka using manual steps, go to STEP 1.2.

Use the following commands to build librdkafka using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/librdkafka/2.8.0/build_librdkafka.sh

# Build librdkafka
bash build_librdkafka.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 2. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2. Install the build dependencies

export SOURCE_ROOT=/<source_root>/

• RHEL (8.8, 8.10, 9.2, 9.4, 9.5)

  sudo yum install -y git openssl-devel cyrus-sasl-devel python3 gcc gcc-c++ zlib-devel binutils make
  sudo yum install -y wget tar perl #for rhel8.x

• SLES 15 SP6

  sudo zypper install -y binutils gcc make libz1 zlib-devel git gcc-c++ openssl-devel
  sudo zypper install -y wget tar 

• Ubuntu (20.04, 22.04, 24.04, 24.10)

  sudo apt-get update
  sudo apt-get install -y git build-essential make zlib1g-dev libpthread-stubs0-dev libssl-dev libsasl2-dev libzstd-dev libcurl4-openssl-dev
  sudo apt-get install -y wget tar #for ubuntu20.04

• Install OpenSSL 3.4.0

# only for rhel8.8, rhel8.10, ubuntu20.04
NAME_OSSL=openssl
VERSION_OSSL=3.4.0

cd $SOURCE_ROOT
wget https://github.com/openssl/openssl/releases/download/openssl-$VERSION_OSSL/openssl-$VERSION_OSSL.tar.gz
tar -xzf openssl-$VERSION_OSSL.tar.gz
cd openssl-$VERSION_OSSL
./config --libdir=/usr/local/lib
make -j$(nproc)
sudo make install
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
openssl version

1.3. Download, configure and install librdkafka

# only for rhel8.8, rhel8.10, ubuntu20.04 
export CFLAGS="-I/usr/local/include"
export LIBS="-L/usr/local/lib"

• Download source

  cd $SOURCE_ROOT
  git clone -b v2.8.0 https://github.com/confluentinc/librdkafka.git
  cd librdkafka/

• Configure

  ./configure --install-deps

• Build and install

  make
  sudo make install

2. Testing (Optional)

2.1. Run the unit and integration tests

cd $SOURCE_ROOT/librdkafka
make -C tests -j1 run_local_quick

3. Verify (Optional)

3.1. set LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

3.2. Create verify_librdkafka.c

#include<stdio.h>
#include<librdkafka/rdkafka.h>

   int main(){
   printf("librdkafka version: %s\n", rd_kafka_version_str());
   return 0;
   }

3.3. Compile and run

gcc -o verify_librdkafka verify_librdkafka.c -lrdkafka
./verify_librdkafka

References:

• https://github.com/confluentinc/librdkafka#documentation