Skip to content

Building CockroachDB

Jump to bottom
linuxonz edited this page Dec 6, 2024 · 34 revisions

Building CockroachDB

The instructions provided below specify the steps to build CockroachDB version 24.1.6 on Linux on IBM Z for the following distributions:

  • RHEL (8.8, 8.10, 9.2, 9.4)
  • Ubuntu (20.04, 22.04, 24.04)

Important Notes:

  • You need to enable the EPEL repositories on RHEL distros. This can be done by installing the epel-release package:

    • RHEL (8.8, 8.10)
    sudo subscription-manager repos --enable codeready-builder-for-rhel-8-$(arch)-rpms
sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
    • RHEL (9.2, 9.4)
    sudo subscription-manager repos --enable codeready-builder-for-rhel-9-$(arch)-rpms
sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

1. Build and Install CockroachDB

1.1. Build using script

If you'd like to build CockroachDB using the manual steps, please go to STEP 1.2.

Use the following commands to build CockroachDB using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/CockroachDB/24.1.6/build_crdb.sh

# Build CockroachDB
bash build_crdb.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 3. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps

1.2. Install the Dependencies

export SOURCE_ROOT=/<source_root>/
export PATCH_URL=https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/CockroachDB/24.1.6/patch

  • RHEL (8.8, 8.10)

    sudo yum install -y gcc-c++ git ncurses-devel make cmake automake bison patch wget tar xz zip unzip java-21-openjdk-devel python3 zlib-devel diffutils libtool libarchive openssl-devel keyutils-libs-devel bsdtar

  • RHEL 9.2

    sudo yum install -y gcc-c++ git ncurses-devel make cmake automake bison patch wget tar xz zip unzip java-21-openjdk-devel python3 ghc-resolv zlib-devel diffutils libtool libarchive keyutils-libs-devel bsdtar

  • RHEL 9.4

    sudo yum install -y gcc-c++ git ncurses-devel make cmake automake bison patch wget tar xz zip unzip java-21-openjdk-devel python3 zlib-devel diffutils libtool libarchive keyutils-libs-devel bsdtar

  • Ubuntu 20.04

    sudo apt-get update && sudo apt-get -y install zip unzip autoconf automake wget make libssl-dev libncurses5-dev bison xz-utils patch g++ curl git python3 libresolv-wrapper libkeyutils-dev openjdk-21-jdk libarchive-tools

  • Ubuntu 22.04

    sudo apt-get update && sudo apt-get -y install zip unzip autoconf automake wget make libssl-dev libncurses5-dev bison xz-utils patch g++ curl git python3 cmake netbase libresolv-wrapper libkeyutils-dev openjdk-21-jdk libarchive-tools

  • Ubuntu 24.04

    sudo apt-get update && sudo apt-get -y install zip unzip autoconf automake wget make libssl-dev libncurses5-dev bison xz-utils patch g++ curl git python3 cmake netbase libresolv-wrapper libkeyutils-dev openjdk-21-jdk bzip2 libarchive-tools

1.3. Build and install GCC 10.5.0 (Only for Ubuntu 24.x)

cd $SOURCE_ROOT
ver=10.5.0
wget https://ftp.gnu.org/gnu/gcc/gcc-$ver/gcc-$ver.tar.gz
tar xzf gcc-$ver.tar.gz
cd gcc-$ver
./contrib/download_prerequisites
mkdir build-gcc
cd build-gcc
../configure --enable-languages=c,c++ --disable-multilib
make -j$(nproc)
sudo make install

1.4. Build and install CMake (Ubuntu 20.04)

cd $SOURCE_ROOT
wget https://github.com/Kitware/CMake/releases/download/v3.29.0/cmake-3.29.0.tar.gz
tar -xzf cmake-3.29.0.tar.gz
cd cmake-3.29.0
./bootstrap
make
sudo make install
cmake --version

1.5. Build and install Go

cd $SOURCE_ROOT
GO_VERSION=1.22.5
mkdir go_bootstrap
cd go_bootstrap
wget https://go.dev/dl/go$GO_VERSION.linux-s390x.tar.gz
tar -xzf go$GO_VERSION.linux-s390x.tar.gz
export GOROOT_BOOTSTRAP=$SOURCE_ROOT/go_bootstrap/go

cd $SOURCE_ROOT
git clone -b go$GO_VERSION https://go.googlesource.com/go goroot
cd goroot
wget -O go.patch https://raw.githubusercontent.com/cockroachdb/cockroach/v24.1.6/build/teamcity/internal/release/build-and-publish-patched-go/diff.patch
git apply --reject --whitespace=fix go.patch
cd src
./make.bash
export PATH="$SOURCE_ROOT/goroot/bin:$PATH"

go version

1.6. Build and Install Bazel

cd $SOURCE_ROOT
git clone -b 7.6.1 https://github.com/bazelbuild/rules_java.git
cd rules_java
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/7.2.1/patch/rules_java_7.6.1.patch | git apply || error "Patch rules_java v7.6.1"
cd $SOURCE_ROOT
git clone -b v0.11.1 https://github.com/sgammon/rules_graalvm
cd rules_graalvm
curl -sSL https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/7.2.1/patch/rules_graalvm_0.11.1.patch | git apply || error "Patch graalvm v0.11.1"
mkdir -p $SOURCE_ROOT/bazel
cd $SOURCE_ROOT/bazel
wget https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-7.2.1-dist.zip
unzip -q bazel-7.2.1-dist.zip
chmod -R +w .
curl -sSLO https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Bazel/7.2.1/patch/bazel.patch
sed -i '368,$d' bazel.patch
sed -i "s#RULES_JAVA_ROOT_PATH#$SOURCE_ROOT#g" bazel.patch
patch -p1 < bazel.patch || error "Patch bazel"
env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" bash ./compile.sh

export PATH=$PATH:$SOURCE_ROOT/bazel/output/
bazel --version

1.7. Build and Install resolv_wrapper (RHEL only)

cd $SOURCE_ROOT
wget https://ftp.samba.org/pub/cwrap/resolv_wrapper-1.1.8.tar.gz
tar zxf resolv_wrapper-1.1.8.tar.gz
cd resolv_wrapper-1.1.8
mkdir obj && cd obj
cmake -DCMAKE_INSTALL_PREFIX=/usr ..
make
sudo make install

1.8. Build and Install Bazel-lib

cd $SOURCE_ROOT
git clone -b v1.42.3 https://github.com/aspect-build/bazel-lib.git
cd bazel-lib/
value=($(sha256sum /usr/bin/bsdtar))
wget -O $SOURCE_ROOT/bazel-lib.patch $PATCH_URL/bazel-lib.patch
sed -i "s#BSDTAR_SHA256SUM#$value#g" $SOURCE_ROOT/bazel-lib.patch
git apply --reject --whitespace=fix $SOURCE_ROOT/bazel-lib.patch
bazel build @aspect_bazel_lib//tools/copy_directory
bazel build @aspect_bazel_lib//tools/copy_to_directory

1.9. Build and Install CockroachDB

  • Download source code

    cd $SOURCE_ROOT
git clone -b v24.1.6 --depth 1 https://github.com/cockroachdb/cockroach.git
cd cockroach

  • Apply patches to code using below commands:

    cd $SOURCE_ROOT/cockroach
mkdir patch && cd patch
cp $SOURCE_ROOT/bazel-lib.patch .
cat <<EOF >BUILD
filegroup(
    name = "bazel-lib.patch",
    srcs = ["bazel-lib.patch"],
)
EOF
cd $SOURCE_ROOT/cockroach
wget -O $SOURCE_ROOT/crdb.patch $PATCH_URL/crdb.patch
wget -O $SOURCE_ROOT/crdb_pebble.patch $PATCH_URL/crdb_pebble.patch
sed -i "s#SOURCE_ROOT_PATH#$SOURCE_ROOT#g" $SOURCE_ROOT/crdb.patch
git apply --reject --whitespace=fix $SOURCE_ROOT/crdb.patch
git apply $SOURCE_ROOT/crdb_pebble.patch

  • Build and install

    cd $SOURCE_ROOT/cockroach
echo 'build --remote_cache=http://127.0.0.1:9867' > .bazelrc.user
echo 'build --config=dev
build --config nolintonbuild' >> .bazelrc.user
echo "test --test_tmpdir=$SOURCE_ROOT/cockroach/tmp" >> .bazelrc.user
./dev doctor
./dev build
bazel build c-deps:libgeos --config force_build_cdeps
sudo cp cockroach /usr/local/bin
sudo mkdir -p /usr/local/lib/cockroach
sudo cp _bazel/bin/c-deps/libgeos_foreign/lib/libgeos.so /usr/local/lib/cockroach/
sudo cp _bazel/bin/c-deps/libgeos_foreign/lib/libgeos_c.so /usr/local/lib/cockroach/
export PATH=$SOURCE_ROOT/cockroach:$PATH

2. Testing (Optional)

cd $SOURCE_ROOT/cockroach
./dev test -v -- --test_timeout=3600

These are currently known test failures on s390x :

Many tests check for exact floating point values however due to optimizations on the s390x platform that cause small differences, tests in these packages will fail.

  • pkg/ccl/logictestccl (TestTenantLogic_geospatial)
  • pkg/geo/geomfn
  • pkg/sql/logictest (TestLogic_geospatial)
  • pkg/sql/opt/exec/execbuilder (TestExecBuild_geospatial)
  • pkg/sql/sem/eval
  • pkg/sql/sem/eval_test
  • pkg/util/metric

Tests in these packages fail because of platform dependent filenames in the stack traces.

  • pkg/util/log/logcrash

Tests in this package fail due to platform dependent hash function values.

  • pkg/sql/colflow

These specific tests in the pkg/sql package fail because the binary encoding of the sql plan is platform dependent but the plan is correct and is decoded correctly.

  • pkg/sql/opt/exec/execbuilder/tests/5node
    • TestExecBuild_dist_vectorize
    • TestExecBuild_explain_analyze_plans
    • TestLogic_distsql_stats
  • pkg/sql/logictest/tests/5node
    • TestLogic_distsql_stats

Test related to upgrade will fail because it downloads CockroachDB binary tar which is not available to s390x.

  • //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test
  • //pkg/ccl/logictestccl/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test

Note:

  • Some test cases might experience time out problem. They should pass after increasing the timeout and running the test again. And remaining test cases will pass on re-run.

3. Verify

cockroach version

Output should look like:

Build Tag:        v24.1.6-dev
Build Time:
Distribution:     CCL
Platform:         linux s390x
Go Version:       go1.22.5 X:nocoverageredesign
C Compiler:       gcc 11.4.1 20231218 (Red Hat 11.4.1-3)

Please visit this link to start a local cluster and CockroachDB SQL.

References:

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally