From a153d4b30bb7c6409487487c6fe43b9d5d983420 Mon Sep 17 00:00:00 2001 From: Tirth Date: Tue, 10 Feb 2026 10:07:31 +0530 Subject: [PATCH 1/3] MINOR: Fix SSL certificate verification failure in system test worker provisioning (#21431) Update CA certificates on Ubuntu 14.04 Vagrant workers to fix SSL certificate verification failures when cloning the kibosh repository during system test setup. ### Problem Starting around February 7-9, 2026, system tests using Vagrant workers began failing during worker provisioning with the following error: ``` fatal: unable to access 'https://github.com/confluentinc/kibosh.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none ``` **Root Cause:** The Vagrant workers use Ubuntu 14.04 (Trusty), which reached end-of-life in April 2019. The CA certificate bundle on these workers is outdated and does not include the certificate authorities needed to verify GitHub's current SSL certificate chain. GitHub (or their CA provider) rotated certificates, causing the verification to fail on systems with older CA bundles. ### Solution Refresh the CA certificate store during worker provisioning by: 1. Installing/updating the `ca-certificates` package 2. Running `update-ca-certificates --fresh` to rebuild the certificate store This ensures workers have an updated certificate store that can verify GitHub's SSL certificate chain. Reviewers: Manikumar Reddy --- vagrant/base.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/vagrant/base.sh b/vagrant/base.sh index d0afb06b5e2e9..bc5cab623fdd8 100755 --- a/vagrant/base.sh +++ b/vagrant/base.sh @@ -96,7 +96,8 @@ get_kafka() { } # Install Kibosh -apt-get update -y && apt-get install -y git cmake pkg-config libfuse-dev +apt-get update -y && apt-get install -y git cmake pkg-config libfuse-dev ca-certificates +update-ca-certificates --fresh pushd /opt rm -rf /opt/kibosh git clone -q https://github.com/confluentinc/kibosh.git From 506f7e74e9793eb7bcb67a051aa7ed382b4ae799 Mon Sep 17 00:00:00 2001 From: Tirth Date: Thu, 12 Feb 2026 08:57:45 +0530 Subject: [PATCH 2/3] MINOR: Fix JDK version and architecture parameters in system test worker provisioning (#21394) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes bugs where `--jdk-version` and `--jdk-arch` parameters were ignored during system test worker provisioning, and refactors `vagrant/base.sh` to support flexible JDK versions without code changes. --- The Vagrant provisioning script (`vagrant/base.sh`) had two bugs that caused JDK version parameters to be ignored: | Bug | Problem | |-----|---------| | **#1: `--jdk-version` ignored** | `JDK_FULL` was hardcoded to `17-linux-x64`, so passing `--jdk-version 25` still downloaded JDK 17 | | **#2: `--jdk-arch` ignored** | Architecture parameter was passed but never used in the S3 download URL | --- - Validate `JDK_MAJOR` and `JDK_ARCH` input parameters with regex - Dynamically construct `JDK_FULL` from `JDK_MAJOR` and `JDK_ARCH` - Update S3 path to use `/jdk/` subdirectory - Add logging for debugging --- | Change | Description | |--------|-------------| | **Input validation** | Added regex validation for `JDK_MAJOR` and `JDK_ARCH` with sensible defaults | | **Dynamic construction** | `JDK_FULL` is now constructed from `JDK_MAJOR` and `JDK_ARCH` if not explicitly provided | | **Updated S3 path** | Changed URL from `/kafka-packages/jdk-{version}.tar.gz` to `/kafka-packages/jdk/jdk-{version}.tar.gz` | | **Logging** | Added debug output for JDK configuration | | **Backward compatibility** | Vagrantfile can still pass `JDK_FULL` directly; the script validates and uses it if valid | --- ``` s3://kafka-packages/jdk-{version}.tar.gz ``` ``` s3://kafka-packages/jdk/jdk-{version}.tar.gz ``` | File | Version | Architecture | |------|---------|--------------| | `jdk-7u80-linux-x64.tar.gz` | 7u80 | x64 | | `jdk-8u144-linux-x64.tar.gz` | 8u144 | x64 | | `jdk-8u161-linux-x64.tar.gz` | 8u161 | x64 | | `jdk-8u171-linux-x64.tar.gz` | 8u171 | x64 | | `jdk-8u191-linux-x64.tar.gz` | 8u191 | x64 | | `jdk-8u202-linux-x64.tar.gz` | 8u202 | x64 | | `jdk-11.0.2-linux-x64.tar.gz` | 11.0.2 | x64 | | `jdk-17-linux-x64.tar.gz` | 17 | x64 | | `jdk-18.0.2-linux-x64.tar.gz` | 18.0.2 | x64 | | `jdk-21.0.1-linux-x64.tar.gz` | 21.0.1 | x64 | | `jdk-21.0.1-linux-aarch64.tar.gz` | 21.0.1 | aarch64 | | `jdk-25-linux-x64.tar.gz` | 25 | x64 | | `jdk-25-linux-aarch64.tar.gz` | 25 | aarch64 | | `jdk-25.0.1-linux-x64.tar.gz` | 25.0.1 | x64 | | `jdk-25.0.1-linux-aarch64.tar.gz` | 25.0.1 | aarch64 | | `jdk-25.0.2-linux-x64.tar.gz` | 25.0.2 | x64 | | `jdk-25.0.2-linux-aarch64.tar.gz` | 25.0.2 | aarch64 | --- > **IMPORTANT: No code changes required for future Java major/minor releases!** The validation regex supports all version formats: - **Major versions**: `17`, `25`, `26` - **Minor versions**: `25.0.1`, `25.0.2`, `26.0.1` - **Legacy format**: `8u144`, `8u202` To add support for a new JDK version (e.g., JDK 26, 25.0.3): 1. Download the JDK tarball from Oracle/Adoptium 2. Rename to follow naming convention: `jdk-{VERSION}-linux-{ARCH}.tar.gz` 3. Upload to S3: `aws s3 cp jdk-{VERSION}-linux-{ARCH}.tar.gz s3://kafka-packages/jdk/` 4. Use in tests: `--jdk-version {VERSION} --jdk-arch {ARCH}` No modifications to `base.sh` or any other scripts are needed. --- | Before | After | |--------|-------| | `--jdk-version` ignored | ✅ Correctly uses specified version | | `--jdk-arch` ignored | ✅ Correctly uses specified architecture | | Only major version support | ✅ Full version support (e.g., `25.0.2`) | | Code change needed for new JDK | ✅ Just upload to S3 and pass version | --- Tested with different JDK versions to confirm the fix works correctly: | Test | JDK_MAJOR | Expected | Actual | Result | Test Report | |------|-----------|----------|--------|--------|-------------| | JDK 17 | `17` | javac 17.0.4 | javac 17.0.4 | ✅ | | JDK 25 | `25` | javac 25.0.2 | javac 25.0.2 | ✅ | --- - **Vagrantfile**: Continues to work as before - **Existing workflows**: Default behavior unchanged (JDK 17 on x64 architecture) - **No breaking changes**: All existing configurations continue to work --- Reviewers: Manikumar Reddy , Chia-Ping Tsai --- vagrant/base.sh | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/vagrant/base.sh b/vagrant/base.sh index bc5cab623fdd8..91ea3d9bbd858 100755 --- a/vagrant/base.sh +++ b/vagrant/base.sh @@ -32,14 +32,35 @@ fetch_jdk_tgz() { if [ ! -e $path ]; then mkdir -p $(dirname $path) - curl --retry 5 -s -L "https://s3-us-west-2.amazonaws.com/kafka-packages/jdk-${jdk_version}.tar.gz" -o $path + curl --retry 5 -s -L "https://s3-us-west-2.amazonaws.com/kafka-packages/jdk/jdk-${jdk_version}.tar.gz" -o $path fi } -JDK_MAJOR="${JDK_MAJOR:-17}" -JDK_FULL="${JDK_FULL:-17-linux-x64}" -echo "JDK_MAJOR=$JDK_MAJOR JDK_ARCH=$JDK_ARCH" -export DEBIAN_FRONTEND=noninteractive +# Validate JDK_MAJOR - must be a version number (e.g., 17, 25.0.1), default to 17 if empty or invalid +if [[ -z "$JDK_MAJOR" ]]; then + JDK_MAJOR="17" +elif [[ ! "$JDK_MAJOR" =~ ^[0-9]+(u[0-9]+|(\.[0-9]+)+)?$ ]]; then + echo "WARNING: Invalid JDK_MAJOR format '$JDK_MAJOR'. Expected format: 17, 21.0.1, 25.0.2. Defaulting to 17." + JDK_MAJOR="17" +fi + +# Validate JDK_ARCH - default to x64 if empty or invalid +if [[ -z "$JDK_ARCH" ]]; then + JDK_ARCH="x64" +elif [[ ! "$JDK_ARCH" =~ ^(x64|aarch64)$ ]]; then + echo "WARNING: Invalid JDK_ARCH format '$JDK_ARCH'. Expected: x64 or aarch64. Defaulting to x64." + JDK_ARCH="x64" +fi + +# Use JDK_FULL if provided and valid, otherwise construct from JDK_MAJOR and JDK_ARCH +if [[ -z "$JDK_FULL" ]]; then + JDK_FULL="${JDK_MAJOR}-linux-${JDK_ARCH}" +elif [[ ! "$JDK_FULL" =~ ^[0-9]+(u[0-9]+|(\.[0-9]+)+)?-linux-(x64|aarch64)$ ]]; then + echo "WARNING: Invalid JDK_FULL format '$JDK_FULL'. Constructing from JDK_MAJOR and JDK_ARCH." + JDK_FULL="${JDK_MAJOR}-linux-${JDK_ARCH}" +fi + +echo "JDK_MAJOR=$JDK_MAJOR JDK_ARCH=$JDK_ARCH JDK_FULL=$JDK_FULL" if [ -z `which javac` ]; then apt-get -y update From c65cd1b3c720e2ef72d3e498afe40510c1846d20 Mon Sep 17 00:00:00 2001 From: tirthooo7 Date: Fri, 13 Feb 2026 11:03:43 +0530 Subject: [PATCH 3/3] MINOR: revert DEBIAN_FRONTEND removal --- vagrant/base.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/vagrant/base.sh b/vagrant/base.sh index 91ea3d9bbd858..7bd22df5cd2f8 100755 --- a/vagrant/base.sh +++ b/vagrant/base.sh @@ -61,6 +61,7 @@ elif [[ ! "$JDK_FULL" =~ ^[0-9]+(u[0-9]+|(\.[0-9]+)+)?-linux-(x64|aarch64)$ ]]; fi echo "JDK_MAJOR=$JDK_MAJOR JDK_ARCH=$JDK_ARCH JDK_FULL=$JDK_FULL" +export DEBIAN_FRONTEND=noninteractive if [ -z `which javac` ]; then apt-get -y update