HBASE-29225 [hbase-thirdparty] Add module for Jetty 12 with EE8 and add support for toolchain based JDK selection

.gitignore

@@ -8,3 +8,4 @@ hbase-shaded-protobuf/src/main/resources
 .idea
 *.iml
 .flattened-pom.xml
+toolchains.xml

README.md

@@ -86,20 +86,129 @@ In case build fails due to protobuf-java version change, we can follow below ste
 
 ---
 
-Note that this project requires JDK8. This is because a bunch of the code we
-have in hbase-unsafe is using old APIs that have been removed from more
-modern JDKs. Due to a bug in JDK, we cannot generate this code using a more
-modern version of the JDK. See
-[HBASE-26773](https://issues.apache.org/jira/browse/HBASE-26773) for details.
+## Dual JDK Requirements
+
+Starting with version 4.1.12, this project requires both JDK 8 and JDK 17 to accommodate different HBase versions:
+
+- **HBase 2.x**: Uses JDK 8 compatible modules (including `hbase-shaded-jetty` with Jetty 9)
+- **HBase 3.x**: Uses JDK 17 modules (including `hbase-shaded-jetty-12-plus-*` with Jetty 12)
+
+Jetty 12 requires JDK 17 for compilation, but HBase 2.x deployments cannot move to Jetty 12 for JDK 8 compatibility. Our solution provides a single release containing modules for both JDK versions, eliminating the need for separate branches or releases.
+
+### JDK 8 Required Modules
+
+This project has specific JDK requirements for different modules:
+
+- **hbase-unsafe**: Must be built with JDK 8 because it uses old APIs that have been removed from more modern JDKs. Due to a bug in JDK, we cannot generate this code using a more modern version of the JDK. See [HBASE-26773](https://issues.apache.org/jira/browse/HBASE-26773) for details.
+- **hbase-shaded-protobuf**: Must be built with JDK 8 because it depends on internal Java APIs such as `sun.misc.Unsafe`. These internal APIs are inaccessible when compiling with newer JDKs but release target set to JDK 8.
+
+### JDK 17 Compilation with JDK 8 Target
+
+**All other modules** (including Jetty 12 modules) use JDK 17 for compilation but with release target set to JDK 8.
+
+### Maven Enforcer Plugin Profiles
+
+The project uses Maven enforcer plugin profiles to ensure these requirements are met:
+- **enforce-jdk8-bytecode**: Ensures hbase-unsafe and hbase-shaded-protobuf are built with JDK 8.
+- **enforce-jdk17-bytecode**: For **Jetty 12 modules** (hbase-shaded-jetty-12-plus-*), the enforcer plugin allows JDK 17 bytecode to be included from Jetty 12 dependencies, while still using JDK 8 as the release target.
+
+### Why toolchains are required?
+
+Maven needs explicit toolchain configuration to automatically select JDK 8 for hbase-unsafe and hbase-shaded-protobuf modules, and JDK 17 for all other modules including Jetty 12 modules. Environment variables alone are insufficient.
+
+### Files
+- `dev-support/generate-toolchains.sh` - Script to generate toolchains.xml with configurable paths
+- `toolchains.xml` - Generated Maven toolchains configuration file (not checked in)
 
 ## Build/Deploy
 
-To build, make sure that your environment uses JDK8, then just run:
+### Local Development Setup
+
+1. **Install both JDK versions**: JDK 8 and JDK 17
+2. **Set environment variables**:
+   ```sh
+   export JAVA8_HOME=/path/to/your/jdk8
+   export JAVA17_HOME=/path/to/your/jdk17
+   ```
+3. **Choose your toolchain setup approach** (see options below)
+
+### Toolchain Setup Options
+
+**Option 1: Project-local toolchains.xml**
+```sh
+# Generate and use project-specific toolchains
+export JAVA8_HOME=/path/to/your/jdk8
+export JAVA17_HOME=/path/to/your/jdk17
+
+# Below command will generate toolchains.xml in project root
+./dev-support/generate-toolchains.sh
+
+# Run build by passing toolchains.xml file to maven
+mvn clean install -t toolchains.xml
+```
 
+**Option 2: Global Maven toolchains setup**
 ```sh
-mvn clean package
+# Setup toolchains in ~/.m2/ directory
+export JAVA8_HOME=/path/to/your/jdk8
+export JAVA17_HOME=/path/to/your/jdk17
+
+# Below command will generate toolchains.xml in project root
+./dev-support/generate-toolchains.sh
+
+# Copy the generated file to global .m2 directory
+cp toolchains.xml ~/.m2/toolchains.xml
+
+# Run build as usual
+mvn clean install
 ```
 
+### CI/Jenkins Setup
+
+The Jenkins CI environment uses a Docker-based build system that automatically handles the dual JDK requirements without any manual configuration.
+
+#### Multi-JDK Docker Environment
+
+The Jenkins build uses a custom Dockerfile (`dev-support/jenkins/Dockerfile`) that:
+- Downloads and installs both JDK 8 (Adoptium Temurin) and JDK 17 (Adoptium Temurin)
+- Places them at standardized paths:
+  - Java 8: `/usr/lib/jvm/java-8`
+  - Java 17: `/usr/lib/jvm/java-17`
+- Includes Maven 3.9.8 for the build process
+
+#### Maven Wrapper with Automatic Toolchains
+
+The Docker image creates a Maven wrapper that automatically handles toolchain configuration by replacing the original `mvn` command with a wrapper that always passes the `-t ${BASEDIR}/dev-support/toolchains-jenkins.xml` parameter to ensure the correct toolchains file is used for every Maven invocation.
+
+#### Automatic JDK Selection
+
+The system uses a pre-configured toolchains file (`dev-support/toolchains-jenkins.xml`) that:
+- Defines JDK 1.8 toolchain pointing to `/usr/lib/jvm/java-8`
+- Defines JDK 17 toolchain pointing to `/usr/lib/jvm/java-17`
+- Allows Maven to automatically select the correct JDK for each module based on the toolchain requirements in their POMs
+
+#### Jenkins Build Process
+
+In Jenkins, the build process is completely automated:
+1. Docker container starts with both JDKs pre-installed
+2. Maven wrapper automatically passes the toolchains configuration
+3. Each module uses the appropriate JDK version:
+   - `hbase-unsafe` and `hbase-shaded-protobuf`: Built with JDK 8
+   - All other modules: Built with JDK 17 (with JDK 8 release target)
+4. No manual toolchain setup or environment configuration required
+
+#### Environment Variables
+
+The Jenkinsfile sets the following environment variables:
+```bash
+SET_JAVA_HOME="/usr/lib/jvm/java-17"    # Default JDK for the build
+JAVA8_HOME="/usr/lib/jvm/java-8"        # JDK 8 location
+JAVA17_HOME="/usr/lib/jvm/java-17"      # JDK 17 location
+```
+
+This automated setup ensures consistent builds across all Jenkins jobs without requiring developers or maintainers to manually configure toolchains in the CI environment.
+
+
 ## Release
 
 To cut a release candidate, update JIRA. The hbase-thirdparty currently uses

dev-support/generate-toolchains.sh

@@ -0,0 +1,62 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script to generate toolchains.xml with configurable Java paths
+
+# Set default paths if environment variables are not set
+if [ -z "$JAVA8_HOME" ]; then
+    echo "Trying to detect Java 8 installation (Jenkins/CI standard path)"
+    if [ -d "/usr/lib/jvm/java-8" ]; then
+        JAVA8_HOME="/usr/lib/jvm/java-8"
+    else
+        echo "Warning: JAVA8_HOME not set and Java 8 not found at /usr/lib/jvm/java-8"
+        echo "For local development, please set JAVA8_HOME environment variable"
+        JAVA8_HOME="/usr/lib/jvm/java-8"
+    fi
+fi
+
+if [ -z "$JAVA17_HOME" ]; then
+    echo "Trying to detect Java 17 installation (Jenkins/CI standard path)"
+    if [ -d "/usr/lib/jvm/java-17" ]; then
+        JAVA17_HOME="/usr/lib/jvm/java-17"
+    else
+        echo "Warning: JAVA17_HOME not set and Java 17 not found at /usr/lib/jvm/java-17"
+        echo "For local development, please set JAVA17_HOME environment variable"
+        JAVA17_HOME="/usr/lib/jvm/java-17"
+    fi
+fi
+
+TEMPLATE_FILE="./dev-support/toolchains-jenkins.xml"
+OUTPUT_FILE="toolchains.xml"
+
+if [ ! -f "$TEMPLATE_FILE" ]; then
+  echo "Template file $TEMPLATE_FILE not found!"
+  exit 1
+fi
+
+echo "Generating toolchains.xml with:"
+echo "  JAVA8_HOME: $JAVA8_HOME"
+echo "  JAVA17_HOME: $JAVA17_HOME"
+
+# Substitute placeholders and write output
+sed \
+  -e "s|/usr/lib/jvm/java-8|$JAVA8_HOME|g" \
+  -e "s|/usr/lib/jvm/java-17|$JAVA17_HOME|g" \
+  "$TEMPLATE_FILE" > "$OUTPUT_FILE"
+
+echo "toolchains.xml generated successfully!"

dev-support/jenkins/Dockerfile

@@ -14,21 +14,112 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-# Dockerfile for hbase-operator-tools pre-commit build.
-# https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build
-
-FROM maven:3.9-eclipse-temurin-8
-
-# hadolint ignore=DL3008
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
-       binutils \
-       git \
-       rsync \
-       shellcheck \
-       patch \
-       wget && \
+# Dockerfile for hbase-thirdparty pre-commit build.
+#
+#
+# Built in multiple stages so as to avoid re-downloading large binaries when
+# tweaking unrelated aspects of the image.
+FROM ubuntu:22.04 AS base_image
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+  DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+    ca-certificates=20211016 \
+    curl='7.81.0-*' \
+    locales='2.35-*' \
+    bash='5.1-*' \
+    binutils='2.38-*' \
+    build-essential=12.9ubuntu3 \
+    git='1:2.34.1-*' \
+    rsync='3.2.3-*' \
+    tar='1.34+dfsg-*' \
+    wget='1.21.2-*' \
+    shellcheck='0.8.0-*' \
+    patch='2.7.6-*' \
+    && \
     apt-get clean && \
-    rm -rf /var/lib/apt/lists/*
+    rm -rf /var/lib/apt/lists/* \
+    && \
+    locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+##
+# download sundry dependencies
+#
+
+FROM base_image AS maven_download_image
+ENV MAVEN_VERSION='3.9.8'
+ENV MAVEN_URL="https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz"
+ENV MAVEN_SHA512='7d171def9b85846bf757a2cec94b7529371068a0670df14682447224e57983528e97a6d1b850327e4ca02b139abaab7fcb93c4315119e6f0ffb3f0cbc0d0b9a2'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl --location --fail --silent --show-error --output /tmp/maven.tar.gz "${MAVEN_URL}" && \
+  echo "${MAVEN_SHA512} */tmp/maven.tar.gz" | sha512sum -c -
+
+FROM base_image AS openjdk8_download_image
+ENV OPENJDK8_URL='https://github.com/adoptium/temurin8-binaries/releases/download/jdk8u412-b08/OpenJDK8U-jdk_x64_linux_hotspot_8u412b08.tar.gz'
+ENV OPENJDK8_SHA256='b9884a96f78543276a6399c3eb8c2fd8a80e6b432ea50e87d3d12d495d1d2808'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl --location --fail --silent --show-error --output /tmp/adoptopenjdk8.tar.gz "${OPENJDK8_URL}" && \
+  echo "${OPENJDK8_SHA256} */tmp/adoptopenjdk8.tar.gz" | sha256sum -c -
+
+FROM base_image AS openjdk17_download_image
+ENV OPENJDK17_URL='https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.11%2B9/OpenJDK17U-jdk_x64_linux_hotspot_17.0.11_9.tar.gz'
+ENV OPENJDK17_SHA256='aa7fb6bb342319d227a838af5c363bfa1b4a670c209372f9e6585bd79da6220c'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl --location --fail --silent --show-error --output /tmp/adoptopenjdk17.tar.gz "${OPENJDK17_URL}" && \
+  echo "${OPENJDK17_SHA256} */tmp/adoptopenjdk17.tar.gz" | sha256sum -c -
+
+##
+# build the final image
+#
+
+FROM base_image
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# hadolint ignore=DL3010
+COPY --from=maven_download_image /tmp/maven.tar.gz /tmp/maven.tar.gz
+RUN tar xzf /tmp/maven.tar.gz -C /opt && \
+  ln -s "/opt/$(dirname "$(tar -tf /tmp/maven.tar.gz | head -n1)")" /opt/maven && \
+  rm /tmp/maven.tar.gz
+
+##
+# ensure JVMs are available under `/usr/lib/jvm` and prefix each installation
+# as `java-` so as to conform with Yetus's assumptions.
+#
+
+# hadolint ignore=DL3010
+COPY --from=openjdk8_download_image /tmp/adoptopenjdk8.tar.gz /tmp/adoptopenjdk8.tar.gz
+RUN mkdir -p /usr/lib/jvm && \
+  tar xzf /tmp/adoptopenjdk8.tar.gz -C /usr/lib/jvm && \
+  ln -s "/usr/lib/jvm/$(basename "$(tar -tf /tmp/adoptopenjdk8.tar.gz | head -n1)")" /usr/lib/jvm/java-8-adoptopenjdk && \
+  ln -s /usr/lib/jvm/java-8-adoptopenjdk /usr/lib/jvm/java-8 && \
+  rm /tmp/adoptopenjdk8.tar.gz
+
+# hadolint ignore=DL3010
+COPY --from=openjdk17_download_image /tmp/adoptopenjdk17.tar.gz /tmp/adoptopenjdk17.tar.gz
+RUN mkdir -p /usr/lib/jvm && \
+  tar xzf /tmp/adoptopenjdk17.tar.gz -C /usr/lib/jvm && \
+  ln -s "/usr/lib/jvm/$(basename "$(tar -tf /tmp/adoptopenjdk17.tar.gz | head -n1)")" /usr/lib/jvm/java-17-adoptopenjdk && \
+  ln -s /usr/lib/jvm/java-17-adoptopenjdk /usr/lib/jvm/java-17 && \
+  rm /tmp/adoptopenjdk17.tar.gz
+
+# configure default environment for Yetus
+ENV MAVEN_HOME='/opt/maven'
+
+# create a wrapper for mvn to always pass the toolchains.xml file
+RUN mv /opt/maven/bin/mvn /opt/maven/bin/mvn-original && \
+    cat > /opt/maven/bin/mvn <<'EOF'
+#!/bin/bash
+TOOLCHAIN="${BASEDIR}/dev-support/toolchains-jenkins.xml"
+if [ -f "$TOOLCHAIN" ]; then
+  echo "Added: -t ${TOOLCHAIN} to mvn flags!"
+  exec ${MAVEN_HOME}/bin/mvn-original "$@" -t "$TOOLCHAIN"
+else
+  echo "Unexpected: ${TOOLCHAIN} file is missing!"
+  exec ${MAVEN_HOME}/bin/mvn-original "$@"
+fi
+EOF
+RUN chmod +x /opt/maven/bin/mvn && cat /opt/maven/bin/mvn
 
 CMD ["/bin/bash"]
 

dev-support/jenkins/Jenkinsfile

@@ -53,7 +53,9 @@ pipeline {
         DOCKERFILE_REL = "${SRC_REL}/dev-support/jenkins/Dockerfile"
         YETUS_DRIVER_REL = "${SRC_REL}/dev-support/jenkins/jenkins_precommit_github_yetus.sh"
         ARCHIVE_PATTERN_LIST = '*.dump'
-        SET_JAVA_HOME = '/opt/java/openjdk'
+        SET_JAVA_HOME = "/usr/lib/jvm/java-17"
+        JAVA8_HOME = "/usr/lib/jvm/java-8"
+        JAVA17_HOME = "/usr/lib/jvm/java-17"
         PLUGINS = 'all'
     }
 
@@ -147,7 +149,7 @@ pipeline {
             }
             tools {
                 // this needs to be set to the jdk that ought to be used to build releases on the branch the Jenkinsfile is stored in.
-                jdk "jdk_1.8_latest"
+                jdk "jdk_17_latest"
             }
             stages {
                 stage ('setup') {

dev-support/toolchains-jenkins.xml

@@ -0,0 +1,41 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<toolchains xmlns="http://maven.apache.org/TOOLCHAINS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xsi:schemaLocation="http://maven.apache.org/TOOLCHAINS/1.1.0 http://maven.apache.org/xsd/toolchains-1.1.0.xsd">
+  <!--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+  <toolchain>
+    <type>jdk</type>
+    <provides>
+      <version>1.8</version>
+    </provides>
+    <configuration>
+      <jdkHome>/usr/lib/jvm/java-8</jdkHome>
+    </configuration>
+  </toolchain>
+  <toolchain>
+    <type>jdk</type>
+    <provides>
+      <version>17</version>
+    </provides>
+    <configuration>
+      <jdkHome>/usr/lib/jvm/java-17</jdkHome>
+    </configuration>
+  </toolchain>
+</toolchains>