Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -568,3 +568,25 @@ jobs:
name: cudf_java_maven_repo
path: ${{ runner.temp }}/maven-repo
if-no-files-found: error

# Publish tagged release candidates to Maven Central via the Sonatype
# Central Publisher Portal. Release path only (vYY.MM.PP tags). Does not
# publish nightlies.
# TODO: add nightly Sonatype snapshot publishing.
java-publish:
needs: [java-gather]
if: ${{ github.ref_type == 'tag' && (inputs.build_type || 'branch') == 'branch' }}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
permissions:
actions: read
contents: read
uses: rapidsai/shared-workflows/.github/workflows/maven-publish.yaml@main
secrets:
GPG_PRIVATE_KEY: ${{ secrets.GPG_PRIVATE_KEY }}
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
MAVEN_DEPLOY_TOKEN: ${{ secrets.MAVEN_DEPLOY_TOKEN }}
with:
publication-type: 'rc'
artifact-name: cudf_java_maven_repo
source-git-sha: ${{ inputs.sha || github.sha }}
# false = validate + drop (safe). true = stage PENDING for manual publish.
stage-for-maven-central-publish: true
3 changes: 3 additions & 0 deletions ci/build_java.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ if [[ -z ${RAPIDS_CUDA_VERSION:-} ]]; then
exit 1
fi

export HOST_UID="${HOST_UID:-$(id -u)}"
export HOST_GID="${HOST_GID:-$(id -g)}"

RAPIDS_CUDA_VERSION="$(cudf_java_normalize_cuda_version "${RAPIDS_CUDA_VERSION}")"
export RAPIDS_CUDA_VERSION
CLASSIFIER="$(cudf_java_maven_classifier "${RAPIDS_CUDA_VERSION}")"
Expand Down
56 changes: 42 additions & 14 deletions java/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,15 @@ so plain `rm -rf` works.
--cuda-version 12.9
```

This compiles the JNI layer against the static libcudf from Step 1 and emits a
single classifier JAR (e.g. `cudf-26.10.0-SNAPSHOT-cuda12.jar`) plus its POM
into a classifier-named subdirectory under `--output-dir`:
Optional `GITHUB_REF` selects release tag vs SNAPSHOT versioning. Unset means
SNAPSHOT. See the versioning section below.

```
This compiles the JNI layer against the static libcudf from Step 1 and emits
the classifier JAR (e.g. `cudf-26.10.0-SNAPSHOT-cuda12.jar`), a
classifier-independent sources jar and javadoc jar, and the POM into a
classifier-named subdirectory under `--output-dir`:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

```text
/tmp/jars/cuda12/
cudf-26.10.0-SNAPSHOT-cuda12.jar
cudf-26.10.0-SNAPSHOT.pom
Expand All @@ -68,21 +72,45 @@ inside the container.
```

This walks every subdirectory of `--jars-dir` (each subdir name IS the
classifier), gathers the per-classifier JAR and shared POM, derives the
artifact version from the JAR filenames (requiring a single unique version
across subdirs), and lays them out as:

```
/tmp/maven-repo/ai/rapids/cudf/26.10.0-SNAPSHOT/
cudf-26.10.0-SNAPSHOT-cuda12.jar
cudf-26.10.0-SNAPSHOT-cuda13.jar
cudf-26.10.0-SNAPSHOT.pom
classifier), gathers the per-classifier JAR, one shared sources jar, one
shared javadoc jar, the shared POM, and seeds an unclassified primary JAR
as a copy of the `cuda12` classifier. Derives the artifact version from
the JAR filenames (requiring a single unique version across subdirs) and
lays them out as:

```text
/tmp/maven-repo/ai/rapids/cudf/<CUDF_VERSION>-SNAPSHOT/
cudf-<CUDF_VERSION>-SNAPSHOT.jar
cudf-<CUDF_VERSION>-SNAPSHOT-cuda12.jar
cudf-<CUDF_VERSION>-SNAPSHOT-cuda13.jar
cudf-<CUDF_VERSION>-SNAPSHOT-sources.jar
cudf-<CUDF_VERSION>-SNAPSHOT-javadoc.jar
cudf-<CUDF_VERSION>-SNAPSHOT.pom
```

The set of classifiers is whatever subdirectories are present under
`--jars-dir`. For a local `x86_64`-only run, populate `/tmp/jars/cuda12/`
and `/tmp/jars/cuda13/`. For the full four-way release build, add
`/tmp/jars/cuda12-arm64/` and `/tmp/jars/cuda13-arm64/`.
`/tmp/jars/cuda12-arm64/` and `/tmp/jars/cuda13-arm64/`. The `cuda12`
subdirectory is required because the unclassified primary JAR is copied from
it, so an `aarch64`-only set of subdirectories is not a valid gather input.

### Release Tag vs SNAPSHOT Versioning

Release tag CI runs (`GITHUB_REF=refs/tags/vYY.MM.PP`) produce release-versioned
JARs (`cudf-<CUDF_VERSION>-*.jar`). All other runs produce `-SNAPSHOT`. Gated by
[`rapids-is-release-build`](https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-is-release-build).
`GITHUB_REF` is optional. Unset or non-tag values stay SNAPSHOT.

To rehearse the release path locally:

```bash
GITHUB_REF=refs/tags/vYY.MM.PP ./java/ci/test_java_build_local.sh
```

Rewrites `java/pom.xml` in place for packaging, then restores it on exit.

### GitHub Actions

In GitHub Actions (`.github/workflows/build.yaml`), the `java-build` matrix job
runs Steps 1-2 per (CUDA x arch) entry and uploads each classifier subdir as a
Expand Down
14 changes: 13 additions & 1 deletion java/ci/assemble_maven_repo.sh
Original file line number Diff line number Diff line change
Expand Up @@ -166,12 +166,24 @@ if [[ -z "${FIRST_VERSION}" ]]; then
exit 1
fi

DEST_DIR="${OUTPUT_DIR}/${GROUP_PATH}/${ARTIFACT_ID}/${FIRST_VERSION}"

# Seed the unclassified primary from cuda12. Maven Central serves this to
# consumers depending on ai.rapids:cudf without a <classifier>.
PRIMARY_SOURCE="${DEST_DIR}/cudf-${FIRST_VERSION}-cuda12.jar"
if [[ ! -f "${PRIMARY_SOURCE}" ]]; then
echo "Error: ${PRIMARY_SOURCE} missing." >&2
exit 1
fi
UNCLASSIFIED="${DEST_DIR}/cudf-${FIRST_VERSION}.jar"
cp -f "${PRIMARY_SOURCE}" "${UNCLASSIFIED}"
echo " + cudf-${FIRST_VERSION}.jar (unclassified primary, copy of cuda12)"

# Sources and javadoc jars are classifier-independent (pure Java, no arch or
# cuda variation). Every classifier subdir produces byte-equivalent copies;
# pick the lexicographically first subdir's copy as canonical. Fail fast if
# any subdir is missing either file - that indicates -Prelease or
# -Pjavadoc-jdk17 did not activate for that classifier's build.
DEST_DIR="${OUTPUT_DIR}/${GROUP_PATH}/${ARTIFACT_ID}/${FIRST_VERSION}"
FIRST_CLASSIFIER_SUBDIR=""
for subdir in "${JARS_DIR}"/*/; do
if [[ -z ${FIRST_CLASSIFIER_SUBDIR} ]]; then
Expand Down
4 changes: 4 additions & 0 deletions java/ci/build_cudf_java_jar.sh
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,10 @@ DOCKER_ARGS=(
--env REPO_ROOT=/repo
)

if [[ -n ${GITHUB_REF:-} ]]; then
DOCKER_ARGS+=(--env GITHUB_REF="${GITHUB_REF}")
fi

if [[ -n ${CMAKE_CUDA_ARCHITECTURES} ]]; then
DOCKER_ARGS+=(--env CMAKE_CUDA_ARCHITECTURES="${CMAKE_CUDA_ARCHITECTURES}")
fi
Expand Down
33 changes: 28 additions & 5 deletions java/ci/build_cudf_java_jar_in_container.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,23 @@ if [[ -z ${RAPIDS_CUDA_VERSION:-} ]]; then
exit 1
fi

_chown_outputs_on_exit() {
if [[ -n ${HOST_UID:-} && -n ${HOST_GID:-} ]]; then
chown -R "${HOST_UID}:${HOST_GID}" "${OUTPUT_DIR}" "${REPO_ROOT}/java/target" 2>/dev/null || true
if [[ -z ${HOST_UID} || -z ${HOST_GID} ]]; then
echo "Error: HOST_UID and HOST_GID must both be set" >&2
exit 1
fi

POM_WAS_REWRITTEN=0
_cleanup_on_exit() {
local prior_status=$?
if [[ ${POM_WAS_REWRITTEN} -eq 1 ]]; then
mv -f "${REPO_ROOT}/java/pom.xml.backup" "${REPO_ROOT}/java/pom.xml"
fi
if ! chown -R "${HOST_UID}:${HOST_GID}" "${OUTPUT_DIR}" "${REPO_ROOT}/java/target"; then
echo "Warning: chown -R ${HOST_UID}:${HOST_GID} on ${OUTPUT_DIR} + ${REPO_ROOT}/java/target failed. Outputs may remain owned by root." >&2
fi
return "${prior_status}"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
}
trap _chown_outputs_on_exit EXIT
trap _cleanup_on_exit EXIT

BUILD_ARG=(
-B
Expand Down Expand Up @@ -83,7 +94,18 @@ BUILD_ARG+=("-Dcmake.ccache.opts=${CMAKE_CCACHE_OPTS[*]}")

cd "${REPO_ROOT}/java"

CUDF_VERSION="$(cudf_java_scl mvn help:evaluate -Dexpression=project.version -q -DforceStdout "${BUILD_ARG[@]}")"
CUDF_VERSION="$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout "${BUILD_ARG[@]}")"

# Release tag builds strip -SNAPSHOT and rewrite the POM so packaged artifacts
# carry the release version. Non-release builds keep -SNAPSHOT. The EXIT trap
# restores java/pom.xml after packaging (the rewritten POM is copied to OUTPUT_DIR).
if rapids-is-release-build; then
CUDF_VERSION="${CUDF_VERSION%-SNAPSHOT}"
cp -p "${REPO_ROOT}/java/pom.xml" "${REPO_ROOT}/java/pom.xml.backup"
POM_WAS_REWRITTEN=1
mvn versions:set -DnewVersion="${CUDF_VERSION}" -DgenerateBackupPoms=false "${BUILD_ARG[@]}"
fi

Comment thread
NvTimLiu marked this conversation as resolved.
Resolved
rapids-logger "Packaging cuDF Java JAR ${CUDF_VERSION}"

# Omit the `clean` goal: java/target may be a bind-mount point, so `mvn clean`
Expand Down Expand Up @@ -136,6 +158,7 @@ done

cp -f "${MAIN_JAR}" "${OUTPUT_DIR}/"
cp -f pom.xml "${OUTPUT_DIR}/cudf-${CUDF_VERSION}.pom"

rapids-logger "Emitted artifacts to ${OUTPUT_DIR}"
if command -v sccache >/dev/null 2>&1; then
sccache --show-adv-stats || true
Expand Down
19 changes: 15 additions & 4 deletions java/ci/build_static_libcudf_in_container.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,20 @@ if [[ -z ${RAPIDS_CUDA_VERSION:-} ]]; then
exit 1
fi

if [[ -z ${HOST_UID} || -z ${HOST_GID} ]]; then
echo "Error: HOST_UID and HOST_GID must both be set" >&2
exit 1
fi

_cleanup_on_exit() {
local prior_status=$?
if ! chown -R "${HOST_UID}:${HOST_GID}" "${INSTALL_PREFIX}"; then
echo "Warning: chown -R ${HOST_UID}:${HOST_GID} on ${INSTALL_PREFIX} failed. Outputs may remain owned by root." >&2
fi
return "${prior_status}"
}
trap _cleanup_on_exit EXIT
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

CMAKE_ARGS=(
-S "${REPO_ROOT}/cpp"
-B "${BUILD_DIR}"
Expand Down Expand Up @@ -75,10 +89,7 @@ cudf_java_scl cmake "${CMAKE_ARGS[@]}"
cmake --build "${BUILD_DIR}" --parallel "${PARALLEL_LEVEL}"
cmake --install "${BUILD_DIR}"

# Hand the install tree back to the host user (host wrapper passes HOST_UID/GID).
if [[ -n ${HOST_UID:-} && -n ${HOST_GID:-} ]]; then
chown -R "${HOST_UID}:${HOST_GID}" "${INSTALL_PREFIX}"
fi
rapids-logger "Emitted static libcudf install tree to ${INSTALL_PREFIX}"
if command -v sccache >/dev/null 2>&1; then
sccache --show-adv-stats || true
fi
Loading