[REVIEW] [Java] Option to build fat-jars with native dependencies included #1296
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,10 +7,18 @@ set -e -u -o pipefail | |
| ARGS="$*" | ||
| NUMARGS=$# | ||
|
|
||
| VERSION="25.10.0" # Note: The version is updated automatically when ci/release/update-version.sh is invoked | ||
| GROUP_ID="com.nvidia.cuvs" | ||
|
|
||
| # Identify CUDA major version. | ||
| CUDA_VERSION_FROM_NVCC=$(nvcc --version | grep -oP 'release [0-9]+' | awk '{print $2}') | ||
| CUDA_MAJOR_VERSION=${CUDA_VERSION_FROM_NVCC:-12} | ||
|
|
||
| # Identify architecture. | ||
| ARCH=$(uname -m) | ||
|
|
||
| BUILD_PROFILE="$ARCH-cuda$CUDA_MAJOR_VERSION" | ||
|
|
||
| if [ -z "${CMAKE_PREFIX_PATH:=}" ]; then | ||
| CMAKE_PREFIX_PATH="$(pwd)/../cpp/build" | ||
| export CMAKE_PREFIX_PATH | ||
|
|
@@ -33,12 +41,12 @@ fi | |
|
|
||
| # Build the java layer | ||
| if [ -z ${LD_LIBRARY_PATH+x} ] | ||
| then export LD_LIBRARY_PATH=$CMAKE_PREFIX_PATH | ||
| else export LD_LIBRARY_PATH=$CMAKE_PREFIX_PATH:${LD_LIBRARY_PATH} | ||
|
Comment on lines
+44
to
+45
Contributor
Author
There was a problem hiding this comment. This was changed so that we don't make assumptions on the build paths. When building Java artifacts for In #1264, the Java build will look like: # First, build libcuvs.so.
LIBCUVS_BUILD_DIR=`pwd`/cpp/build/cuda13 build.sh libcuvs
# Next, build Java, using the libraries built above.
CMAKE_PREFIX_PATH=`pwd`/cpp/build/cuda13 build.sh java
Contributor
There was a problem hiding this comment. In general I recommend C++ builds be installed before being used downstream, instead of being used from the build directory. Ensure that downstream projects only use public API.... |
||
| fi | ||
|
|
||
| cd cuvs-java | ||
| mvn clean verify "${MAVEN_VERIFY_ARGS[@]}" -P "$BUILD_PROFILE" \ | ||
| && mvn install:install-file -Dfile=./target/cuvs-java-$VERSION.jar -DgroupId=$GROUP_ID -DartifactId=cuvs-java -Dversion=$VERSION -Dpackaging=jar \ | ||
| && mvn install:install-file -Dfile=./target/cuvs-java-$VERSION-"$BUILD_PROFILE".jar -DgroupId=$GROUP_ID -DartifactId=cuvs-java -Dversion=$VERSION -Dclassifier="$BUILD_PROFILE" -Dpackaging=jar \ | ||
|
Contributor
There was a problem hiding this comment. Nice, and thank you for keeping the slim jar too! |
||
| && cp pom.xml ./target/ | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <!-- | ||
| /* | ||
| * Copyright (c) 2025, NVIDIA CORPORATION. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| --> | ||
| <assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0" | ||
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0 | ||
| http://maven.apache.org/xsd/assembly-2.1.0.xsd"> | ||
|
|
||
| <id>${native.classifier}</id> | ||
| <formats> | ||
| <format>jar</format> | ||
| </formats> | ||
|
|
||
| <includeBaseDirectory>false</includeBaseDirectory> | ||
|
|
||
| <!-- Include all dependencies --> | ||
| <dependencySets> | ||
| <dependencySet> | ||
| <outputDirectory>/</outputDirectory> | ||
| <useProjectArtifact>true</useProjectArtifact> | ||
| <unpack>true</unpack> | ||
| <scope>runtime</scope> | ||
| </dependencySet> | ||
| </dependencySets> | ||
|
|
||
| <!-- Include native libraries from separate directory --> | ||
| <fileSets> | ||
| <fileSet> | ||
| <directory>${project.build.directory}/native-libs</directory> | ||
|
Contributor
Author
There was a problem hiding this comment. This is the assembly file that packages native-libraries into the native-jar artifact. |
||
| <outputDirectory>/</outputDirectory> | ||
| <includes> | ||
| <include>**/*</include> | ||
| </includes> | ||
| </fileSet> | ||
| </fileSets> | ||
|
|
||
| </assembly> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,6 +31,10 @@ | |
|
|
||
| final class JDKProvider implements CuVSProvider { | ||
|
|
||
| static { | ||
| OptionalNativeDependencyLoader.loadLibraries(); | ||
|
Contributor
Author
There was a problem hiding this comment. This is the call into the optional native-dependency loader. If the jar includes native dependency libraries, they will be loaded at startup. If not, the load is skipped, and it runs as before (i.e. depending on
Contributor
There was a problem hiding this comment. Is there a way to pre-load specific paths? With the conda-pack option, we'd need to force pre-load the bundled libstdc++ library. I'm mentioning this because this looks like it might be the right place to do that pre-loading. If you just rely on LD_LIBRARY_PATH, you would likely end up with the system libstdc++, which will be too old for the conda packages. |
||
| } | ||
|
|
||
| private static final MethodHandle createNativeDataset$mh = createNativeDatasetBuilder(); | ||
|
|
||
| static MethodHandle createNativeDatasetBuilder() { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason to have two variables here? My inclination would be to error out if the inner expression fails. The pipefail setting at the top of the script takes care of propagating the status.