Skip to content
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ci/build_cpp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ RAPIDS_PACKAGE_VERSION=$(rapids-generate-version)
export RAPIDS_PACKAGE_VERSION

# populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array
source ./ci/use_conda_packages_from_prs.sh
source rapids-rattler-channel-string

# --no-build-id allows for caching with `sccache`
Expand Down
1 change: 1 addition & 0 deletions ci/build_docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
set -euo pipefail

rapids-logger "Downloading artifacts from previous jobs"
source ./ci/use_conda_packages_from_prs.sh
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuml cuml --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cuml cuml --stable --cuda "$RAPIDS_CUDA_VERSION")")

Expand Down
1 change: 1 addition & 0 deletions ci/build_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ RAPIDS_PACKAGE_VERSION=$(head -1 ./VERSION)
export RAPIDS_PACKAGE_VERSION

# populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array
source ./ci/use_conda_packages_from_prs.sh
source rapids-rattler-channel-string

rapids-logger "Prepending channel ${CPP_CHANNEL} to RATTLER_CHANNELS"
Expand Down
2 changes: 2 additions & 0 deletions ci/build_wheel_cuml.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ package_dir="python/cuml"

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"

source ./ci/use_wheels_from_prs.sh

# Download the libcuml wheel built in the previous step and make it
# available for pip to find.
#
Expand Down
2 changes: 2 additions & 0 deletions ci/build_wheel_libcuml.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ rapids-dependency-file-generator \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};cuda_suffixed=true" \
| tee /tmp/requirements-build.txt

source ./ci/use_wheels_from_prs.sh

rapids-logger "Installing build requirements"
rapids-pip-retry install \
-v \
Expand Down
5 changes: 5 additions & 0 deletions ci/test_cpp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,18 @@ rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

rapids-logger "Downloading artifacts from previous jobs"
source ./ci/use_conda_packages_from_prs.sh
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuml cuml --cuda "$RAPIDS_CUDA_VERSION")")

rapids-logger "Generate C++ testing dependencies"
source ./ci/use_conda_packages_from_prs.sh
rapids-dependency-file-generator \
--output conda \
--file-key test_cpp \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" \
--prepend-channel "${LIBRAFT_CHANNEL}" \
--prepend-channel "${RAFT_CHANNEL}" \
--prepend-channel "${LIBCUVS_CHANNEL}" \
--prepend-channel "${CPP_CHANNEL}" | tee env.yaml

rapids-mamba-retry env create --yes -f env.yaml -n test
Expand Down
5 changes: 5 additions & 0 deletions ci/test_notebooks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,19 @@ rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

rapids-logger "Downloading artifacts from previous jobs"
source ./ci/use_conda_packages_from_prs.sh
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuml cuml --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cuml cuml --stable --cuda "$RAPIDS_CUDA_VERSION")")

rapids-logger "Generate Notebook testing dependencies"
source ./ci/use_conda_packages_from_prs.sh
rapids-dependency-file-generator \
--output conda \
--file-key test_notebooks \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" \
--prepend-channel "${LIBRAFT_CHANNEL}" \
--prepend-channel "${RAFT_CHANNEL}" \
--prepend-channel "${LIBCUVS_CHANNEL}" \
--prepend-channel "${CPP_CHANNEL}" \
--prepend-channel "${PYTHON_CHANNEL}" | tee env.yaml

Expand Down
5 changes: 5 additions & 0 deletions ci/test_python_common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,19 @@ rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

rapids-logger "Downloading artifacts from previous jobs"
source ./ci/use_conda_packages_from_prs.sh
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcuml cuml --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cuml cuml --stable --cuda "$RAPIDS_CUDA_VERSION")")

rapids-logger "Generate Python testing dependencies"
source ./ci/use_conda_packages_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Remove duplicate sourcing of PR conda helper.

Line 18 re-sources ./ci/use_conda_packages_from_prs.sh after it was already sourced at Line 13. Since this helper mutates conda channel state, running it twice can duplicate prepended channels and cause unnecessary artifact/config churn. Source it once and reuse the resulting variables.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 18-18: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_python_common.sh` at line 18, The conda PR helper is being sourced
twice in the test script, which can re-apply its channel mutations and create
duplicate state. Update the logic in the test setup around source
./ci/use_conda_packages_from_prs.sh so it is only sourced once, and reuse the
variables it sets for the rest of the script instead of re-invoking it.

rapids-dependency-file-generator \
--output conda \
--file-key "${DEPENDENCY_FILE_KEY:-test_python}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};dependencies=${RAPIDS_DEPENDENCIES}" \
--prepend-channel "${LIBRAFT_CHANNEL}" \
--prepend-channel "${RAFT_CHANNEL}" \
--prepend-channel "${LIBCUVS_CHANNEL}" \
--prepend-channel "${CPP_CHANNEL}" \
--prepend-channel "${PYTHON_CHANNEL}" | tee env.yaml

Expand Down
1 change: 1 addition & 0 deletions ci/test_wheel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ set -euo pipefail

source rapids-init-pip

source ./ci/use_wheels_from_prs.sh
LIBCUML_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_cpp libcuml cuml --cuda "$RAPIDS_CUDA_VERSION")")
CUML_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuml cuml --stable --cuda "$RAPIDS_CUDA_VERSION")")

Expand Down
1 change: 1 addition & 0 deletions ci/test_wheel_dask.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ set -euo pipefail

source rapids-init-pip

source ./ci/use_wheels_from_prs.sh
LIBCUML_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_cpp libcuml cuml --cuda "$RAPIDS_CUDA_VERSION")")
CUML_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_python cuml cuml --stable --cuda "$RAPIDS_CUDA_VERSION")")
RAPIDS_TESTS_DIR=${RAPIDS_TESTS_DIR:-"${PWD}/test-results"}
Expand Down
32 changes: 32 additions & 0 deletions ci/use_conda_packages_from_prs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-License-Identifier: Apache-2.0

# download CI artifacts
LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
LIBCUVS_CHANNEL=$(rapids-get-pr-artifact cuvs 2227 cpp conda)
Comment on lines +6 to +8

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check if rapids-get-pr-artifact exists and how it handles invalid inputs
command -v rapids-get-pr-artifact

# Search for error handling patterns in similar CI helper scripts
rg -n -A 3 -B 1 'rapids-get-pr-artifact' --type sh -g 'ci/**/*.sh'

# Look for validation patterns after command substitution in CI scripts
rg -n -A 2 '\$\(rapids-get-pr-artifact.*\)' --type sh -g 'ci/**/*.sh'

Repository: rapidsai/cuml

Length of output: 2069


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the target script
echo "---- ci/use_conda_packages_from_prs.sh (top 120 lines) ----"
nl -ba ci/use_conda_packages_from_prs.sh | head -n 120

# Inspect the related wheels script (for consistency/precedent)
echo "---- ci/use_wheels_from_prs.sh (top 140 lines) ----"
nl -ba ci/use_wheels_from_prs.sh | head -n 140

# Find all uses of rapids-get-pr-artifact and whether any validation exists nearby
echo "---- occurrences + nearby lines (validation/error handling) ----"
rg -n -A 5 -B 2 'rapids-get-pr-artifact' --type sh 'ci/**/*.sh' || true

# Specifically look for empty-string validation after command substitution assignments
echo "---- empty-string / non-empty validation after rapids-get-pr-artifact ----"
rg -n '\[\[\s*-n\s+"\$\{?(LIB|RAFT|LIBCUVS)[A-Z0-9_]*\}?"\s*\]\]' ci/**/*.sh || true
rg -n '\[\[\s*-n\s+"\$\([^)]*rapids-get-pr-artifact' --type sh ci/**/*.sh || true
rg -n 'rapids-get-pr-artifact.*\n.*\[\[\s*-n' --type sh ci/**/*.sh || true

# Look for existing checks on variables populated from rapids-get-pr-artifact
echo "---- checks of variables populated from rapids-get-pr-artifact ----"
rg -n 'LIBRAFT_CHANNEL|RAFT_CHANNEL|LIBCUVS_CHANNEL' ci/use_conda_packages_from_prs.sh || true

Repository: rapidsai/cuml

Length of output: 178


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "---- ci/use_conda_packages_from_prs.sh (first 120 lines) ----"
sed -n '1,120p' ci/use_conda_packages_from_prs.sh

echo "---- ci/use_wheels_from_prs.sh (first 140 lines) ----"
sed -n '1,140p' ci/use_wheels_from_prs.sh

echo "---- occurrences + nearby lines (validation/error handling) ----"
rg -n -A 6 -B 2 'rapids-get-pr-artifact' --type sh 'ci/**/*.sh' || true

echo "---- checks of variables populated from rapids-get-pr-artifact ----"
rg -n 'LIBRAFT_CHANNEL|RAFT_CHANNEL|LIBCUVS_CHANNEL' ci/use_conda_packages_from_prs.sh || true

echo "---- empty-string / non-empty validation patterns in CI scripts ----"
rg -n '\[\[\s*-n\s+"\$\{?(LIB|RAFT|LIBCUVS)[A-Z0-9_]*\}?"\s*\]\]' --type sh 'ci/**/*.sh' || true
rg -n '\[\[\s*-n\s+"\$\([^)]*rapids-get-pr-artifact' --type sh 'ci/**/*.sh' || true

Repository: rapidsai/cuml

Length of output: 2901


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "---- locate rapids-get-pr-artifact in repo ----"
fd -a 'rapids-get-pr-artifact' . || true
rg -n --files-with-matches 'rapids-get-pr-artifact' . || true

echo "---- find likely script locations on PATH (from repo tooling) ----"
command -v rapids-get-pr-artifact || true
type rapids-get-pr-artifact || true

echo "---- search all CI shell scripts for rapids-get-pr-artifact usage + nearby validation ----"
rg -n -A 6 -B 2 'rapids-get-pr-artifact' --type sh ci || true

echo "---- search for non-empty checks tied to rapids-get-pr-artifact outputs ----"
rg -n '\[\[\s*-n\s+"\$\{?[^}]*\}?\]|\[\[\s*-n\s+"\$\([^)]*\)\]' --type sh ci || true

echo "---- inspect rapids-get-pr-artifact script if it exists in PATH and is readable ----"
PTH="$(command -v rapids-get-pr-artifact 2>/dev/null || true)"
if [[ -n "$PTH" && -r "$PTH" ]]; then
  echo "---- contents: $PTH (first 160 lines) ----"
  sed -n '1,160p' "$PTH"
else
  echo "---- cannot read rapids-get-pr-artifact from PATH (not found or not readable) ----"
fi

Repository: rapidsai/cuml

Length of output: 2637


Add validation for non-empty conda channels after rapids-get-pr-artifact
ci/use_conda_packages_from_prs.sh assigns LIBRAFT_CHANNEL, RAFT_CHANNEL, and LIBCUVS_CHANNEL from command substitution with no checks, and the script doesn’t enable set -euo pipefail, so empty strings can be added as conda channels.

🛡️ Proposed validation to add
 # download CI artifacts
 LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
+[[ -n "${LIBRAFT_CHANNEL}" ]] || { echo "ERROR: Failed to download libraft channel"; exit 1; }
 RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
+[[ -n "${RAFT_CHANNEL}" ]] || { echo "ERROR: Failed to download raft channel"; exit 1; }
 LIBCUVS_CHANNEL=$(rapids-get-pr-artifact cuvs 2227 cpp conda)
+[[ -n "${LIBCUVS_CHANNEL}" ]] || { echo "ERROR: Failed to download libcuvs channel"; exit 1; }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
LIBCUVS_CHANNEL=$(rapids-get-pr-artifact cuvs 2227 cpp conda)
LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
[[ -n "${LIBRAFT_CHANNEL}" ]] || { echo "ERROR: Failed to download libraft channel"; exit 1; }
RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
[[ -n "${RAFT_CHANNEL}" ]] || { echo "ERROR: Failed to download raft channel"; exit 1; }
LIBCUVS_CHANNEL=$(rapids-get-pr-artifact cuvs 2227 cpp conda)
[[ -n "${LIBCUVS_CHANNEL}" ]] || { echo "ERROR: Failed to download libcuvs channel"; exit 1; }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/use_conda_packages_from_prs.sh` around lines 6 - 8, Enable strict shell
failure at the top of ci/use_conda_packages_from_prs.sh (set -euo pipefail) and
validate the results of the rapids-get-pr-artifact calls by checking that
LIBRAFT_CHANNEL, RAFT_CHANNEL, and LIBCUVS_CHANNEL are non-empty after
assignment; if any is empty, print a clear error mentioning the variable name
and the rapids-get-pr-artifact invocation that failed, and exit with a non-zero
status so an empty string is never passed as a conda channel.


# For `rattler` builds:
#
# Add these channels to the array checked by 'rapids-rattler-channel-string'.
# This ensures that when conda packages are built with strict channel priority enabled,
# the locally-downloaded packages will be preferred to remote packages (e.g. nightlies).
#
RAPIDS_PREPENDED_CONDA_CHANNELS=(
"${LIBRAFT_CHANNEL}"
"${RAFT_CHANNEL}"
"${LIBCUVS_CHANNEL}"
)
export RAPIDS_PREPENDED_CONDA_CHANNELS

# For tests and `conda-build` builds:
#
# Add these channels to the system-wide conda configuration.
# This results in PREPENDING them to conda's channel list, so
# these packages should be found first if strict channel priority is enabled.
#
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
conda config --system --add channels "${_channel}"
done
Comment on lines +35 to +38

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Guard against adding empty channels to conda config.

If any channel variable is empty (e.g., due to artifact download failure), conda config --system --add channels "" may fail or behave unexpectedly. Add a check to skip empty values.

🛡️ Proposed fix to skip empty channels
 for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
 do
+  [[ -n "${_channel}" ]] || continue
    conda config --system --add channels "${_channel}"
 done
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
conda config --system --add channels "${_channel}"
done
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
[[ -n "${_channel}" ]] || continue
conda config --system --add channels "${_channel}"
done
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/use_conda_packages_from_prs.sh` around lines 29 - 32, The loop over
RAPIDS_PREPENDED_CONDA_CHANNELS may call conda config with an empty string;
update the loop that iterates _channel to skip empty values by testing _channel
is non-empty (e.g., using a shell check like [[ -n "$_channel" ]] or if [ -z ...
] continue) before running conda config --system --add channels "${_channel}" so
only valid channel strings are added.

20 changes: 20 additions & 0 deletions ci/use_wheels_from_prs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-License-Identifier: Apache-2.0

# initialize PIP_CONSTRAINT
source rapids-init-pip

RAPIDS_PY_CUDA_SUFFIX=$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")

# download wheels, store the directories holding them in variables
LIBRAFT_WHEELHOUSE=$(rapids-get-pr-artifact raft 3052 cpp wheel)
PYLIBRAFT_WHEELHOUSE=$(rapids-get-pr-artifact raft 3052 python wheel --pkg_name pylibraft)
LIBCUVS_WHEELHOUSE=$(rapids-get-pr-artifact cuvs 2227 cpp wheel)

# write a pip constraints file saying e.g. "whenever you encounter a requirement for 'librmm-cu12', use this wheel"
cat > "${PIP_CONSTRAINT}" <<EOF
libraft-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBRAFT_WHEELHOUSE}"/libraft_*.whl)
raft-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${PYLIBRAFT_WHEELHOUSE}"/pylibraft_*.whl)
libcuvs-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUVS_WHEELHOUSE}"/libcuvs_*.whl)
EOF
Loading