-
Notifications
You must be signed in to change notification settings - Fork 667
[DO NOT MERGE] Test RAFT 3052 #8261
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
09bb6e9
de52107
abad232
d8f8518
b56792d
fdd209f
df4fbfc
9d80a1b
6699b06
1491a5d
fd22c28
239a366
212fa25
ec03ec7
c824478
3b50f8c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,38 @@ | ||||||||||||||||||||
| #!/bin/bash | ||||||||||||||||||||
| # SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION. | ||||||||||||||||||||
| # SPDX-License-Identifier: Apache-2.0 | ||||||||||||||||||||
|
|
||||||||||||||||||||
| # download CI artifacts | ||||||||||||||||||||
| LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda) | ||||||||||||||||||||
| RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda) | ||||||||||||||||||||
| LIBCUVS_CHANNEL=$(rapids-get-pr-artifact cuvs 2227 cpp conda) | ||||||||||||||||||||
|
Comment on lines
+6
to
+8
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Check if rapids-get-pr-artifact exists and how it handles invalid inputs
command -v rapids-get-pr-artifact
# Search for error handling patterns in similar CI helper scripts
rg -n -A 3 -B 1 'rapids-get-pr-artifact' --type sh -g 'ci/**/*.sh'
# Look for validation patterns after command substitution in CI scripts
rg -n -A 2 '\$\(rapids-get-pr-artifact.*\)' --type sh -g 'ci/**/*.sh'Repository: rapidsai/cuml Length of output: 2069 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Inspect the target script
echo "---- ci/use_conda_packages_from_prs.sh (top 120 lines) ----"
nl -ba ci/use_conda_packages_from_prs.sh | head -n 120
# Inspect the related wheels script (for consistency/precedent)
echo "---- ci/use_wheels_from_prs.sh (top 140 lines) ----"
nl -ba ci/use_wheels_from_prs.sh | head -n 140
# Find all uses of rapids-get-pr-artifact and whether any validation exists nearby
echo "---- occurrences + nearby lines (validation/error handling) ----"
rg -n -A 5 -B 2 'rapids-get-pr-artifact' --type sh 'ci/**/*.sh' || true
# Specifically look for empty-string validation after command substitution assignments
echo "---- empty-string / non-empty validation after rapids-get-pr-artifact ----"
rg -n '\[\[\s*-n\s+"\$\{?(LIB|RAFT|LIBCUVS)[A-Z0-9_]*\}?"\s*\]\]' ci/**/*.sh || true
rg -n '\[\[\s*-n\s+"\$\([^)]*rapids-get-pr-artifact' --type sh ci/**/*.sh || true
rg -n 'rapids-get-pr-artifact.*\n.*\[\[\s*-n' --type sh ci/**/*.sh || true
# Look for existing checks on variables populated from rapids-get-pr-artifact
echo "---- checks of variables populated from rapids-get-pr-artifact ----"
rg -n 'LIBRAFT_CHANNEL|RAFT_CHANNEL|LIBCUVS_CHANNEL' ci/use_conda_packages_from_prs.sh || trueRepository: rapidsai/cuml Length of output: 178 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "---- ci/use_conda_packages_from_prs.sh (first 120 lines) ----"
sed -n '1,120p' ci/use_conda_packages_from_prs.sh
echo "---- ci/use_wheels_from_prs.sh (first 140 lines) ----"
sed -n '1,140p' ci/use_wheels_from_prs.sh
echo "---- occurrences + nearby lines (validation/error handling) ----"
rg -n -A 6 -B 2 'rapids-get-pr-artifact' --type sh 'ci/**/*.sh' || true
echo "---- checks of variables populated from rapids-get-pr-artifact ----"
rg -n 'LIBRAFT_CHANNEL|RAFT_CHANNEL|LIBCUVS_CHANNEL' ci/use_conda_packages_from_prs.sh || true
echo "---- empty-string / non-empty validation patterns in CI scripts ----"
rg -n '\[\[\s*-n\s+"\$\{?(LIB|RAFT|LIBCUVS)[A-Z0-9_]*\}?"\s*\]\]' --type sh 'ci/**/*.sh' || true
rg -n '\[\[\s*-n\s+"\$\([^)]*rapids-get-pr-artifact' --type sh 'ci/**/*.sh' || trueRepository: rapidsai/cuml Length of output: 2901 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "---- locate rapids-get-pr-artifact in repo ----"
fd -a 'rapids-get-pr-artifact' . || true
rg -n --files-with-matches 'rapids-get-pr-artifact' . || true
echo "---- find likely script locations on PATH (from repo tooling) ----"
command -v rapids-get-pr-artifact || true
type rapids-get-pr-artifact || true
echo "---- search all CI shell scripts for rapids-get-pr-artifact usage + nearby validation ----"
rg -n -A 6 -B 2 'rapids-get-pr-artifact' --type sh ci || true
echo "---- search for non-empty checks tied to rapids-get-pr-artifact outputs ----"
rg -n '\[\[\s*-n\s+"\$\{?[^}]*\}?\]|\[\[\s*-n\s+"\$\([^)]*\)\]' --type sh ci || true
echo "---- inspect rapids-get-pr-artifact script if it exists in PATH and is readable ----"
PTH="$(command -v rapids-get-pr-artifact 2>/dev/null || true)"
if [[ -n "$PTH" && -r "$PTH" ]]; then
echo "---- contents: $PTH (first 160 lines) ----"
sed -n '1,160p' "$PTH"
else
echo "---- cannot read rapids-get-pr-artifact from PATH (not found or not readable) ----"
fiRepository: rapidsai/cuml Length of output: 2637 Add validation for non-empty conda channels after 🛡️ Proposed validation to add # download CI artifacts
LIBRAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 cpp conda)
+[[ -n "${LIBRAFT_CHANNEL}" ]] || { echo "ERROR: Failed to download libraft channel"; exit 1; }
RAFT_CHANNEL=$(rapids-get-pr-artifact raft 3052 python conda)
+[[ -n "${RAFT_CHANNEL}" ]] || { echo "ERROR: Failed to download raft channel"; exit 1; }
LIBCUVS_CHANNEL=$(rapids-get-pr-artifact cuvs 2227 cpp conda)
+[[ -n "${LIBCUVS_CHANNEL}" ]] || { echo "ERROR: Failed to download libcuvs channel"; exit 1; }📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||
| CUVS_CHANNEL=$(rapids-get-pr-artifact cuvs 2227 python conda) | ||||||||||||||||||||
| LIBNVFOREST_CHANNEL=$(rapids-get-pr-artifact nvforest 156 cpp conda) | ||||||||||||||||||||
| NVFOREST_CHANNEL=$(rapids-get-pr-artifact nvforest 156 python conda) | ||||||||||||||||||||
|
|
||||||||||||||||||||
| # For `rattler` builds: | ||||||||||||||||||||
| # | ||||||||||||||||||||
| # Add these channels to the array checked by 'rapids-rattler-channel-string'. | ||||||||||||||||||||
| # This ensures that when conda packages are built with strict channel priority enabled, | ||||||||||||||||||||
| # the locally-downloaded packages will be preferred to remote packages (e.g. nightlies). | ||||||||||||||||||||
| # | ||||||||||||||||||||
| RAPIDS_PREPENDED_CONDA_CHANNELS=( | ||||||||||||||||||||
| "${LIBRAFT_CHANNEL}" | ||||||||||||||||||||
| "${RAFT_CHANNEL}" | ||||||||||||||||||||
| "${LIBCUVS_CHANNEL}" | ||||||||||||||||||||
| "${CUVS_CHANNEL}" | ||||||||||||||||||||
| "${LIBNVFOREST_CHANNEL}" | ||||||||||||||||||||
| "${NVFOREST_CHANNEL}" | ||||||||||||||||||||
| ) | ||||||||||||||||||||
| export RAPIDS_PREPENDED_CONDA_CHANNELS | ||||||||||||||||||||
|
|
||||||||||||||||||||
| # For tests and `conda-build` builds: | ||||||||||||||||||||
| # | ||||||||||||||||||||
| # Add these channels to the system-wide conda configuration. | ||||||||||||||||||||
| # This results in PREPENDING them to conda's channel list, so | ||||||||||||||||||||
| # these packages should be found first if strict channel priority is enabled. | ||||||||||||||||||||
| # | ||||||||||||||||||||
| for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}" | ||||||||||||||||||||
| do | ||||||||||||||||||||
| conda config --system --add channels "${_channel}" | ||||||||||||||||||||
| done | ||||||||||||||||||||
|
Comment on lines
+35
to
+38
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Guard against adding empty channels to conda config. If any channel variable is empty (e.g., due to artifact download failure), 🛡️ Proposed fix to skip empty channels for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
+ [[ -n "${_channel}" ]] || continue
conda config --system --add channels "${_channel}"
done📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # initialize PIP_CONSTRAINT | ||
| source rapids-init-pip | ||
|
|
||
| RAPIDS_PY_CUDA_SUFFIX=$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}") | ||
|
|
||
| # download wheels, store the directories holding them in variables | ||
| LIBRAFT_WHEELHOUSE=$(rapids-get-pr-artifact raft 3052 cpp wheel) | ||
| PYLIBRAFT_WHEELHOUSE=$(rapids-get-pr-artifact raft 3052 python wheel --pkg_name pylibraft) | ||
| RAFT_DASK_WHEELHOUSE=$(rapids-get-pr-artifact raft 3052 python wheel --pkg_name raft_dask) | ||
| LIBCUVS_WHEELHOUSE=$(rapids-get-pr-artifact cuvs 2227 cpp wheel) | ||
| CUVS_WHEELHOUSE=$(rapids-get-pr-artifact cuvs 2227 python wheel) | ||
| LIBNVFOREST_WHEELHOUSE=$(rapids-get-pr-artifact nvforest 156 cpp wheel) | ||
| NVFOREST_WHEELHOUSE=$(rapids-get-pr-artifact nvforest 156 python wheel) | ||
|
|
||
| # write a pip constraints file saying e.g. "whenever you encounter a requirement for 'librmm-cu12', use this wheel" | ||
| cat > "${PIP_CONSTRAINT}" <<EOF | ||
| libraft-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBRAFT_WHEELHOUSE}"/libraft_*.whl) | ||
| pylibraft-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${PYLIBRAFT_WHEELHOUSE}"/pylibraft_*.whl) | ||
| raft-dask-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${RAFT_DASK_WHEELHOUSE}"/raft_dask_*.whl) | ||
| libcuvs-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUVS_WHEELHOUSE}"/libcuvs_*.whl) | ||
| cuvs-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${CUVS_WHEELHOUSE}"/cuvs_*.whl) | ||
| libnvforest-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBNVFOREST_WHEELHOUSE}"/libnvforest_*.whl) | ||
| nvforest-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${NVFOREST_WHEELHOUSE}"/nvforest_*.whl) | ||
| EOF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Remove duplicate sourcing of PR conda helper.
Line 18 re-sources
./ci/use_conda_packages_from_prs.shafter it was already sourced at Line 13. Since this helper mutates conda channel state, running it twice can duplicate prepended channels and cause unnecessary artifact/config churn. Source it once and reuse the resulting variables.🧰 Tools
🪛 Shellcheck (0.11.0)
[info] 18-18: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).
(SC1091)
🤖 Prompt for AI Agents