Repository navigation
Rapidsmpf logger API changes #22972
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rapidsmpf logger API changes #22972
Changes from 3 commits
8eda57a
6d0439a
3da4723
5215713
fca333d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION. | ||
| # SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| set -euo pipefail | ||
|
|
@@ -15,6 +15,9 @@ rapids-logger "Create test conda environment" | |
| rapids-logger "Configuring conda strict channel priority" | ||
| conda config --set channel_priority strict | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_conda_packages_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Same unguarded PR-artifact fetch as other build scripts. Same concern as flagged in 🧰 Tools🪛 Shellcheck (0.11.0)[info] 19-19: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| ENV_YAML_DIR="$(mktemp -d)" | ||
|
|
||
| rapids-logger "Downloading artifacts from previous jobs" | ||
|
|
@@ -28,6 +31,7 @@ rapids-dependency-file-generator \ | |
| --prepend-channel "${CPP_CHANNEL}" \ | ||
| --prepend-channel "${PYTHON_CHANNEL}" \ | ||
| --prepend-channel "${PYTHON_NOARCH_CHANNEL}" \ | ||
| "${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" \ | ||
| --matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee "${ENV_YAML_DIR}/env.yaml" | ||
|
|
||
| rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n docs | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,9 @@ CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libc | |
| RAPIDS_PACKAGE_VERSION=$(head -1 ./VERSION) | ||
| export RAPIDS_PACKAGE_VERSION | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_conda_packages_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Same unguarded PR-only artifact-fetch pattern. Same concern as the sibling build scripts: this unconditionally sources 🧰 Tools🪛 Shellcheck (0.11.0)[info] 25-25: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| # populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array | ||
| source rapids-rattler-channel-string | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,9 @@ echo "libcudf-streaming-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUDF_STRE | |
| echo "libcudf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUDF_WHEELHOUSE}"/libcudf_*.whl)" >> "${PIP_CONSTRAINT}" | ||
| echo "pylibcudf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${PYLIBCUDF_WHEELHOUSE}"/pylibcudf_*.whl)" >> "${PIP_CONSTRAINT}" | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_wheels_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n 'build_wheel_cudf_streaming\.sh' .github/workflows -g '*.yml' -g '*.yaml'
rg -n 'build_type' .github/workflows -g '*.yml' -g '*.yaml' -B5 -A5 | rg -n 'cudf.streaming' -B10
cat -n ci/use_wheels_from_prs.sh 2>/dev/null | sed -n '1,80p'Repository: rapidsai/cudf Length of output: 10771 Gate PR-only wheel sourcing behind 🧰 Tools🪛 Shellcheck (0.11.0)[info] 25-25: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| rapids-logger "Generating build requirements" | ||
|
|
||
| rapids-dependency-file-generator \ | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,6 +17,9 @@ RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")" | |
| LIBCUDF_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")") | ||
| echo "libcudf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUDF_WHEELHOUSE}"/libcudf_*.whl)" >> "${PIP_CONSTRAINT}" | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_wheels_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n 'build_wheel_libcudf_streaming\.sh' .github/workflows -g '*.yml' -g '*.yaml'Repository: rapidsai/cudf Length of output: 316 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Inspect the wheel script and its workflow callers.
printf '\n== ci/build_wheel_libcudf_streaming.sh ==\n'
cat -n ci/build_wheel_libcudf_streaming.sh | sed -n '1,120p'
printf '\n== .github/workflows/build.yaml around caller ==\n'
sed -n '170,230p' .github/workflows/build.yaml
printf '\n== .github/workflows/pr.yaml around caller ==\n'
sed -n '530,610p' .github/workflows/pr.yamlRepository: rapidsai/cudf Length of output: 8271 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Inspect the related wheel script for comparison.
printf '\n== ci/build_wheel_cudf_streaming.sh ==\n'
cat -n ci/build_wheel_cudf_streaming.sh | sed -n '1,120p'Repository: rapidsai/cudf Length of output: 4072 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Find where use_wheels_from_prs.sh is sourced and whether it's guarded.
rg -n 'use_wheels_from_prs\.sh|build_type|nightly|branch|pull_request|PR' ci .github/workflows -g '*.sh' -g '*.yml' -g '*.yaml'Repository: rapidsai/cudf Length of output: 21243 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== ci/build_wheel_libcudf_streaming.sh ==\n'
cat -n ci/build_wheel_libcudf_streaming.sh | sed -n '1,120p'
printf '\n== ci/use_wheels_from_prs.sh ==\n'
cat -n ci/use_wheels_from_prs.sh | sed -n '1,200p'
printf '\n== .github/workflows/build.yaml top section ==\n'
sed -n '1,120p' .github/workflows/build.yaml
printf '\n== .github/workflows/pr.yaml top section ==\n'
sed -n '1,120p' .github/workflows/pr.yamlRepository: rapidsai/cudf Length of output: 11617 🏁 Script executed: #!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
for path, start, end in [
(Path('.github/workflows/build.yaml'), 180, 220),
(Path('.github/workflows/pr.yaml'), 540, 590),
]:
print(f"\n== {path} {start}-{end} ==")
lines = path.read_text().splitlines()
for i in range(start-1, min(end, len(lines))):
print(f"{i+1}: {lines[i]}")
PYRepository: rapidsai/cudf Length of output: 4366 Guard 🧰 Tools🪛 Shellcheck (0.11.0)[info] 21-21: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| rapids-logger "Generating build requirements" | ||
|
|
||
| rapids-dependency-file-generator \ | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION. | ||
| # SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| set -euo pipefail | ||
|
|
@@ -18,6 +18,9 @@ CUDF_POLARS_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name whe | |
| # generate constraints (possibly pinning to oldest support versions of dependencies) | ||
| rapids-generate-pip-constraints py_test_cudf_polars "${PIP_CONSTRAINT}" | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_wheels_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n 'test_cudf_polars_polars_tests\.sh' .github/workflows -g '*.yml' -g '*.yaml'Repository: rapidsai/cudf Length of output: 317 🏁 Script executed: #!/bin/bash
set -euo pipefail
# Inspect the polars test script and the PR/test workflow jobs that call it.
sed -n '1,120p' ci/test_cudf_polars_polars_tests.sh
printf '\n--- pr.yaml excerpt ---\n'
sed -n '660,730p' .github/workflows/pr.yaml
printf '\n--- test.yaml excerpt ---\n'
sed -n '260,320p' .github/workflows/test.yaml
printf '\n--- helper script ---\n'
sed -n '1,200p' ci/use_wheels_from_prs.shRepository: rapidsai/cudf Length of output: 9684 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- test.yaml top ---'
sed -n '1,120p' .github/workflows/test.yaml
printf '\n%s\n' '--- pr.yaml top ---'
sed -n '1,120p' .github/workflows/pr.yaml
printf '\n%s\n' '--- references to test.yaml ---'
rg -n 'test\.yaml@main|build_type: .*branch|build_type: .*nightly|build_type: .*pull-request|cudf-polars-polars-tests' .github/workflows ci -g '*.yml' -g '*.yaml' -g '*.sh'Repository: rapidsai/cudf Length of output: 12440 Guard the PR-only wheel sourcing behind a build-type check 🧰 Tools🪛 Shellcheck (0.11.0)[info] 22-22: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| rapids-logger "Install libcudf, pylibcudf and cudf_polars" | ||
|
|
||
| # notes: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,6 +9,9 @@ set -euo pipefail | |
| rapids-logger "Configuring conda strict channel priority" | ||
| conda config --set channel_priority strict | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_conda_packages_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== ci/test_java.sh ==\n'
cat -n ci/test_java.sh | sed -n '1,120p'
printf '\n== ci/cudf_pandas_scripts/third-party-integration/test.sh ==\n'
cat -n ci/cudf_pandas_scripts/third-party-integration/test.sh | sed -n '1,120p'
printf '\n== references to use_conda_packages_from_prs.sh ==\n'
rg -n "use_conda_packages_from_prs\.sh|RAPIDS_BUILD_TYPE" ci -g '!**/build/**'Repository: rapidsai/cudf Length of output: 7947 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== ci/use_conda_packages_from_prs.sh ==\n'
cat -n ci/use_conda_packages_from_prs.sh | sed -n '1,220p'
printf '\n== java-related CI references ==\n'
rg -n "test_java\.sh|java test|RAPIDS_BUILD_TYPE|pull-request" ci .github -g '!**/build/**'Repository: rapidsai/cudf Length of output: 9454 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== .github/workflows/test.yaml around ci/test_java.sh ==\n'
sed -n '130,175p' .github/workflows/test.yaml | cat -n
printf '\n== .github/workflows/pr.yaml around ci/test_java.sh ==\n'
sed -n '470,510p' .github/workflows/pr.yaml | cat -nRepository: rapidsai/cudf Length of output: 4034 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n== .github/workflows/test.yaml top ==\n'
sed -n '1,90p' .github/workflows/test.yaml | cat -n
printf '\n== invocations of test.yaml and build_type assignments ==\n'
rg -n "build_type: (branch|nightly|pull-request)|uses: .*\/\.github\/workflows\/test\.yaml@|uses: .*test\.yaml" .github/workflows -g '!**/build/**'Repository: rapidsai/cudf Length of output: 3749 Guard the PR-only conda bootstrap in 🧰 Tools🪛 Shellcheck (0.11.0)[info] 13-13: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| rapids-logger "Downloading artifacts from previous jobs" | ||
| CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")") | ||
|
|
||
|
|
@@ -20,6 +23,7 @@ rapids-dependency-file-generator \ | |
| --output conda \ | ||
| --file-key test_java \ | ||
| --prepend-channel "${CPP_CHANNEL}" \ | ||
| "${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" \ | ||
| --matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" | tee "${ENV_YAML_DIR}/env.yaml" | ||
|
|
||
| rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n test | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2020-2026, NVIDIA CORPORATION. | ||
| # SPDX-FileCopyrightText: Copyright (c) 2020-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| set -euo pipefail | ||
|
|
@@ -9,6 +9,9 @@ set -euo pipefail | |
| rapids-logger "Configuring conda strict channel priority" | ||
| conda config --set channel_priority strict | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_conda_packages_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n## ci/test_notebooks.sh\n'
cat -n ci/test_notebooks.sh | sed -n '1,120p'
printf '\n## Search RAPIDS_BUILD_TYPE and use_conda_packages_from_prs.sh\n'
rg -n "RAPIDS_BUILD_TYPE|use_conda_packages_from_prs\.sh" ci -g '*.sh' -g '*.bash'
printf '\n## Related integration test script\n'
if [ -f ci/cudf_pandas_scripts/third-party-integration/test.sh ]; then
cat -n ci/cudf_pandas_scripts/third-party-integration/test.sh | sed -n '1,220p'
fiRepository: rapidsai/cudf Length of output: 9383 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n## ci/use_conda_packages_from_prs.sh\n'
cat -n ci/use_conda_packages_from_prs.sh | sed -n '1,220p'
printf '\n## Search workflow/job references to ci/test_notebooks.sh\n'
rg -n "test_notebooks\.sh|test_notebooks" .github ci -g '*.yml' -g '*.yaml' -g '*.sh'
printf '\n## Search for RAPIDS_BUILD_TYPE setup in notebook-related CI\n'
rg -n "RAPIDS_BUILD_TYPE" .github ci -g '*.yml' -g '*.yaml' -g '*.sh'Repository: rapidsai/cudf Length of output: 3483 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n## .github/workflows/test.yaml relevant section\n'
sed -n '130,210p' .github/workflows/test.yaml
printf '\n## .github/workflows/pr.yaml relevant section around test_notebooks\n'
sed -n '200,240p' .github/workflows/pr.yaml
printf '\n## .github/workflows/pr.yaml relevant job section around test_notebooks script\n'
sed -n '490,525p' .github/workflows/pr.yamlRepository: rapidsai/cudf Length of output: 5731 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '\n## top of .github/workflows/test.yaml\n'
sed -n '1,120p' .github/workflows/test.yaml
printf '\n## references to conda-notebook-tests\n'
rg -n "conda-notebook-tests|build_type:" .github/workflows -g '*.yml' -g '*.yaml'
printf '\n## search for workflow_call / workflow_dispatch in test.yaml\n'
rg -n "workflow_call|workflow_dispatch|schedule|push:" .github/workflows/test.yamlRepository: rapidsai/cudf Length of output: 9082 Guard PR-only conda package sourcing in notebook tests. 🧰 Tools🪛 Shellcheck (0.11.0)[info] 13-13: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| rapids-logger "Downloading artifacts from previous jobs" | ||
| CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")") | ||
| PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cudf cudf --stable --cuda "$RAPIDS_CUDA_VERSION")") | ||
|
|
@@ -24,6 +27,7 @@ rapids-dependency-file-generator \ | |
| --prepend-channel "${CPP_CHANNEL}" \ | ||
| --prepend-channel "${PYTHON_CHANNEL}" \ | ||
| --prepend-channel "${PYTHON_NOARCH_CHANNEL}" \ | ||
| "${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" \ | ||
| --matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee "${ENV_YAML_DIR}/env.yaml" | ||
|
|
||
| rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n test | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION. | ||
| # SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # Common setup steps shared by Python test jobs | ||
|
|
@@ -11,6 +11,9 @@ set -euo pipefail | |
| rapids-logger "Configuring conda strict channel priority" | ||
| conda config --set channel_priority strict | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_conda_packages_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Same unguarded PR-artifact fetch, shared across all Python test jobs. Since this is common setup sourced by all Python test jobs (per the file's own comment), unconditionally sourcing 🧰 Tools🪛 Shellcheck (0.11.0)[info] 15-15: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| rapids-logger "Downloading artifacts from previous jobs" | ||
| CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")") | ||
| PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cudf cudf --stable --cuda "$RAPIDS_CUDA_VERSION")") | ||
|
|
@@ -30,7 +33,16 @@ done | |
| CMD="${CMD} \ | ||
| --prepend-channel \"${CPP_CHANNEL}\" \ | ||
| --prepend-channel \"${PYTHON_CHANNEL}\" \ | ||
| --prepend-channel \"${PYTHON_NOARCH_CHANNEL}\" \ | ||
| --prepend-channel \"${PYTHON_NOARCH_CHANNEL}\"" | ||
|
|
||
| # TODO: Remove before merging. Prepend rapidsmpf PR channels (see | ||
| # ci/use_conda_packages_from_prs.sh) ahead of the default channels in the | ||
| # generated env file so the solver prefers them over rapidsai-nightly. | ||
| for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"; do | ||
| CMD="${CMD} --prepend-channel \"${_channel}\"" | ||
| done | ||
|
|
||
| CMD="${CMD} \ | ||
| --matrix \"cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};dependencies=${RAPIDS_DEPENDENCIES}\"" | ||
|
|
||
| eval ${CMD} | tee "${ENV_YAML_DIR}/env.yaml" | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,6 +20,9 @@ CUDF_STREAMING_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name | |
| # generate constraints (possibly pinning to oldest support versions of dependencies) | ||
| rapids-generate-pip-constraints py_test_cudf_polars "${PIP_CONSTRAINT}" | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_wheels_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Same unguarded PR-only wheel fetch pattern. Sourcing 🧰 Tools🪛 Shellcheck (0.11.0)[info] 24-24: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| read -r -a VERSIONS <<< "$(python ci/utils/get_matrix_values.py dependencies.yaml test_cudf_polars_compat polars_compat_version)" | ||
|
|
||
| if [[ "${POLARS_VERSIONS:-all}" == "endpoints" ]] && [[ ${#VERSIONS[@]} -ge 2 ]]; then | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION. | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| set -euo pipefail | ||
|
|
@@ -17,6 +17,9 @@ PYLIBCUDF_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel | |
| # generate constraints (possibly pinning to oldest support versions of dependencies) | ||
| rapids-generate-pip-constraints py_test_cudf_streaming "${PIP_CONSTRAINT}" | ||
|
|
||
| # TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081. | ||
| source ./ci/use_wheels_from_prs.sh | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Same unguarded PR-only wheel fetch pattern. Same concern as 🧰 Tools🪛 Shellcheck (0.11.0)[info] 21-21: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x). (SC1091) 🤖 Prompt for AI Agents |
||
| rapids-logger "Install cudf_streaming and its dependencies" | ||
|
|
||
| rapids-pip-retry install \ | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # download CI artifacts | ||
| LIBRAPIDSMPF_CHANNEL=$(rapids-get-pr-artifact rapidsmpf 1081 cpp conda) | ||
| RAPIDSMPF_CHANNEL=$(rapids-get-pr-artifact rapidsmpf 1081 python conda) | ||
|
|
||
| # For `rattler` builds: | ||
| # | ||
| # Add these channels to the array checked by 'rapids-rattler-channel-string'. | ||
| # This ensures that when conda packages are built with strict channel priority enabled, | ||
| # the locally-downloaded packages will be preferred to remote packages (e.g. nightlies). | ||
| # | ||
| RAPIDS_PREPENDED_CONDA_CHANNELS=( | ||
| "${LIBRAPIDSMPF_CHANNEL}" | ||
| "${RAPIDSMPF_CHANNEL}" | ||
| ) | ||
| export RAPIDS_PREPENDED_CONDA_CHANNELS | ||
|
|
||
| # For tests and `conda-build` builds: | ||
| # | ||
| # Add these channels to the system-wide conda configuration. | ||
| # This results in PREPENDING them to conda's channel list, so | ||
| # these packages should be found first if strict channel priority is enabled. | ||
| # | ||
| for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}" | ||
| do | ||
| conda config --system --add channels "${_channel}" | ||
| done | ||
|
|
||
| # For `conda env create -f <env.yaml>` jobs (e.g. C++/Python/Java/notebook | ||
| # tests, docs builds): | ||
| # | ||
| # `conda env create` gives the channels listed in the environment file a HIGHER | ||
| # priority than the channels in the system/user `.condarc`. So the | ||
| # `conda config --system --add channels` calls above are NOT sufficient on their | ||
| # own: with strict channel priority the solver still prefers e.g. | ||
| # `rapidsai-nightly` (which is listed in the generated env file) over these PR | ||
| # channels, pulling in the wrong build of `librapidsmpf`/`rapidsmpf`. | ||
| # | ||
| # Expose the channels as `--prepend-channel` arguments so callers can inject | ||
| # them into the generated environment file ahead of the default channels. | ||
| RAPIDS_PREPENDED_CHANNEL_ARGS=() | ||
| for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}" | ||
| do | ||
| RAPIDS_PREPENDED_CHANNEL_ARGS+=(--prepend-channel "${_channel}") | ||
| done | ||
| export RAPIDS_PREPENDED_CHANNEL_ARGS |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # NOTE: This script assumes the caller has already run `source rapids-init-pip` | ||
| # (so PIP_CONSTRAINT is set) and has finished generating any of its own | ||
| # constraints. It APPENDS to PIP_CONSTRAINT so it must not be sourced before | ||
| # `rapids-generate-pip-constraints`, which overwrites that file. | ||
|
|
||
| RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")" | ||
|
|
||
| # download wheels, store the directories holding them in variables | ||
| LIBRAPIDSMPF_WHEELHOUSE=$(rapids-get-pr-artifact rapidsmpf 1081 cpp wheel) | ||
| RAPIDSMPF_WHEELHOUSE=$(rapids-get-pr-artifact rapidsmpf 1081 python wheel) | ||
|
|
||
| # append to the pip constraints file saying e.g. "whenever you encounter a | ||
| # requirement for 'librapidsmpf-cu12', use this wheel" | ||
| cat >> "${PIP_CONSTRAINT}" <<EOF | ||
| librapidsmpf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBRAPIDSMPF_WHEELHOUSE}"/librapidsmpf_*.whl) | ||
| rapidsmpf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${RAPIDSMPF_WHEELHOUSE}"/rapidsmpf_*.whl) | ||
| EOF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please disregard all changes in .sh files. They will be reverted.