Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion ci/build_cpp.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please disregard all changes in .sh files. They will be reverted.

# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand All @@ -22,6 +22,9 @@ RAPIDS_ARTIFACTS_DIR=${RAPIDS_ARTIFACTS_DIR:-"${PWD}/artifacts"}
mkdir -p "${RAPIDS_ARTIFACTS_DIR}"
export RAPIDS_ARTIFACTS_DIR

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
# populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array
source rapids-rattler-channel-string

Expand Down
6 changes: 5 additions & 1 deletion ci/build_docs.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand All @@ -15,6 +15,9 @@ rapids-logger "Create test conda environment"
rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Same unguarded PR-artifact fetch as other build scripts.

Same concern as flagged in ci/build_python_noarch.sh: sourcing this unconditionally will attempt to fetch rapidsmpf#1081 artifacts even on non-PR (branch/nightly) runs of this job.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 19-19: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/build_docs.sh` around lines 18 - 20, The unconditional source call in the
build_docs script will try to fetch PR artifacts even on branch/nightly runs.
Update the logic around the ci/use_conda_packages_from_prs.sh sourcing to match
the guarded pattern used in ci/build_python_noarch.sh, using the same job/PR
condition check so it only runs for PR builds and skips artifact fetching
otherwise.

ENV_YAML_DIR="$(mktemp -d)"

rapids-logger "Downloading artifacts from previous jobs"
Expand All @@ -28,6 +31,7 @@ rapids-dependency-file-generator \
--prepend-channel "${CPP_CHANNEL}" \
--prepend-channel "${PYTHON_CHANNEL}" \
--prepend-channel "${PYTHON_NOARCH_CHANNEL}" \
"${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee "${ENV_YAML_DIR}/env.yaml"

rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n docs
Expand Down
3 changes: 3 additions & 0 deletions ci/build_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libc
RAPIDS_PACKAGE_VERSION=$(head -1 ./VERSION)
export RAPIDS_PACKAGE_VERSION

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Same unguarded PR-only artifact-fetch pattern.

Same concern as the sibling build scripts: this unconditionally sources ci/use_conda_packages_from_prs.sh, which will fail on non-PR (branch/nightly) runs of this job since it fetches hardcoded rapidsmpf#1081 artifacts.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 25-25: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/build_python.sh` around lines 24 - 26, The build_python.sh setup
unconditionally sources ci/use_conda_packages_from_prs.sh, which will break
non-PR runs by trying to fetch PR-only artifacts. Guard this logic the same way
as the other build scripts by checking the PR context before sourcing, and keep
the existing artifact helper invocation under that condition so branch/nightly
jobs skip it. Use the ci/use_conda_packages_from_prs.sh call site in the
build_python.sh flow as the target for the fix.

# populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array
source rapids-rattler-channel-string

Expand Down
5 changes: 4 additions & 1 deletion ci/build_python_noarch.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand All @@ -21,6 +21,9 @@ PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_pytho
RAPIDS_PACKAGE_VERSION=$(head -1 ./VERSION)
export RAPIDS_PACKAGE_VERSION

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
# populates `RATTLER_CHANNELS` array and `RATTLER_ARGS` array
source rapids-rattler-channel-string

Expand Down
3 changes: 3 additions & 0 deletions ci/build_wheel_cudf_streaming.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ echo "libcudf-streaming-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUDF_STRE
echo "libcudf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUDF_WHEELHOUSE}"/libcudf_*.whl)" >> "${PIP_CONSTRAINT}"
echo "pylibcudf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${PYLIBCUDF_WHEELHOUSE}"/pylibcudf_*.whl)" >> "${PIP_CONSTRAINT}"

# TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081.
source ./ci/use_wheels_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
rg -n 'build_wheel_cudf_streaming\.sh' .github/workflows -g '*.yml' -g '*.yaml'
rg -n 'build_type' .github/workflows -g '*.yml' -g '*.yaml' -B5 -A5 | rg -n 'cudf.streaming' -B10
cat -n ci/use_wheels_from_prs.sh 2>/dev/null | sed -n '1,80p'

Repository: rapidsai/cudf

Length of output: 10771


Gate PR-only wheel sourcing behind RAPIDS_BUILD_TYPE == "pull-request". ci/build_wheel_cudf_streaming.sh is also used by branch/nightly wheel builds, but source ./ci/use_wheels_from_prs.sh depends on PR artifacts. Wrap it in the same guard used elsewhere so non-PR builds don’t fail.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 25-25: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/build_wheel_cudf_streaming.sh` around lines 24 - 26, Gate the PR-only
wheel sourcing in build_wheel_cudf_streaming.sh behind the same
RAPIDS_BUILD_TYPE == "pull-request" check used elsewhere, since source
./ci/use_wheels_from_prs.sh only works with PR artifacts. Update the script
around the use_wheels_from_prs.sh import so branch and nightly builds skip it
and only pull-request builds execute it.

rapids-logger "Generating build requirements"

rapids-dependency-file-generator \
Expand Down
3 changes: 3 additions & 0 deletions ci/build_wheel_libcudf_streaming.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"
LIBCUDF_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")")
echo "libcudf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBCUDF_WHEELHOUSE}"/libcudf_*.whl)" >> "${PIP_CONSTRAINT}"

# TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081.
source ./ci/use_wheels_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
rg -n 'build_wheel_libcudf_streaming\.sh' .github/workflows -g '*.yml' -g '*.yaml'

Repository: rapidsai/cudf

Length of output: 316


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the wheel script and its workflow callers.
printf '\n== ci/build_wheel_libcudf_streaming.sh ==\n'
cat -n ci/build_wheel_libcudf_streaming.sh | sed -n '1,120p'

printf '\n== .github/workflows/build.yaml around caller ==\n'
sed -n '170,230p' .github/workflows/build.yaml

printf '\n== .github/workflows/pr.yaml around caller ==\n'
sed -n '530,610p' .github/workflows/pr.yaml

Repository: rapidsai/cudf

Length of output: 8271


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the related wheel script for comparison.
printf '\n== ci/build_wheel_cudf_streaming.sh ==\n'
cat -n ci/build_wheel_cudf_streaming.sh | sed -n '1,120p'

Repository: rapidsai/cudf

Length of output: 4072


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find where use_wheels_from_prs.sh is sourced and whether it's guarded.
rg -n 'use_wheels_from_prs\.sh|build_type|nightly|branch|pull_request|PR' ci .github/workflows -g '*.sh' -g '*.yml' -g '*.yaml'

Repository: rapidsai/cudf

Length of output: 21243


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== ci/build_wheel_libcudf_streaming.sh ==\n'
cat -n ci/build_wheel_libcudf_streaming.sh | sed -n '1,120p'

printf '\n== ci/use_wheels_from_prs.sh ==\n'
cat -n ci/use_wheels_from_prs.sh | sed -n '1,200p'

printf '\n== .github/workflows/build.yaml top section ==\n'
sed -n '1,120p' .github/workflows/build.yaml

printf '\n== .github/workflows/pr.yaml top section ==\n'
sed -n '1,120p' .github/workflows/pr.yaml

Repository: rapidsai/cudf

Length of output: 11617


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
for path, start, end in [
    (Path('.github/workflows/build.yaml'), 180, 220),
    (Path('.github/workflows/pr.yaml'), 540, 590),
]:
    print(f"\n== {path} {start}-{end} ==")
    lines = path.read_text().splitlines()
    for i in range(start-1, min(end, len(lines))):
        print(f"{i+1}: {lines[i]}")
PY

Repository: rapidsai/cudf

Length of output: 4366


Guard ci/use_wheels_from_prs.sh behind PR builds. ci/build_wheel_libcudf_streaming.sh runs in both build.yaml and pr.yaml, but ci/use_wheels_from_prs.sh hardcodes PR artifact 1081. Unconditional sourcing will break branch/nightly wheel builds when those artifacts don’t exist.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 21-21: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/build_wheel_libcudf_streaming.sh` around lines 20 - 22, The unconditional
source of ci/use_wheels_from_prs.sh in build_wheel_libcudf_streaming.sh will
break non-PR wheel builds because it assumes PR artifact 1081 exists. Update the
script to only source ci/use_wheels_from_prs.sh when running in PR builds, and
leave branch/nightly builds to use the normal wheel sources; use the
build_wheel_libcudf_streaming.sh entrypoint and its existing PR context checks
to gate this behavior.

rapids-logger "Generating build requirements"

rapids-dependency-file-generator \
Expand Down
8 changes: 6 additions & 2 deletions ci/cudf_pandas_scripts/third-party-integration/test.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Common setup steps shared by Python test jobs
Expand Down Expand Up @@ -27,6 +27,9 @@ main() {
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cudf cudf --stable --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_NOARCH_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cudf cudf --pure --cuda "$RAPIDS_CUDA_VERSION" --arch any)")

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh
fi

ANY_FAILURES=0
Expand All @@ -46,7 +49,8 @@ main() {
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" \
--prepend-channel "${CPP_CHANNEL}" \
--prepend-channel "${PYTHON_CHANNEL}" \
--prepend-channel "${PYTHON_NOARCH_CHANNEL}" | tee env.yaml
--prepend-channel "${PYTHON_NOARCH_CHANNEL}" \
"${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" | tee env.yaml
else
rapids-logger "Generate Python testing dependencies"
rapids-dependency-file-generator \
Expand Down
6 changes: 5 additions & 1 deletion ci/test_cpp_common.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand All @@ -9,6 +9,9 @@ set -euo pipefail
rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")")

Expand All @@ -20,6 +23,7 @@ rapids-dependency-file-generator \
--output conda \
--file-key test_cpp \
--prepend-channel "${CPP_CHANNEL}" \
"${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" | tee "${ENV_YAML_DIR}/env.yaml"

rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n test
Expand Down
5 changes: 4 additions & 1 deletion ci/test_cudf_polars_polars_tests.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand All @@ -18,6 +18,9 @@ CUDF_POLARS_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name whe
# generate constraints (possibly pinning to oldest support versions of dependencies)
rapids-generate-pip-constraints py_test_cudf_polars "${PIP_CONSTRAINT}"

# TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081.
source ./ci/use_wheels_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
rg -n 'test_cudf_polars_polars_tests\.sh' .github/workflows -g '*.yml' -g '*.yaml'

Repository: rapidsai/cudf

Length of output: 317


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the polars test script and the PR/test workflow jobs that call it.
sed -n '1,120p' ci/test_cudf_polars_polars_tests.sh
printf '\n--- pr.yaml excerpt ---\n'
sed -n '660,730p' .github/workflows/pr.yaml
printf '\n--- test.yaml excerpt ---\n'
sed -n '260,320p' .github/workflows/test.yaml
printf '\n--- helper script ---\n'
sed -n '1,200p' ci/use_wheels_from_prs.sh

Repository: rapidsai/cudf

Length of output: 9684


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- test.yaml top ---'
sed -n '1,120p' .github/workflows/test.yaml
printf '\n%s\n' '--- pr.yaml top ---'
sed -n '1,120p' .github/workflows/pr.yaml
printf '\n%s\n' '--- references to test.yaml ---'
rg -n 'test\.yaml@main|build_type: .*branch|build_type: .*nightly|build_type: .*pull-request|cudf-polars-polars-tests' .github/workflows ci -g '*.yml' -g '*.yaml' -g '*.sh'

Repository: rapidsai/cudf

Length of output: 12440


Guard the PR-only wheel sourcing behind a build-type check ci/test_cudf_polars_polars_tests.sh:21-23 sources ./ci/use_wheels_from_prs.sh unconditionally, but .github/workflows/test.yaml can run this job with build_type: branch or nightly. Those runs will still try to fetch PR 1081 artifacts and can fail outside pull requests.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 22-22: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_cudf_polars_polars_tests.sh` around lines 21 - 23, The PR-only wheel
sourcing in ci/test_cudf_polars_polars_tests.sh is unconditional, so branch and
nightly runs can still try to fetch PR artifacts. Update the script around the
source ./ci/use_wheels_from_prs.sh call to only run when the build type
indicates a pull request, using the existing build_type handling or equivalent
check in this script. Keep the PR-wheel setup isolated to the same conditional
path that already distinguishes build_type values like branch and nightly.

rapids-logger "Install libcudf, pylibcudf and cudf_polars"

# notes:
Expand Down
4 changes: 4 additions & 0 deletions ci/test_java.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ set -euo pipefail
rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== ci/test_java.sh ==\n'
cat -n ci/test_java.sh | sed -n '1,120p'

printf '\n== ci/cudf_pandas_scripts/third-party-integration/test.sh ==\n'
cat -n ci/cudf_pandas_scripts/third-party-integration/test.sh | sed -n '1,120p'

printf '\n== references to use_conda_packages_from_prs.sh ==\n'
rg -n "use_conda_packages_from_prs\.sh|RAPIDS_BUILD_TYPE" ci -g '!**/build/**'

Repository: rapidsai/cudf

Length of output: 7947


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== ci/use_conda_packages_from_prs.sh ==\n'
cat -n ci/use_conda_packages_from_prs.sh | sed -n '1,220p'

printf '\n== java-related CI references ==\n'
rg -n "test_java\.sh|java test|RAPIDS_BUILD_TYPE|pull-request" ci .github -g '!**/build/**'

Repository: rapidsai/cudf

Length of output: 9454


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== .github/workflows/test.yaml around ci/test_java.sh ==\n'
sed -n '130,175p' .github/workflows/test.yaml | cat -n

printf '\n== .github/workflows/pr.yaml around ci/test_java.sh ==\n'
sed -n '470,510p' .github/workflows/pr.yaml | cat -n

Repository: rapidsai/cudf

Length of output: 4034


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== .github/workflows/test.yaml top ==\n'
sed -n '1,90p' .github/workflows/test.yaml | cat -n

printf '\n== invocations of test.yaml and build_type assignments ==\n'
rg -n "build_type: (branch|nightly|pull-request)|uses: .*\/\.github\/workflows\/test\.yaml@|uses: .*test\.yaml" .github/workflows -g '!**/build/**'

Repository: rapidsai/cudf

Length of output: 3749


Guard the PR-only conda bootstrap in ci/test_java.sh — test.yaml runs this job for branch and nightly too, but ci/use_conda_packages_from_prs.sh always calls rapids-get-pr-artifact, so non-PR Java runs can fail here. Add the same RAPIDS_BUILD_TYPE == pull-request guard used in ci/cudf_pandas_scripts/third-party-integration/test.sh.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 13-13: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_java.sh` around lines 12 - 14, The PR-only conda bootstrap in
test_java.sh is being sourced unconditionally, which can break branch and
nightly runs because it depends on PR artifacts. Add the same RAPIDS_BUILD_TYPE
pull-request guard used in
ci/cudf_pandas_scripts/third-party-integration/test.sh around the source of
ci/use_conda_packages_from_prs.sh so it only runs for PR builds.

rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")")

Expand All @@ -20,6 +23,7 @@ rapids-dependency-file-generator \
--output conda \
--file-key test_java \
--prepend-channel "${CPP_CHANNEL}" \
"${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" | tee "${ENV_YAML_DIR}/env.yaml"

rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n test
Expand Down
6 changes: 5 additions & 1 deletion ci/test_notebooks.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2020-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2020-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand All @@ -9,6 +9,9 @@ set -euo pipefail
rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## ci/test_notebooks.sh\n'
cat -n ci/test_notebooks.sh | sed -n '1,120p'

printf '\n## Search RAPIDS_BUILD_TYPE and use_conda_packages_from_prs.sh\n'
rg -n "RAPIDS_BUILD_TYPE|use_conda_packages_from_prs\.sh" ci -g '*.sh' -g '*.bash'

printf '\n## Related integration test script\n'
if [ -f ci/cudf_pandas_scripts/third-party-integration/test.sh ]; then
  cat -n ci/cudf_pandas_scripts/third-party-integration/test.sh | sed -n '1,220p'
fi

Repository: rapidsai/cudf

Length of output: 9383


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## ci/use_conda_packages_from_prs.sh\n'
cat -n ci/use_conda_packages_from_prs.sh | sed -n '1,220p'

printf '\n## Search workflow/job references to ci/test_notebooks.sh\n'
rg -n "test_notebooks\.sh|test_notebooks" .github ci -g '*.yml' -g '*.yaml' -g '*.sh'

printf '\n## Search for RAPIDS_BUILD_TYPE setup in notebook-related CI\n'
rg -n "RAPIDS_BUILD_TYPE" .github ci -g '*.yml' -g '*.yaml' -g '*.sh'

Repository: rapidsai/cudf

Length of output: 3483


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## .github/workflows/test.yaml relevant section\n'
sed -n '130,210p' .github/workflows/test.yaml

printf '\n## .github/workflows/pr.yaml relevant section around test_notebooks\n'
sed -n '200,240p' .github/workflows/pr.yaml
printf '\n## .github/workflows/pr.yaml relevant job section around test_notebooks script\n'
sed -n '490,525p' .github/workflows/pr.yaml

Repository: rapidsai/cudf

Length of output: 5731


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## top of .github/workflows/test.yaml\n'
sed -n '1,120p' .github/workflows/test.yaml

printf '\n## references to conda-notebook-tests\n'
rg -n "conda-notebook-tests|build_type:" .github/workflows -g '*.yml' -g '*.yaml'

printf '\n## search for workflow_call / workflow_dispatch in test.yaml\n'
rg -n "workflow_call|workflow_dispatch|schedule|push:" .github/workflows/test.yaml

Repository: rapidsai/cudf

Length of output: 9082


Guard PR-only conda package sourcing in notebook tests. ci/test_notebooks.sh always sources ci/use_conda_packages_from_prs.sh, but .github/workflows/test.yaml runs this job with build_type values like nightly and branch too. That helper is PR-specific and will try to pull PR artifacts in non-PR runs, so gate it on RAPIDS_BUILD_TYPE == "pull-request" like the third-party integration test does.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 13-13: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_notebooks.sh` around lines 12 - 14, The notebook test script
unconditionally sources the PR-only package helper, which breaks non-PR runs
like nightly and branch builds. Update ci/test_notebooks.sh to guard the source
of use_conda_packages_from_prs.sh behind a check that RAPIDS_BUILD_TYPE is
"pull-request", matching the pattern used by the third-party integration test,
so the helper only runs for PR builds.

rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cudf cudf --stable --cuda "$RAPIDS_CUDA_VERSION")")
Expand All @@ -24,6 +27,7 @@ rapids-dependency-file-generator \
--prepend-channel "${CPP_CHANNEL}" \
--prepend-channel "${PYTHON_CHANNEL}" \
--prepend-channel "${PYTHON_NOARCH_CHANNEL}" \
"${RAPIDS_PREPENDED_CHANNEL_ARGS[@]}" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee "${ENV_YAML_DIR}/env.yaml"

rapids-mamba-retry env create --yes -f "${ENV_YAML_DIR}/env.yaml" -n test
Expand Down
16 changes: 14 additions & 2 deletions ci/test_python_common.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Common setup steps shared by Python test jobs
Expand All @@ -11,6 +11,9 @@ set -euo pipefail
rapids-logger "Configuring conda strict channel priority"
conda config --set channel_priority strict

# TODO: Remove before merging. Use rapidsmpf conda packages from rapidsai/rapidsmpf#1081.
source ./ci/use_conda_packages_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Same unguarded PR-artifact fetch, shared across all Python test jobs.

Since this is common setup sourced by all Python test jobs (per the file's own comment), unconditionally sourcing ci/use_conda_packages_from_prs.sh here risks breaking every non-PR (branch/nightly) Python test job that calls this script, not just one.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 15-15: Not following: ./ci/use_conda_packages_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_python_common.sh` around lines 14 - 16, The shared Python test setup
currently sources ci/use_conda_packages_from_prs.sh unconditionally, which
affects every job that uses this script. Update the shared setup in
test_python_common.sh to only source that helper when running PR-based jobs,
using the existing job/context detection logic already available in the Python
test flow. Keep the change localized around the sourcing step so branch and
nightly jobs skip the PR-artifact fetch while PR jobs still use it.

rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_cpp libcudf cudf --cuda "$RAPIDS_CUDA_VERSION")")
PYTHON_CHANNEL=$(rapids-download-from-github "$(rapids-artifact-name conda_python cudf cudf --stable --cuda "$RAPIDS_CUDA_VERSION")")
Expand All @@ -30,7 +33,16 @@ done
CMD="${CMD} \
--prepend-channel \"${CPP_CHANNEL}\" \
--prepend-channel \"${PYTHON_CHANNEL}\" \
--prepend-channel \"${PYTHON_NOARCH_CHANNEL}\" \
--prepend-channel \"${PYTHON_NOARCH_CHANNEL}\""

# TODO: Remove before merging. Prepend rapidsmpf PR channels (see
# ci/use_conda_packages_from_prs.sh) ahead of the default channels in the
# generated env file so the solver prefers them over rapidsai-nightly.
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"; do
CMD="${CMD} --prepend-channel \"${_channel}\""
done

CMD="${CMD} \
--matrix \"cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};dependencies=${RAPIDS_DEPENDENCIES}\""

eval ${CMD} | tee "${ENV_YAML_DIR}/env.yaml"
Expand Down
3 changes: 3 additions & 0 deletions ci/test_wheel_cudf_polars.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ CUDF_STREAMING_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name
# generate constraints (possibly pinning to oldest support versions of dependencies)
rapids-generate-pip-constraints py_test_cudf_polars "${PIP_CONSTRAINT}"

# TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081.
source ./ci/use_wheels_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Same unguarded PR-only wheel fetch pattern.

Sourcing ci/use_wheels_from_prs.sh unconditionally will attempt to fetch rapidsmpf#1081 wheel artifacts even on non-PR (branch/nightly) runs of this test job, mirroring the conda-helper concern raised elsewhere in this stack.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 24-24: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_wheel_cudf_polars.sh` around lines 23 - 25, The wheel-fetch setup in
the test script is unguarded and always sources ci/use_wheels_from_prs.sh, which
will try to pull PR-only artifacts on non-PR runs. Update the logic around the
ci/use_wheels_from_prs.sh invocation to only run when the job is executing for a
PR, using the same gating approach as other PR-only wheel helpers in this stack,
so branch and nightly runs skip the rapidsmpf#1081 fetch path.

read -r -a VERSIONS <<< "$(python ci/utils/get_matrix_values.py dependencies.yaml test_cudf_polars_compat polars_compat_version)"

if [[ "${POLARS_VERSIONS:-all}" == "endpoints" ]] && [[ ${#VERSIONS[@]} -ge 2 ]]; then
Expand Down
5 changes: 4 additions & 1 deletion ci/test_wheel_cudf_streaming.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail
Expand All @@ -17,6 +17,9 @@ PYLIBCUDF_WHEELHOUSE=$(rapids-download-from-github "$(rapids-artifact-name wheel
# generate constraints (possibly pinning to oldest support versions of dependencies)
rapids-generate-pip-constraints py_test_cudf_streaming "${PIP_CONSTRAINT}"

# TODO: Remove before merging. Use rapidsmpf wheels from rapidsai/rapidsmpf#1081.
source ./ci/use_wheels_from_prs.sh

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Same unguarded PR-only wheel fetch pattern.

Same concern as ci/test_wheel_cudf_polars.sh: unconditionally sourcing ci/use_wheels_from_prs.sh will fail to find rapidsmpf#1081 wheel artifacts on non-PR (branch/nightly) test runs.

🧰 Tools
🪛 Shellcheck (0.11.0)

[info] 21-21: Not following: ./ci/use_wheels_from_prs.sh was not specified as input (see shellcheck -x).

(SC1091)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test_wheel_cudf_streaming.sh` around lines 20 - 22, The unguarded source
of use_wheels_from_prs.sh in the streaming wheel test script can break non-PR
branch/nightly runs by trying to fetch rapidsmpf#1081 artifacts that do not
exist there. Update the test setup so the wheel fetch in
test_wheel_cudf_streaming.sh is only invoked for PR builds, following the same
guard pattern used in the related test_wheel_cudf_polars.sh flow, and keep the
conditional centered around the use_wheels_from_prs.sh call.

rapids-logger "Install cudf_streaming and its dependencies"

rapids-pip-retry install \
Expand Down
49 changes: 49 additions & 0 deletions ci/use_conda_packages_from_prs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# download CI artifacts
LIBRAPIDSMPF_CHANNEL=$(rapids-get-pr-artifact rapidsmpf 1081 cpp conda)
RAPIDSMPF_CHANNEL=$(rapids-get-pr-artifact rapidsmpf 1081 python conda)

# For `rattler` builds:
#
# Add these channels to the array checked by 'rapids-rattler-channel-string'.
# This ensures that when conda packages are built with strict channel priority enabled,
# the locally-downloaded packages will be preferred to remote packages (e.g. nightlies).
#
RAPIDS_PREPENDED_CONDA_CHANNELS=(
"${LIBRAPIDSMPF_CHANNEL}"
"${RAPIDSMPF_CHANNEL}"
)
export RAPIDS_PREPENDED_CONDA_CHANNELS

# For tests and `conda-build` builds:
#
# Add these channels to the system-wide conda configuration.
# This results in PREPENDING them to conda's channel list, so
# these packages should be found first if strict channel priority is enabled.
#
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
conda config --system --add channels "${_channel}"
done

# For `conda env create -f <env.yaml>` jobs (e.g. C++/Python/Java/notebook
# tests, docs builds):
#
# `conda env create` gives the channels listed in the environment file a HIGHER
# priority than the channels in the system/user `.condarc`. So the
# `conda config --system --add channels` calls above are NOT sufficient on their
# own: with strict channel priority the solver still prefers e.g.
# `rapidsai-nightly` (which is listed in the generated env file) over these PR
# channels, pulling in the wrong build of `librapidsmpf`/`rapidsmpf`.
#
# Expose the channels as `--prepend-channel` arguments so callers can inject
# them into the generated environment file ahead of the default channels.
RAPIDS_PREPENDED_CHANNEL_ARGS=()
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
RAPIDS_PREPENDED_CHANNEL_ARGS+=(--prepend-channel "${_channel}")
done
export RAPIDS_PREPENDED_CHANNEL_ARGS
21 changes: 21 additions & 0 deletions ci/use_wheels_from_prs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# NOTE: This script assumes the caller has already run `source rapids-init-pip`
# (so PIP_CONSTRAINT is set) and has finished generating any of its own
# constraints. It APPENDS to PIP_CONSTRAINT so it must not be sourced before
# `rapids-generate-pip-constraints`, which overwrites that file.

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"

# download wheels, store the directories holding them in variables
LIBRAPIDSMPF_WHEELHOUSE=$(rapids-get-pr-artifact rapidsmpf 1081 cpp wheel)
RAPIDSMPF_WHEELHOUSE=$(rapids-get-pr-artifact rapidsmpf 1081 python wheel)

# append to the pip constraints file saying e.g. "whenever you encounter a
# requirement for 'librapidsmpf-cu12', use this wheel"
cat >> "${PIP_CONSTRAINT}" <<EOF
librapidsmpf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${LIBRAPIDSMPF_WHEELHOUSE}"/librapidsmpf_*.whl)
rapidsmpf-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo "${RAPIDSMPF_WHEELHOUSE}"/rapidsmpf_*.whl)
EOF
10 changes: 6 additions & 4 deletions cpp/libcudf_streaming/benchmarks/bench_shuffle.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
#include <rapidsmpf/bootstrap/bootstrap.hpp>
#include <rapidsmpf/bootstrap/utils.hpp>
#include <rapidsmpf/communicator/communicator.hpp>
#include <rapidsmpf/communicator/logger.hpp>
#include <rapidsmpf/error.hpp>
#include <rapidsmpf/memory/spill.hpp>
#include <rapidsmpf/nvtx.hpp>
Expand Down Expand Up @@ -516,6 +517,8 @@ int main(int argc, char** argv)
auto& stat_enabled_mr = br->device_mr_adaptor();
rmm::mr::set_current_device_resource(stat_enabled_mr);

auto log = rapidsmpf::Logger::from_options(options);

std::shared_ptr<rapidsmpf::Communicator> comm;
auto progress_thread = std::make_shared<rapidsmpf::ProgressThread>(stats);
if (args.comm_type == "mpi") {
Expand All @@ -527,7 +530,7 @@ int main(int argc, char** argv)
return 1;
}
rapidsmpf::mpi::init(&argc, &argv);
comm = std::make_shared<rapidsmpf::MPI>(MPI_COMM_WORLD, options, progress_thread);
comm = std::make_shared<rapidsmpf::MPI>(MPI_COMM_WORLD, progress_thread, log);
#else
std::cerr << "Error: MPI communicator is not available in this build." << std::endl;
return 1;
Expand All @@ -537,11 +540,11 @@ int main(int argc, char** argv)
if (use_bootstrap) {
// Launched with rrun - use bootstrap backend
comm = rapidsmpf::bootstrap::create_ucxx_comm(
progress_thread, rapidsmpf::bootstrap::BackendType::AUTO, options);
progress_thread, rapidsmpf::bootstrap::BackendType::AUTO, options, log);
} else {
#ifdef CUDF_STREAMING_HAVE_MPI
// Launched with mpirun - use MPI bootstrap
comm = rapidsmpf::ucxx::init_using_mpi(MPI_COMM_WORLD, options, progress_thread);
comm = rapidsmpf::ucxx::init_using_mpi(MPI_COMM_WORLD, options, progress_thread, log);
#else
std::cerr << "Error: UCXX without MPI support requires bootstrap mode." << std::endl;
return 1;
Expand All @@ -558,7 +561,6 @@ int main(int argc, char** argv)

args.pprint(*comm);

auto& log = comm->logger();
rmm::cuda_stream_view stream = cudf::get_default_stream();

// Print benchmark/hardware info.
Expand Down
Loading
Loading