rapids-rattler-channel-string: add support for prepending channels

[jameslamb]

Contributes to https://github.com/rapidsai/build-infra/issues/237

Contributes to rapidsai/build-planning#14

Splitting this off from #/173

Overview

As of this PR, if a bash array RAPIDS_PREPENDED_CONDA_CHANNELS is detected whenever rapids-rattler-channel-string is sourced, it prepends each element of that array the channel list passed to rattler-build (RATTLER_CHANNELS).

Notes for Reviewers

Benefits of these changes

Helps with simplifying "use packages from one PR in another PR's CI" (related: rapidsai/docs#601).

With this, you could create a script like ci/use_conda_packages_from_prs.sh and source it at the top of every ci/build*.sh, without any other changes to ci/build*.sh scripts required:

# download CI artifacts
LIBRAFT_CHANNEL=$(rapids-get-pr-conda-artifact raft 789 python)
LIBRMM_CHANNEL=$(rapids-get-pr-conda-artifact rmm 1909 cpp)

# For `rattler` builds:
#
# Add these channels to the array checked by 'rapids-rattler-channel-string'.
RAPIDS_PREPENDED_CONDA_CHANNELS=(
    "${LIBRAFT_CHANNEL}"
    "${LIBRMM_CHANNEL}"
)
export RAPIDS_PREPENDED_CONDA_CHANNELS

# For tests and `conda-build` builds:
#
# Add these channels to the system-wide conda configuration.
for _channel in "${RAPIDS_PREPENDED_CONDA_CHANNELS[@]}"
do
   conda config --system --add channels "${_channel}"
done

More Details

rapids-rattler-channel-string (first added in #143) defines a bash array, RATTLER_CHANNELS, containing a list of conda channels to be used when building the build/host/test environments in rattler-build builds. Like this:

source rapids-rattler-channel-string
rattler-build build --recipe conda/recipes/cuml \
                    "${RATTLER_ARGS[@]}" \
                    "${RATTLER_CHANNELS[@]}"

(cuml code link)

The ordering in that channel list is important... for any projects using conda "strict" channel priority, those channels are searched in order for packages, and the first package found is used.

When testing something like "use packages from this cuvs PR in CI on this cuml PR", it's desirable to download the upstream PR's CI artifacts and tell rattler-build to use them. One useful mechanism for that is to put the "channels" (local directories) holding those downloaded files into the channel list. It's important that you be able to prepend them, so strict channel priority will result in the locally-downloaded packages being chosen.

This change makes that possible for rattler-build builds (see above and rapidsai/docs#601).

How I tested this

Tested this as part of #173, via NVIDIA/cudf#18747

Evidence that it worked: NVIDIA/cudf#18747 (comment)

[gforsyth]

One typo, but this looks great, approving so you can merge at your discretion!

[gforsyth]

Thanks for documenting this!!

[jameslamb]

No prob, no prob 😊