update docs on managing CI artifacts

CONTRIBUTING.md

@@ -2,37 +2,31 @@
 
 ## Testing Locally
 
-Before opening a pull-request, it is recommended to test all changes locally.
+Before opening a pull request, it is recommended to test all changes locally.
 
 The command below is a quick way to set up a disposable test environment for the `gha-tools` repository:
 
 ```sh
 docker run \
   --pull=always \
-  --rm -it \
-  --network=host \
+  --rm \
   --gpus all \
-  -v $HOME/.aws:/root/.aws:ro \
-  -v $HOME/gha-tools/tools:/root/.local/bin:ro \
+  -v "${HOME}/repos/gha-tools/tools":/usr/local/bin/gha-tools:ro \
   -v $PWD:/work \
   -w /work \
-  rapidsai/ci-conda:latest
+  -it rapidsai/ci-conda:latest
 ```
 
 This command makes the following assumptions:
 
-- The `gha-tools` repository is checked out to `$HOME/gha-tools`
-- The current working directory is a RAPIDS repository that has artifacts on [downloads.rapids.ai](https://downloads.rapids.ai) (NVIDIA VPN connectivity is required to access this site), like `cugraph`.
+- The `gha-tools` repository is checked out to `${HOME}/repos/gha-tools`
+- The current working directory is a RAPIDS repository that publishes CI artifacts, like `cugraph`.
   - This is important for testing how changes to artifact download scripts will affect local invocations (see note below)
-- The AWS credentials in `$HOME/.aws` have access to the `rapids-downloads` bucket
-  - This is important for testing how changes to artifact download scripts will affect CI (see note below)
-
-> **Note:** CI interacts with S3 directly, whereas local `gha-tools` script invocations get artifacts through [downloads.rapids.ai](https://downloads.rapids.ai).
 
 Once the container above is running, change the `PATH` variable inside of it so that the scripts in the volume mounted `gha-tools` directory take precedence:
 
 ```sh
-export PATH=/root/.local/bin:$PATH
+export PATH="/usr/local/binn/gha-tools/tools:${PATH}"
 ```
 
 Now, the volume mounted scripts will be run any time a `gha-tools` script is invoked.
@@ -45,26 +39,21 @@ Here is an example test workflow:
 # Set up environment variables for `gha-tools` scripts below
 export RAPIDS_BUILD_TYPE=branch
 export RAPIDS_REPOSITORY=rapidsai/cugraph
-export RAPIDS_REF_NAME=branch-23.08
-export RAPIDS_SHA=3f66966ec6beb678d531ec01713292e67ca1b290
+export RAPIDS_REF_NAME=branch-25.06
+
+# latest commit on that branch
+export RAPIDS_SHA="$(git ls-remote https://github.com/${RAPIDS_REPOSITORY}.git refs/heads/${RAPIDS_REF_NAME} | awk '{print $1}')"
 
 # Test how the scripts will work in CI
-# These invocations will interact with S3 directly and therefore use the AWS credentials that were volume mounted in
 export CI=true
-rapids-download-conda-from-s3 python
-rapids-get-artifact ci/cugraph/branch/branch-23.08/3f66966/cugraph_conda_python_cuda11_310_aarch64.tar.gz
+rapids-download-conda-from-github python
 
 # Test how the scripts will work locally
 export CI=false
-rapids-download-conda-from-s3 python
-rapids-get-artifact ci/cugraph/branch/branch-23.08/3f66966/cugraph_conda_python_cuda11_310_aarch64.tar.gz
+rapids-download-conda-from-github python
 ```
 
-Since the current working directory is a standard RAPIDS repository, the CI scripts can also be run directly to test the changes:
-
-```sh
-./ci/test_python.sh
-```
+For more details on how to extend this to fully reproducing CI locally, see "Reproducing CI Locally" ([link](https://docs.rapids.ai/resources/reproducing-ci/)).
 
 ## Testing in CI
 

README.md

@@ -44,14 +44,33 @@ The `gpuci_*` tools in this project are wrappers around the new tools for backwa
 1. They print a deprecation warning to use the `rapids-*` equivalents
 2. They re-export `GPUCI_*` env vars to the new `RAPIDS_*` equivalents
 
-### S3 tools for downloads.rapids.ai
+### Managing CI artifacts
 
-Some enhancements have been made to the S3 tools for interacting with [downloads.rapids.ai](https://github.com/rapidsai/downloads):
-* Added support for uploading and downloading wheel tarballs (built with cibuildwheel) using `rapids-upload-wheels-to-s3` and `rapids-download-wheels-from-s3`
-* Added support for misc one-off file or directory uploads by calling `rapids-upload-to-s3` directly
-* Print the human-browsable `https://downloads.rapids.ai/...` URL in the logs for convenience
-* `rapids-package-name` takes a package type and generates the name (e.g. `conda_cpp` -> `rmm_conda_cpp_x86_64.tar.gz`)
+This project contains some scripts for managing CI artifacts.
+
+* `rapids-download-{conda,wheels}-from-github`: download conda packages and wheels from the GitHub Actions artifact store
+* `rapids-upload-to-anaconda-github`: downloads conda packages from GitHub Actions artifact store and uploads conda channels on anaconda.org 
+* `rapids-wheels-anaconda-github`: downloads wheels from GitHub Actions artifact store and uploads them to the RAPIDS nightly index at https://pypi.anaconda.org/rapidsai-wheels-nightly/simple/
+* `rapids-package-name`: takes a package type and generate the artifact name (e.g. `conda_cpp` -> `rmm_conda_cpp_x86_64.tar.gz`)
+
+It also contains some scripts for working with CI artifacts on `downloads.rapids.ai`.
+
+* `rapids-upload-to-s3`: upload arbitrary files to the `downloads.rapids.ai` S3 bucket
+
+For more details, see the RAPIDS maintain docs:
+
+* "GitHub Actions" ([link](https://docs.rapids.ai/resources/github-actions/))
+* "Reproducing CI Locally" ([link](https://docs.rapids.ai/resources/reproducing-ci/))
+
+#### (deprecated) Storing conda packages and wheels on `downloads.rapids.ai`
+
+Support for storing conda packages and wheels on `downloads.rapids.ai` is considered **deprecated**.
+Switch those workloads to using the the GitHub Actions artifact store.
+But for backwards-compatibility, this project still contains some tools for doing that:
+
+* `rapids-download-{conda,wheels}-from-s3`: download conda packages and wheels from the `downloads.rapids.ai` S3 bucket
+* `rapids-upload-{conda,wheels}-to-s3`: upload conda packages and wheels to the `downloads.rapids.ai` S3 bucket
 
 ### Testing Scripts Locally
 
-See [CONTRIBUTING.md](CONTRIBUTING.md) for instructions on how to test these scripts locally.
+See [CONTRIBUTING.md](CONTRIBUTING.md) for instructions on how to test these scripts locally.

tools/rapids-configure-conda-channels

@@ -1,5 +1,10 @@
 #!/bin/bash
-# A utility script that configures conda channels
+# Selectively removes conda channels from the system-wide conda configuration,
+# based on runtime context.
+#
+# WARNING: Some projects use "strict channel priority" (https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-channels.html#strict-channel-priority),
+# and so rely on the assumption that this script may remove but NEVER PREPENDS channels.
+#
 
 set -euo pipefail
 

tools/rapids-extract-conda-files

@@ -1,6 +1,5 @@
 #!/bin/bash
-# A utility script that extracts all conda packages
-# after being downloaded by rapids-download-conda-from-s3
+# Extracts all conda packages in a directory into another directory that only includes conda packages.
 set -eo pipefail
 export RAPIDS_SCRIPT_NAME="rapids-extract-conda-files"
 

tools/rapids-wheel-ctk-name-gen

@@ -1,11 +1,11 @@
 #!/bin/bash
-# A utility script that generates CUDA suffix in the format "cu${VER}" where `$VER`
+# Generates CUDA suffix in the format "cu${VER}" where `$VER`
 # is the CUDA major version (for example, "cu11").
 # Positional Arguments:
 #   1) ctk tag
 set -eu -o pipefail
 
-if [ -z "$1" ]; then
+if [ -z "${1-}" ]; then
   rapids-echo-stderr "Must specify input argument: CTK_TAG"
   exit 1
 fi