Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions tools/rapids-get-pr-conda-artifact-github
Original file line number Diff line number Diff line change
Expand Up @@ -23,20 +23,23 @@ commit="${4:-}"

# If commit is not provided, get the latest commit on the PR
if [[ -z "${commit}" ]]; then
commit=$(rapids-retry gh pr view "${pr}" --repo rapidsai/"${repo}" --json headRefOid --jq '.headRefOid')
commit=$(git ls-remote https://github.com/rapidsai/"${repo}".git refs/heads/pull-request/"${pr}" | cut -f1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two seem the same -- is there a reason to change this and miss out on the retries?

@jayavenkatesh19 jayavenkatesh19 Apr 30, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refer to the comment above about the rapids-retry PR. (#164).

However, using gh lets us access PR artifacts from private repositories, so I will switch it back to using gh pr view

fi

# Use a subshell to isolate environment variable changes
(
# Set up environment variables needed for GitHub artifact download
export RAPIDS_REPOSITORY="rapidsai/${repo}"
export RAPIDS_BUILD_TYPE="pull-request"
export RAPIDS_SHA="${commit}"
export RAPIDS_REF_NAME="pull-request/${pr}"
RAPIDS_REPOSITORY="rapidsai/${repo}"
RAPIDS_SHA="${commit}"
RAPIDS_REF_NAME="pull-request/${pr}"

# Generate the artifact name
artifact_name="$(RAPIDS_NO_PKG_EXTENSION=true rapids-package-name "conda_${package_type}")"
pkg_name="$(RAPIDS_NO_PKG_EXTENSION=true rapids-package-name "conda_${package_type}")"
github_run_id=$(gh run list --repo "${RAPIDS_REPOSITORY}" --branch "${RAPIDS_REF_NAME}" --commit "${RAPIDS_SHA}" --json databaseId --jq '.[0] | .databaseId')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to use rapids-retry in case the GitHub API fails? Will the $(...) parse the output correctly if it is retried?

Suggested change
github_run_id=$(gh run list --repo "${RAPIDS_REPOSITORY}" --branch "${RAPIDS_REF_NAME}" --commit "${RAPIDS_SHA}" --json databaseId --jq '.[0] | .databaseId')
github_run_id=$(rapids-retry gh run list --repo "${RAPIDS_REPOSITORY}" --branch "${RAPIDS_REF_NAME}" --commit "${RAPIDS_SHA}" --json databaseId --jq '.[0] | .databaseId')

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR (#164) changes the way rapids-retry works with stdout. The --quiet flag being added will ensure that that none of the rapids-logger messages are captured by this variable.

Once those changes have been approved and merged, I will make those changes here.

unzip_dest="${RAPIDS_UNZIP_DIR:-$(mktemp -d)}"

unzip_dir=$(rapids-download-from-github "${artifact_name}")
echo -n "${unzip_dir}"
rapids-echo-stderr "Downloading and decompressing ${pkg_name} from Run ID ${github_run_id} into ${unzip_dest}"
rapids-retry gh run download "${github_run_id}" --repo "${RAPIDS_REPOSITORY}" --name "${pkg_name}" --dir "${unzip_dest}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it seems like this is a workaround for the current behavior of RAPIDS_BUILD_TYPE=pull-request in rapids-github-run-id -- should we change that instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In rapids-github-run-id, the default value of the Run ID is the GITHUB_RUN_ID environment variable set by Github. Making this change there wouldn't help in the usecase where PR artifacts are needed as part of a Github Workflow run.

Making this change here would be the best option for both Actions and for reproducing CI locally.


echo -n "${unzip_dest}"
)
19 changes: 11 additions & 8 deletions tools/rapids-get-pr-wheel-artifact-github

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this logic be refactored into a common file? See https://github.com/rapidsai/gha-tools/blob/main/tools/_rapids-download-from-s3 for an example.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's a great idea.

Refactored the logic into a common file called _rapids-get-pr-artifact-github, mirroring the PR artifact flow used earlier.

Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,23 @@ fi

# If commit is not provided, get the latest commit on the PR
if [[ -z "${commit}" ]]; then
commit=$(rapids-retry gh pr view "${pr}" --repo rapidsai/"${repo}" --json headRefOid --jq '.headRefOid')
commit=$(git ls-remote https://github.com/rapidsai/"${repo}".git refs/heads/pull-request/"${pr}" | cut -f1)
fi

# Use a subshell to isolate environment variable changes
(
# Set up environment variables needed for GitHub artifact download
export RAPIDS_REPOSITORY="rapidsai/${repo}"
export RAPIDS_BUILD_TYPE="pull-request"
export RAPIDS_SHA="${commit}"
export RAPIDS_REF_NAME="pull-request/${pr}"
RAPIDS_REPOSITORY="rapidsai/${repo}"
RAPIDS_SHA="${commit}"
RAPIDS_REF_NAME="pull-request/${pr}"

# Generate the artifact name
artifact_name="$(RAPIDS_NO_PKG_EXTENSION=true rapids-package-name "wheel_${package_type}")"
pkg_name="$(RAPIDS_NO_PKG_EXTENSION=true rapids-package-name "wheel_${package_type}")"
github_run_id=$(gh run list --repo "${RAPIDS_REPOSITORY}" --branch "${RAPIDS_REF_NAME}" --commit "${RAPIDS_SHA}" --json databaseId --jq '.[0] | .databaseId')
unzip_dest="${RAPIDS_UNZIP_DIR:-$(mktemp -d)}"

unzip_dir=$(rapids-download-from-github "${artifact_name}")
echo -n "${unzip_dir}"
rapids-echo-stderr "Downloading and decompressing ${pkg_name} from Run ID ${github_run_id} into ${unzip_dest}"
rapids-retry gh run download "${github_run_id}" --repo "${RAPIDS_REPOSITORY}" --name "${pkg_name}" --dir "${unzip_dest}"

echo -n "${unzip_dest}"
)