Skip to content

Add 'build_workflow_name' input, to allow overriding where scripts look for artifacts - #331

Merged
jameslamb merged 9 commits into
branch-25.08from
nightly-fix-for-custom-workflows
May 28, 2025
Merged

Add 'build_workflow_name' input, to allow overriding where scripts look for artifacts#331
jameslamb merged 9 commits into
branch-25.08from
nightly-fix-for-custom-workflows

Conversation

@jayavenkatesh19

@jayavenkatesh19 jayavenkatesh19 commented Apr 28, 2025

Copy link
Copy Markdown
Contributor

Uniquely identifying an artifact from the GitHub Actions artifact store requires identifying the workflow run that produced it.

In PR CI, this is trivial because the uploading-artifacts and downloading-artifacts jobs are all running as part of the same workflow.

In other setups though, that isn't true. For example, RAPIDS uses 2 separate nightly workloads:

  • build.yaml = build packages and publish the package artifacts
  • test.yaml = pull those artifacts and run tests with them

For this case, in the relevant gha-tools scripts we default to assuming that branch or nightly runs want to pull artifacts from runs of a workflow literally called build.yaml:

run_id=$(
  RAPIDS_RETRY_SLEEP=30 \
    rapids-retry --quiet gh run list \
      ... \
      --workflow "${RAPIDS_BUILD_WORKFLOW_NAME:-build.yaml}" \
      ...
)

ref: https://github.com/rapidsai/gha-tools/blob/570fc211e6bf1bfb91098aa1bc4823ef7bb71fc2/tools/rapids-github-run-id#L66

There might be cases in the future where that assumption doesn't hold, and people want to pull artifacts in one workflow run from some other workflow which is NOT called build.yaml.

This PR + rapidsai/shared-actions#52 add support for specifying the name of that "some other workflow" as an input, overriding the build.yaml default.

For more context, see rapidsai/gha-tools#162 (comment)

How I tested this

Tested with an rmm PR: rapidsai/rmm#1909

@jayavenkatesh19 jayavenkatesh19 self-assigned this Apr 28, 2025
@jayavenkatesh19
jayavenkatesh19 requested a review from a team as a code owner April 28, 2025 21:37
@jayavenkatesh19 jayavenkatesh19 added the improvement Improves an existing functionality label Apr 28, 2025
@jayavenkatesh19 jayavenkatesh19 added the non-breaking Introduces a non-breaking change label Apr 28, 2025
@jameslamb
jameslamb changed the base branch from branch-25.06 to branch-25.08 May 22, 2025 21:27
@jameslamb

Copy link
Copy Markdown
Member

I've added context for why we're doing this in the description, and re-targeted this at branch-25.08. I need to run right now, but when I return to this I'll get it updated and merge it.

@jameslamb jameslamb changed the title specify a custom workflow to download artifacts from for tests [DO NOT MERGE] specify a custom workflow to download artifacts from for tests May 23, 2025
jameslamb added a commit to rapidsai/shared-actions that referenced this pull request May 23, 2025
Adding a new optional input to specify a custom workflow (only if
required, defaults to build.yaml) to download artifacts from, primarily
for nightly tests (see
rapidsai/gha-tools#162 (comment))

Complementary changes in `shared-workflows`:
rapidsai/shared-workflows#331

---------

Co-authored-by: James Lamb <jaylamb20@gmail.com>
@jameslamb jameslamb changed the title [DO NOT MERGE] specify a custom workflow to download artifacts from for tests [DO NOT MERGE] Add 'build_workflow_name' input, to allow overriding where scripts look for artifacts May 28, 2025
@jameslamb

Copy link
Copy Markdown
Member

I just pushed some changes in c34104a

  • custom-job.yaml should be included as well.. it's sometimes used to run tests
  • clarified the description for the new build_workflow_name input

I've also changed the PR title to more accurately reflect the goal here.

@jameslamb jameslamb changed the title [DO NOT MERGE] Add 'build_workflow_name' input, to allow overriding where scripts look for artifacts Add 'build_workflow_name' input, to allow overriding where scripts look for artifacts May 28, 2025
@jameslamb
jameslamb merged commit cff1b66 into branch-25.08 May 28, 2025
@jameslamb
jameslamb deleted the nightly-fix-for-custom-workflows branch May 28, 2025 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants