Skip to content

[REVIEW][OPS] gpuCI Script Updates - #6329

Merged
mike-wendt merged 61 commits into
branch-0.16from
enh-gpuci-updates
Oct 16, 2020
Merged

[REVIEW][OPS] gpuCI Script Updates#6329
mike-wendt merged 61 commits into
branch-0.16from
enh-gpuci-updates

Conversation

@mike-wendt

@mike-wendt mike-wendt commented Sep 25, 2020

Copy link
Copy Markdown
Contributor

Overview

First step before "Project Flash" to improve gpuCI scripts and allow for use of different node types for CPU builds. Post "Project Flash" there will be another round of improvements and updates with the goal to make these scripts easier to maintain.

Changes

  • Consolidates some of the CPU gpuCI scripts, reducing number of scripts to maintain
  • Standardizes naming of the upload script to upload.sh
  • Changes logging to use gpuci_logger where practical
  • Updates comments and documentation where necessary
  • Enables gpuCI to override environment variables based on node types
  • Reorders upload script to save ~5 mins or more in testing time

@mike-wendt mike-wendt added the 2 - In Progress Currently a work in progress label Sep 25, 2020
@mike-wendt mike-wendt self-assigned this Sep 25, 2020
@GPUtester

Copy link
Copy Markdown
Collaborator

Please update the changelog in order to start CI tests.

View the gpuCI docs here.

@mike-wendt mike-wendt changed the title [OPS] gpuCI Script Updates [OPS] gpuCI Script Updates [skip-ci] Sep 25, 2020
@mike-wendt
mike-wendt requested a review from a team as a code owner October 15, 2020 08:01
@mike-wendt mike-wendt added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Oct 15, 2020
@mike-wendt mike-wendt changed the title [WIP][OPS] gpuCI Script Updates [REVIEW][OPS] gpuCI Script Updates Oct 15, 2020
@mike-wendt

Copy link
Copy Markdown
Contributor Author

Once tests pass will merge in morning

Comment thread ci/gpu/build.sh
Comment thread ci/gpu/build.sh
@mike-wendt

Copy link
Copy Markdown
Contributor Author

With rapidsai/gpuci-build-environment#135 and rapidsai/gpuci-tools#12 merged, A100 tests should pass now

@mike-wendt

Copy link
Copy Markdown
Contributor Author

Incoming fix for the conda builds, waiting on gpuci-tools to build again

@mike-wendt

Copy link
Copy Markdown
Contributor Author

Woot A100 CUDA 11 tests work with all of the above patches. Running a test with a branch build to verify uploads work off of this branch. That combined with these tests can merge this PR

@mike-wendt

Copy link
Copy Markdown
Contributor Author

Will merge first thing in the morning after final ops review

@mike-wendt

Copy link
Copy Markdown
Contributor Author

Successful branch test and two survived 504 errors during upload https://gpuci.gpuopenanalytics.com/job/rapidsai/job/gpuci/job/cudf/job/prb/job/cudf-cpu-build/69425/console

May want to retry 3 times instead. Will get feedback before submitting a change

@ajschmidt8 ajschmidt8 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two small things, otherwise 👍

Comment thread ci/cpu/upload.sh Outdated
Comment thread ci/docs/build.sh Outdated
All of these pkgs are already installed from  and  is in the gpuCI images already
@mike-wendt

Copy link
Copy Markdown
Contributor Author

Cancelled tests, admin merging due to recent changes changes being a config tweak, doc update, and duplicate code removal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 - Ready for Review Ready for review by team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants