Skip to content

[TheRock-CI] Add a code coverage workflow - #4441

Closed
reboss wants to merge 44 commits into
ROCm:developfrom
reboss:therock-codecov
Closed

reboss wants to merge 44 commits into
ROCm:developfrom
reboss:therock-codecov

Conversation

@reboss

@reboss reboss commented Feb 9, 2026 •

Copy link
Copy Markdown
Member

Motivation

This PR adds a GitHub workflow to generate code coverage reports using TheRock CI.

Technical Details

Currently a work in progress. Current assumptions are:

  1. TheRock super build is able to pass project specific cmake defines to the project_to_test using extra_cmake_options variable
  2. The project_to_test will expose the coverage target somewhere within the build, currently it points to /build/TheRock/rocm-libraries/${{ inputs.project_to_test }}/build, which probably is incorrect
  3. We can add codecov.io secrets to the project to be consumed by this workflow

Test Plan

We need to verify this CI works for a PR that touches one or multiple projects. We can make a small change that can be reverted before merge to trigger this pipeline to test the two scenarios.

Test Result

Submission Checklist

Comment thread .github/workflows/therock-ci-linux.yml Outdated
@@ -5,7 +5,7 @@ on:
inputs:
cmake_options:
type: string
projects_to_test:
project_to_test:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please pick up the latest changes to the workflow files. This has been renamed to projects_to_test with c124e5c

Comment thread .github/workflows/therock-ci-linux.yml Outdated
Comment on lines +104 to +111
set -euo pipefail
EXTRA_OPTS="-DTHEROCK_ROCM_LIBRARIES_SOURCE_DIR=../ ${{ inputs.cmake_options }}"
PROJECT_KEY="$(echo "${{inputs.project_to_test }}" | tr '[:lower:]' '[:upper:]')"
if [[ "${{ inputs.test_type }}" == "coverage" ]]; then
EXTRA_OPTS="${EXTRA_OPTS} -D${PROJECT_KEY_UPPER}_ENABLE_COVERAGE=ON"
echo "Coverage enabled for ${PROJECT_KEY_UPPER}: ${EXTRA_OPTS}"
fi
export extra_cmake_options="${EXTRA_OPTS}"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move as much of this into the env block of this step, or if some processing needs to be done, It can be a prior, separate step that produces output values that can be fed into env.

Comment thread .github/workflows/therock-ci.yml Outdated
secrets: inherit
with:
cmake_options: ${{ matrix.projects.cmake_options }}
project_to_test: ${{ matrix.projects.project_to_test }}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see previous comment on projects_to_test. This input might contain multiple projects.

@geomin12 geomin12 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idk if this PR is relevant anymore after our discussoin, but some comments

Comment thread .github/workflows/therock-ci-linux.yml Outdated
@@ -101,6 +101,14 @@ jobs:
extra_cmake_options: "-DTHEROCK_ROCM_LIBRARIES_SOURCE_DIR=../ ${{ inputs.cmake_options }}"
BUILD_DIR: build
run: |
set -euo pipefail

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this apply for windows too? or not yet?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't do code coverage on Windows. I don't think there would be any added value to doing this on Windows either

Comment thread .github/workflows/therock-ci-linux.yml Outdated
@@ -101,6 +101,14 @@ jobs:
extra_cmake_options: "-DTHEROCK_ROCM_LIBRARIES_SOURCE_DIR=../ ${{ inputs.cmake_options }}"
BUILD_DIR: build
run: |
set -euo pipefail
EXTRA_OPTS="-DTHEROCK_ROCM_LIBRARIES_SOURCE_DIR=../ ${{ inputs.cmake_options }}"
PROJECT_KEY="$(echo "${{inputs.project_to_test }}" | tr '[:lower:]' '[:upper:]')"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we get a quick comment explaining this logic?

Comment thread .github/workflows/therock-ci-linux.yml Outdated
run: |
BUILD_DIR="/build/TheRock/rocm-libraries/${{ inputs.project_to_test }}/build"
cmake --build "${BUILD_DIR}" --target coverage || make -C "${BUILD_DIR}" coverage
- name: Upload coverage to Codecov

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: new line

@codecov-commenter

codecov-commenter commented Mar 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files
@@             Coverage Diff              @@
##           develop    #4441       +/-   ##
============================================
+ Coverage    70.89%   80.50%    +9.61%     
============================================
  Files         1106      388      -718     
  Lines       203899    49053   -154846     
  Branches     30141     6783    -23358     
============================================
- Hits        144553    39490   -105063     
+ Misses       47226     7286    -39940     
+ Partials     12120     2277     -9843     
Flag Coverage Δ
hipBLAS ?
hipBLASLt ?
hipCUB ?
hipDNN 80.50% <ø> (-1.92%) ⬇️
hipSOLVER ?
rocFFT ?
rocSPARSE ?
see 735 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@reboss
reboss force-pushed the therock-codecov branch from 9613ed1 to d731dc2 Compare April 2, 2026 15:39
@reboss
reboss force-pushed the therock-codecov branch 5 times, most recently from 38e4fcd to aed410b Compare April 18, 2026 02:08
@adickin-amd

Copy link
Copy Markdown
Contributor

Waiting on ROCm/TheRock#4931

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been inactive for 25 days and will be marked as stale.

If you would like to keep this PR open, please:

  • Add new commits
  • Add a comment explaining why it should remain open

This PR will be automatically closed in 5 days if no further activity occurs.

@github-actions github-actions Bot added the Stale PR has no activity for 25+ days label Jun 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically closed due to inactivity (30 days with no updates).

If you'd like to continue working on this, feel free to reopen the PR or create a new one.

@github-actions github-actions Bot closed this Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github actions organization: ROCm project: hipdnn project: none Does not target any component Stale PR has no activity for 25+ days

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants