Skip to content

Jainprad/codecoverage lib - #8427

Closed
jainprad wants to merge 61 commits into
ROCm:developfrom
jainprad:jainprad/codecoverage_lib
Closed

jainprad wants to merge 61 commits into
ROCm:developfrom
jainprad:jainprad/codecoverage_lib

Conversation

@jainprad

Copy link
Copy Markdown

code coverage

Adds a TheRock-based code coverage CI flow and enables per-project coverage configuration.

Changes:

  • .github/scripts/therock_configure_coverage.py — configure coverage jobs per project
  • .github/workflows/therock-ci-coverage.yml — standalone coverage workflow
  • .github/workflows/therock-ci.yml, therock-ci-linux.yml — wire coverage into CI
  • projects/hipdnn/CMakeLists.txt, projects/hipdnn/cmake/CheckToolVersion.cmake — coverage build adjustments

Made with Cursor

reboss added 30 commits May 22, 2026 15:32
The current therock_configure_ci.py doesn't have a clean way to only
configure the project that has changed, and not other components that
the changed component has been grouped together with.  For code coverage
reports, it does not make sense to even test downstream components.
jainprad and others added 2 commits June 16, 2026 08:32
The coverage build matrix (e.g. hipDNN) passes THEROCK_ENABLE_IREE_LIBS=ON,
which makes CMake configure the iree-compiler/fusilli/fusilliprovider
subprojects. The coverage workflow's fetch step did not fetch the IREE
sources, so the build failed with a missing iree-libs/fusilli/CMakeLists.txt.

Mirror therock-ci-linux.yml by conditionally adding --include-iree-libs to
fetch_sources.py when the project's cmake options enable IREE libs.

Co-authored-by: Cursor <cursoragent@cursor.com>
…erage_lib

Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	projects/hipdnn/cmake/CheckToolVersion.cmake
@codecov-commenter

codecov-commenter commented Jun 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

❌ Your project status has failed because the head coverage (77.83%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #8427   +/-   ##
========================================
  Coverage    65.17%   65.17%           
========================================
  Files         2597     2597           
  Lines       404636   404636           
  Branches     60332    60332           
========================================
+ Hits        263690   263695    +5     
+ Misses      121086   121082    -4     
+ Partials     19860    19859    -1     
Flag Coverage Δ *Carryforward flag
TensileLite 31.04% <ø> (ø) Carriedforward from d974af3
hipBLAS 90.65% <ø> (ø) Carriedforward from d974af3
hipBLASLt 41.20% <ø> (ø) Carriedforward from d974af3
hipCUB 82.68% <ø> (ø) Carriedforward from d974af3
hipDNN 86.69% <ø> (+0.01%) ⬆️
hipFFT 50.97% <ø> (ø) Carriedforward from d974af3
hipRAND 76.12% <ø> (ø) Carriedforward from d974af3
hipSOLVER 69.18% <ø> (ø) Carriedforward from d974af3
hipSPARSE 86.55% <ø> (ø) Carriedforward from d974af3
rocBLAS 48.10% <ø> (ø) Carriedforward from d974af3
rocFFT 49.48% <ø> (ø) Carriedforward from d974af3
rocRAND 57.02% <ø> (ø) Carriedforward from d974af3
rocSOLVER 77.83% <ø> (ø) Carriedforward from d974af3
rocSPARSE 72.62% <ø> (ø) Carriedforward from d974af3
rocThrust 91.34% <ø> (ø) Carriedforward from d974af3

*This pull request uses carry forward flags. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jainprad
jainprad requested a review from a team as a code owner June 16, 2026 15:57
jainprad and others added 5 commits June 16, 2026 16:28
Switch the coverage matrix to target hipRAND, and make primary-project
selection deterministic by preferring the first coverage-enabled entry in
projects_to_test. This is required for multi-project groups (e.g. the rand
group's projects_to_test is rocrand,hiprand) where the group key never
matches an individual test project name.

Co-authored-by: Cursor <cursoragent@cursor.com>
In merged "enable all" groups, projects_to_test contains both hiprand and
hipdnn. The changed-project override was re-selecting hipdnn (a changed but
no-longer-coverage-enabled project), emptying the coverage matrix. Require the
overriding changed project to be coverage-enabled so hiprand stays selected.

Co-authored-by: Cursor <cursoragent@cursor.com>
The coverage job inherited the merged mega-group cmake options (when a workflow
change enables all projects, rand is absorbed into the miopen group), so the
HIPRAND build was building hipdnn/providers and failing on an unrelated
hipblaslt-provider clang-tidy step. Pin the coverage build to its own options
(-DTHEROCK_ENABLE_RAND=ON -DTHEROCK_ENABLE_ALL=OFF).

Also make the hipRAND coverage report step robust in the TheRock dist build:
honor LLVM_TOOLS_SEARCH_PREFIX when locating llvm-profdata/llvm-cov, and use
$<TARGET_FILE:hiprand> instead of the hardcoded debug-suffixed libhiprand-d.so.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the GPU test/report stage from the coverage workflow, leaving only the
coverage build job.

Co-authored-by: Cursor <cursoragent@cursor.com>
--build-dir TheRock/build-coverage \
--upload

therock-coverage-test:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be replaced with the test flow from therock-ci-test-component

# mega-group options that would otherwise build unrelated components.
COVERAGE_PROJECT_METADATA = {
"hiprand": ("hipRAND", "ml-libs/hipRAND"),
"hiprand": ("hipRAND", "ml-libs/hipRAND", "-DTHEROCK_ENABLE_RAND=ON -DTHEROCK_ENABLE_ALL=OFF"),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is necessary, since these should get added in an earlier step, for building. I could be wrong though

"-DTHEROCK_ENABLE_RAND=ON -DTHEROCK_ENABLE_ALL=OFF"

jainprad and others added 9 commits June 17, 2026 18:58
…stage

Wire the coverage build into the same build -> test-packages -> test-component
flow used by mainline CI, then merge/report:

- therock-test-component.yml / therock-test-packages.yml: add an opt-in
  `coverage` input (default false). When set, the component test job exports
  LLVM_PROFILE_FILE and uploads the resulting profraw files as an artifact.
  Mainline CI is unaffected since it does not pass coverage.
- therock-ci-coverage.yml: the build job stages the instrumented libraries and
  version-matched llvm-profdata/llvm-cov into an artifact; add a test job that
  reuses therock-test-packages.yml with coverage=true, and a report job that
  merges the profraw files and runs llvm-cov report/show/export.
- therock-ci.yml: forward projects_to_test and test_type to the coverage flow.
- therock_configure_coverage.py: pin projects_to_test to the coverage project
  so the test stage matches the pinned (single-project) build.

Co-authored-by: Cursor <cursoragent@cursor.com>
In container jobs the github.workspace expression resolves to the host path
(/home/runner/_work/...) while upload-artifact uses the container workspace
(/__w/...), so the staged coverage objects and profraw files were written to a
different path than the upload step read, failing with "No files were found".

Use $GITHUB_WORKSPACE (container path) when writing files and relative paths in
upload-artifact so both sides agree. Set LLVM_PROFILE_FILE at runtime via
GITHUB_ENV using $GITHUB_WORKSPACE for the same reason.

Co-authored-by: Cursor <cursoragent@cursor.com>
This PR now targets hipRAND code coverage only; hipdnn is no longer a
coverage-enabled project. Revert the projects/hipdnn build changes so the PR
touches no hipdnn sources, which also stops the unrelated HipDNN Superbuild CI
from running on this PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
The staged llvm-profdata/llvm-cov are dynamically linked against
libLLVM.so.<ver>, which is absent on a plain runner, so the merge step failed
with "error while loading shared libraries: libLLVM.so". Stage libLLVM.so next
to the tools, run the report job in the build container, and point
LD_LIBRARY_PATH at the staged libs.

Co-authored-by: Cursor <cursoragent@cursor.com>
The build stage needs the (unmerged) export ref that ships llvm-cov/llvm-profdata
in the dist, but that ref predates build_tools/github_actions/cleanup_processes.sh,
so the shared test flow failed in its post-job cleanup. Run the test stage on a
current TheRock commit (2d417fa) that has the cleanup script. The report stage
uses the build-staged llvm tools, so it is unaffected by the test ref.

Co-authored-by: Cursor <cursoragent@cursor.com>
The merge/report step uses bash-only constructs (mapfile, process
substitution), but ran under the container's default sh, failing with
"syntax error near unexpected token '<'". Pin the step shell to bash.

Co-authored-by: Cursor <cursoragent@cursor.com>
llvm-profdata/llvm-cov link libLLVM plus TheRock's bundled sysdeps
(librocm_sysdeps_z, etc.), so the report job failed with "error while loading
shared libraries: librocm_sysdeps_z.so.1". Stage the whole llvm lib dir and all
librocm_sysdeps_* next to the tools, and add ldd diagnostics to both stages.

Co-authored-by: Cursor <cursoragent@cursor.com>
The sysdeps copy used find -type f, which skipped the SONAME symlinks
(librocm_sysdeps_z.so.1 -> ...1.3.2), so the loader still reported
"librocm_sysdeps_z.so.1: cannot open shared object file" even though the
versioned file was staged. Drop -type f so the symlinks are staged too.

Co-authored-by: Cursor <cursoragent@cursor.com>
- Check out the sources in the report job so llvm-cov show can annotate the
  HTML report (previously the source file could not be found).
- Make the lcov paths repo-relative and upload coverage.info to Codecov under
  the project flag (non-fatal), so the report is browsable online.
- Generate coverage-report-links.txt listing the report outputs, instrumented
  source/objects, tooling libraries, tests executed, the TheRock build/pack
  location, and the report upload/download locations; also surface a summary
  with the run/Codecov links in the job summary.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been inactive for 25 days and will be marked as stale.

If you would like to keep this PR open, please:

  • Add new commits
  • Add a comment explaining why it should remain open

This PR will be automatically closed in 5 days if no further activity occurs.

@github-actions github-actions Bot added the Stale PR has no activity for 25+ days label Jul 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically closed due to inactivity (30 days with no updates).

If you'd like to continue working on this, feel free to reopen the PR or create a new one.

@github-actions github-actions Bot closed this Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants