Skip to content

[ci] Adding test dependency script to rocm-libraries#7219

Open
geomin12 wants to merge 10 commits into
developfrom
users/geomin12/test-dependency
Open

[ci] Adding test dependency script to rocm-libraries#7219
geomin12 wants to merge 10 commits into
developfrom
users/geomin12/test-dependency

Conversation

@geomin12
Copy link
Copy Markdown
Collaborator

@geomin12 geomin12 commented May 8, 2026

The script from ROCm/TheRock#3491 is applied to rocm-libraries. Currently, the script only works for rocblas and hipsparse

Changes:

As we chat with component teams, we will determine what projects are impacted by components.

Progress on ROCm/TheRock#3491

AIHPBLAS-3530

geomin12 and others added 4 commits May 6, 2026 20:35
Integrate determine_rocm_test_dependencies.py from TheRock to
dynamically determine which components to test based on
TEST_SUBPROJECTS declarations in CMakeLists.txt files.

When multiple components are touched (e.g., projects/rocblas and
projects/rocprim), their test dependencies are combined. Dynamic
deps are also combined with hardcoded projects_to_test.

Falls back to hardcoded projects_to_test if TheRock script
is not available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Query TheRock for test dependencies on each component individually,
since some components have dynamic deps while others don't yet.
Components with dynamic deps use those; components without fall back
to hardcoded projects_to_test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When TheRock's script returns only the component itself (e.g., ["hipblaslt"]
for hipblaslt), treat this as "no tests available" and fall back to
project_map's projects_to_test.

Added unit tests for dynamic deps behavior with mocked TheRock responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@geomin12 geomin12 requested a review from a team as a code owner May 8, 2026 21:15
@github-actions github-actions Bot added github actions project: none Does not target any component labels May 8, 2026
@TorreZuk
Copy link
Copy Markdown
Contributor

TorreZuk commented May 8, 2026

I think we can also contrast this to ROCm/TheRock#4817

@TorreZuk
Copy link
Copy Markdown
Contributor

TorreZuk commented May 8, 2026

The rocblas downstream runner content looks good:
{'linux_projects': '[{"cmake_options": "-DTHEROCK_ENABLE_BLAS=ON -DTHEROCK_ENABLE_ALL=OFF", "projects_to_test": "rocsolver,rocblas,hipblas"}]', 'test_type': 'standard'}
But it didn't run the tests.

@geomin12
Copy link
Copy Markdown
Collaborator Author

geomin12 commented May 8, 2026

The rocblas downstream runner content looks good: {'linux_projects': '[{"cmake_options": "-DTHEROCK_ENABLE_BLAS=ON -DTHEROCK_ENABLE_ALL=OFF", "projects_to_test": "rocsolver,rocblas,hipblas"}]', 'test_type': 'standard'} But it didn't run the tests.

i cancelled them as the projects_to_test was proof

@geomin12
Copy link
Copy Markdown
Collaborator Author

geomin12 commented May 8, 2026

I think we can also contrast this to ROCm/TheRock#4817

I'm not a fan of the hard-coded dictionary, but this is a great reference to use as we continue to add more project dependencies

Copy link
Copy Markdown
Collaborator

@jayhawk-commits jayhawk-commits left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

collect_projects_to_run() first collects tests_per_component for every changed subtree at therock_matrix.py#L205-L218. Later, inside the loop that emits each project matrix row, it unconditionally adds every dependency list from tests_per_component into that row's projects_to_test at therock_matrix.py#L287-L299.

That is okay only when the changed components all collapse into a single build row. When the PR touches CI files or otherwise selects multiple independent project groups, each row receives the global dependency union even though its cmake_options still build only that row's project group. The current PR CI shows the failure mode: a Linux test job tried to run ./build/bin/hipblas-test and failed immediately with FileNotFoundError because that test executable was not present in the artifacts downloaded for that row:

INFO:root:++ Exec [...]$ ./build/bin/hipblas-test ...
FileNotFoundError: [Errno 2] No such file or directory: './build/bin/hipblas-test'

Public job evidence: https://github.com/ROCm/rocm-libraries/actions/runs/25692469298/job/75435412230?pr=7219

geomin12 and others added 4 commits May 12, 2026 09:54
Previously, tests_per_component collected test dependencies for all
changed components, but then added ALL of them to EVERY project row.
This caused failures when multiple independent project groups were
selected (e.g., CI file changes), as each row received tests for
components it didn't build.

Now track which components belong to which project and only add test
dependencies for components that belong to the current project row.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@TorreZuk TorreZuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good once we can see it run the dependency tests instead of default ones. It did print a list for rocblas that looks correct but didn't correctly run them so must be bugs to fix.

@geomin12
Copy link
Copy Markdown
Collaborator Author

Looks good once we can see it run the dependency tests instead of default ones. It did print a list for rocblas that looks correct but didn't correctly run them so must be bugs to fix.

ah so in this CI job, due to this being a CI update, it runs all of them. It will work for specific subcomponents such as this: https://github.com/ROCm/rocm-libraries/actions/runs/25573695018/job/75075609400#step:7:16

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants