[ci] Adding test dependency script to rocm-libraries#7219
Conversation
Integrate determine_rocm_test_dependencies.py from TheRock to dynamically determine which components to test based on TEST_SUBPROJECTS declarations in CMakeLists.txt files. When multiple components are touched (e.g., projects/rocblas and projects/rocprim), their test dependencies are combined. Dynamic deps are also combined with hardcoded projects_to_test. Falls back to hardcoded projects_to_test if TheRock script is not available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Query TheRock for test dependencies on each component individually, since some components have dynamic deps while others don't yet. Components with dynamic deps use those; components without fall back to hardcoded projects_to_test. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When TheRock's script returns only the component itself (e.g., ["hipblaslt"] for hipblaslt), treat this as "no tests available" and fall back to project_map's projects_to_test. Added unit tests for dynamic deps behavior with mocked TheRock responses. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
I think we can also contrast this to ROCm/TheRock#4817 |
|
The rocblas downstream runner content looks good: |
i cancelled them as the projects_to_test was proof |
I'm not a fan of the hard-coded dictionary, but this is a great reference to use as we continue to add more project dependencies |
jayhawk-commits
left a comment
There was a problem hiding this comment.
collect_projects_to_run() first collects tests_per_component for every changed subtree at therock_matrix.py#L205-L218. Later, inside the loop that emits each project matrix row, it unconditionally adds every dependency list from tests_per_component into that row's projects_to_test at therock_matrix.py#L287-L299.
That is okay only when the changed components all collapse into a single build row. When the PR touches CI files or otherwise selects multiple independent project groups, each row receives the global dependency union even though its cmake_options still build only that row's project group. The current PR CI shows the failure mode: a Linux test job tried to run ./build/bin/hipblas-test and failed immediately with FileNotFoundError because that test executable was not present in the artifacts downloaded for that row:
INFO:root:++ Exec [...]$ ./build/bin/hipblas-test ...
FileNotFoundError: [Errno 2] No such file or directory: './build/bin/hipblas-test'
Public job evidence: https://github.com/ROCm/rocm-libraries/actions/runs/25692469298/job/75435412230?pr=7219
Previously, tests_per_component collected test dependencies for all changed components, but then added ALL of them to EVERY project row. This caused failures when multiple independent project groups were selected (e.g., CI file changes), as each row received tests for components it didn't build. Now track which components belong to which project and only add test dependencies for components that belong to the current project row. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…OCm/rocm-libraries into users/geomin12/test-dependency
TorreZuk
left a comment
There was a problem hiding this comment.
Looks good once we can see it run the dependency tests instead of default ones. It did print a list for rocblas that looks correct but didn't correctly run them so must be bugs to fix.
ah so in this CI job, due to this being a CI update, it runs all of them. It will work for specific subcomponents such as this: https://github.com/ROCm/rocm-libraries/actions/runs/25573695018/job/75075609400#step:7:16 |
The script from ROCm/TheRock#3491 is applied to rocm-libraries. Currently, the script only works for rocblas and hipsparse
Changes:
By default, we try to use the script to pull proper dependencies to test.
If no return, we use the old method of project_map_data
Test with rocblas only: https://github.com/ROCm/rocm-libraries/actions/runs/25573695018/job/75075609400#step:7:16
Test with hipblaslt only: https://github.com/ROCm/rocm-libraries/actions/runs/25573338826/job/75074289228#step:7:16
Test with both rocblas and hipblaslt: https://github.com/ROCm/rocm-libraries/actions/runs/25573843203/job/75076019926#step:7:16
As we chat with component teams, we will determine what projects are impacted by components.
Progress on ROCm/TheRock#3491
AIHPBLAS-3530